Delay Resource. More...

#include <MprDelay.h>

Inheritance diagram for MprDelay:

Collaboration diagram for MprDelay:

Classes
class	MprDelayFifo
	Fifo for Delay resource. More...

Public Types
enum	{ DEFAULT_FIFO_CAPACITY = 1000 }

Public Types inherited from MpResource
enum	VisitState { NOT_VISITED, IN_PROGRESS, FINISHED }
	Graph traversal states that are used when running a topological sort to order resources within a flow graph. More...

enum	{ ASSOCIATED_LATENCY =-1, INF_LATENCY =-1 }

Protected Member Functions
UtlBoolean	doDelay (MpBufPtr &inBuf, MpBufPtr &outBuf, int samplesPerFrame, int samplesPerSecond)

UtlBoolean	handleMessage (MpResourceMsg &rMsg)
	Handles an incoming flowgraph message for this media processing object. More...

UtlBoolean	doProcessFrame (MpBufPtr inBufs[], MpBufPtr outBufs[], int inBufsSize, int outBufsSize, UtlBoolean isEnabled, int samplesPerFrame, int samplesPerSecond)
	This method does the real work for the media processing resource and must be defined in each class derived from this one. More...

Protected Member Functions inherited from MpResource
virtual UtlBoolean	handleMessage (MpFlowGraphMsg &fgMsg)
	Handles an incoming flowgraph message for this media processing object. More...

virtual UtlBoolean	handleEnable ()
	perform the enable operation on the resource More...

virtual UtlBoolean	handleDisable ()
	perform the disable operation on the resource More...

void	setInputBuffer (int inPortIdx, const MpBufPtr &pBuf)
	If there already is a buffer stored for this input port, delete it. Then store `pBuf` for the indicated input port. More...

OsStatus	postMessage (MpFlowGraphMsg &rMsg)
	Post a message from this resource. More...

OsStatus	postMessage (MpResourceMsg &rMsg)
	Post a message for this resource. More...

UtlBoolean	pushBufferDownsream (int outPortIdx, const MpBufPtr &pBuf)
	Makes `pBuf` available to resource connected to the `outPortIdx` output port of this resource. More...

virtual OsStatus	setFlowGraph (MpFlowGraphBase *pFlowGraph)
	Associates this resource with the indicated flow graph. More...

virtual OsStatus	setNotificationsEnabled (UtlBoolean enable)
	Sets whether or not this resource should send notifications. More...

virtual UtlBoolean	connectInput (MpResource &rFrom, int fromPortIdx, int toPortIdx)
	Connects the `toPortIdx` input port on this resource to the `fromPortIdx` output port of the `rFrom` resource. More...

virtual UtlBoolean	connectOutput (MpResource &rTo, int toPortIdx, int fromPortIdx)
	Connects the `fromPortIdx` output port on this resource to the `toPortIdx` input port of the `rTo` resource. More...

virtual UtlBoolean	disconnectInput (int inPortIdx)
	Removes the connection to the `inPortIdx` input port of this resource. More...

virtual UtlBoolean	disconnectOutput (int outPortIdx)
	Removes the connection to the `outPortIdx` output port of this resource. More...

void	setName (const UtlString &rName)
	Sets the name that is associated with this resource. More...

Private Types
enum	MprDelayState { MPDS_WAITING, MPDS_BUFFERING, MPDS_PLAYING, MPDS_PLAYING_WITHOUT_DELAY }

enum	AddlMsgTypes { MPRM_DELAY_START_PLAY = MpResourceMsg::MPRM_EXTERNAL_MESSAGE_START, MPRM_DELAY_STOP_PLAY }

Private Attributes
MprDelayState	mState
	State of delay resource. More...

UtlBoolean	mIsQuiescenceNotified
	Have FIFO quiescent state been notified? More...

MprDelayFifo	mFifo
	FIFO for the stored data. More...

Creators
	MprDelay (const UtlString &rName, int capacity=DEFAULT_FIFO_CAPACITY)
	Constructor. More...

	~MprDelay ()
	Destructor. More...

Manipulators
static UtlBoolean	startPlay (const UtlString &namedResource, OsMsgQ &fgQ)
	Send message to start play. More...

static UtlBoolean	stopPlay (const UtlString &namedResource, OsMsgQ &fgQ)
	Send message to stop play. More...

Inquiry
int	getDelayMs ()
	Get delay in milliseconds. More...

int	getDelaySamples ()
	Get delay in samples. More...

int	getDelayFrames ()
	Get delay in frames. More...

Additional Inherited Members
Public Member Functions inherited from MpAudioResource
	MpAudioResource (const UtlString &rName, int minInputs, int maxInputs, int minOutputs, int maxOutputs)
	Constructor. More...

virtual	~MpAudioResource ()
	Destructor. More...

virtual UtlBoolean	processFrame (void)
	Wrapper around doProcessFrame(). More...

Public Member Functions inherited from MpResource
	MpResource (const UtlString &rName, int minInputs, int maxInputs, int minOutputs, int maxOutputs)
	Constructor. More...

virtual	~MpResource ()
	Destructor. More...

virtual UtlBoolean	disable ()
	Disable this resource. More...

virtual UtlBoolean	enable ()
	Enable this resource. More...

virtual void	reset ()
	This method is invoked for resources that care about stream discontinuities. More...

UtlBoolean	handleMessages (OsMsgQ &msgQ)
	Handles a queue full of incoming messages for this media processing object. More...

void	setVisitState (int newState)
	Sets the visit state for this resource. More...

OsStatus	sendNotification (MpResNotificationMsg::RNMsgType msgType)
	Send a notification with the given message type if notifications are enabled. More...

OsStatus	sendNotification (MpResNotificationMsg &msg)
	Send the given notification message if notifications are enabled. More...

virtual void	setConnectionId (MpConnectionID connectionId)
	Set the ID of a connection this resource belongs to. More...

virtual void	setStreamId (int streamId)
	Set the ID of a stream inside of the connection this resource belongs to. More...

virtual int	getStreamId (void)
	Get the ID of a stream within the connection this resource belongs to. More...

virtual OsStatus	pushBuffer (int inputPort, MpBufPtr &inputBuffer)
	Receive buffer asynchronously from resource at given input port. More...

virtual void	reassignSSRC (void)
	Broadcast announcement that we are changing our SSRC. More...

MpFlowGraphBase *	getFlowGraph () const
	Returns parent flowgraph. More...

void	getInputInfo (int inPortIdx, MpResource *&rpUpstreamResource, int &rUpstreamPortIdx)
	Returns information about the upstream end of a connection. More...

const UtlString &	getName () const
	Returns the name associated with this resource. More...

void	getOutputInfo (int outPortIdx, MpResource *&rpDownstreamResource, int &rDownstreamPortIdx)
	Returns information about the downstream end of a connection. More...

int	getVisitState ()
	Returns the current visit state for this resource. More...

int	maxInputs () const
	Returns the maximum number of inputs supported by this resource. More...

int	maxOutputs () const
	Returns the maximum number of outputs supported by this resource. More...

int	minInputs () const
	Returns the minimum number of inputs required by this resource. More...

int	minOutputs () const
	Returns the minimum number of outputs required by this resource. More...

int	numInputs () const
	Returns the number of resource inputs that are currently connected. More...

int	numOutputs () const
	Returns the number of resource outputs that are currently connected. More...

virtual MpConnectionID	getConnectionId (void) const
	Get the ID of a connection this resource belongs to. More...

int	reserveFirstUnconnectedInput ()
	Find the first unconnected input port and reserve it. More...

int	reserveFirstUnconnectedOutput ()
	Find the first unconnected output port and reserve it. More...

UtlContainableType	getContainableType () const
	Get the ContainableType for a UtlContainable derived class. More...

virtual OsStatus	getCurrentLatency (int &latency, int input=0, int output=0) const
	Get current input to output latency (in samples) More...

UtlBoolean	isEnabled () const
	Returns TRUE is this resource is currently enabled, FALSE otherwise. More...

UtlBoolean	isInputConnected (int portIdx)
	Returns TRUE if portIdx is valid and the indicated input is connected, FALSE otherwise. More...

UtlBoolean	isInputUnconnected (int portIdx)
	Returns TRUE if portIdx is valid and the indicated input is not connected, FALSE otherwise. More...

UtlBoolean	isOutputConnected (int portIdx)
	Returns TRUE if portIdx is valid and the indicated output is connected, FALSE otherwise. More...

UtlBoolean	isOutputUnconnected (int portIdx)
	Returns TRUE if portIdx is valid and the indicated output is not connected, FALSE otherwise. More...

UtlBoolean	areNotificationsEnabled () const
	Returns TRUE if notification sending is enabled on this resource. More...

virtual UtlBoolean	isAsynchInput (int inputIndex)
	Takes asynchronous input (pushBuffer). More...

Static Public Member Functions inherited from MpResource
static OsStatus	disable (const UtlString &namedResource, OsMsgQ &fgQ)
	Post a message to disable the resource named. More...

static OsStatus	enable (const UtlString &namedResource, OsMsgQ &fgQ)
	Post a message to enable the resource named. More...

static OsStatus	setNotificationsEnabled (UtlBoolean enable, const UtlString &namedResource, OsMsgQ &fgQ)
	Post a message to enable or disable resource notifications on the named resource. More...

static void	resourceInfo (MpResource *pResource, int index)
	Displays information on the console about the specified resource. More...

Static Public Attributes inherited from MpResource
static const UtlContainableType	TYPE = "MpResource"
	Class name, used for run-time checks. More...

Protected Attributes inherited from MpResource
MpFlowGraphBase *	mpFlowGraph
	flow graph this resource belongs to More...

MpConnectionID	mConnectionId
	The ID of connection this resource belongs to. More...

int	mStreamId

UtlBoolean	mIsEnabled
	TRUE if resource is enabled, FALSE otherwise. More...

OsRWMutex	mRWMutex
	reader/writer lock for synchronization More...

MpBufPtr *	mpInBufs
	input buffers for this resource More...

MpBufPtr *	mpOutBufs
	output buffers for this resource More...

Conn *	mpInConns
	input connections for this resource More...

Conn *	mpOutConns
	output connections for this resource More...

int	mMaxInputs
	maximum number of inputs More...

int	mMaxOutputs
	maximum number of outputs More...

int	mMinInputs
	number of required inputs More...

int	mMinOutputs
	number of required outputs More...

int	mNumActualInputs
	actual number of connected inputs More...

int	mNumActualOutputs
	actual number of connected outputs More...

int	mVisitState
	(used by flow graph topological sort alg.) More...

UtlBoolean	mNotificationsEnabled
	Whether we should send notifications or not. More...

OsBSem	mLock
	used mainly to make safe changes to ports More...

Static Protected Attributes inherited from MpResource
static const OsTime	sOperationQueueTimeout = OsTime::OS_INFINITY
	The timeout for message operations for all resources when posting to the flowgraph queue. More...

Detailed Description

Delay Resource.

This resource is intended to add user-controlled delay to audio stream. Such a strange feature may be useful in many advanced use-cases, e.g. when our network may be not ready to accept data at the moment data becomes ready.

Internal state machine description

Delay resource can be found in four internal states: 1) WAITING. This is initial state of resource after creation and also is the state after stopPlay() call. WAITING state means that resource is idle and do not appear to have active audio input. No frames are produced on output.

WAITING state will be changed to BUFFERING automatically when frame with active audio is received, and it can be manually changed to PLAYING state by calling startPlay() method.

    active voice
WAITING ---------------> BUFFERING
   |
   | startPlay()
   v
PLAYING

       stopPlay()
WAITING <--------------- BUFFERING
   ^ ^______________________
   |        stopPlay()      |
   | stopPlay()             |
PLAYING            PLAYING_WITHOUT_DELAY

2) BUFFERING. In this state resource stores all incoming frames to internal circular buffer and does not produce any output. Thus the longer resource remains in this state, the longer delay will be introduced.

This state can only be reached from WAITING state in case active voice has been received. Calling startPlay() in BUFFERING state leads to transition to PLAYING state, calling stopPlay() moves resource back to WAITING state.

    active voice
WAITING ---------------> BUFFERING
   ^______________________|  |
          stopPlay()         |
   __________________________|
   |      startPlay()
   v
PLAYING

3) PLAYING. In this state resource pops recorded data from the head of internal buffer and sends it to output, adding newly arrived data to the tail of internal buffer. I.e. data is played back with constant delay, which depends on the time spent in BUFFERING state.

Resource can be put to PLAYING state only manually by calling startPlay() method. Calling stopPlay() method when in PLAYING state will move resource to WAITING state. If internal buffer is empty then resource will automatically fall through to PLAYING_WITHOUT_DELAY state.

4) PLAYING_WITHOUT_DELAY. In this case every input frame is directly forwarded to the output, i.e. no delay is introduced. At this moment this state is hardly useful, but it is thought to become useful when delay resource is able to "catch up", i.e. reduce delay to zero using e.g. timescaling technique. From application perspective this state is indistinguishable from PLAYING state.

This state can only be reached from PLAYING state if internal buffer is empty. Calling stopPlay() will move resource to WAITING state.

          stopPlay()
WAITING <------------------------.
                                 |
                                 |
         empty buffer            |
PLAYING --------------> PLAYING_WITHOUT_DELAY

Notifications

1) When state changes from WAITING to BUFFERING (i.e. when active audio is detected on input) resource emits MPRNM_DELAY_SPEECH_STARTED notification. User should respond to this notification by calling startPlay() when he thinks relevant. User can also call stopPlay() to put resource into WAITING state again. Until one of this action is performed resource will buffer input data and thus increase induced delay.

2) When resource in PLAYING state detects, that its buffer contain only NULL frames and frames with inactive audio it emits MPRNM_DELAY_QUIESCENCE for user to perform any relevant actions. E.g. user can call stopPlay() and power down network interface, if there are no other active audio sources. Note, that at in current implementation MPRNM_DELAY_QUIESCENCE notification is not emited in PLAYING_WITHOUT_DELAY state.

3) When resource detects active audio after quescent situation have been recently detected resource emits MPRNM_DELAY_SPEECH_STARTED notification. This should warn user that resource's buffer contain active audio again.

4) When resource in PLAYING state detects empty internal buffer it emits MPRNM_DELAY_NO_DELAY notification and enters PLAYING_WITHOUT_DELAY state.

How to control Delay resource

If you're using flowgraph with Delay resources in it you must control them with care. Use case deemed to be most useful consists of three steps repeated foreveris as outlined below:

1) Initially Delay resource is in WAITING state. User is listenning for notifications and honestly expects MPRNM_DELAY_SPEECH_STARTED to appear soon. 2) MPRNM_DELAY_SPEECH_STARTED is received by user which means that resource is now in BUFFERING state. It will remain in this state until user calls either startPlay() or stopPlay(). Once user thinks it's the time to start playback (e.g. network interface have woken) he calls startPlay() (goto (3) then) and resource is put into PLAYING state. If for any reason user thinks buffered audio should be discarded he calls stopPlay() to put resource to WAITING state again (goto (1) then). 3) In PLAYING state user can call stopPlay() at any time to discard buffered frames and put resource into WAITING state (goto (1) then). But it's most reasonable to wait for MPRNM_DELAY_QUIESCENCE notification and call stopPlay() only them. This will minimize impact on audio quality caused by dropped frames.

Member Enumeration Documentation

anonymous enum

Enumerator
DEFAULT_FIFO_CAPACITY

enum MprDelayState

private

Enumerator
MPDS_WAITING
MPDS_BUFFERING
MPDS_PLAYING
MPDS_PLAYING_WITHOUT_DELAY

enum AddlMsgTypes

private

Enumerator
MPRM_DELAY_START_PLAY
MPRM_DELAY_STOP_PLAY

Constructor & Destructor Documentation

MprDelay	(	const UtlString &	rName,
		int	capacity = `DEFAULT_FIFO_CAPACITY`
	)

Constructor.

~MprDelay ( )

Destructor.

Member Function Documentation

UtlBoolean startPlay	(	const UtlString &	namedResource,
		OsMsgQ &	fgQ
	)

static

Send message to start play.

UtlBoolean stopPlay	(	const UtlString &	namedResource,
		OsMsgQ &	fgQ
	)

static

Send message to stop play.

int getDelayMs ( )

Get delay in milliseconds.

int getDelaySamples ( )

Get delay in samples.

int getDelayFrames ( )

Get delay in frames.

UtlBoolean doDelay	(	MpBufPtr &	inBuf,
		MpBufPtr &	outBuf,
		int	samplesPerFrame,
		int	samplesPerSecond
	)

protected

UtlBoolean handleMessage ( MpResourceMsg & rMsg )

protectedvirtual

Handles an incoming flowgraph message for this media processing object.

Returns: TRUE if the message was handled, otherwise FALSE.

Reimplemented from MpResource.

UtlBoolean doProcessFrame	(	MpBufPtr	inBufs[],
		MpBufPtr	outBufs[],
		int	inBufsSize,
		int	outBufsSize,
		UtlBoolean	isEnabled,
		int	samplesPerFrame,
		int	samplesPerSecond
	)

protectedvirtual

This method does the real work for the media processing resource and must be defined in each class derived from this one.

Parameters

inBufs	(in) array of pointers to input buffers for the resource
outBufs	(out) array of pointers to output buffers produce by the resource
inBufsSize	(in) size of the inBufs array
outBufsSize	(in) size of the outBufs array
isEnabled	(in) indicates whether this resource has been enabled
samplesPerFrame	(in) samples to produce per frame processing interval
samplesPerSecond	(in) samples to produce per second

Returns: TRUE if successful, FALSE otherwise.

Implements MpAudioResource.

Member Data Documentation

MprDelayState mState

private

State of delay resource.

UtlBoolean mIsQuiescenceNotified

private

Have FIFO quiescent state been notified?

MprDelayFifo mFifo

private

FIFO for the stored data.

Classes

Public Types

Protected Member Functions

Private Types

Private Attributes

Creators

Manipulators

Inquiry

Additional Inherited Members

Detailed Description

Internal state machine description

Notifications

How to control Delay resource

Member Enumeration Documentation

Constructor & Destructor Documentation

Member Function Documentation

Member Data Documentation