sipxmedialib  Version 3.3
Classes | Public Types | Protected Member Functions | Private Types | Private Attributes | List of all members
MprDelay Class Reference

Delay Resource. More...

#include <MprDelay.h>

Inheritance diagram for MprDelay:
Inheritance graph
[legend]
Collaboration diagram for MprDelay:
Collaboration graph
[legend]

Classes

class  MprDelayFifo
 Fifo for Delay resource. More...
 

Public Types

enum  { DEFAULT_FIFO_CAPACITY = 1000 }
 
- Public Types inherited from MpResource
enum  VisitState { NOT_VISITED, IN_PROGRESS, FINISHED }
 Graph traversal states that are used when running a topological sort to order resources within a flow graph. More...
 
enum  { ASSOCIATED_LATENCY =-1, INF_LATENCY =-1 }
 

Protected Member Functions

UtlBoolean doDelay (MpBufPtr &inBuf, MpBufPtr &outBuf, int samplesPerFrame, int samplesPerSecond)
 
UtlBoolean handleMessage (MpResourceMsg &rMsg)
 Handles an incoming flowgraph message for this media processing object. More...
 
UtlBoolean doProcessFrame (MpBufPtr inBufs[], MpBufPtr outBufs[], int inBufsSize, int outBufsSize, UtlBoolean isEnabled, int samplesPerFrame, int samplesPerSecond)
 This method does the real work for the media processing resource and must be defined in each class derived from this one. More...
 
- Protected Member Functions inherited from MpResource
virtual UtlBoolean handleMessage (MpFlowGraphMsg &fgMsg)
 Handles an incoming flowgraph message for this media processing object. More...
 
virtual UtlBoolean handleEnable ()
 perform the enable operation on the resource More...
 
virtual UtlBoolean handleDisable ()
 perform the disable operation on the resource More...
 
void setInputBuffer (int inPortIdx, const MpBufPtr &pBuf)
 If there already is a buffer stored for this input port, delete it. Then store pBuf for the indicated input port. More...
 
OsStatus postMessage (MpFlowGraphMsg &rMsg)
 Post a message from this resource. More...
 
OsStatus postMessage (MpResourceMsg &rMsg)
 Post a message for this resource. More...
 
UtlBoolean pushBufferDownsream (int outPortIdx, const MpBufPtr &pBuf)
 Makes pBuf available to resource connected to the outPortIdx output port of this resource. More...
 
virtual OsStatus setFlowGraph (MpFlowGraphBase *pFlowGraph)
 Associates this resource with the indicated flow graph. More...
 
virtual OsStatus setNotificationsEnabled (UtlBoolean enable)
 Sets whether or not this resource should send notifications. More...
 
virtual UtlBoolean connectInput (MpResource &rFrom, int fromPortIdx, int toPortIdx)
 Connects the toPortIdx input port on this resource to the fromPortIdx output port of the rFrom resource. More...
 
virtual UtlBoolean connectOutput (MpResource &rTo, int toPortIdx, int fromPortIdx)
 Connects the fromPortIdx output port on this resource to the toPortIdx input port of the rTo resource. More...
 
virtual UtlBoolean disconnectInput (int inPortIdx)
 Removes the connection to the inPortIdx input port of this resource. More...
 
virtual UtlBoolean disconnectOutput (int outPortIdx)
 Removes the connection to the outPortIdx output port of this resource. More...
 
void setName (const UtlString &rName)
 Sets the name that is associated with this resource. More...
 

Private Types

enum  MprDelayState { MPDS_WAITING, MPDS_BUFFERING, MPDS_PLAYING, MPDS_PLAYING_WITHOUT_DELAY }
 
enum  AddlMsgTypes { MPRM_DELAY_START_PLAY = MpResourceMsg::MPRM_EXTERNAL_MESSAGE_START, MPRM_DELAY_STOP_PLAY }
 

Private Attributes

MprDelayState mState
 State of delay resource. More...
 
UtlBoolean mIsQuiescenceNotified
 Have FIFO quiescent state been notified? More...
 
MprDelayFifo mFifo
 FIFO for the stored data. More...
 

Creators

 MprDelay (const UtlString &rName, int capacity=DEFAULT_FIFO_CAPACITY)
 Constructor. More...
 
 ~MprDelay ()
 Destructor. More...
 

Manipulators

static UtlBoolean startPlay (const UtlString &namedResource, OsMsgQ &fgQ)
 Send message to start play. More...
 
static UtlBoolean stopPlay (const UtlString &namedResource, OsMsgQ &fgQ)
 Send message to stop play. More...
 

Inquiry

int getDelayMs ()
 Get delay in milliseconds. More...
 
int getDelaySamples ()
 Get delay in samples. More...
 
int getDelayFrames ()
 Get delay in frames. More...
 

Additional Inherited Members

- Public Member Functions inherited from MpAudioResource
 MpAudioResource (const UtlString &rName, int minInputs, int maxInputs, int minOutputs, int maxOutputs)
 Constructor. More...
 
virtual ~MpAudioResource ()
 Destructor. More...
 
virtual UtlBoolean processFrame (void)
 Wrapper around doProcessFrame(). More...
 
- Public Member Functions inherited from MpResource
 MpResource (const UtlString &rName, int minInputs, int maxInputs, int minOutputs, int maxOutputs)
 Constructor. More...
 
virtual ~MpResource ()
 Destructor. More...
 
virtual UtlBoolean disable ()
 Disable this resource. More...
 
virtual UtlBoolean enable ()
 Enable this resource. More...
 
virtual void reset ()
 This method is invoked for resources that care about stream discontinuities. More...
 
UtlBoolean handleMessages (OsMsgQ &msgQ)
 Handles a queue full of incoming messages for this media processing object. More...
 
void setVisitState (int newState)
 Sets the visit state for this resource. More...
 
OsStatus sendNotification (MpResNotificationMsg::RNMsgType msgType)
 Send a notification with the given message type if notifications are enabled. More...
 
OsStatus sendNotification (MpResNotificationMsg &msg)
 Send the given notification message if notifications are enabled. More...
 
virtual void setConnectionId (MpConnectionID connectionId)
 Set the ID of a connection this resource belongs to. More...
 
virtual void setStreamId (int streamId)
 Set the ID of a stream inside of the connection this resource belongs to. More...
 
virtual int getStreamId (void)
 Get the ID of a stream within the connection this resource belongs to. More...
 
virtual OsStatus pushBuffer (int inputPort, MpBufPtr &inputBuffer)
 Receive buffer asynchronously from resource at given input port. More...
 
virtual void reassignSSRC (void)
 Broadcast announcement that we are changing our SSRC. More...
 
MpFlowGraphBasegetFlowGraph () const
 Returns parent flowgraph. More...
 
void getInputInfo (int inPortIdx, MpResource *&rpUpstreamResource, int &rUpstreamPortIdx)
 Returns information about the upstream end of a connection. More...
 
const UtlString & getName () const
 Returns the name associated with this resource. More...
 
void getOutputInfo (int outPortIdx, MpResource *&rpDownstreamResource, int &rDownstreamPortIdx)
 Returns information about the downstream end of a connection. More...
 
int getVisitState ()
 Returns the current visit state for this resource. More...
 
int maxInputs () const
 Returns the maximum number of inputs supported by this resource. More...
 
int maxOutputs () const
 Returns the maximum number of outputs supported by this resource. More...
 
int minInputs () const
 Returns the minimum number of inputs required by this resource. More...
 
int minOutputs () const
 Returns the minimum number of outputs required by this resource. More...
 
int numInputs () const
 Returns the number of resource inputs that are currently connected. More...
 
int numOutputs () const
 Returns the number of resource outputs that are currently connected. More...
 
virtual MpConnectionID getConnectionId (void) const
 Get the ID of a connection this resource belongs to. More...
 
int reserveFirstUnconnectedInput ()
 Find the first unconnected input port and reserve it. More...
 
int reserveFirstUnconnectedOutput ()
 Find the first unconnected output port and reserve it. More...
 
UtlContainableType getContainableType () const
 Get the ContainableType for a UtlContainable derived class. More...
 
virtual OsStatus getCurrentLatency (int &latency, int input=0, int output=0) const
 Get current input to output latency (in samples) More...
 
UtlBoolean isEnabled () const
 Returns TRUE is this resource is currently enabled, FALSE otherwise. More...
 
UtlBoolean isInputConnected (int portIdx)
 Returns TRUE if portIdx is valid and the indicated input is connected, FALSE otherwise. More...
 
UtlBoolean isInputUnconnected (int portIdx)
 Returns TRUE if portIdx is valid and the indicated input is not connected, FALSE otherwise. More...
 
UtlBoolean isOutputConnected (int portIdx)
 Returns TRUE if portIdx is valid and the indicated output is connected, FALSE otherwise. More...
 
UtlBoolean isOutputUnconnected (int portIdx)
 Returns TRUE if portIdx is valid and the indicated output is not connected, FALSE otherwise. More...
 
UtlBoolean areNotificationsEnabled () const
 Returns TRUE if notification sending is enabled on this resource. More...
 
virtual UtlBoolean isAsynchInput (int inputIndex)
 Takes asynchronous input (pushBuffer). More...
 
- Static Public Member Functions inherited from MpResource
static OsStatus disable (const UtlString &namedResource, OsMsgQ &fgQ)
 Post a message to disable the resource named. More...
 
static OsStatus enable (const UtlString &namedResource, OsMsgQ &fgQ)
 Post a message to enable the resource named. More...
 
static OsStatus setNotificationsEnabled (UtlBoolean enable, const UtlString &namedResource, OsMsgQ &fgQ)
 Post a message to enable or disable resource notifications on the named resource. More...
 
static void resourceInfo (MpResource *pResource, int index)
 Displays information on the console about the specified resource. More...
 
- Static Public Attributes inherited from MpResource
static const UtlContainableType TYPE = "MpResource"
 Class name, used for run-time checks. More...
 
- Protected Attributes inherited from MpResource
MpFlowGraphBasempFlowGraph
 flow graph this resource belongs to More...
 
MpConnectionID mConnectionId
 The ID of connection this resource belongs to. More...
 
int mStreamId
 
UtlBoolean mIsEnabled
 TRUE if resource is enabled, FALSE otherwise. More...
 
OsRWMutex mRWMutex
 reader/writer lock for synchronization More...
 
MpBufPtrmpInBufs
 input buffers for this resource More...
 
MpBufPtrmpOutBufs
 output buffers for this resource More...
 
ConnmpInConns
 input connections for this resource More...
 
ConnmpOutConns
 output connections for this resource More...
 
int mMaxInputs
 maximum number of inputs More...
 
int mMaxOutputs
 maximum number of outputs More...
 
int mMinInputs
 number of required inputs More...
 
int mMinOutputs
 number of required outputs More...
 
int mNumActualInputs
 actual number of connected inputs More...
 
int mNumActualOutputs
 actual number of connected outputs More...
 
int mVisitState
 (used by flow graph topological sort alg.) More...
 
UtlBoolean mNotificationsEnabled
 Whether we should send notifications or not. More...
 
OsBSem mLock
 used mainly to make safe changes to ports More...
 
- Static Protected Attributes inherited from MpResource
static const OsTime sOperationQueueTimeout = OsTime::OS_INFINITY
 The timeout for message operations for all resources when posting to the flowgraph queue. More...
 

Detailed Description

Delay Resource.

This resource is intended to add user-controlled delay to audio stream. Such a strange feature may be useful in many advanced use-cases, e.g. when our network may be not ready to accept data at the moment data becomes ready.

Internal state machine description

Delay resource can be found in four internal states: 1) WAITING. This is initial state of resource after creation and also is the state after stopPlay() call. WAITING state means that resource is idle and do not appear to have active audio input. No frames are produced on output.

WAITING state will be changed to BUFFERING automatically when frame with active audio is received, and it can be manually changed to PLAYING state by calling startPlay() method.

    active voice
WAITING ---------------> BUFFERING
   |
   | startPlay()
   v
PLAYING
       stopPlay()
WAITING <--------------- BUFFERING
   ^ ^______________________
   |        stopPlay()      |
   | stopPlay()             |
PLAYING            PLAYING_WITHOUT_DELAY

2) BUFFERING. In this state resource stores all incoming frames to internal circular buffer and does not produce any output. Thus the longer resource remains in this state, the longer delay will be introduced.

This state can only be reached from WAITING state in case active voice has been received. Calling startPlay() in BUFFERING state leads to transition to PLAYING state, calling stopPlay() moves resource back to WAITING state.

    active voice
WAITING ---------------> BUFFERING
   ^______________________|  |
          stopPlay()         |
   __________________________|
   |      startPlay()
   v
PLAYING

3) PLAYING. In this state resource pops recorded data from the head of internal buffer and sends it to output, adding newly arrived data to the tail of internal buffer. I.e. data is played back with constant delay, which depends on the time spent in BUFFERING state.

Resource can be put to PLAYING state only manually by calling startPlay() method. Calling stopPlay() method when in PLAYING state will move resource to WAITING state. If internal buffer is empty then resource will automatically fall through to PLAYING_WITHOUT_DELAY state.

4) PLAYING_WITHOUT_DELAY. In this case every input frame is directly forwarded to the output, i.e. no delay is introduced. At this moment this state is hardly useful, but it is thought to become useful when delay resource is able to "catch up", i.e. reduce delay to zero using e.g. timescaling technique. From application perspective this state is indistinguishable from PLAYING state.

This state can only be reached from PLAYING state if internal buffer is empty. Calling stopPlay() will move resource to WAITING state.

          stopPlay()
WAITING <------------------------.
                                 |
                                 |
         empty buffer            |
PLAYING --------------> PLAYING_WITHOUT_DELAY

Notifications

1) When state changes from WAITING to BUFFERING (i.e. when active audio is detected on input) resource emits MPRNM_DELAY_SPEECH_STARTED notification. User should respond to this notification by calling startPlay() when he thinks relevant. User can also call stopPlay() to put resource into WAITING state again. Until one of this action is performed resource will buffer input data and thus increase induced delay.

2) When resource in PLAYING state detects, that its buffer contain only NULL frames and frames with inactive audio it emits MPRNM_DELAY_QUIESCENCE for user to perform any relevant actions. E.g. user can call stopPlay() and power down network interface, if there are no other active audio sources. Note, that at in current implementation MPRNM_DELAY_QUIESCENCE notification is not emited in PLAYING_WITHOUT_DELAY state.

3) When resource detects active audio after quescent situation have been recently detected resource emits MPRNM_DELAY_SPEECH_STARTED notification. This should warn user that resource's buffer contain active audio again.

4) When resource in PLAYING state detects empty internal buffer it emits MPRNM_DELAY_NO_DELAY notification and enters PLAYING_WITHOUT_DELAY state.

How to control Delay resource

If you're using flowgraph with Delay resources in it you must control them with care. Use case deemed to be most useful consists of three steps repeated foreveris as outlined below:

1) Initially Delay resource is in WAITING state. User is listenning for notifications and honestly expects MPRNM_DELAY_SPEECH_STARTED to appear soon. 2) MPRNM_DELAY_SPEECH_STARTED is received by user which means that resource is now in BUFFERING state. It will remain in this state until user calls either startPlay() or stopPlay(). Once user thinks it's the time to start playback (e.g. network interface have woken) he calls startPlay() (goto (3) then) and resource is put into PLAYING state. If for any reason user thinks buffered audio should be discarded he calls stopPlay() to put resource to WAITING state again (goto (1) then). 3) In PLAYING state user can call stopPlay() at any time to discard buffered frames and put resource into WAITING state (goto (1) then). But it's most reasonable to wait for MPRNM_DELAY_QUIESCENCE notification and call stopPlay() only them. This will minimize impact on audio quality caused by dropped frames.

Member Enumeration Documentation

anonymous enum
Enumerator
DEFAULT_FIFO_CAPACITY 
enum MprDelayState
private
Enumerator
MPDS_WAITING 
MPDS_BUFFERING 
MPDS_PLAYING 
MPDS_PLAYING_WITHOUT_DELAY 
enum AddlMsgTypes
private
Enumerator
MPRM_DELAY_START_PLAY 
MPRM_DELAY_STOP_PLAY 

Constructor & Destructor Documentation

MprDelay ( const UtlString &  rName,
int  capacity = DEFAULT_FIFO_CAPACITY 
)

Constructor.

~MprDelay ( )

Destructor.

Member Function Documentation

UtlBoolean startPlay ( const UtlString &  namedResource,
OsMsgQ &  fgQ 
)
static

Send message to start play.

UtlBoolean stopPlay ( const UtlString &  namedResource,
OsMsgQ &  fgQ 
)
static

Send message to stop play.

int getDelayMs ( )

Get delay in milliseconds.

int getDelaySamples ( )

Get delay in samples.

int getDelayFrames ( )

Get delay in frames.

UtlBoolean doDelay ( MpBufPtr inBuf,
MpBufPtr outBuf,
int  samplesPerFrame,
int  samplesPerSecond 
)
protected
UtlBoolean handleMessage ( MpResourceMsg rMsg)
protectedvirtual

Handles an incoming flowgraph message for this media processing object.

Returns
TRUE if the message was handled, otherwise FALSE.

Reimplemented from MpResource.

UtlBoolean doProcessFrame ( MpBufPtr  inBufs[],
MpBufPtr  outBufs[],
int  inBufsSize,
int  outBufsSize,
UtlBoolean  isEnabled,
int  samplesPerFrame,
int  samplesPerSecond 
)
protectedvirtual

This method does the real work for the media processing resource and must be defined in each class derived from this one.

Parameters
inBufs(in) array of pointers to input buffers for the resource
outBufs(out) array of pointers to output buffers produce by the resource
inBufsSize(in) size of the inBufs array
outBufsSize(in) size of the outBufs array
isEnabled(in) indicates whether this resource has been enabled
samplesPerFrame(in) samples to produce per frame processing interval
samplesPerSecond(in) samples to produce per second
Returns
TRUE if successful, FALSE otherwise.

Implements MpAudioResource.

Member Data Documentation

MprDelayState mState
private

State of delay resource.

UtlBoolean mIsQuiescenceNotified
private

Have FIFO quiescent state been notified?

MprDelayFifo mFifo
private

FIFO for the stored data.