Streaming Video to Multiple Receivers

Manage Multimedia Data Distribution with Connext DDS

Introduction

This use case applies if you are trying to stream video, audio, or other streaming media to one or more receivers. This solution supports use cases with one or more data sources, and one or more receivers of data streams. Some examples of this use case include:

  • Monitoring videos streams from security cameras
  • Receiving video streams from multiple UAVs
  • Sending news feed video to many receivers
  • Streaming movies to many subscribers

RTI Connext DDS is a high-performance, peer-to-peer middleware that can send real-time data over a network. RTI Connext DDS supports a variety of Quality of Service (QoS) policies, including reliable delivery of data over multicast. In addition, RTI Connext DDS provides a built-in dynamic discovery mechanism that allows applications to discover sources of videos, which videos are available, and metadata about video streams. RTI Connext DDS is also available on a variety of platforms, so it can be used to seamlessly send data from embedded systems to desktop machines.

What This Example Does

Video publisher and subscribers communicating on the RTI Connext DDS bus

This example opens a video and sends that data stream from a single Video Publisher application to one or more Video Subscriber applications which display the video. By default, this demo sends data over unicast, but you can use command-line parameters to stream the video over multicast to multiple subscribers at the same time. This provides a benefit over TCP because RTI Connext DDS can provide reliability over UDP multicast, allowing for high-efficiency streaming of video data to many subscribers at once.

Video publisher detail

Video Publisher

  • Discovers Video Subscribers
  • Checks metadata about the stream to see if the Video Subscriber supports compatible codec(s)
  • Publishes streaming video data if one or more compatible subscribers are online

Video subscriber detail

Video Subscriber

  • Configures metadata with descriptions of supported codec(s)
  • Subscribes to streaming video
  • Displays streaming video

Running the Example

Download the Example

Download the example files [ Windows  |  Linux ] and extract them into a location of your choice. We'll refer to this location in this document as EXAMPLE_HOME.

To view and download the example source code without pre-built executables, visit the RTI Community DDS Use Cases repository in GitHub.

This download includes:

  • Pre-built applications you can run that load a video and send it over the network
  • Example source code and project files for Windows and Linux systems

Download RTI Connext DDS Professional

If you do not already have RTI Connext DDS Professional installed, download and install it now. You can use a 30-day trial license to try out the product. Your download will include the libraries that are required to run the example, and tools you can use to visualize and debug your distributed system.

Linux Only: Download the GStreamer Framework

On Ubuntu, you can install GStreamer and its development plugins:

sudo apt-get install gstreamer0.10 libgstreamer-plugins-base0.10-dev

On CentOS, you can install GStreamer and its development plugins:

yum install gstreamer gstreamer-devel gstreamer-plugins-base-devel

Run the Example

These examples can be run on the same machine or on multiple machines. The video file will be loaded by the Video Publisher application and sent to one or more Video Subscriber applications. If the applications are running on the same machine, they are sending data over shared memory. If they are running on multiple machines, they are sending data over the network.

To Start the Video Publisher Application:

Run the batch file: scripts\VideoPublisher.bat or the shell script: scripts/VideoPublisher.sh

To Start the Video Subscriber Application:

Run the batch file: scripts\VideoSubscriber.bat or the shell script: scripts/VideoSubscriber.sh

To Start the Video Subscriber With Multicast Enabled:

Optionally, you can enable the video data to be sent over multicast. To do this, run the batch file: scripts\VideoSubscriber.bat --multicast or the shell script: scripts/VideoSubscriber.sh --multicast

Example Output

When you start the Video Publisher example, you will see the following text in the console:

--- Starting publisher application. ---
This application will read a video file, and publish it over RTI Connext DDS
middleware
Waiting for a compatible video subscriber to come online
Waiting for a compatible video subscriber to come online
Waiting for a compatible video subscriber to come online

Once it discovers a compatible Video Subscriber application, it prints:

Initializing and starting video source
Video Source: VideoSource started

When you start the Video Subscriber example, you will see the following text in the console:

--- Starting subscriber application. ---
This application will subscribe to a video feed over RTI Connext DDS
middleware

Once it discovers a Video Publisher application, it will print the following text and you should see a video playing:

. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

Streaming Video Example: Running on Linux [Download video file]

Building the Example

Directory Overview

EXAMPLE_HOME
|-- Docs
`-- ExampleCode
    |-- make
    |-- win32
    |-- scripts
    `-- src
        |-- CommonInfrastructure
        |-- Config
        |-- Generated
        |-- Idl
        `-- . . .

The source code is divided into:

  • make - Linux makefiles
  • win32 - Windows project and solution files
  • scripts - Scripts to run applications
  • src - Source code
    • CommonInfrastructure - The code that all applications use to start using RTI Connext DDS to send or receive data
    • Config - XML QoS configuration files
    • Generated - Source files generated from Idl
    • Idl - Describes the data types that are sent over the network
    • Other directories - Source code for specific applications. RTI Connext DDS publishing and subscribing code is in FooInterface.h and FooInterface.cxx

Building the Example

On all platforms, the first thing you must do is set an environment variable called NDDSHOME. This environment variable must point to the ndds.5.x.x directory inside your RTI Connext DDS installation. For more information on how to set an environment variable, please see the RTI Core Libraries and Utilities Getting Started Guide.

Windows Systems

On a Windows system, start by opening the EXAMPLE_HOME\ExampleCode\win32\StreamingVideoExample-<compilerver>.sln file. This code is made up of a combination of libraries, source, and IDL files that represent the interface to the application. The Visual Studio solution files are set up to automatically generate the necessary code and link against the required libraries.

Streaming Video Example: How to build on Windows [Download video file]

Linux Systems

To build the applications on a Linux system, change directories to the ExampleCode directory and use the command:

gmake –f make/Makefile.<platform>

The platform you choose will be the combination of your processor, OS, and compiler version. Please review the selection of Makefiles in the make directory to find the closest match to your system.

Streaming Video Example: How to build on Linux [Download video file]

Under the Hood

Data Model and QoS Considerations

Video publisher IDL data types

Video streams consist of a combination of video images and metadata such as the video format, video width, and video height. These can be divided into multiple streams, or combined into a single stream. In this example, the image data and the metadata are encoded in the same stream, or Topic. Metadata updates and video image updates have different sizes, so this example uses an octet sequence to represent the updates.

RTI Connext DDS uses IDL to describe the format of data that will be sent over the network. The IDL that is used to encode the video stream looks like this:

// 1024 * 1024 is the maximum frame size
// Data will be allocated at this size, but
// only the actual size of the frame will
// be sent.
const long MAX_BUFFER_SIZE = 1048576;

struct VideoStream {
    // This allows a subscriber to differentiate between different video
    // publishers that are publishing the same data on the same Topic.
    long stream_id; //@key

    // Some video formats may require metadata to be sent with the binary
    // video data, such as flags or a frame sequence_number.
    unsigned long flags;
    unsigned long sequence_number;

    // This contains the video buffers
    sequence <octet, MAX_BUFFER_SIZE> frame; 
};

The key piece is that a variable-length binary sequence is defined that contains the video frames:

sequence <octet, MAX_BUFFER_SIZE> frame;

Many video codecs have variable sizes of data – some packets contain a full keyframe, while other packets contain smaller updates to the image that is shown on-screen. DDS sequences support this variable-length data. RTI Connext DDS will allocate the full maximum size of one or more VideoStream structures up front (allocating a number up to the size of the queue, which is controlled by QoS described below). However, if a particular frame is smaller than the maximum, only the data in that frame is sent over the network. This combination of preallocating the data to a maximum size and sending only the actual data size leads to low-latency performance while conserving bandwidth.

Writing and Reading Streaming Video Data

These applications have a simple data interface. The Video Publisher application has a single DDS DataWriter object, which is an object that is associated with a single Topic and data type, and sends streaming video data over the network, or shared memory. The Video Subscriber has a single DDS DataReader object, which is an object that is associated with a single Topic and data type, and receives streaming video data over the network. A DataWriter and DataReader must have a matching Topic to communicate

Most applications both send and receive data using multiple DataWriters, DataReaders, and Topics, but these simple applications are only sending or receiving streaming video data on a single Topic.

Streaming Video QoS

Video publisher XML QoS configuration

RTI Connext DDS allows you to tune Quality of Service (QoS) parameters in an XML format. This allows you to separate your application logic from your network capabilities and rapidly reconfigure your application for new deployment scenarios. The configuration files for this example are located in ExampleCode/src/Config.

This data is sent as a single stream, with no reliability enabled. This means that when data is sent from the Video Publisher to the Video Subscriber, it doesn't resend data if it is dropped over the network. If there is a network disruption, any data that was sent during that time is not stored or re-sent by the middleware.

Discovering Video Formats

Not all video feeds will necessarily be readable by all applications. This application sends meta-information about the video codecs that are used to encode the data, and uses that to decide whether to send data to a particular application. It does this by using a QoS policy that allows applications to send metadata during the RTI Connext DDS discovery process.

The Video Publisher application opens a file in WebM video format – a royalty-free, open video compression for use with HTML5 video. The WebM file format contains up to three streams: video, audio, and subtitles. After opening the file, the application demuxes the WebM format and removes just the video data for streaming. It sends just the video part of the file in a VP8 video compression format. This video is built using GStreamer, a framework for doing various processing of media data.

The Video Subscriber application is expecting a VP8 stream with a size of 640x360 pixels. This information is part of the metadata that the Video Subscriber sends over the network. If the Video Publisher is providing a video that is a different format or size, it will not send to the Video Subscriber.

The Video Subscriber configures this metadata when it creates the DataReader object. The DataReader in the Video Subscriber application is configured with a copy of this metadata in the USER_DATA QoS. The USER_DATA QoS allows the application developer to send application-specific data as a part of discovery.

DDS_DataReaderQos readerQos;
. . .
readerQos.user_data.value.from_array(
reinterpret_cast<const DDS_Octet *>(videoMetadata.c_str()), videoMetadata.length());

This metadata about the supported codecs is a string that the application code queries from the GStreamer framework it is built on.

On the Video Publisher side, the application installs a listener on the DomainParticipant, so it can be notified when the DomainParticipant discovers DataReaders. In the listener, the application checks whether the discovered DataReader has put data into the USER_DATA QoS and whether that data describes a codec that is compatible with the Video Publisher's codec.

void VideoPublisherDiscoveryListener::on_data_available(
        DDS::DataReader *reader)
{
    . . .
    // Access the discovery information about remote DataReaders
    retcode = builtinReader->take(...)
}

In this version, the Video Publisher and the Video Subscriber applications are always compatible.

Common DDS Infrastructure for all Applications

Now let's look at the code that you will write once and use in every DDS application. The code in CommonInfrastructure/DDSCommunicator.h/.cxx creates the basic objects that start DDS communications. The DDSCommunicator class encapsulates the creation and initialization of the DDS DomainParticipant object.

All applications need at least one DomainParticipant to discover other RTI Connext DDS applications and to create other DDS Entities. Typically, an application has only one DomainParticipant.

In the source code, you can see this in the class DDSCommunicator, in CommonInfrastructure/DDSCommunicator.cxx:

DomainParticipant* CreateParticipant(long domain,
        char *participantQosLibrary, char *participantQosProfile) {

  _participant = TheParticipantFactory->create_participant_with_profile(
          domain, participantQosLibrary, participantQosProfile,
	  NULL, STATUS_MASK_NONE);
  ...
}

The DomainParticipant's QoS is loaded from one or more XML files. The profile to load is specified by the participantQosLibrary and participantQosProfile. The full list of DomainParticipant QoS is described in the HTML API Documentation.

Next Steps

Extra Credit – Make the Video Subscriber Codec Incompatible

You can easily change the Video Subscriber application to request a codec that is incompatible with the Video Publisher application. The easiest way to make this change is by changing the properties that the Video Subscriber is expecting. You do this by editing VideoOutput.cxx:

_displayPipeline = (GstPipeline *)gst_parse_launch(
    "appsrc name=\"src\" is-live=\"true\" do-timestamp=\"true\" "
    "caps=\"video/x-vp8, width=(int)640, height=(int)360, "
    "pixel-aspect-ratio=(fraction)1/1, framerate=(fraction)1000/1\" ! "
    "queue2 ! vp8dec ! queue2 ! "
    "videorate ! video/x-raw-yuv,framerate=25/1 ! "
    "ffmpegcolorspace ! "
    "directdrawsink name=\"sink\"",
    NULL);

Change video/x-vp8 to video/MP2T. When you make this change and rebuild, the Video Publisher application will recognize that the Video Subscriber is expecting an incompatible codec and will not send to it. You will see the following output:

Waiting for a compatible video subscriber to come online
Waiting for a compatible video subscriber to come online
Discovered a DataReader with an incompatible codec. Ignoring it (not sending it any data)
Waiting for a compatible video subscriber to come online

The Video Publisher application calls the DDS ignore_subscription() API to ignore the Video Subscriber's DataReader.

void VideoPublisherDiscoveryListener::on_data_available(DDSDataReader *reader)
{
    . . .
    retcode = participant->ignore_subscription(info_seq[i].instance_handle);
}

In your real application, you can take this farther by providing video with different resolutions or quality, and sending that information as a part of the discovery data. Based on this discovery information, you can publish different data.

Extra Credit – Make the Video Reliable

If you want to make the video data reliable, you can edit the video_stream_multicast.xml or video_stream_no_multicast.xml file (depending on whether you are running with no multicast available.) Edit the file to change BEST_EFFORT_RELIABILITY_QOS to RELIABLE_RELIABILITY_QOS.

<datareader_qos>
    <!-- Streaming video data can be reliable or best-effort -->
    <!-- depending on network characteristics                -->
    <reliability>
    <kind>BEST_EFFORT_RELIABILITY_QOS</kind>
    </reliability>
</datareader_qos>

The DataWriter is already configured to send data reliably if the DataReader is reliable. In the example, we adjusted the reliability protocol settings to repair dropped samples in a timely manner. Please refer to the XML in the example for detailed settings.

The DataWriter is configured with "non-strict reliability." This means that a cache of data is kept on the DataWriter. The data in the DataWriter's cache is delivered reliably. However, if there is a network disruption, the DataWriter is allowed to overwrite data that the DataReader has not yet received. This offers some reliability for a network that may be busy, but allows the DataWriter to continue sending data even if a DataReader has not received it. This is good for a system like a UAV, where it may be more important to send the current video rather than ensuring that the entire video is received.

<!-- Video data can be best-effort or reliable, depending -->
<!--  on network characteristics                          -->
<reliability>
    <kind>RELIABLE_RELIABILITY_QOS</kind>
</reliability>

<!-- Reliably deliver 50 video frames, but do  not block -->
<!-- if the reader does not receive data                 -->
<history>
    <!-- If you need strict reliability, this should be  -->
    <!-- changed to keep all history.                    -->
    <kind>KEEP_LAST_HISTORY_QOS</kind>
    <depth>50</depth>
</history>

You can make the data strictly reliable by changing the QoS from KEEP_LAST_HISTORY_QOS to KEEP_ALL_HISTORY_QOS.

Join the Community

Post questions on the RTI Community Forum.

Contribute to our Case + Code examples on RTI Community GitHub using these instructions.

The video shipped with this example is Big Buck Bunny, (c) copyright 2008, Blender Foundation / www.bigbuckbunny.orgVideo format is WebM, a royalty-free open source video codec.