GStreamer Programming - My Experience

This is no official documentation or any kind of FAQ or HOWTO. I only want to record and share my experience with GStreamer programming and the solutions I found for the issues I ran into. Before trying to use any of the information presented on this page, please read the official GStreamer Application Development Manual. A lot of information came from the folks on the #gstreamer IRC channel on freenode. Thanks a lot!

Official GStreamer Homepage

I don't doubt that some of the stuff I'm going to put up here is either wrong, inefficient or can be done easier if I only knew how. Please feel free to comment on them or correct mistakes or misstatements. I don't want to bash on GStreamer, I want to learn more about how to use it and at the same time document my findings for others to read. This page probably will never be well-organized or structured, as I will add bits and pieces here whenever I feel the need to. Still, I hope it's going to be useful for some of you. Mind you, I am no professional software engineer, nor did I study Computer Science. So please don't hold my lack of "serious coding methods" or "project planning" against me. I just enjoy creating software ;)

When you start digging into GStreamer programming you will probably wonder how to achieve common tasks, such as "Playback an audio file, keep track of it's current playing position and be able to stop / restart / pause / seek it at any time". The documentation doesn't help you there very much, and reading the core API is not too helpful either, because even a simple thing as "stopping" or "pausing" the stream takes several steps for it to work as expected.

Using GStreamer with C++ certainly doesn't help matters :)

The API documentation is extensive and probably complete by means of "accurate". But it severely lacks references to code examples, "see also" references, (e.g. the command to link elements together should contain a reference to the command to unlink them later, too).

All code examples on this page are licenced under the GNU General Public Licence V2.

Compiling and Linking

To successfully compile your project with GStreamer, you need to have the right includes in your source code:

#include <gst/gst.h>

For the linking stage, the compiler needs to know the location of the GStreamer libraries. Usually this is done by using the pkg-config tool. For example, in kdevelop, open your Project Options dialog and add the following to the Configure Options:

Here's the lines for you to copy and paste:

$(pkg-config --cflags gstreamer-0.10) $(pkg-config --libs gstreamer-0.10)

Example Pipeline

Here is an example setup for a GStreamer pipeline I am using in my first application (click to enlarge):

You can see the toplevel pipeline, a bin for each of the actual sound files to be played and a number of elements making up the stream chain for file reading, a stream adder, and in the end the sound output alsasink. The process of connecting elements and forming a stream chain is called linking.

A sink is a pad that will accept data, a source is a pad, that will deliver data. Elements can have either of both, and some element types have more than one sink and/or source. In this example, the filesrc Element is a source element, it only delivers data to the stream. The alsasink element in the end of the main pipeline is a sink element - it only accepts data. The adder element is an example of an element with multiple sinks, which adds up all the stream data coming in and sends them out to its single source element.

You might have noticed the dashed arrow that leads to the pad at the end of each bin. This is called a ghost pad. You need ghost pads to make element pads accessible to the outside of the container they are placed in. Linking pads together does only work, when the pads' owning elements are inside of the same container. So, in order to get the stream data out of the file's volume element's source pad and link it to one of the adder element's sink pads, I need to "export" the pad to the bin - creating a ghost pad.

Seeking

If you are going to write any sort of media player, you will without a doubt want to seek the files you're playing. Fast forward, rewind, restart, all these are essential functions for these types of applications. Going by the example setup shown above, you would think the right way to do it would be trying to seek on the bin that's holding all the other items of the particular file, having the container pass on the seek to its children. However, this is not working. Instead, you seek on one of the children inside of the bin. I used the volume element for my application, and it seems to work quite well. The actual seek code I use to skip two seconds backwards looks similar to this:

qDebug("SoundObject:rewindPressed(): seeking backwards"); if(!gst_element_seek(volumeElement, 1.0, GST_FORMAT_TIME, GST_SEEK_FLAG_FLUSH, GST_SEEK_TYPE_CUR, -2000000000, GST_SEEK_TYPE_NONE, -1 ) ) { qDebug("SoundObject::rewindPressed(): seek failed"); } qDebug("SoundObject::rewindPressed(): seek done.");

GStreamer Callbacks in C++

Since the C++ Language Bindings are still under development, I had to use GStreamer's native C interface. With normal programming, there's no real problem with it, but when it comes to message transport from GStreamer elements to the application, it exposes a variety of problems. GStreamer uses callbacks to communicate with the application, and naturally, object oriented design and callbacks don't really go well together. But since I had the same issue at another point of my application (decoding sound files using the MAD library), I had a working solution for it.

Let's say you built a bin of elements and want to catch bus signals. What I found working to some extent was something similar to the following code:

gboolean soundObjectBusCall(GstBus* bus,GstMessage* message,gpointer data) { // create a proper object pointer from the supplied data argument SoundObject* self=(SoundObject*) data; switch(GST_MESSAGE_TYPE(message)) { // ... handle element messages here case GST_MESSAGE_ELEMENT: { // take appropriate action inside the calling method self->someFunction(); } // case // ... handle more cases here, like GST_MESSAGE_ERROR etc. default: break; } // tell the system we have dealt with the messages properly return true; }

Note that this function is not a member of any class. It's just a global function that handles the signals coming in from the bus and calls the appropriate object's method. The data parameter is used to find the object's address in memory. To set up a callback on the bus, I use something like this:

// get a reference of this pipeline's bus GstBus* bus=gst_pipeline_get_bus(GST_PIPELINE(mainPipeline)); // insert a bus watcher with our global function as callback and *this* pointer as data guint watchId=gst_bus_add_watch(bus,soundObjectBusCall,this); // remove reference on the bus from memory, since we don't need it anymore gst_object_unref(bus);

One disadvantage of this solution is that all the methods and class members you want to access from the callback function need to be declared public, A possible way to prevent this would be a general purpose public entry method which in turn can use the internal, protected or private methods of the class.

The watchId is a reference counter for the inserted watch callback. You can use it to remove the bus watch later like this:

// deactivate bus callback g_source_remove(watchId);