OpenSLES | Android audio interface through c

Open SL | ES (Open Sound Library for Embedded Systems) is a software library created by khronos group. It is a royalty free, cross platform, hardware accelerated c - language 2D and 3D audio API. This has been adopted as the standard in android since API level 9 [gingerbread]. It follows the object and interface model. Rest of this post details creating a simple c program to play sound.

I will be using the 8bit pcm file that came with android ndk as my sound source.  We can play other popular streams too, but for this article the program will just say 'hello android'. The api calls for an 'engine' object to be created at first. This diagram illustrates the required relationship

Before we start anything let's include the necessary headers

#include <stdio.h>

#include <assert.h>

#include <string.h>

#include <SLES/OpenSLES.h>

#include <SLES/OpenSLES_Android.h>

The pcm 8 bit values to say hello are encoded in the file hello_clip.h [found it one of the examples in android ndk]. The values look like this.

"\x01\x00\x03\x00\x06\x00\x0a\x00\x0a\x00\x09\x00\x04\x00\x04\x00"

"\x06\x00\x06\x00\x00\x00\xff\xff\x05\x00\x08\x00\x01\x00\xfe\xff"

"\xff\xff\x03\x00\x04\x00\xfe\xff\xf9\xff\xfd\xff\x04\x00\xfe\xff"

"\x03\x00\x04\x00\x01\x00\xfb\xff\xfb\xff\xfc\xff\xfb\xff\x03\x00"

"\xfc\xff\xf9\xff\xfc\xff\x01\x00\x06\x00\x00\x00\xf9\xff\xfa\xff"

"\x04\x00\x06\x00\xfe\xff\xfa\xff\xfd\xff\x01\x00\xfe\xff\xfe\xff"

"\xfe\xff\xfd\xff\xfd\xff\xfd\xff\xfe\xff\xff\xff\xfd\xff\xfa\xff"

"\xfe\xff\x00\x00\x03\x00\xfe\xff\xfc\xff\xfb\xff\xfe\xff\x01\x00"

        .............................................................

So lets create an object of engine and realize it. Don't worry its not that complicated stuff. assert() is used to check if everything is all right on creation of engine and its realisation. 

     SLresult result;

     static SLObjectItf engineObject = NULL;

     

     //First we need to create an engine and then check if it was successful

     result = slCreateEngine(&engineObject, 0, NULL, 0, NULL, NULL);

     assert(SL_RESULT_SUCCESS == result);

     

     //We realize the engine object

     result = (*engineObject)->Realize(engineObject, SL_BOOLEAN_FALSE);

     assert(SL_RESULT_SUCCESS == result);

Now we get the engine interface [observe that we need two things to control the engine ObjectItf and EngineItf].

     static SLEngineItf engineEngine;

     

     //Get the engine's interface

     result = (*engineObject)->GetInterface(engineObject, SL_IID_ENGINE, &engineEngine);

     assert(SL_RESULT_SUCCESS == result);

Now we create the output mix

     static SLObjectItf outputMixObject = NULL;

     const SLInterfaceID ids[1] = {SL_IID_ENVIRONMENTALREVERB};

     const SLboolean req[1] = {SL_BOOLEAN_FALSE};

     result = (*engineEngine)->CreateOutputMix(engineEngine, &outputMixObject, 1, ids, req);

     assert(SL_RESULT_SUCCESS == result);

Observe that output mix is used to control the quality of the output. It maintains the equalisation, virtualiser, etc,. The ids and req help the function in determining the type of the output mix. Here we tell the function that we don't want the environmental reverberation settings. We also need to realise this 

     // realize the output mix

     result = (*outputMixObject)->Realize(outputMixObject, SL_BOOLEAN_FALSE);

     assert(SL_RESULT_SUCCESS == result);

Next we create a buffer queue, set data type  and configure the source.

     // configure audio source

     SLDataLocator_AndroidSimpleBufferQueue loc_bufq =   {SL_DATALOCATOR_ANDROIDSIMPLEBUFFERQUEUE, 2};

     SLDataFormat_PCM format_pcm = {SL_DATAFORMAT_PCM, 1, SL_SAMPLINGRATE_8,

         SL_PCMSAMPLEFORMAT_FIXED_16, SL_PCMSAMPLEFORMAT_FIXED_16,

         SL_SPEAKER_FRONT_CENTER, SL_BYTEORDER_LITTLEENDIAN};

     SLDataSource audioSrc = {&loc_bufq, &format_pcm};

Next create an audio sink

     SLDataLocator_OutputMix loc_outmix = {SL_DATALOCATOR_OUTPUTMIX, outputMixObject};

     SLDataSink audioSnk = {&loc_outmix, NULL};

Now its the time we can create our audio player. We mention that Buffer queue, Effect send and volume objects are required 

     const SLInterfaceID ids2[3] = {SL_IID_BUFFERQUEUE, SL_IID_EFFECTSEND,

         SL_IID_VOLUME};

     const SLboolean req2[3] = {SL_BOOLEAN_TRUE, SL_BOOLEAN_TRUE,

         SL_BOOLEAN_TRUE};

     result = (*engineEngine)->CreateAudioPlayer(engineEngine, &bqPlayerObject, &audioSrc, &audioSnk,

                                                 3, ids2, req2);

     assert(SL_RESULT_SUCCESS == result);

We need to realise this too.

     result = (*bqPlayerObject)->Realize(bqPlayerObject, SL_BOOLEAN_FALSE);

     assert(SL_RESULT_SUCCESS == result);

Now we need to the player interface and the buffer queue interface

     result = (*bqPlayerObject)->GetInterface(bqPlayerObject, SL_IID_PLAY, &bqPlayerPlay);

     assert(SL_RESULT_SUCCESS == result);

     

     // get the buffer queue interface

     result = (*bqPlayerObject)->GetInterface(bqPlayerObject, SL_IID_BUFFERQUEUE,

                                              &bqPlayerBufferQueue);

     assert(SL_RESULT_SUCCESS == result);

At the end of each frame next frame has to be loaded. There is a call back functionality available. This function is implemented and linked to the main program. Let us create a function first.

void bqPlayerCallback(SLAndroidSimpleBufferQueueItf bq, void *context)

{

    assert(bq == bqPlayerBufferQueue);

    assert(NULL == context);

        if (NULL != nextBuffer && 0 != nextSize) {

        SLresult result;

        result = (*bqPlayerBufferQueue)->Enqueue(bqPlayerBufferQueue, nextBuffer, nextSize);

        assert(SL_RESULT_SUCCESS == result);

        nextBuffer = NULL;

        nextSize = NULL;

    }

}

We will use this to play a second frame and then stop. The second frame will be saying 'android'. It was taken from the same example. We shall register this callback 

     result = (*bqPlayerBufferQueue)->RegisterCallback(bqPlayerBufferQueue, bqPlayerCallback, NULL);

     assert(SL_RESULT_SUCCESS == result);

Then we set the player state to playing

     result = (*bqPlayerPlay)->SetPlayState(bqPlayerPlay, SL_PLAYSTATE_PLAYING);

     assert(SL_RESULT_SUCCESS == result);

Now we put the buffers out to play

     nextBuffer = (short *) hello;

     nextSize = sizeof(hello);

     result = (*bqPlayerBufferQueue)->Enqueue(bqPlayerBufferQueue, nextBuffer, nextSize);

     assert(SL_RESULT_SUCCESS != result);

     nextBuffer = (short *) android;

     nextSize = sizeof(android);

     sleep(2);

This will play hello first and then call the call back function, which will play android as it is queued next. There is a delay to complete the playing.

     if (bqPlayerObject != NULL) {

         (*bqPlayerObject)->Destroy(bqPlayerObject);

         bqPlayerObject = NULL;

         bqPlayerPlay = NULL;

         bqPlayerBufferQueue = NULL;

     }

     

     if (outputMixObject != NULL) {

         (*outputMixObject)->Destroy(outputMixObject);

         outputMixObject = NULL;

         outputMixEnvironmentalReverb = NULL;

     }

     

     if (engineObject != NULL) {

         (*engineObject)->Destroy(engineObject);

         engineObject = NULL;

         engineEngine = NULL;

     }

We destroy all the objects created, and then exit the program.

Download the binaries and code from here. You will have to run it through either adb or terminal emulator by copying the binaries to convenient place, like /data/local/tmp. Try it out, let me know if the code or binary doesn't work. Should work flawlessly on a jelly bean device. Haven't checked it on any others