FFMPEG on Android | Command line tool

Got the ffmpeg command line tool working on android today. I was working on a screenshot binary for android. And the way you take a screenshot on android is to cat /dev/graphics/fb0. Now this file is in RGB565 format [on Emulator]. Initially I managed it by pulling the file to my computer and then using ffmpeg to convert it to jpg. Though porting ffmpeg for android was not my initial idea, and it has already been done before. But I didn't find any ffmpeg binaries online that I could directly download and run over my phone. Therefore I decided to see if I could build it from source and it took me a couple of hours to get it working. The binary runs all the user commands except the common options like "-h","-v" [For some reason there was no definitions to cover those in the source I used]. You can download the binaries here [Need to copy the library 'libffmpeg.so' to the '/system/lib' folder]. I did, even copy the binary to my 'system/bin' so that I can comfortably run it from the shell.
To finish of the work I wrote a shell script to do rest of the work

#Shell script to take a screenshot
cat /dev/graphics/fb0 > /sdcard/raw-input
ffmpeg -vcodec rawvideo -f rawvideo -pix_fmt rgb565 -s 480x800 -i /sdcard/raw-input -f image2 -vcodec mjpeg /sdcard/screenshot.jpg
rm /sdcard/raw-input
echo "Screen shot saved at scard/screenshot.jpg"

I do know that there is an inbuilt screenshot binary in android that does the work instead of this lousy process. But I am working on something more interesting. To be revealed in the days to come.

Have fun with ffmpeg

Here are some screenshots that were pulled from the device

PS: I am not sure if it works on all versions of android. I've only tried it on KitKat...

Audio Amplifier | TDA2050

A quick update on the audio amplifiers. This is a higher power IC, TDA2050. The data sheet claims that it can do up to 32W. I simply put up a application circuit from the datasheet. I put it into a temporary PCB layout that I designed. I've plans to design a complete 5.1 channel amp, more updates on it later. The circuit produces awesome amount of sound, enough for your neighbors to complain. And I was only running a small speaker that I had designed.

Here is a video of it working, [My room's a mess :(]

Check the same on youtube for better quality

I'll update with the circuit layout very soon

OpenSLES | Android audio interface through c

Open SL | ES (Open Sound Library for Embedded Systems) is a software library created by khronos group. It is a royalty free, cross platform, hardware accelerated c - language 2D and 3D audio API. This has been adopted as the standard in android since API level 9 [gingerbread]. It follows the object and interface model. Rest of this post details creating a simple c program to play sound.

I will be using the 8bit pcm file that came with android ndk as my sound source.  We can play other popular streams too, but for this article the program will just say 'hello android'. The api calls for an 'engine' object to be created at first. This diagram illustrates the required relationship

Before we start anything let's include the necessary headers

#include <stdio.h>

#include <assert.h>

#include <string.h>

#include <SLES/OpenSLES.h>

#include <SLES/OpenSLES_Android.h>

The pcm 8 bit values to say hello are encoded in the file hello_clip.h [found it one of the examples in android ndk]. The values look like this.

"\x01\x00\x03\x00\x06\x00\x0a\x00\x0a\x00\x09\x00\x04\x00\x04\x00"

"\x06\x00\x06\x00\x00\x00\xff\xff\x05\x00\x08\x00\x01\x00\xfe\xff"

"\xff\xff\x03\x00\x04\x00\xfe\xff\xf9\xff\xfd\xff\x04\x00\xfe\xff"

"\x03\x00\x04\x00\x01\x00\xfb\xff\xfb\xff\xfc\xff\xfb\xff\x03\x00"

"\xfc\xff\xf9\xff\xfc\xff\x01\x00\x06\x00\x00\x00\xf9\xff\xfa\xff"

"\x04\x00\x06\x00\xfe\xff\xfa\xff\xfd\xff\x01\x00\xfe\xff\xfe\xff"

"\xfe\xff\xfd\xff\xfd\xff\xfd\xff\xfe\xff\xff\xff\xfd\xff\xfa\xff"

"\xfe\xff\x00\x00\x03\x00\xfe\xff\xfc\xff\xfb\xff\xfe\xff\x01\x00"

        .............................................................

So lets create an object of engine and realize it. Don't worry its not that complicated stuff. assert() is used to check if everything is all right on creation of engine and its realisation. 

     SLresult result;

     static SLObjectItf engineObject = NULL;

     

     //First we need to create an engine and then check if it was successful

     result = slCreateEngine(&engineObject, 0, NULL, 0, NULL, NULL);

     assert(SL_RESULT_SUCCESS == result);

     

     //We realize the engine object

     result = (*engineObject)->Realize(engineObject, SL_BOOLEAN_FALSE);

     assert(SL_RESULT_SUCCESS == result);

Now we get the engine interface [observe that we need two things to control the engine ObjectItf and EngineItf].

     static SLEngineItf engineEngine;

     

     //Get the engine's interface

     result = (*engineObject)->GetInterface(engineObject, SL_IID_ENGINE, &engineEngine);

     assert(SL_RESULT_SUCCESS == result);

Now we create the output mix

     static SLObjectItf outputMixObject = NULL;

     const SLInterfaceID ids[1] = {SL_IID_ENVIRONMENTALREVERB};

     const SLboolean req[1] = {SL_BOOLEAN_FALSE};

     result = (*engineEngine)->CreateOutputMix(engineEngine, &outputMixObject, 1, ids, req);

     assert(SL_RESULT_SUCCESS == result);

Observe that output mix is used to control the quality of the output. It maintains the equalisation, virtualiser, etc,. The ids and req help the function in determining the type of the output mix. Here we tell the function that we don't want the environmental reverberation settings. We also need to realise this 

     // realize the output mix

     result = (*outputMixObject)->Realize(outputMixObject, SL_BOOLEAN_FALSE);

     assert(SL_RESULT_SUCCESS == result);

Next we create a buffer queue, set data type  and configure the source.

     // configure audio source

     SLDataLocator_AndroidSimpleBufferQueue loc_bufq =   {SL_DATALOCATOR_ANDROIDSIMPLEBUFFERQUEUE, 2};

     SLDataFormat_PCM format_pcm = {SL_DATAFORMAT_PCM, 1, SL_SAMPLINGRATE_8,

         SL_PCMSAMPLEFORMAT_FIXED_16, SL_PCMSAMPLEFORMAT_FIXED_16,

         SL_SPEAKER_FRONT_CENTER, SL_BYTEORDER_LITTLEENDIAN};

     SLDataSource audioSrc = {&loc_bufq, &format_pcm};

Next create an audio sink

     SLDataLocator_OutputMix loc_outmix = {SL_DATALOCATOR_OUTPUTMIX, outputMixObject};

     SLDataSink audioSnk = {&loc_outmix, NULL};

Now its the time we can create our audio player. We mention that Buffer queue, Effect send and volume objects are required 

     const SLInterfaceID ids2[3] = {SL_IID_BUFFERQUEUE, SL_IID_EFFECTSEND,

         SL_IID_VOLUME};

     const SLboolean req2[3] = {SL_BOOLEAN_TRUE, SL_BOOLEAN_TRUE,

         SL_BOOLEAN_TRUE};

     result = (*engineEngine)->CreateAudioPlayer(engineEngine, &bqPlayerObject, &audioSrc, &audioSnk,

                                                 3, ids2, req2);

     assert(SL_RESULT_SUCCESS == result);

We need to realise this too.

     result = (*bqPlayerObject)->Realize(bqPlayerObject, SL_BOOLEAN_FALSE);

     assert(SL_RESULT_SUCCESS == result);

Now we need to the player interface and the buffer queue interface

     result = (*bqPlayerObject)->GetInterface(bqPlayerObject, SL_IID_PLAY, &bqPlayerPlay);

     assert(SL_RESULT_SUCCESS == result);

     

     // get the buffer queue interface

     result = (*bqPlayerObject)->GetInterface(bqPlayerObject, SL_IID_BUFFERQUEUE,

                                              &bqPlayerBufferQueue);

     assert(SL_RESULT_SUCCESS == result);

At the end of each frame next frame has to be loaded. There is a call back functionality available. This function is implemented and linked to the main program. Let us create a function first.

void bqPlayerCallback(SLAndroidSimpleBufferQueueItf bq, void *context)

{

    assert(bq == bqPlayerBufferQueue);

    assert(NULL == context);

        if (NULL != nextBuffer && 0 != nextSize) {

        SLresult result;

        result = (*bqPlayerBufferQueue)->Enqueue(bqPlayerBufferQueue, nextBuffer, nextSize);

        assert(SL_RESULT_SUCCESS == result);

        nextBuffer = NULL;

        nextSize = NULL;

    }

}

We will use this to play a second frame and then stop. The second frame will be saying 'android'. It was taken from the same example. We shall register this callback 

     result = (*bqPlayerBufferQueue)->RegisterCallback(bqPlayerBufferQueue, bqPlayerCallback, NULL);

     assert(SL_RESULT_SUCCESS == result);

Then we set the player state to playing

     result = (*bqPlayerPlay)->SetPlayState(bqPlayerPlay, SL_PLAYSTATE_PLAYING);

     assert(SL_RESULT_SUCCESS == result);

Now we put the buffers out to play

     nextBuffer = (short *) hello;

     nextSize = sizeof(hello);

     result = (*bqPlayerBufferQueue)->Enqueue(bqPlayerBufferQueue, nextBuffer, nextSize);

     assert(SL_RESULT_SUCCESS != result);

     nextBuffer = (short *) android;

     nextSize = sizeof(android);

     sleep(2);

This will play hello first and then call the call back function, which will play android as it is queued next. There is a delay to complete the playing.

     if (bqPlayerObject != NULL) {

         (*bqPlayerObject)->Destroy(bqPlayerObject);

         bqPlayerObject = NULL;

         bqPlayerPlay = NULL;

         bqPlayerBufferQueue = NULL;

     }

     

     if (outputMixObject != NULL) {

         (*outputMixObject)->Destroy(outputMixObject);

         outputMixObject = NULL;

         outputMixEnvironmentalReverb = NULL;

     }

     

     if (engineObject != NULL) {

         (*engineObject)->Destroy(engineObject);

         engineObject = NULL;

         engineEngine = NULL;

     }

We destroy all the objects created, and then exit the program.

Download the binaries and code from here. You will have to run it through either adb or terminal emulator by copying the binaries to convenient place, like /data/local/tmp. Try it out, let me know if the code or binary doesn't work. Should work flawlessly on a jelly bean device. Haven't checked it on any others