Showing posts with label linux. Show all posts
Showing posts with label linux. Show all posts

Tuesday, November 22, 2016

Building Racket for Android

I've been using Racket for academic purposes and decided to try it running on android. I was able to get it built for android. This post details the efforts made and is also meant to be sort of a log entry.

Download the source from github. Use this script to compile it on linux/mac.

It's simple to compile it for android. However the real trouble lies in getting it to run, because you need to have all the startup files and also the binary should know where to find them. So first time you run it on a android device it shows this error message

Welcome to Racket v6.7.0.3.
standard-module-name-resolver: collection not found
  for module path: (submod (lib "racket/init") configure-runtime)
  collection: "racket"
  in collection directories:
   /data/.racket/6.7.0.3/collects
   /home/regnarts/workspace/scheme/android/armv5/racket/collects
standard-module-name-resolver: collection not found
  for module path: racket/interactive
  collection: "racket"
  in collection directories:
   /data/.racket/6.7.0.3/collects
  /home/regnarts/workspace/scheme/android/armv5/racket/collects
standard-module-name-resolver: collection not found
  for module path: racket/base
  collection: "racket"
  in collection directories:
   /data/.racket/6.7.0.3/collects
   /home/regnarts/workspace/scheme/android/armv5/racket/collects

Which is expected. Now you can fix this in two ways one is by using '-S' to specify the path that the interpreter is expected to look at. Or as the README suggests, edit the binary to change the default collects directory path name!!. You might want to go through the README and this post to get it working .

Once that is done the interpreter should start-up and behave as it would do on any other platform. However the boot time is notoriously long. Got to figure out a way to reduce it. This is not a main project but something I did to kill time. So I am not going to look into it now. However it would be really nice to get something like DRRacket to work on Android. This can be made into a app with moderate effort. But that's the story for another day.

Finally I could get my interpreter to run on racket running on Android.

Cheers!!

Friday, January 9, 2015

The Androux project

Finally, I've completed the linux support for the ports. I have decided to call the project androux. A play on words android and linux. You can grab the source from github. Finally "make" works and the compile script works on linux x86_64. The build scripts still won't run if you are using 32 bit machine. Will try to bring support for that in future. The android shell is not fit to run the configure scripts and hence a port of few utilities like sed and bash shell is required. I am working on it.

As of now you can create a Makefile and compile your projects on the device. I have tried a basic test and make works fine.

Visit github for instructions on compiling.

After compiling, you need to copy system folder created to the system folder of your device. You also need to copy the include and lib folders from platform directory corresponding to android version that your device runs.

The final system image will be quite big and I am quite sure that the partitions that are created in your phone by default are not enough to hold it all. There are two ways of approaching this problem, either extend the system partition, if the inbuilt flash can hold it or create a new partition in the sdcard and change the mount scripts to mount that partition as system instead.

When writing install scripts in make file we need to take care that system partition is mounted r/w. By default the system partition is read only in android. Or you could choose another install location.

I still haven't had time or resources to test on an actual device. Please let me know if any of you can do so and share the results

Cheers ! :-)
Stay Awesome  

Sunday, May 18, 2014

Android Kernel Compile | Galaxy Fit S5670 | ARM Basics

Very recently I was asked by one of my professors to compile a custom kernel to his phone. He gave me an old Samsung Galaxy Fit phone.  This post documents the process carried out to do the same.

The phone runs ARMv6-compatible processor rev 5 (v6l) and supports features like



swp [SWaP] & swpb [SWaPByte]

This instruction is deprecated as of ARMv7 even in some later versions of ARMv6

This instruction was used to implement exclusive access to semaphores themselves.
Semaphores are used to manage access to a shared resource. Depending on the type of semaphore, one or more clients may be granted access.
Before accessing a resource, a client must read the semaphore value and check that it indicates whether the client can proceed, or whether it must wait. When the client is about to proceed it must change the semaphore value to inform other clients.
A fundamental issue with semaphores is that they themselves are shared resources, which – as we just learned – must be protected by semaphores.
SWP (Swap) and SWPB (Swap Byte) provide a method for software synchronization that does not require disabling interrupts. This is achieved by performing a special type of memory access, reading a value into a processor register and writing a value to a memory location as an atomic operation. The example code below shows the implementation of simple mutex functions using the SWPinstruction. SWP and SWPB are not supported in the Thumb instruction set, so the example must be assembled for ARM.
binary mutex functions
    EXPORT lock_mutex_swp
lock_mutex_swp PROC
    LDR r2, =locked
    SWP r1, r2, [r0]       ; Swap R2 with location [R0], [R0] value placed in R1
    CMP r1, r2             ; Check if memory value was ‘locked’
    BEQ lock_mutex_swp     ; If so, retry immediately
    BX  lr                 ; If not, lock successful, return
    ENDP

    EXPORT unlock_mutex_swp
unlock_mutex_swp
    LDR r1, =unlocked
    STR r1, [r0]           ; Write value ‘unlocked’ to location [R0]
    BX  lr
    ENDP
In the SWP instruction in above example, R1 is the destination register that receives the value from the memory location, and R2 is the source register that is written to the memory location. You can use the same register for destination and source. For example 
        SWP r2, r2, [r0]       ; Swap R2 with location [R0], [R0] value placed in R2
is a valid instruction.

But there are certain limitations to this. If an interrupt triggers while a swap operation is taking place, the processor must complete both the load and the store part of the instruction before taking the interrupt, increasing interrupt latency. Because Load-Exclusive and Store-Exclusive are separate instructions, this effect is reduced when using the new synchronization primitives. In our source for Galaxy Fit we will find that swp or swpb aren't used to create atomic operations but Load-Exclusive and Store-Exclusive. Here is an example 

static inline void atomic_add(int i, atomic_t *v)
{
unsigned long tmp;
int result;

__asm__ __volatile__("@ atomic_add\n"
"1: ldrex %0, [%3]\n"
" add %0, %0, %4\n"
" strex %1, %0, [%3]\n"
" teq %1, #0\n"
" bne 1b"
: "=&r" (result), "=&r" (tmp), "+Qo" (v->counter)
: "r" (&v->counter), "Ir" (i)
: "cc");

}

This is taken from atomic.h of /arch/arm/include/asm/ directory.
In a multi-core system, preventing access to main memory for all processors for the duration of a swap instruction can reduce overall system performance. This is especially true in a multi-core system where processors operate at different frequencies but share the same main memory.

half [Half - Precision Floating Point Support]

In the ARM's VFP [Vector Floating Point Co-Processor] the support for 'half' means that it supports 16 bit floating point numbers and conversions between 16-32 bit floating point numbers. Half-precision floating-point numbers are provided as an optional extension to the VFPv3 architecture.
Half-precision floating-point format

Where:
   S (bit[15]):      Sign bit
   E (bits[14:10]):  Biased exponent
   T (bits[9:0]):    Mantissa.
The meanings of these fields depend on the format that is selected.

thumb [The Thumb Instruction set]

The Thumb instruction set is a subset of the most commonly used 32-bit ARM instructions. Thumb instructions are each 16 bits long, and have a corresponding 32-bit ARM instruction that has the same effect on the processor model. Thumb instructions operate with the standard ARM register configuration, allowing excellent interoperability between ARM and Thumb states.
On execution, 16-bit Thumb instructions are transparently decompressed to full 32-bit ARM instructions in real time, without performance loss.
Thumb has all the advantages of a 32-bit core:
  • 32-bit address space
  • 32-bit registers
  • 32-bit shifter, and Arithmetic Logic Unit (ALU)
  • 32-bit memory transfer.
Thumb therefore offers a long branch range, powerful arithmetic operations, and a large address space.
Thumb code is typically 65% of the size of ARM code, and provides 160% of the performance of ARM code when running from a 16-bit memory system.
The availability of both 16-bit Thumb and 32-bit ARM instruction sets gives designers the flexibility to emphasize performance or code size on a subroutine level, according to the requirements of their applications. For example, critical loops for applications such as fast interrupts and DSP algorithms can be coded using the full ARM instruction set then linked with Thumb code

fastmult [Fast Multiplication]

This refers to the fact that the processor can do 32 bit multiplication. The processor implementing this provides hardware to preform 32bit x 32 bit = 64 bit operation. This is a common feature in almost all the processors nowadays. But still they guys who built the kernel felt the need to mention it.

vfp [Vector Floating Point Co-Processor]

The VFP coprocessor supports floating point arithmetic operations and is a functional block within.
The VFP has its own bank of 32 registers for single-precision operands that you can:
  • use in pairs for double-precision operands
  • operate loads and stores of VFP registers in parallel with arithmetic operations.
The VFP supports a wide range of single and double precision operations, including ABS, NEG, COPY, MUL, MAC, DIV, and SQRT. The VFP effectively executes most of these in a single cycle. Sometime in future I promise to do a tutorial on this and NEON instructions.

edsp [DSP extensions]

The ARM DSP instruction set extensions increase the DSP processing capability of ARM solutions in high-performance applications, while offering the low power consumption required by portable, battery-powered devices. DSP extensions are optimized for a broad range of software applications including servo motor control, Voice over IP (VOIP) and video & audio codecs, where the extensions increase the DSP performance to enable efficient processing of the required tasks.

Features

  • Single-cycle 16x16 and 32x16 MAC implementations
  • 2-3 x DSP performance improvement over ARM7™ processor-based CPU products
  • Zero overhead saturation extension support
  • New instructions to load and store pairs of registers, with enhanced addressing modes
  • New CLZ instruction improves normalization in arithmetic operations and improves divide performance
  • Full support in the ARMv5TE, ARMv6 and ARMv7 architectures

Applications

  • Audio encode/decode (MP3: AAC, WMA)
  • Servo motor control (HDD/DVD)
  • MPEG4 decode
  • Voice and handwriting recognition
  • Embedded control
  • Bit exact algorithms (GSM-AMR)

java [Jazelle]

Jazelle DBX (Direct Bytecode eXecution) allows some ARM processors to execute Java bytecode in hardware as a third execution state alongside the existing ARM and Thumb modes. Jazelle functionality was specified in the ARMv5TEJ architecture and the first processor with Jazelle technology was the ARM926EJ-S. Jazelle is denoted by a 'J' appended to the CPU name, except for post-v5 cores where it is required (albeit only in trivial form) for architecture conformance.
These are the extra features that the processor supports over the regular ones. The kernel can only be complied over linux or Mac. For this one I will be using linux. First we need to install a few requirements and then download the source.


You will find the following files in the kernel source. Next install the required programs to compile the source. Run the following line in your terminal.

sudo apt-get install git-core gnupg flex bison gperf libsdl-dev libesd0-dev libwxgtk2.6-dev build-essential zip curl libncurses5-dev zlib1g-dev valgrind
Extract the kernel source. And also download the toolchain. Now extract the toolchain. and place it in a folder. You'll find the following in the toolchain folder.

Go to the kernel source and edit the Makefile to add the CROSS_COMPILE to point to the toolchain. It must look like this


Make sure the path is right. For me the toolchain was in /home/regnarts/S3/arm-2011.03 check the full path before updating.

Now to make this kernel there is a small problem. The original kernel source that was released by the samsung didn't have any support for Ext filesystem. That was because the original rom for this phone runs on samsung's proprietary RFS file system. So if you want to only run the stock rom over the kernel then just jump this part. Else you will need to enable the support for this in the config file. Find /arch/arm/configs/beni_rev01_defconfig and change the File System settings as shown below.


There is another important thing. The phone uses proprietary wifi/wireless drivers. So their source code is not provided. Therefore we need to just use the already existing ones. But the problem is that insmod checks for the kernel version before loading the modules. So if there is a mismatch. The kernel won't load the modules. Therefore you'll need to name the kernel as the module wishes to see it. Run

make menuconfig

This opens up the menu to configure the kernel.

In general setup append the name "-pref-CL928291" to the Local version. Now you are good to go. Run the make script that came with the kernel make_kernel_GT-S5670.sh. Or run

make beni_rev01_defconfig
make

That should give us the kernel. You will find it as /arch/arm/boot/zImage. Congratulations you've built the kernel for your phone. However you will need to still put it up in to the phone. For this we shall make the android's very popular "recovery flashable" zip file.

Download a sample file. Extract the zip, you'll find this

Now the file boot.img is the kernel+ramdisk file. But in order to build a boot.img you'll first need a ramdisk. We shall take the ramdisk from the sample previously downloaded. Use boot-tools to unpack and use the boot.img. The following must help
To unpack the boot.img

tools/unpackbootimg -i source_img/boot.img -o unpack

To unpack ramdisk (If you want to)

gzip -dc ../unpack/boot.img-ramdisk.gz | cpio -i

Replace boot.img-zImage with your own zImage and then run

tools/mkbootimg --kernel unpack/boot.img-zImage --ramdisk unpack/boot.img-ramdisk-new.gz -o target_img/boot.img --base `cat unpack/boot.img-base`

With the new boot.img replace it in the update.zip and zip it again.
This will flash in most of the recoveries if you turn of the signature verification. If you are unable to do it in your recovery. Then you'll need to sign it. Use this to sign it from linux.

export CLASSPATH=$CLASSPATH:./lib.jar
java testsign update.zip update-finished.zip

Now flash the zip file from recovery.
I'll discuss overclocking this phone in the next blog post.






Sunday, December 8, 2013

FFMPEG on Android | Command line tool

Got the ffmpeg command line tool working on android today. I was working on a screenshot binary for android. And the way you take a screenshot on android is to cat /dev/graphics/fb0. Now this file is in RGB565 format [on Emulator]. Initially I managed it by pulling the file to my computer and then using ffmpeg to convert it to jpg. Though porting ffmpeg for android was not my initial idea, and it has already been done before. But I didn't find any ffmpeg binaries online that I could directly download and run over my phone. Therefore I decided to see if I could build it from source and it took me a couple of hours to get it working. The binary runs all the user commands except the common options like "-h","-v" [For some reason there was no definitions to cover those in the source I used]. You can download the binaries here [Need to copy the library 'libffmpeg.so' to the '/system/lib' folder]. I did, even copy the binary to my 'system/bin' so that I can comfortably run it from the shell.
To finish of the work I wrote a shell script to do rest of the work

#Shell script to take a screenshot
cat /dev/graphics/fb0 > /sdcard/raw-input
ffmpeg -vcodec rawvideo -f rawvideo -pix_fmt rgb565 -s 480x800 -i /sdcard/raw-input -f image2 -vcodec mjpeg /sdcard/screenshot.jpg

rm /sdcard/raw-input
echo "Screen shot saved at scard/screenshot.jpg"


I do know that there is an inbuilt screenshot binary in android that does the work instead of this lousy process. But I am working on something more interesting. To be revealed in the days to come.

Have fun with ffmpeg

Here are some screenshots that were pulled from the device
PS: I am not sure if it works on all versions of android. I've only tried it on KitKat...

Thursday, August 9, 2012

Android native C programs | hello, world ! ARM LINUX | Dynamic Linking

Well, there are a lot of websites online that tell you how to write a android program. But they all are written in java and are compiled to a dalvik executable. Lets get to the basics and write one that runs over your processor rather than the dalvik virtual machine. First of all you need a toolchain. CodeSourcery has a lite edition that is available for free download (linux link, There is a windows version available too). After you download the toolchain (Make sure you have downloaded ARM/GNU Linux version). Untar it, Install it or do whatever you have to get it running (I suggest you to just download the tarball instead of installer). This must give you a folder arm-20xx.xx. We need to copy the libraries required. Decide on your workspace. Create a folder to hold all your work call it /exp. Copy the tool chain to the same folder. Now connect your phone to the computer open shell, command prompt, whichever you are using to access adb and pull /system/lib from it to some new folder in /exp say /exp/lib.

$ adb pull /system/lib /exp/lib

This must pull all the libraries to that folder. Now write your first code hello.c in the root of /exp

#include<stdio.h>
int main()
{
         printf("hello, world! \n ");
         return 0;
}



Now is the time to compile it. Compile it with arm-none-linux-gnueabi-gcc

$./arm-2011.03/bin/arm-none-linux-gnueabi-gcc -c hello.c

This should output a file hello.o, this is our compiled file. But is still not a executable. So now as to get a executable we need to link the file (dynamically) to the libraries. We have already copied libraries to the file /lib. So let us run the loader with required flags to get our executable. What flags do we require???. We need one to tell our program where to start running on execution --entry flag takes care of that. We also need to tell the program where the dynamic linker is when being executed on your phone so we use --dynamic-linker flag. -nostdlib is required so that the compiler doesn't include the standard libraries of linux (The ones in android are quite different). Its is also required to give the path the program should search for its libraries while running on the phone, so the first -rpath. The second one does the work of pointing the linker to the library files as to where the files are stored now (the ones you copied from phone), this is to help the loaded libraries to search for its dependencies. -L, this specifies the path where our library is stored in the computer, the library that is specified by the -l option. -o gives the file name of the output executable. And hello.o is the input file.


$./arm-2011.03/bin/arm-none-linux-gnueabi-ld --entry=main --dynamic-linker /system/bin/linker -nostdlib -rpath /system/lib -rpath ~/exp/lib -L ~/exp/lib -l c -o hello hello.o 

Now we only need to push the file into the phone and execute it.


We can observer that there is a problem in this program. It doesn't exit like the normal programs we used to work with. Instead it has to be terminated manually. How to overcome this defect??. This occurs because we have not asked the program to exit at all. Let us do that now by creating another function that exits the program. We modify the program


#include<stdio.h>
#include<stdlib.h>
int main()
{
         printf("hello, world! \n ");
         return 0;
}

void _close()
{
        exit(main());
}


Now we need to start the program from the _close point so that it calls the main program and then exits it. so compile it same way. But while linking it use --entry flag for _close. 

./arm-2011.03/bin/arm-none-linux-gnueabi-ld --entry=_close --dynamic-linker /system/bin/linker -nostdlib -rpath /system/lib -rpath ~/exp/lib -L ~/exp/lib -l c -o hello hello.o 

Now push this the same way into the phone and execute it


Now the program should exit normally. Cheers!!!

Wednesday, July 25, 2012

Android | Linux "Sysfs" interface in kernel [Xperia X10]

Before reading this, make sure you have read this post and this one.
Sysfs is a virtual file system provided by the linux kernel. It is exports information about devices and drivers, and is also used for configuration. We used this in previous post to set scaling_governor and scaling_max_freq. All those files are part of Sysfs interface. This post will outline on creating a new file in the same old cpufreq folder to display all possible frequencies that the cpu can run on. This was done on Xperia X10, and I have good reasons to believe that The interface files are created from the source in /drivers folder of the /kernel. This particular one we are searching for must be in /cpufreq. Inside the folder we find the source files. Open up cpufreq.c. 



Go to that part of the code that says sysfs interface. And find suitable place to create our file. As you can see I found mine.



Now that you have added a function to display the frequencies. Its time to create the file in the interface that would display them


They can be created by adding a line to the descriptors that create a file. Just add

define_one_ro(scaling_available_frequencies);


We need to add a line to the attributes structure also. Add

&scaling_available_frequencies.attr,


to the lines.


Now the driver file is ready. But if you have observed, we have made use of an external function by name


acpuclock_get_available_frequencies_str


This is not defined in acpuclock file. So we need to head over to the file


../kernel/arch/arm/mach-msm/acpuclock-8x50.c


and add this function to output the value of frequencies as a string buffer.


The function plainly alters the string in *buf to hold the values of the frequencies. This completes the work. Now compile the kernel and load it to the phone, and discover your own new interface that shows all the possible frequency values.