[Android Software Development] midnjerry:
Okay, I'm writing this because I want to help any other newbies trying to learn how to reverse engineer. The technical details involved in this are extremely daunting, so the purpose of this tutorial is to first explain in layman terms exactly what you're trying to accomplish and what to expect. Then we'll go over the details. That way you're not completely blind going into this. I'm fairly new to the scene, so I'm not as knowledgeable as everyone else. If you see any errors in my post, let me know so I can change. I'm going to assume you know a little bit of Java, can find your way around a computer, and know nothing about Android. The techniques used should work with other Android phones. For this tutorial I'm using Windows 7, Cygwin, and my stock (not rooted) EVO 4G mobile phone.

The FM tuner for the Evo is run by a Broadcom chip: BCM4329. This chip is pretty amazing in that it does wireless, bluetooth, and it has an FM receiver/transmitter. We're interested in the FM receiver / transmitter.

Now, all android phones are based on a Linux kernel. Basically they're Linux running computers. The Android operating system is then installed onto the linux system. Every app is then run off of Android.

Android is based on Java but it is not a Java system. It uses a virtual machine called Dalvik. Google did this to get around licensing issues with Sun Microsystems. So they pretty much invented their own machine language (called byte code) for the Java language. This makes things complicated for the reverse engineer because from what I've read, once Java is converted into this machine language or byte code, it can't be converted back.

So let's rehash.

If you were programming strictly in Java, you would see these extensions:

Java source code = .java

Compiled Java source code = Java byte code = .class

Compressed file to package your program = .jar (Java Archive)

But since you're programming in Android and Dalvik, you will see these:

Java source code = .java

Compiled Java source code = Dalvik byte code = .dex

Compressed file to package your program = .apk

(I haven't mentioned this, but HTC further Optimizes their .dex code)

Optimized Dalvik byte code = .odex

I'm writing all of these down because it's very easy to get confused with all of the extensions. (for me at least!). remember how I said once you go dex, you can't go back to java? That's where JesusFreke comes in. He's a senior member of XDA, and he created "baksmali" and "smali", two programs that can convert the Dalvik code back into a human readable format. These files have extensions of .smali

Decompiled Dalvik byte code = .smali

But what can you do with .smali files? That's where this other senior member, brut.all comes in: He developed apktool. apktool takes JesusFreke's work to the next level. This program in conjunction with NetBeans, actually lets you trace through any program using the .smali code taken from JesusFreke's programs!

apktool does this by converting those .smali files into "fake" .java files that can be used by the NetBeans (program that compiles and makes java programs) IDE. I say "fake" because apktool embeds the .smali code into java files as comments. However, once you attach a debugger to NetBeans, you'll see that the debugger will follow line by line every execution statement found in the smali code!

So...... you can take the program you want, plug it into Net Beans using a debugger (using the default ddms command provided by Android SDK), and you can trace everything you do in the program. I have it connected to my phone, so whenever I push a button while running my HTC FMRadio app or unplug my headphones,I see the corresponding response to the HTCFMRadio code I have loaded in NetBeans. I can now see in real-time how the program operates from my own interactions... JAM.

Technical Aspects: How to get from ground zero to tracing HTCFMRadio?

1.) Download Android SDK - Go to google development site and follow instructions: Make sure to download the latest Java JDK. Once that is installed, download NetBeans 6.8. Unfortunately, smali debugging does not work with the lastest versions of NetBeans.

Download the "Java SE" version for minimal space

http://netbeans.org/downloads/6.8/index.html

You can follow the rest of Google walkthrough and download Eclipse and ADT plugin, but it's not pertinent to this. You're going to be using adb and ddms from the android SDK extensively, so make sure the path for </android SDK/tools> is included in the PATH variable in your ENVIRONMENT SETTINGS. To get here, right click My computer, click properties, Advanced Settings, ENVIRONMENT SETTINGS.

2.) Search for 7z and download it. It is an awesome and free compression tool that will be extremely useful. It can be used to "unzip" .jar, .apk, and other compressed formats.

3.) Get the Radio app. You can do this by going to "shipped-roms" website, downloading the latest Supersonic image, and following the directions in the unlockr tutorial for HTC kitchens at the unlockr website... (once you have extracted the files from the image, you can look in the system/app and system/framework directories to get the files listed below) or:

you can pull the following files from your phone:

Using the command prompt type (and with phone plugged in, and with USB debugging enabled on phone):

adb pull /system/app/HtcFMRadio.odex

adb pull /system/app/HtcFMRadio.apk

adb pull /system/framework ./framework

This will put HtcFMRadio.odex and HtcFMRadio.apk in the current directory and create a framework directory with more files. A couple of the files in the framework are needed for the HtcFMRadio app, but for simplicity, we're just going to pull the whole directory.

Now that we have the files, we have to make a few changes to make the app installable and to be viewable by the debugger. To do this we have to decompile the .odex format into a human readable format we can edit. That brings us to:

3.) Download baksmali and smali from Project Hosting on Google Code (google search smali).

Usually an Android application is made up of one file, an apk file. Inside the apk file is an AndroidManifest.xml file, a classes.dex file (compiled Java code for the program), and other folders. The other folders contain either graphics or other .xml files that tell the program how it should look to the user. We don't have to worry about those for now. This is important because APKTOOL only opens programs set up this way. But wait up? We didn't download one .apk file, we downloaded an .apk file and an .odex file! What gives? Well, if you right click the apk file and open it (using 7z), you'll see that it's missing the classes.dex file. The dex file for the app is actually the HtcFMRadio.odex file we downloaded. So, to make this system app more like a nominal app, we have to find a way to convert the HtcFMRadio.odex to a classes.dex file. That's easy with baksmali and smali!

Once you download goto command prompt and type:

java -jar baksmali-<version>.jar -d framework -x HtcFMRadio.odex

(Remember to match baksmali-<version>.jar with the filename of baksmali you downloaded)

If done correctly, you should see a newly created \out directory

This creates an out\com\htc\fm directory with many .smali files.

Now let's reverse the process and put it back as a dex file. Type at command prompt:

java -jar smali-<version>.jar out -o classes.dex

If done correctly you'll see a newly created classes.dex.

now, right click on HtcFMRadio.apk (select 7z and open). Drag classes.dex into the file. Say yes to the prompt. Now you have a normal apk file APKTOOL can read!

4.) Download APKTOOL from Project Hosting on Google Code and the helper apps for your OS. (If you're extracting files for windows OS you should have apktool.bat and aapt.exe). Extract (again using 7z, don't you love this program?) apktool.jar (keep it as a jar file, don't extract the stuff inside of it), apktool.bat, and aapt.exe to the directory you're working on. To make things neat, you can also delete HtcFMRadio.odex (you don't need it anymore) and classes.dex (make sure you put it in the HtcFMRadio.apk file first!)

If this is the first time you're using apktool, then you have to install the htc framework so apktool can baksmali the Radio app. You only have to do this once:

apktool if ./framework/com.htc.resources.apk

Alright, at the command prompt:

apktool d -d HtcFMRadio.apk

This extracts the contents of HtcFMRadio.apk and places them in the HtcFMRadio directory. However, there are two major differences between this content and the content created in step 3. If you go into the smali directory you'll see that instead of .smali files, you'll see .java files. And if you go back and edit the AndroidManifest.xml file, you will also see that it's in text! Android applications convert their xml files to binary format. Now that APKTOOL has converted everything to an IDE friendly format, we can use NetBeans to edit everything. The first thing we're going to do is edit AndroidManifest.xml (using notepad) and add the following:

android:debuggable="true" to the Application tag.

IT should now look like this:

<application android:theme="@android:style/Theme.Black.NoTitleBar" android:label="@string/fm_app_name" android:icon="@drawable/fm_radio" android:taskAffinity="android.task.fmradio" android:description="@string/htc_corp" android:allowTaskReparenting="true" android:debuggable="true">

This permission lets the debugger watch the program while it's running on the phone.

We are going to run into two problems if we try to install this program. One is that Android doesn't let you install more than one copy of a system app. The second issue is that if we change the signature of our system app, then we'll have to change the signatures of our other system apps as well! Ahh.... So, to get around that, we're going to trick Android into thinking we have a completely new program. We're going to do that by renaming the com.htc.fm class to com.htc.modradio class. Next step:

5.) Cygwin (or Linux virtual machine)

The easiest way that I can think of to replace strings in multiple files is by using linux. You can most definitely do it in WIndows, but I dont know how. If you let me know how, I can put it in this tutorial.

(update: you can use Notepad++ to easily find/replace strings in multiple files for Windows. You still, however, want to download Cygwin if you're going to develop with Android-NDK.)

For now, just search for Cygwin (Cygwin is a program that lets you run Linux commands from a command prompt using your Windows directories), and install it. Make sure to have the Perl option selected. You'll need Perl to make the following commands work.

Once you get Cygwin up and running

cd <to your HtcFMRadio directory>

in my case it's

cd /cygdrive/c/Users/Jerry/Desktop/HtcFMRadio

now type the following commands in this order:

this command changes all occurances of htc/fm to htc/modradio in your xml and .java files.

find ./ -type f | xargs perl -pi -e 's/htc\/fm/htc\/modradio/g'

this command changes all occurances of htc.fm to htc.modradio

find ./ -type f | xargs perl -pi -e 's/htc.fm/htc.modradio/g'

If you don't follow this order, your source code will get messed up.

If using cygwin, a bunch of .bak files will be created. Using windows search, find all .bak files in your HtcFMRadio directory, then select them all and delete them (Make sure they are only files with .bak!)

Now just rename the fm directory to modradio. It is located in HtcFMRadio/smali/com/htc

Now go to your windows command prompt and type:

apktool b -d .\HtcFMRadio modradio.apk

Now sign and install modradio.apk on your phone.

adb install modradio.apk

If you have never signed before, then you need to use keytool and jarsigner. These two files are in your JDK directory, so make sure you include your JDK directory in the PATH variable of your ENVIRONMENT SETTINGS. (To get here, right click on My Computer, click Properties, Advanced Settings, Environment Variables. Once you make change, open up a new COMMAND prompt to see changes).

cd to the directory which has modradio.apk

now type:

keytool -genkeypair

Answer all questions, then use the same password for all password prompts.

Next type:

jarsigner -verbose modradio.apk mykey

Type in the password you created in the above step. Your apk should now be signed.

Next install:

adb install modradio.apk

Success!

6.) Testing the app on phone

Go to your phone and you'll now see a new FMRadio icon next to your first. Click on it and watch it open. It should now be able to play music. Keep it open.

7.) Using Netbeans

Go into HtcFMRadio and delete the build directory created by APKTOOL.

Now open up Net Beans and click on File, New Project, Select Java Project with Existing Sources, click on Next

Select HtcFMRadio directory for Project Folder, rename Project Name to whatever you want. Let's type in ModRadio. click on Next

Next to "Source Package Folders" click on "Add Folder" and select the smali directory.

Click Finish. For a quick tutorial by Brut.all, search APKTOOL in youtube and click on: Apktool Demo 2 - Smali improvements

Right click on Libraries. Click on "Add Jar / Folder". You want to add Android.Jar. Since I have Android 2.1 loaded I went to /platforms/android-7 located in my android SDK directory.

Your project is now ready for editting!

8.) Running the Debugger to trace through program.

Next go back to Windows command prompt and type ddms. This runs the Dalvik Debug Monitor. A window should open up. In the left hand side you should see com.htc.modradio. That's our app! To the right you're going to see 2 numbers, you're interested in the one to the right, 4 cells away from com.htc.modradio. This number is a port number, and you're going to use it to communicate with NetBeans. (In my case it is 8603)

Go back to NetBeans and click on Debug, Attach Debugger.

In the host field type: localhost

In the Port field: type in the second number you saw. (8603)

If everything is working you'll see a bug appear next to com.htc.modradio in the Dalvik Debug Monitor. Look at the bottom bar of NetBeans for feedback. If you get errors make sure the numbers match, or try port 8700 and make sure you select com.htc.modradio in the Dalvik Debug Monitor. Port 8700 is the default port used for whatever program you select in Dalvik Debug Monitor.

9.) Setting a breakpoint

I'm making this a seperate step because it is completely arbitrary. When creating a break point be sure to follow this rule:

You must select line with some instruction, you can't set breakpoint on lines starting with ".", ":" or "#".

Rather than looking for a spot to breakpoint, though, I'll tell you where to put one so you can quickly see how the debugger traces through the code. You aren't "REQUIRED" to do the next step, but if you want to trace you have to put a breakpoint somewhere.

In Net Beans click on the Project tab, click on Source Packages, com.htc.modradio, and then doubleclick on BroadcomFMTuner.java

We're going to insert a breakpoint. Scroll down to line 3226 and on your keyboard press: CTRL-SHIFT-F8, select line in dropdown box and hit ok. (To keep it simple, I usually look for "invoke" instructions to set breakpoints at)

Now go to your phone and click on the physical "back" button on your phone. This will clear the radio,(you should still be able to listen to music). Drag your status bar down. You should see a radio icon. Click on it again. The radio backgroudn will appear, but you wont' see any text or anything. Now go back to your netbeans application. You should now see debug options highlighted! Click on Step Over (F8) to step through!