[Android Software/Hacking General [Developers Only]] geckoneer:
Persisting system.img and trying to make Android Market application to work

Actually, there is a way to persist system partition.

Built-in command line help shows that there is an optional persistent system-qemu.img. If it worked, all the hoops below would be not necessary.

If it try to create system-qemu.img in .avd directory as a copy of system.img from one the platforms in the SDK, emulator fails with an error

ko:Missing initial system image path!

. My guess is that emulator was deliberately crippled to not support persistent /system partition out of the box. If someone has a method to make system-qemu.img to actually work, please post.

I quickly looked through the emulator sources, the patch to enable system-qemu.img would be a pretty easy task for someone who has built android emulator from sources at least once. Any volunteers?

Read on for a hack to get system-qemu.img functionality.

When emulator starts, it creates a copy of system.img in %temp%/AndroidEmulator with some random name like TMP9323.tmp. If you do

adb -e remount

to mount /system as rw, you will get the writable system, but with an empty free space. Everything you do to /system is saved into this .tmp file.

The next trick is to start emulator instance with more virtual sectors than there are actually are in the stock system.img. You do that by starting emulator with the aptly named emulator command like so

emulator @AvdName -partition-size 256

If look later at free space with df, you will get something like this

adb -e shell df
Filesystem Size Used Free Blksize
/dev 252M 32K 252M 4096
/mnt/asec 252M 0K 252M 4096
/mnt/obb 252M 0K 252M 4096
/system 248M 96M 151M 4096
/data 248M 34M 213M 4096
/cache 64M 1M 62M 4096
/mnt/sdcard 252M 1K 252M 512
/mnt/secure/asec 252M 1K 252M 512

Note that you now have some free space in /system.

Later you do

adb remount

from your host shell. This is essentially the same as running

#mount -o remount,rw /dev/block/mtdblock0 /system

from emulator shell.

Now you get the fully writable /system filesystem. You can copy with

adb push

any files you like into the /system and they will be successfully written.

After you are happy with your new /system, from

adb -e shell

you sync filesystems and make system read-only:

#sync
#mount -o remount,ro /dev/block/mtdblock0 /system

After that you can hot copy the .tmp to some other place and close the emulator with an [x] button from the window title bar. I do not use snapshots, so closing emulator is akin to removing the battery from a real device.

And using snapshots to get a "persistent" /system is impossible without rebooting. Currently it is not possible to shutdown via emulated power button (F7). The emulator manual says just to close the emulator window. If you do that with snapshots enabled, you will get into the state you were when you closed the window. Also it is not possible to reboot, as there are no documented way to reset the virtual processor. The reboot command does not work. It just hangs the emulator-arm.exe with 100% use of the CPU core it happens to run on. That is another observation for emulator was deliberately crippled or not developed to a point when resetting the virtual processor is possible.

Later you put your .tmp file into your .avd directory with a name of system.img. And when you start your emulator (even from AVD Manager GUI) you will get your new /system.

If you later need to make some modification to /system, you need to go through all the steps again.

When starting emulator with a new /system for the first time, I recommend to check "Wipe user data" to recreate the file and directory structure in the /data partition, especially if you pushed new .apk files into /system/app or pushed some other system wide components like .jar (framework) or .so (library) files.

Wiping a /data partition can be a lengthy process because it leads to recreation of dalvik-cache, the most CPU demanding process. In most cases, you can avoid the /data wipe. But if you push Google Apps to a pristine AVD, you should do a wipe so that Google initialization program with "Touch Droid to begin" kicks in. It is a first step screen the user sees at least on a real, Google "enhanced" device based on Android 2.3.

You can even observe how userdata-qemu.img file gets bigger as dalvik-cache and other parts of /data get populated. You may even use the

top

command to see the processes on the emulated device.

If you are running emulator from the command line you can add -wipe-data option to wipe the /data partition.

Optionally you can remove cache.img from .avd directory to clear the cache. cache.img will be recreated from scratch in this case.

After all these manipulations, the only way to launch AVD would be from command line as -partition-size 256 must be specified every time, otherwise you will get a kernel panic when mounting the /system partition.

The GUI option "Ideal size of system partition" is actually ignored. Its value is not copied into hardware-qemu.ini from config.ini, but stays at the size of initial system.img. Just another observation that emulator was deliberately crippled.

If emulator hangs with textual (not graphical) "A N D R O I D" on its screen, it is likely that the new system.img is corrupted and/or the partition size of 256MB has not been specified for not stock system.img. You can verify this by launching AVD from the command line with -show-kernel option to get all the kernel output in real time. You will likely find a kernel panic there.

If emulator hangs with a graphical ANDROID animation (aka boot loop), you can use

adb -e logcat

tool to look for any errors in the start up process. Examples below.

But that was an easy part. Now we need to install Vending.apk and its dependencies.

As my goal is Android 2.3.3, as it is a close to current majority of real devices as it can be. I tried to work with the version of Google Apps for CyanogenMod. I own an HTC Desire with CyanogneMod 7.1-nightly. I downloaded the initial gapps-gb-20110828-signed.zip from the link in CM wiki. This .zip, when flashed through recovery works without any issues on CM. Also I pulled from HTC Desire an updated Android Market .apk. Also I generated a demo app which shows a text edit and a button, but does nothing just to check that .apks can be installed both as system and 3rd party apps.

Here is what I tried.

0. Started with a Google API (API Level 10) emulator images. Pushed unpacked gapps. Everything worked: initial Google account registration, Talk, Maps, Latitude, Navigaton, Places, News and Weather (widget + app), Search (including widget). But not Market. There were no icon for it. Fail.

1. Started with Android 2.3.3 (API Level 10) emulator images. Pushed unpacked gapps. Got a boot loop with an exception in logcat related to contacts migration/upgrade. Strange, Google API 10 image may have something that plain image does not have. Fail.

2. Started with Android 2.3.3 (API Level 10) emulator images. Pushed upgraded Vending.apk to /system/app. No Market icon after a reset. Even with data wipe. Fail.

3. Started with a Google API (API Level 10) emulator images. Tried to install through adb the latest Vending.apk as a third-party app. adb install failed with "INSTALL_FAILED_ALREADY_EXISTS". Fail.

4. Started with a Google API (API Level 10) emulator images. Deleted /system/app/SdkSetup.apk. Tried to install latest Vending.apk with adb. Same result as in 3. Fail.

5. Started with a Google API (API Level 10) emulator images. Deleted /system/app/SdkSetup.apk. Pushed latest Vending.apk to /system/app. After reboot no icon. Also pushed own app, it got the icon. Vending.apk got no entry in /data/dalvik-cache, but demo app got it. Something prevents the Vending.apk to be recognized as a valid app. Fail.

6. Same as 5, but pushed not as Vending.apk, but as foobar.apk. Same result. No icon after reboot, no entry in dalvik-cache. Fail.

Spent the last 4 hours for figuring out the method for /system persistence and Vending.apk experiments.

Some possible reasons for Market Application to not show up in dalvik cache and no icon in launcher application list:

1. Some metadata in .apk prevents it from being processed. SDK level?

2. Crypto keys mismatch. Firmware signing keys != .apk signing keys.

3. Apk contents conflict with something existing in the firmware. (INSTALL_FAILED_ALREADY_EXISTS).

4. Permission issues.

5. System image configuration variables like ro.secure and ro.allow.mock.location.

The next step for me would be to read 3 books:

1. Learning Android.

2. Application Security for the Android Platform

3. Android Forensics: Investigation, Analysis and Mobile Security for Google Android

As I plan to fork or contribute to pffmod (http://xdaforums.com/showthread.php?t=996726), I had to read them anyway.

If someone has a working method to put recent Android Market into Android 2.3.3 (API level 10) or Google API (API level 10) emulator images, please post.