[Android Software/Hacking General [Developers Only]] Michael Spencer:
INTRODUCTION

This document describes a procedure for installing Ubuntu 11.10 Oneiric Ocelot onto your Android device to run alongside an existing Android rom. My instructions assume you are running CyanogenMod 7. I will try to provide workarounds for other roms.

Ubuntu on a mobile device, running alongside Android, is somewhat difficult to use and is useless for most users. You should not attempt this procedure unless you have a need to run arbitrary Linux software from a mobile device. Many mobile computing needs can be met by installing remote access software on a traditional desktop computer. Conversely if you find the idea of having a general purpose Linux system in your pocket to be freaking awesome, whether you need it or not, please read on.

CAUTION

The following procedure assumes familiarity with Linux fundamentals, and basic familiarity with Android's Linux underpinnings. Data loss is possible, especially for the data on your SD card, so please make backups. Some commercial Android software is recommended, specifically Tasker and Better Terminal Emulator Pro. A large, fast SD card is also recommended. (I <3 my PNY 32 GB class 10 card)

Some optional steps are proposed that may interfere with how your Android device manages its SD card. You may be required to manually run a script before certain Android features, like mounting your device's SD card on a desktop PC via USB cable, will function properly.

Each step will have, whenever possible, a recommended procedure and one or more alternative procedures. If the recommended procedure doesn't work for you, the alternative procedure might require some research, manual-page reading, or other outside assistance to accomplish.

Shell commands should be executed on a root shell on your Android device. Use the built-in Terminal Emulator, ConnectBot in local mode, Better Terminal Emulator Pro, "adb shell" from a connected PC, or your favorite terminal emulator.

OVERVIEW

Too long; didn't read? Here's a quick summary of the steps involved:

Partition your SD card, creating a 2 GB or 4 GB Linux partition

Confirm the new partition is mounted at /sd-ext

Unpack http://cdimage.ubuntu.com/ubuntu-core/daily/current/oneiric-core-armel.tar.gz to /sd-ext

Create an empty file /sd-ext/.nomedia

Edit /system/etc/init.d/05mountsd to bind proc, dev, and sys to /sd-ext

Optionally edit /system/etc/init.d/05mountsd to bind /data, /system, and /cache to /sd-ext/mnt

Prepare a script to open a chroot shell with correct environment variables

Update /etc/apt/sources.list and /etc/resolv.conf according to your preference

Overwrite /sbin/start with /sbin/true

Copy or link important files into /etc and /lib

Initialize the package manager and begin downloading packages

(optional) Integrate Ubuntu services with Tasker profiles

STEP 1: Partition your SD Card

While a basic Ubuntu system can be installed in as little as 200 MB of disk space, many software packages require more space. Also many Linux tasks require unusual file attributes, permissions, or types, which cannot be placed on your SD card's regular FAT32 / VFAT file system.

Recommended: Use an existing Ubuntu system, or even an Ubuntu live CD, and a USB card reader to repartition your SD card without losing any data. Be sure the first primary partition remains FAT32 / VFAT, and create a second primary partition of type ext3 or ext4.

Alternative 1: Use any desktop system to back up all of the data on your SD card, either with a card reader or using your Android device. Use your phone's recovery partition to destructively repartition your SD card with a 2 or 4 GB ext3 or ext4 partition, and NO swap partition. Then copy your SD card contents back to your SD card.

Alternative 2: If you don't want to partition your SD card, you can create large empty files on your SD card and mount those. Some of the later steps will work differently and you may have to improvise.

Create large empty files on your SD card to contain your Ubuntu filesystem: dd if=/dev/zero of=/mnt/sdcard/UBUNTU.IMG bs=32768 count=65536

Identify a free loopback device number that isn't currently in use and won't likely be occupied by Android Apps2SD in the future: ls /dev/block

Create a new loop device for your file. Note the first number is the loop filename number you observed wasn't in use (at least a few dozen higher than the highest number you see), then a b and a 7, and then a number one less than the filename number. For example: mknod /dev/block/loop200 b 7 199

Associate the loop device with your new empty file: losetup /mnt/block/loop200 /mnt/sdcard/UBUNTU.IMG

Find which versions of mkfs are on your system: ls /system/xbin/mkfs*

Using whatever version you have (ext4 or ext3 preferred, but ext2 is acceptable) run: mkfs.ext# /dev/block/loop200

Save these commands in a script for later use. The dd and mkfs commands were only used this once, but the mknod and losetup commands will have to be run manually from a script each time you boot your phone.

STEP 2: Mount your Linux Partition

To avoid causing problems for your Android system, we shouldn't tamper with any of Android's critical files or directories. We will mount our Linux partition in a new directory that Android isn't using. With any luck, our rom already contains scripts that will automatically mount this for us.

Recommended: If you have partitioned your SD card and you are running CyanogenMod 7, use a root shell or file manager to confirm /sd-ext contains at least a directory called lost+found. If lost+found is present in /sd-ext then this step is done and you should continue to step 3.

Alternative 1: Use a root shell to navigate to /etc/init.d and look for a file called 05mountsd. If that file doesn't exist, you may need to create it or use another alternative method. If it does exist, run it with ./05mountsd and observe any error messages. If fsck or mount are failing because the filesystem type doesn't match what was expected, you can edit the file and change the filesystem type in the script.

Hint: you can do a quick search-and-replace using sed and a regular expression:

mv 05mountsd 05mountsd-old
cat 05mountsd-old | sed -e s/ext3/ext4/ > 05mountsd
diff 05mountsd-old 05mountsd
#changes look good to you? then proceed:
rm 05mountsd-old
ls -al
chown root.shell 05mountsd
chmod 750 05mountsd
#adapt owner (root), group (shell) and permissions (-rwxr-x---) to match the other files in that directory

Alternative 2: If your rom doesn't create, or attempt to mount, anything called /sd-ext you might have to create your own mount point.

mount -o remount,rw /system
mkdir /system/sd-ext
mount -o remount,ro /system

Then, in the following steps, replace any reference to /sd-ext with /system/sd-ext.

STEP 3: Install the Base System

Recommended: Download this file: http://cdimage.ubuntu.com/ubuntu-core/daily/current/oneiric-core-armel.tar.gz (Should be under 32 MB, and is the only potentially-insecure download required because the file is unsigned.)

Store it on your SD card somewhere, like in /mnt/sdcard/download/. (If you use your Android device's browser, it will probably land there automatically.)

If you're nervous about making a mistake, double-check your system partition is read-only:

mount | grep -v block/dm- | grep \(rw
#If anything there looks like it shouldn't be read-write, make it read-only:
mount -o remount,ro /system

Open a root shell and navigate to /sd-ext. Use pwd (present working directory) to confirm you really are in /sd-ext. Then go ahead and unpack:

tar zxvf /mnt/sdcard/download/oneiric-core-armel.tar.gz

STEP 4: Create a .nomedia file

Android may try to scan your Linux partition for images, videos, and pictures. The Linux partition contains many files and directories and this scan can take a long time. To avoid these lengthy scans, we will add a file to the Linux partition that blocks well-behaved Android services from scanning.

Recommended: Use a root shell to navigate to /sd-ext. Create the file and set its permissions:

echo > .nomedia
chown root.root .nomedia
chmod 644 .nomedia

STEP 5: Bind /proc, /dev, and /sys to the Linux partition

Recent versions of Linux provide a way to clone a partition, so when changes are made to one copy of the partition the changes take effect immediately in the other copy. We will use this to make the /proc, /dev, and /sys partitions used by Android available in our Linux partition as well.

This is necessary because when we run Ubuntu we will be using chroot (change root) to make /sd-ext the only filesystem Ubuntu can see. Ubuntu will need to be able to see /proc, /dev, and /sys, as well as some of the files in /etc, /vendor/lib, and /system/lib. We will meet some of these needs with mount --bind

Additionally we will need these bindings to be recreated every time the /sd-ext partition is mounted. So we will add some commands to the run-parts script that mounts /sd-ext, located at /etc/init.d/05mountsd.

Recommended: Open a root shell and navigate to /etc/init.d, and use a text editor to open 05mountsd.

Near the end of the file you should see some commands that operate on /sd-ext:

$BB chown 1000:1000 $SD_EXT_DIRECTORY;
$BB chmod 770 $SD_EXT_DIRECTORY;

Edit the chmod command and change the permission mask number from 770 to 775, which grants non-root users the ability to read the contents of that directory. Then add the following commands immediately after the chown and chmod commands already present:

$BB mount --rbind /dev /sd-ext/dev
$BB mount --rbind /proc /sd-ext/proc
$BB mount --rbind /sys /sd-ext/sys

STEP 6: (OPTIONAL) Bind /system, /data, and /cache to /sd-ext

This step is optional, and provides a way to operate on Android system files from within Ubuntu. If you're concerned about a stray command destroying your Ubuntu system and you want to make sure it doesn't take any Android system files with it, you should skip this step.

Recommended: Open a root shell and navigate to /sd-ext/mnt, and create some directories which will serve as mount points:

pwd
# you should be in /sd-ext/mnt
mkdir -m 700 system data cache
chown root.root system data cache
ls -al
# Each of system, data, and cache should show drwx------ and root for both owner and group

Then navigate to /etc/init.d and edit 05mountsd. Add the following under the existing mount --rbind commands you added earlier:

$BB mount --rbind /system /sd-ext/mnt/system
$BB mount --rbind /data /sd-ext/mnt/data
$BB mount --rbind /cache /sd-ext/mnt/cache

STEP 7: Prepare a script to open a chroot shell with proper environment variables

Now you will need an easy way to create a first process, a shell, in the new environment you've created. This shell will start life in a different world than regular Android shell processes live in: it will use a different root filesystem and will be able to see different files and directories, and it will have access to a vast number of full-featured programs and services.

We will be using the Bourne Again Shell, bash. We will launch bash with a program called env, which sets some environment variables and then launches a program with that new environment. Finally, we will be launching env with a program called chroot, which changes the root filesystem seen by a program before it launches it.

Option 1: Better Terminal Emulator Pro: I don't want to say "recommended" because it involves buying software that's not really necessary. I'm partial to Better Terminal Emulator Pro because it seems to have more available options (compared to the built-in Terminal Emulator) and it doesn't kill your shell unexpectedly when network connectivity changes (unlike ConnectBot). I'm not affiliated with the author of that software. Also, after I bought that I stopped looking for alternative shells, so there may exist other alternative shells that work better for this.

Use Better Terminal Emulator Pro and add the commands necessary for starting your chroot shell to BTEP's options. Once you do this, BTEP will essentially be a dedicated Ubuntu console for you, and may be difficult to use for other tasks.

My settings are:

Shell: Android Shell

Command Line: /system/bin/sh

Initial Command:

/system/xbin/chroot /sd-ext /usr/bin/env PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin TERM=linux /bin/bash -l; exit
clear

Start as root: yes

Option 2: Terminal Emulator, and a shell script: Open a root shell and create a file:

#!/system/bin/sh
/system/xbin/chroot /sd-ext /usr/bin/env PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin TERM=linux /bin/bash -l
exit

Save the file with whatever name you like, perhaps "ubuntu", and put it somewhere in your path. You could add it to /system/bin: mount -o remount,rw /system; mv ubuntu /system/bin/ubuntu; mount -o remount,ro /system

Option 3: ConnectBot: I don't recommend ConnectBot for this because, while it does offer "local" connection types, if your connectivity changes (even wifi to cell) it will close your "connection" immediately.

That said: create a new connection with type local, and give it the following initial command:

su -
/system/xbin/chroot /sd-ext /usr/bin/env PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin TERM=linux /bin/bash -l; exit
clear

STEP 8: Update /etc/apt/sources.list and /etc/resolv.conf

Ubuntu uses a package manager which automatically finds, downloads, verifies, and installs software for you. The package manager requires a list of trusted repositories which contain binaries built for ARM CPUs, like the one in your Android device.

FIXME: Later I'll discuss the pros and cons of each of these selections. For now I'll just provide my sources.list file.

Open your Ubuntu shell, using the method from step 7, and issue the following commands:

echo nameserver 8.8.8.8 > /etc/resolv.conf
# you can use a nameserver of your choice if you like.
echo nameserver 8.8.4.4 >> /etc/resolv.conf
# notice the >> instead of > which means we're appending, not overwriting.
apt-get install nano
# follow the prompts - say yes if it asks if you want to proceed.
nano -w /etc/apt/sources.list

And here are the contents of my sources.list file, containing binary sources for ARM CPUs:

deb http://ports.ubuntu.com/ubuntu-ports/ oneiric main restricted universe multiverse
deb http://ports.ubuntu.com/ubuntu-ports/ oneiric-security main restricted universe multiverse
deb http://ports.ubuntu.com/ubuntu-ports/ oneiric-updates main restricted universe multiverse
deb http://ports.ubuntu.com/ubuntu-ports/ oneiric-backports main restricted universe multiverse

deb http://archive.canonical.com/ oneiric partner
deb http://archive.canonical.com/ oneiric-security partner
deb http://archive.canonical.com/ oneiric-updates partner
deb http://archive.canonical.com/ oneiric-backports partner

STEP 9: Overwrite /sbin/start with /sbin/true

Nothing too complicated here. Some Ubuntu packages install services, which the system will usually want to automatically start and end when the system boots and shuts down.

However, on an Android device Android is responsible for booting and shutting down the system, and it doesn't know much about Ubuntu services.

Later (FIXME: when?) we'll discuss how to use an Android program like Tasker to more intelligently manage Ubuntu services. But for now, we need to trick Ubuntu's package installation scripts into believing it has successfully started a service. We will use a useless program called true, which does nothing and reports that it was successful, in place of a program called start, which will be unable to start a service because it has no access to Android's init system.

Open your Ubuntu shell and do this: cp /sbin/true /sbin/start

STEP 10: Copy or link important files into /etc and /lib

FIXME: I haven't finished this section yet. Some of my software remains broken until I fix this.

STEP 11: Initialize the package manager and begin downloading packages

Now we can begin using our Ubuntu system's package manager to install any other packages we might need.

Open your Ubuntu shell and do the following:

apt-get update
# This will take a short while, and will download updated software catalogs from each of the update servers
apt-get upgrade
# This looks for updates to any currently-installed packages. There aren't many packages installed so this should be quick.
apt-get install PACKAGE NAMES GO HERE

Then keep using apt-get install to install whatever you need.

Here are some common packages and package sets:

ubuntu-dev-tools: development tools including a compiler and linker

tightvncserver: simple X server you can connect to with androidVNC

FIXME: I need to put a lot more here.