[Android Software/Hacking General [Developers Only]] dk_zero-cool:
Before reading.

This article requires that your phone is Rooted and has Busybox installed.

HTC owners, S-off your phone as well.

Android is a great operating system for SmartPhones. Only problem with it (From a more technical users point of view) is the small useless bad excuse of a linux system running behind the UI. Some of the smaller issues can be fixed by installing Busybox that provides a better basic set of tools to work with, but it's not enough. What I would like is to be able to use almost any real linux cli application, daemon, library etc. Also some compilers would be great. I'm using an HTC desire which has more than enough power to do things like compiling (I know, I have done it.), and so has a lot of other SmartPhones. Why would one need this when there is cross compilers you can use on regular and faster machines? Well, because It's fun, why else. I did not spend all that money on a SmartPhones just to make a few phone calls and write a few text messages. My old Sony Ericsson did that quite well. I spend that money to get a phone with many possibilities that I could play with.

I tried to search the web for some helping instructions on how to port a real linux system to this phone. What I mostly got, was people who ported linux systems that would run as an extra OS on the phone, some even with an Xserver accessible via VNC. Sounds fun, but not what I was after. I did not want an extra OS using CPU and unnecessary RAM. I just wanted some extra tools, libs etc to work with and the regular linux user management. I managed to find an old article from someone using Debians armel version on an old G1 phone. I thought that if it worked on that phone, it would be possible to do so on mine. So I started playing around with that.

The way this works is by installing a bare minimum debian on an second partition. This will not boot anything up since it will just work as extra tools and lib using the already booted android linux system.. You will still only have one OS but more tools (Including the nice APT package system) to work with.

Now there is two ways of using debian on android.

The first way is to setup the Debian in an chroot environment. This has some flaws like I have not yet been able to make the chroot start second jobs, which means that you will be able to start applications and daemons running in the current shell session. But you will not be able to make them run in the background. But as an development environment it will still work great, as you will be able to use the Debian libs and compilers.

The second way (The way I was after) is to setup Debian using the UAFS kernel module. This is used to merge directories into one with different rules for each. For an example you would be able to merge /debian_root/etc and /system/etc into a shared /etc with RW on Debians etc and RO on Androids forcing any changes to be made in /debian_root/etc. Using this option you will use Debian to extend the Android Linux part. Android UI with Debian Shell.

This article is split into 3 parts. The first part is the setup of Debian. Must be done whether you want chroot or a merged system. In the second part we will cover the setup of the chroot environment.

The third part is the setup of the merged system. This will require one thing in order to work.

You will need a kernel with the uafs module which is not default in most android kernels. You will need to compile your own kernel. People using the Thalamus kernel, can download a kernel with this module kernel-2.6.37.2_AUFS.zip. A big thanks to Thalamus for providing this. Note that this kernel was a special request from me that he was kind enough to provide. It comes with absolutely no guaranties. Do not email spam him about errors you might encounter using it as this is not an official release.

Note that once you mount the Debian system, you will not be able to mount the SD Card when connecting it to your PC. You can download the App “Multi Mount SD-Card” from Market which will be able to mount your SD Card on your PC while mounted on your phone. This app should also speed up transfers on 2.6.37 kernels, or so I have heard.

Part 1: Building Debian

We are going to run debian from a file containing a ext2 filesystem. If your SD Card is big enough you might be able to create an extra partition and force it to mount as something else other than ext-sd, but in this article we are going to use a partitioned file.

Also, this will require that you have the tool “debootstrap” on your PC. This is found in Ubuntu and Debian, dont know about other Linux distro's or Operating systems. If you don't use Linux or an distro with debootstrap you can download a Ubuntu Live CD from the Ubuntu website. This can be run on CD, DVD and USB disks without installing it. For this article I have used Ubuntu 10.10.

Open a terminal on your Ubuntu/Debian/LiveCD PC and cd to the folder where you want to temp. store your debian. I will call this path “debian_dir”

The first thing we will do is to create the partitioned file.

I'm creating a file with 1.5GB of space. You can create the size you want.

Don't use 0 in the size, dd don't like that.

cd debian_dir
dd if=/dev/zero of=debian.img seek=1499999999 bs=1 count=1
mke2fs -F debian.img
mkdir debian
mount -o loop debian.img debian/

Now that we have created the file and mounted it, we should download and place debian inside of it. First we will install debootstrap.

apt-get install debootstrap
debootstrap --arch armel --foreign squeeze debian http://ftp.us.debian.org/debian
umount debian/

Debian Squeeze is Debian 6.0 which replaces the old Lenny.

Copy the debian.img file to your SD Card on your phone. I placed mine in the root of the SD Card.

Now open a terminal on your phone. You can download one from market. There are some free once, but the paid “Better Terminal” is the best to work with in my opinion.

Note: All shell work is done on the phone from now on.

First we need some temp. exports.

Type the fallowing:

export PATH=/bin:/sbin:/usr/bin:/usr/sbin:/usr/local/bin:/usr/local/sbin:$PATH
export HOME=/root
export TERM=linux
export DM=/data/local/debian
export DI=/sdcard/debian.img

Now we will create the Debian dir and then create a loop device which we will use to mount the Debian image to the Debian dir.

mkdir -d $DM
mknod /dev/loop0 b 7 0
mount -o loop,noatime $DI $DM
Now that we have debian mounted in our debian folder, we need to finalize the installation. Not all files as been installed yet. Don't forget to activate Mobile Network or Wifi.
chroot $DM /debootstrap/debootstrap –second-stage

We also need to insert the debian repository and a DNS (I use OpenDNS).

You can use whatever you want for the DNS, you routers IP for an example. It should have some default Ips configured.

echo “deb http://ftp.us.debian.org/debian squeeze main contrib non-free” > $DM/etc/apt/sources.list
echo “208.67.222.222” > $DM/etc/resolv.conf

Now change the root to the debian dir and set a root password and make sure that certain folders already exists.

chroot $DM /bin/bash
passwd root
mkdir root
exit

Now we need to make sure that certain variables exists and that the values is as they should be. So we will alter the /system/bin/sh that android uses as login shell.

cd /system/bin
mv sh sh0

Create a new /system/bin/sh file with the fallowing content.

This will ensure that these variables is created and/or changed every time you enter a new shell session. You can call another shell like bash or ash instead /system/xbin/sh, just make sure to use ash or sh at the top, since bash does not seam to understand $@

Warning: It is very important that the /system/bin/sh file is made correct. Android will not boot if anything is wrong with this file. It should be exactly like the one I posted, and you cannot make it on your PC and push it to the phone. Don't know why, it works fine with other shell scripts, but not that one.

Enter an adb shell from your PC and use vi or nano (You can download nano here here) to copy and paste the content directly into the ADB Shell.

Note: ADB from the recovery will always work. It does not use /system/bin/sh

#!/system/xbin/sh
/system/xbin/sh --login $@

Make sure that it has execute rights. Then type sh in the terminal to make sure that it works before you close your ADB Shell session.

chmod a+x /system/bin/sh
sh

sh0 will not be used, it's up to you wether you want to save it or not. Just make sure that /system/xbin/sh exists, otherwise

copy /system/bin/sh0 to /system/xbin/sh.

Now we need the profile file that is needed by this method. This file will make sure that we have all our variables and the data that they should contain.

Android does not keep very good track of these things. Place this data in the file /system/etc/profile

export TERM=linux
export TERMINFO=/etc/terminfo
export PATH=/bin:/sbin:/usr/bin:/usr/sbin:/usr/local/bin:/usr/local/sbin:/vendor/bin:/system/bin:/system/xbin

if [ "$EUID" == "0" ]; then
if [ -z "$HOME" ] || [ -z "$USER" ] || [ -z "$GROUPS" ]; then
export HOME=/root
export USER=root
export GROUPS=0
fi

else
# Make sure that this user id variable exists.
if [ -z $EUID ]; then
export EUID=$(echo $(id | cut -d "(" -f 1) | cut -d "=" -f 2)
fi

# If we have an /etc/passwd file available,
# we should also make sure to get name, group and home info from that file and
# store in the default linux variables.
if [ -f /etc/passwd ]; then
# Create the user info variables based on user id and /etc/passwd
export USER=$(grep "x:${EUID}:" /etc/passwd | cut -d : -f 1)
export GROUPS=$(grep "x:${EUID}:" /etc/passwd | cut -d : -f 4)
export HOME=$(grep "x:${EUID}:" /etc/passwd | cut -d : -f 6)
fi
fi
export ADID=$EUID

if [ "$PS1" ]; then
if [ "$BASH" ]; then
# The file bash.bashrc already sets the default PS1.
if [ ! -z $HOME ] && [ -f $HOME/.bashrc ]; then
. $HOME/.bashrc

elif [ -f /etc/bash.bashrc ]; then
. /etc/bash.bashrc
else
export PS1='\h:\W\$ '
fi
else
if [ "$EUID" == "0" ]; then
PS1='\w # '
else
PS1='\w $ '
fi
fi
else
PS1='\h:\W\$ '
fi

if [ -d /etc/profile.d ]; then
for i in /etc/profile.d/*.sh; do
if [ -r $i ]; then
. $i
fi
done
unset i
fi

We will also need an /etc/bash.bashrc which is needed because /etc/profile not always gets executed.

if [ "$ADID" != "$EUID" ]; then
# Some times /etc/profile is not executed (For an example when using Androids su command).
# /etc/bash.bashrc however it used instead. So we just execute it manually in these cases.
# If you create a local bashrc ($HOME/.bashrc), remember to include this check.
# /etc/bash.bachrc is not used when a local file exists.
if [ -f /etc/profile ]; then
. /etc/profile

# Since /etc/profile links back to this file,
# we will make sure not to start an unending loop.
# the profile file should update this, but it's better to be on the safe side.
if [ "$ADID" != "$EUID" ]; then
export ADID=$EUID
fi
fi
fi

PS1='\h:\W\$ '

And last we have some user and group configurations in order to make Debian better work with Android. Open your Android terminal emulator and make sure it opens as user and not root. Now type "/system/xbin/id".

What we need here is the uid, gid and the ids of each accosiated groups along with the names of it all. In my terminal I have "uid=10048(app_48) gid=10048(app_48) groups=1015(sdcard_rw),3003(inet)".

What we will do it create this user along with the groups in our Debian inveroment. However, I use the uid name and gid name "bterm" instead of app_48, but still with the original IDs. The inet and sdcard_rw groups will be created as system groups. Do the same for your app user and group if you don't want any login possibilities with this user. I however have assigned home and password to this user.

This needs to be doen in chroot

chroot $DM /bin/bash
groupadd --system --gid 1015 sdcard_rw
groupadd --system --gid 3003 inet
groupadd --gid 10048 bterm
adduser --home /home/bterm --gid 10048 --uid 10048 bterm
gpasswd -a bterm sdcard_rw
gpasswd -a bterm inet
exit

Now your ready for the next step. Remember that the user you have created, is the user used by your terminal emulator when you are not logged on as root. This ID is only assigned to that specific app, if you install another terminal, it will get another ID.

Part 2 – Setting up a chroot environment

The best way of doing this is to make two small scripts and place them in /system/bin or /system/xbin. One script for doing the mount after phone boot. And one script for entering the chroot when open a new shell session.

Here is the mounting script that I call mkdebian.

This needs to be executed once every time the phone has rebooted. If your ROM supports init scripts, you can add it there for auto-mount on boot.

#!/system/bin/sh
export DM=/data/local/debian
export DI=/sdcard/debian.img

# Create the loop device and mount the debian image
mknod /dev/loop0 b 7 0
mount -o loop,noatime $DI $DM

# Let's make sure that we can access our sd card from within the chroot
if [ ! -d "$DM/mnt/sdcard" ]; then
mkdir $DM/mnt/sdcard
fi
mount --bind /mnt/sdcard $DM/mnt/sdcard

# Now we make sure that the chroot has access to all the device maps
if [ ! -d "$DM/dev" ]; then
mkdir $DM/dev
fi
mount --bind /dev $DM/dev

# We also need to make proc, sys etc available from within the chroot
chroot $DM mount -t proc proc /proc
chroot $DM mount -t sysfs sysfs /sys

# And last fix the mtab
chroot $DM rm -f /etc/mtab
chroot $DM ln -s /proc/mounts /etc/mtab

echo “Debian was mounted in $DM/”

Now for the script that I called “debian”.

This script will make you enter the chroot environment using one single command. Every time you exist the debian chroot, you can enter back using the command “debian”.

#!/system/bin/sh
chroot /data/local/debian/ /bin/bash

And that is it. Open a terminal, type debian and start using your debian environment. It might be a good idea to run aptitude update or apt-get update before you start using the package manager.

Part 3 – Settings up a merged Android/Debian environment

This is the fun part to play with. A merged Android/Debian environment without the usage of chroot. This only requires a script to be executed at boot, or after boot.

Remember that this will not work unless your kernel has the aufs module.

#!/system/xbin/sh

export DM=/data/local/debian
export DI=/mnt/sdcard/debian.img

# Now we mount the debian image to our debian folder.
mknod /dev/loop0 b 7 0
mount -o loop,noatime $DI $DM

# These two directories exists in both android and debian,
# so these are going to be merged together.
mount -t aufs -o dirs=$DM/etc=rw:/system/etc=ro aufs /system/etc

# For some reason, mounting /sbin with aufs does not work.
# So we make a little cheat with symbolic links instead.
mount -o remount,rw /
mv /sbin /sbin0
ln -s $DM/sbin /
rm -rf /sbin/adbd
rm -rf /sbin/ueventd
ln -s /sbin0/adbd /sbin/
ln -s /sbin0/ueventd /sbin/

# Let's get all the debian root directories out to the real root dir.
rm -rf /root
for x in tmp root lib bin boot home media opt selinux srv usr var
do
ln -s $DM/$x /
done

if [ -f /system/xbin/bash ]; then
mount --bind /bin/bash /system/xbin/bash
mount --bin /system/xbin/bash /system/xbin/sh
else
mount --bind /bin/bash /system/xbin/sh
fi

echo "Debian has been merged with Android..."

Run this at startup and start using all of debians possibilities (Don't run it yet).

The / dir is left with RW permissions. If this is set to RO then apt-get will not work. It writes tmp log files to this dir when running. It does not matter since all changes to this dir is restored to default on every boot.

Also dont forget to run apt-get update or aptitude update to update the package list.

You are now able to install and run almost anything (CLI Only) that runs on debian systems and other linux dists. Personally I'm currently running OpenSSH Server and Lighttpd with PHP. A perfect small transportable web server.

You can use either /etc/init.d/[your service] start

or

service [your service] start

to start daemons you have installed.

Now you can use the "mkdebian" command to merge your new Debian with Android.

Note: The command “su” from debian works differently than the android “su” command.

The debian command uses the regular linux management to provide access which means that it will ask for your root password that you created in the first part. The android command will just provide you with su rights without a password. Both commands provides su rights to both android and debian, so it is possible to remove the android command and create a new Android/Java based SU app for the Android UI that will require root password.

Note: There is a small problem with the Android "su" command. It does not recheck the /etc/profile.

That means that if you log on using the regular user, and then uses the "su" command, user variables does not get updated with su paths.

Using the Debian "su" command will work fine, just note that this command will promp for password. Also logging in using the root account

directly works fine.

Good luck :)

[Edit: 2011-03-09 11:06]

- Changed the /etc/profile script in Part 1

- Added /etc/bash.bashrc script in Part 1

- Changed User and Group setup and moved it from Part 3 to Part 1

- Fixed error in /system/bin/sh

All of this will fix some permission problems with Debian

[Edit: 2011-03-08 11:21]

- Fixed problems with user management by adding an /etc/profile and changing /system/bin/sh and the debian merge script. Using the "login" command is no longer needed.

[Edit: 2011-03-07 08:49]

- Fixed bug in Part 1 sh script.

[Edit: 2011-03-07 00:20]

- Added a new /system/bin/sh script to Part 1

- Added the way we create missing users and groups in Part 3

[Edit: 2011-03-06 19:36]

- Inserted a warning text in part 1 about avoiding problems when creating a new /system/bin/sh.