[G1 Android Development] Scrip:
This Guide is intended to fast-track the curious Windows user into the world of Compiling Android With Linux

It does not cover Linux in depth and does not cover hacking the code.

It does include an introduction to git by way of an example (replacing the default Kernel)

I hope it proves helpful

Below I shall run through setting up VirtualBox and installing Lubuntu.

Lubuntu is a lighter alternative to Ubuntu targeted at older/low powered hardware which makes it ideally suited to running in a VM.

Before we start, I would like to make it clear that unless you have a powerful,memory rich PC, compiling Android inside a VM is not ideal.

Having said that, You can probably get some mileage on a reasonably spec'ed Machine. And this should prove helpful to those who are not quite ready to take the plunge and install a native linux.​

The latter half of the guide detailing Lubuntu will also work running native on the bare metal. Just ignore the VB driver parts.

If you haven't already, Download an iso from http://cdimages.ubuntu.com/lubuntu/releases/11.10/release/

I will be working through the desktop version

if you have a 64-bit PC *AND* your current OS is 64-bit get the lubuntu-11.10-desktop-amd64.iso

if your PC or OS is 32-bit then lubuntu-11.10-desktop-1386.iso is what you want.

As well as a {L}ubuntu disk, you will need:

VirtualBox https://www.virtualbox.org/ optionally grab the extension pack for usb2 support

At least 25 GB free disk space with large file support

A working internet connection, the CM source is 9GB so have that in mind.

VirtualBox Installation

Bit of a no brainer, just install it like any other program

At some point it may prompt you to download Additions, not to be confused with the extension pack.

The Additions Pack is an .iso containing drivers for the guest OS's, video, clipboard etc.

VirtualBox Setup

Once VirtualBox has installed open it

If you downloaded the extensions pack install it as follows:

file >> preferences >> extensions

click the add package icon and navigate to the download

Now create a virtual machine

click new >> next

give it a name like "AndroidCompiler"

OS Type "Linux"

Version "Ubuntu" >> next

The Version is not particularly important, as far as I know it is just for the icon.​

increase the memory , keeping it in the green >> next

create new disk >> next

VDI >> next

Fixed size >> next

select the location (for example your second hard drive X: or whatever)

change the size to 5.00 GB, if the slider is a pita just type it >> next

If you can afford it a larger disk will not hurt, 10GB should be plenty​

review and >> create >> create again

The basic VM is now complete

Fine tune the VM Settings

Highlight the new VM and hit Settings

system

Motherboard tab

in boot order select hdd and 'move' it to the top of the list with the up icon

Processor tab

tick enable PAE/NX

select number of CPUs

Note that if you use all of your cpu cores it will slow down your host​

Display

Video tab

tick enable 3D

Network

Adapter 1 tab

Attached to Bridge Adaptor

Name < select your NIC >

Advanced >

Promiscuous Mode : Allow All

Storage

highlight the empty CD

click the cd icon to the far right

Choose a virtual cd/dvd, (your iso d/l)

highlight the sata Controller

add hard disk (3 stacked disks icon) >> create new disk

create a new disk called AndroidSrc or something, it must be at least 20GB (40GB recommended)

if you plan on playing with ics then triple it​

Shared Folders

add new folder

tick auto mount

navigate to a folder (or create a new one)

this will be used to transfer files between your OS and the VM​

Now click OK, and then start

Installation of Lubuntu

You will be presented with a rather ugly language selection, pick one

Then select install Lubuntu

Ignore any errors about smbpii4 bios blah blah..

Language selection (again but nicer looking) >> continue

Tick download updates while installing >> continue

select something else >> continue

select /dev/sda >> new partition table

select /dev/sda free space >> add

type = logical

size = 555

location = end

use as = swap area

>> OK

Normally Linux Distos create a swap the same size as physical memory.

This is so physical memory can be dumped to swap when going into hibernation (save state with no power use)

Since VirtualBox has it's own Save State feature we don't need it.

And for what it's worth I have always found the resume after hibernate to be painfully slow
​

select /dev/sda free space >> add

type = primary

size = <what it says>

location = beginning

use as ext4

mount point = /

>> ok

tick format for /dev/sda2

select /dev/sdb >> new partition table

select /dev/sdb free space >> add

type = primary

size = <what it says>

location = beginning

use as ext4

mount point = /Android

>> ok

tick format for /dev/sdb1

unless you are re-using an old Android source disk​

>> install now

Where are You?

Lubuntu guesses from your IP, if needed correct it​

>> continue

Keyboard setup >> continue

username / password >> continue

wait for it to finish

>> restart now! -- press enter when prompted

You should now be booted to Lubuntu

Ignore the update manager, we will get the script to deal with that.

VirtualBox Drivers

Next to the clock should be a pci card icon

click it

install drivers

activate >> input your password

wait .....

when finished >> close

bottom left is some strange icon, it is the main menu

click it >> logout >> logout (no need to reboot yet) >> login again

Shared Folders

Main Menu >> system tools >> users and groups

Manage Groups >> vboxsf >> Properties >> tick your username

Main Menu >> Accessories >> LXTerminal (right click >> add to desktop, if you like)

The reason for the earlier logout and back in was to activate the VB host to VM clipboard

The below script will install the required packages

It will work for both 32 and 64 bit

The script is based on instructions for 11.10

It will probably work on 11.04 and may even work on 10.xx but I have only tested against Lubuntu 11.10 both 32 and 64 bit

It is not specific to Lubuntu and should work with any Ubuntu 11.10 derivative

To paste with the keyboard ctrl+shift+v *NOT* ctrl+v

ctrl+shift+c to copy
​

copy and paste this to the term.

cat > ~/setup.sh << "EOF"
#!/bin/bash

sudo apt-get update

# Reference : http://source.android.com/source/initializing.html
case $(uname -m) in
x86_64) # packages for 64-bit
sudo apt-get -y install git-core gnupg flex bison gperf build-essential \
zip curl zlib1g-dev libc6-dev lib32ncurses5-dev ia32-libs \
x11proto-core-dev libx11-dev lib32z-dev \
libgl1-mesa-dev g++-multilib mingw32 tofrodos python-markdown \
libxml2-utils xsltproc

sudo apt-get -y install lib32readline-gplv2-dev || \
sudo apt-get -y install lib32readline5-dev

case $(awk -F"=" '/DISTRIB_RELEASE/ {print $NF}' /etc/lsb-release) in
11.10|11.04) sudo apt-get -y install libx11-dev:i386 ;;
10.10|10.04) sudo ln -s /usr/lib32/mesa/libGL.so.1 /usr/lib32/mesa/libGL.so ;;
esac ;;
i?86) # packages for 32-bit
sudo apt-get -y install git-core gnupg flex bison gperf build-essential \
zip curl zlib1g-dev libc6-dev libncurses5-dev x11proto-core-dev \
libx11-dev libreadline6-dev libgl1-mesa-dev tofrodos python-markdown \
libxml2-utils xsltproc ;;
*) echo "Error: Machine type not recognised" && exit 1;;
esac

# Extras for CyanogenMod builds
sudo apt-get -y install squashfs-tools \
pngcrush libsdl1.2-dev \
libesd0-dev \
libwxgtk2.6-dev \
schedtool
# system update
sudo apt-get -y upgrade

# install sun java jdk
# Reference : top google hit for " ubuntu 11.10 sun-java6-jdk "
# http://www.gaggl.com/2011/10/installing-java6-jdk-on-ubuntu-11-10/

case $(awk -F"=" '/DISTRIB_RELEASE/ {print $NF}' /etc/lsb-release) in
11.10) codename=oneiric;;
11.04) codename=natty;;
10.10) codename=maverick;;
11.10) codename=lucid;;
*) echo "Error: Unrecognised ubuntu version";exit 1;;
esac

test -e /etc/apt/sources.list.d/ferramroberto-java-${codename}.list || \
echo | sudo add-apt-repository ppa:ferramroberto/java
sudo apt-get update
sudo apt-get -y install sun-java6-jdk sun-java6-plugin

# The default lubuntu install does not include java
# but other ubuntu derivatives may include openjdk

Java6Sun=$(echo | update-alternatives --config java | awk '/java-6-sun/ && ($1 !~ /*|^There/){print $1}')
test "$Java6Sun" != "" && echo $Java6Sun | sudo update-alternatives --config java

# create ~/bin if it does not exist
install -v -d ~/bin

# clean up
rm $0
EOF

Now run the script

bash ~/setup.sh

It will ask for your password

eventually you will be asked to read and accept the Sun Java licence

Once all that is done..

The updates will have installed a new kernel, and we still haven't loaded the VMs special video/shared folder drivers yet so it is time for a reboot

Like the logout we did earlier, but reboot.

Now that you have rebooted lets fix the permissions for our source partition

open a term

sudo chown $USER:$USER /Android

replace Android with the mount point you gave for your source drive

make a directory for the CM src

mkdir /Android/CM
cd /Android/CM

Now to get the src

#

# http://source.android.com/source/downloading.html

# http://wiki.cyanogenmod.com/wiki/HTC_Dream_&_Magic:_Compile_CyanogenMod_(Linux)

curl https://dl-ssl.google.com/dl/googlesource/git-repo/repo > ~/bin/repo
chmod a+x ~/bin/repo
repo init -u git://github.com/CyanogenMod/android.git -b gingerbread
repo sync -c -j16

The CM wiki advises repo sync -j16

You will be downloading 9 GB, depending on the speed of your connection this may take some time

Reduce the number of threads(16) if you don't want to saturate your connection

NEW : 2012-03-10

added -c to the repo sync.

-c tells repo to only fetch the current branch which will save time and diskspace

If you are interested in comparing branches then leave out -c

don't forget to include -c with all future repo syncs

using -c will save approx 2GB diskspace as well as network bandwidth (7GB instead of 9GB)
​

use this table to get an approximation of how long it will take

10GB @ 60 Mbit/s 0.46 hrs ( 28 mins)
10GB @ 40 Mbit/s 0.69 hrs ( 42 mins)
10GB @ 20 Mbit/s 1.39 hrs ( 83 mins)
10GB @ 10 Mbit/s 2.78 hrs ( 167 mins)
10GB @ 5 Mbit/s 5.56 hrs ( 333 mins)
10GB @ 2.5 Mbit/s 11.11 hrs ( 667 mins)
10GB @ 1 Mbit/s 27.78 hrs ( 1667 mins)

Assumes Max throughput

When setting up the VM we created a bridge to one of your NICs

If you are on a laptop and switch between wire and wireless, you can also with the VM

Devices >> network adaptors (the VirtualBox Menu)

You could try setting up a 2nd adapter, one for wire and the other wireless, net traffic should be routed automatically

You can only add adaptors while the VM is Powered Off​

When repo sync has completed

Setting up the Environment

Setting up the Environment needs to be done with each new shell session

make sure you are in the repo root

A quick glance at your command prompt should tell you where you are

source build/envsetup.sh

alternative:

. build/envsetup.sh

that is <dot><space>bui​

Setting up the build

ccache

not to be confused with compcache, ccache can help speed up recompilations

More information can be found on cm's wiki here

and in google's http://source.android.com/source/initializing.html

export USE_CCACHE=1
# for reboots and new shell sessions
echo "export USE_CCACHE=1" >> ~/.bashrc

by default ccache uses ~/.ccache we won't have enough space

export CCACHE_DIR=/Android/.ccache
# for reboots and new shell sessions
echo "export CCACHE_DIR=/Android/.ccache" >> ~/.bashrc

The default max cache size is 976.6 MB (that is what is reported,could be a / 1024 screwup ), this will just about fit our min sized 20GB drive

http://source.android.com/source/initializing.html recommends between 50 and 100 GB. I imagine this is to accommodate multiple targets

space permitting between 5 and 10 GB should be fine for a single device

if you have a 20GB Android drive then leave it at the default

If you opted for a larger drive, adapt to suit

for 5 GB :

prebuilt/linux-x86/ccache/ccache -M 5G

The setting is stored in CCACHE_DIR so unless you want to change it forget about it.

Lets do lunch

don't ask me why it is called lunch. My best guess is a play on Menu

lunch

and select your target

when you get bored of seeing that lunch menu

lunch cyanogen_dream_sapphire-eng

Proprietary Files

The Proprietary Files need to be put into the vendor tree

You have two options:

Extract from a zip

Extract from your device

I'm going to walk through Option 1

Remember the Shared Folder we created?

Well we are going to use that now.

On your Host system copy a recent ROM zip to the folder your shared with the VM

good choices are : ezginger, ADS_magpie, dw9906

in your VM's terminal,

ls /media/

you should see a folder sf_<Name> and in that folder your rom zip

There are two scripts in device/htc/dream_sapphire/

unzip-files.sh and extract-files.sh

Unfortunately the unzip-files.sh script is not up to date with extract

You can manually fix it or use my script which converts the extract script to an unzip script

to use it:

set a environment var like so

ROM=/me<tab>s<tab><tab>

<tab> is the tab key, please don't type <tab>

read this to get an idea of what is happening
​

eventually you should have something like

ROM=/media/sf_5gb/ADS_magpie-V1.0-signed.zip

pushd device/htc/dream_sapphire
#awk '/adb pull/{sub(/adb pull/,"unzip '$ROM'")sub(/\.\./,"-o ..")};{print}' extract-files.sh | sh
# 2012-03-19 corrected the above script, it should be :
awk '/adb pull/{sub(/adb pull \//,"unzip -o '$ROM' ")sub(/.*/,"-d ../../../vendor/htc/$DEVICE/proprietary/",$NF)};{print}' extract-files.sh | sh
popd

pushd and popd

The above step only needs to be done the once

that is unless the proprietary files require changes, for instance a switch to open sourced versions

Kernel

The DS kernel in CM is old so I added ezterry's latest

I shall show you how, but we are taking the scenic route

We shall create our own git repo and add that to a local_manifest

Configure git

This is a global config, you should only need to do it once per user

git config --global user.name "Your Name Comes Here"
git config --global user.email you@yourdomain.example.com

taken from file:///usr/share/doc/git-doc/gittutorial.html

Install git docs with

sudo apt-get install git-doc

Create a git repo and push to it

git init /Android/my_git_repos/device_htc_dream_sapphire
cd /Android/CM/device/htc/dream_sapphire
git push file:///Android/my_git_repos/device_htc_dream_sapphire github/gingerbread

Now let's have a look

cd /Android/my_git_repos/device_htc_dream_sapphire
ls

you will see it is empty

But wait.. is it empty?

ls .<tab><tab>

git branch -a
git checkout gingerbread
ls
git log # q to exit
git log -p

now lets make some changes

Remember the shared folder?

Well I hope you have ezterry kernel pack in there

unzip /media/sf_<REPLACEME>/ezgb-2636-v1.5.2_S.zip *2708*
mv 2708-zImage kernel
mv mod-2708/modules.sqf prebuilt/
rmdir mod-2708/
git status

edit full_dream_sapphire.mk

change

device/htc/dream_sapphire/prebuilt/wlan.ko:/system/lib/modules/wlan.ko \

to

device/htc/dream_sapphire/prebuilt/modules.sqf:/system/lib/modules/modules.sqf \

we can do this with a simple sed

sed -i -e 's[/wlan.ko[/modules.sqf[g' full_dream_sapphire.mk

now

git diff

git add full_dream_sapphire.mk kernel prebuilt/modules.sqf

Since the wlan.ko is of no use to us (we replaced the kernel it was for) we need to remove it

but not just from the filesystem but git as well

git rm prebuilt/wlan.ko

Have another look at git status and git diff

notice that git diff produces no output

now try git diff --cached

commit the staged changes with

git commit

The default editor for git is nano

^ == ctrl

^O == ctrl+o

write commit notes

Updated Kernel and kmodules to 2708 ezgb-2636-v1.5.2

http://xdaforums.com/showpost.php?p=14768272&postcount=2427

to save Ctrl+o<enter>

to exit Ctrl+x

if you prefer vim

git config --global core.editor vim

don’t forget to install vim​

Adding a local_manifest.xml

cat > /Android/CM/.repo/local_manifest.xml << "EOF"
<?xml version="1.0" encoding="UTF-8"?>
<manifest>
<remote name="Local"
fetch="file:///Android/my_git_repos/" />

<project path="device/htc/dream_sapphire" remote="Local" name="device_htc_dream_sapphire" revision="gingerbread" />
</manifest>
EOF

Apologies My initial post had an error in the above codebox

it should be fetch="file:///Android/my_git_repos/" not fetch="file:///Android/my_git_repos/device_htc_dream_sapphire/"
​

make sure you are in /Android/CM/

repo sync

Then look in device/htc/dream_sapphire to see if your changes were applied

pushd device/htc/dream_sapphire
git branch -a # look for remotes/m/gingerbread -> Local/gingerbread
git log -p # you should see your commit notes
popd

Remove RomManager

Optionally remove RomManager from the build

sed -i '/RomManager/d' vendor/cyanogen/products/common.mk

The above is not permanent, when you repo sync it will be reverted

That is unless you commit the change​

if you want RomManager then

./vendor/cyanogen/get-rommanager

finally build it

mka bacon

It is going to take a while depending on the speed of your machine and the memory/cpu cores you gave the VM (if you are using a VM)

Lets hope you don't get any build errors

If all goes well you should end up with a zipfile in /Android/CM/out/target/product/dream_sapphire/

Don't forget your /media/sf_ folder, cp the completed zip to that and from there you can treat it like any other rom

I hope you had fun following this