[ONE General] jgcaap:
Hello everyone.

I’ve decided to build this tutorial in order to assist all developers and builders into the ROM and kernel development. I’ve felt motivating into writing this small post because I feel that the current information available is confusing and also do not provide the set of skills and information to help someone in the first steps.

This tutorial will be divided on:

The Machine Configuration

The tools of development

The Source

Hardware

Software

What to do to be able to Innovate

The Machine Configuration

In order to have a good machine configuration you should select your distro based on the packages which are provided to developers. In my opinion the best distros are Fedora and Ubuntu. It is important to learn a bit how Linux works in order to fully understand what we are editing as Android is nothing more than equivalent Linux system where you run java on a virtual machine. In other words, you’ll understand with time that many parts which are used on Linux Operating system it is also used on Android Operating System (example: bionic, kernel libs).

After you installed your Linux system you’ll need to prepare your machine for building. For that you’ll need to install the following packages:

sudo apt-get install git ccache automake lzop bison gperf build-essential zip curl zlib1g-dev zlib1g-dev:i386 g++-multilib python-networkx libxml2-utils bzip2 libbz2-dev libbz2-1.0 libghc-bzlib-dev squashfs-tools pngcrush schedtool dpkg-dev liblz4-tool make optipng maven
sudo apt-get install openjdk-7-jdk

sudo apt-get install android androidsdk-uiautomatorviewer android-copyright android-src-vendor android-emulator android-tools-adb android-headers android-tools-adbd androidsdk-ddms android-tools-fastboot androidsdk-hierarchyviewer android-tools-fsutils androidsdk-traceview

After you have the basic packages installed we are going to add the repo instructions to you git in your machine. Basically this will interpret the source information when you are syncing new code from google repos and we’ll add to the folder /bin/repo.

To do that we’ll run.

[B]mkdir -p ~/bin
curl https://storage.googleapis.com/git-repo-downloads/repo > ~/bin/repo
chmod a+x ~/bin/repo[/B]

And run the /.profile. Basically this command will reload information of the new settings for your profile, where it’ll read the folder ~/bin/repo information for all situations. In the future only makes sense to load again in case you have added a new repo instruction.

[B]. ~/.profile[/B]

Now we are going to sync the source code. I’ll use here as an example CyanogenMod as I believe to be the easiest for anyone which is starting. Lets imagine we are going to put all the source on a folder named cm, you’ll have to just run the following commands.

[B]mkdir ~/cm
cd ~/cm
repo init -u git://github.com/CyanogenMod/android.git -b cm-13.0[/B]

So in order to explain what we’ve done here by steps. The repo init –u basically is a command instructing to go to the GitHub on that specific location and extract what is available on the cm-13.0 branch regarding to manifests. So basically it’ll automatically push the file which has all CyanogenMod source to .repo/ folder with the name manifest.xml

After you have loaded this command you’ll run the repo sync , basically this command consists on downloading all the code into your building folder (in this case cm folder).

After the repo sync is complete you’ll have interest on having a local_manifests. Basically on this local_manifests.xml , where some developers prefer to name it as roomservice.xml it is where you’ll add the changes to the manifest.xml or even sources which you wish to add to your building folder. This is a solution to you never edit the manifest.xml.

Usually it is added on the local_manifests the folders related to your device tree. I’ll give in this example how to build for the OnePlus One (bacon).

A device tree consists on configurations where you set information related to your device. Example if is qualcom, usually is android_device_qcom_common (where is the general settings) together with android_device_oneplus_bacon (where are the specific settings). When I say settings, for you to understand is like compile with the folder hardware/qcom/display for the display. You should take some time to explore to understand what is in it.

So you’ll add your local_manifests.xml this way.

[B]
mkdir .repo/local_manifests
nano .repo/local_manifests/local_manifests.xm[/B]l

You now add:

<?xml version="1.0" encoding="UTF-8"?>
<manifest>
<project name="CyanogenMod/android_device_oneplus_bacon" path="device/oneplus/bacon" remote="github" revision="cm-13.0" />
<project name="CyanogenMod/android_device_oppo_common" revision="cm-13.0" remote="github" path="device/oppo/common"/>
<project name="CyanogenMod/android_device_qcom_common" path="device/qcom/common" remote="github" revision="cm-13.0" />
<project name="CyanogenMod/android_kernel_oneplus_msm8974" path="kernel/oneplus/msm8974" remote="github" revision="cm-13.0" />
<project name="TheMuppets/proprietary_vendor_oneplus" path="vendor/oneplus" remote="github" revision="cm-13.0" />
</manifest>

Now you need to run the repo sync command again.

This has to do separately the first time or it won’t sync all the files you need. In case the repo sync fails with an error saying you need to do –force-sync just run repo sync –force-sync (this happens only when you change common git sources with your manifest.

After the sync is done you are ready to build!

To start building you’ll need to run the .envsetup.sh , which will load all the source code into memory. After that breakfast bacon userdebug (in case it is for the OnePlus One device), then later make bacon (the word bacon it is to be used on all devices , here it is just a coincidence with OnePlus One)

[B]. build/envsetup.sh
breakfast bacon
make bacon[/B]

After it compiled your rom will be located at /cm/out/target/product/bacon/

Easy until here? I hope so.

Possible errors :

Q: Receive error: fatal: duplicate path xxx/xxx in ~/cm/.repo/manifest.xml

A: edit .repo/local_manifests/local_manifests.xml to remove the item which matches the item located in bold.

Q: How big is the repo?

A: 50GB just source code and around 10GB when built per device. Therefore you should aim to have about 60GB free on your system.

The tools of development

Git

Probably many of you question about what it is git, read about it on the github manuals etc and still cannot figure out how to work (I took two weeks to understand that it is really simple). So imagine that you have a diary, but instead of being separated by dates it is separated by titles, which we call commits. And what is a commit? Basically it is a portion of code. So when you make a change and want to record that change, you create a commit and in the end it’ll be a portion of code. This portion of code, known as commit, can be easily ported between different sources.

Here is an example:

https://github.com/jgcaaprom/androi...mmit/9f16b3cd79ad7bb8c821a518ca73725f19c38478

The commit number is: 9f16b3cd79ad7bb8c821a518ca73725f19c38478

This commit has recorded a change of IZAT_DEBUG_LEVEL = 2 .

Now how to bring other commits from different sources into our source.

The process comes in 3 phases:

First we’ll have to add the source where the commits that interests us are with git add source_name link

Second we’ll download that source by using git fetch source command

Thirdly we’ll cherry-pick (bring the portion of code recorded in the commit) into our source in order to make the changes automatically.

So lets see a real scenario. Imagin you are interested on cherry-picking this source:

https://github.com/CyanogenMod/android_device_oneplus_bacon

What you’ll have to do is.

Step one: git add cm https://github.com/CyanogenMod/android_device_oneplus_bacon.git

Step two: git fetch cm

Step three: (imagine this is the commit you want:

https://github.com/CyanogenMod/andr...mmit/58027fcbea4dd9fbb5aaeb8ae5f6e32bdf613573 )

The commit number is: 58027fcbea4dd9fbb5aaeb8ae5f6e32bdf613573

So you’ll git cherry-pick 58027fcbea4dd9fbb5aaeb8ae5f6e32bdf613573

And voila! You’ve done your cherry-pick . Now it is important to have in mind how the cherry-pick process works which I’ll explain on the next step.

Cherry-picking

Cherry-picking is a process of importing code from one source to another. So the process that git does basically is, compares your present files in the source with the files that was on the source you are cherry-picking and if everything is a match it’ll change the code automatically for you.

Now lets pick up the example above and imagine that the changes you are doing instead of being exactly the same as described here:

https://github.com/CyanogenMod/andr...mmit/58027fcbea4dd9fbb5aaeb8ae5f6e32bdf613573

Your source has a difference which doesn’t have this line:

32 private static String CONTROL_PATH = "/proc/touchpanel/keypad_enable";

You’ll notice that it’ll give a conflict. The reason for that is because the git when comparing understood that there was no match between the sources. When that happens means basically that there might be changes you should do in order for the code to be compatible, so you’ll have to review that code.

Now the review process is very simple. For you to see which files are in conflict you’ll have to run git status ( basically this command informs you of the situation of the commits which are being cherry-picked and the changes which you’ve done. )

You’ll notice that the files which need to be edited/review are in red. In case it appears files in green, means that there was a perfect match and doesn’t need a review.

Pretty useful right?

After you finish editing all the code you’ll do git add . and for this exercise do git status again just for you to see what happened . Everything is green. To finish the cherry-pick you’ll now do git commit and done!

How to record my own changes and create a commit with it.

Basically this process is exatly the same as the previous one, the only difference is that after you made the changes, you can make git status, then git add filename in case you want to commit only one file or git add . in case you wish to commit all the files and to record the commit you write git commit, or git commit –m “commit message”.

Congratulations! You’ve just recorded your commit!

Git push/pull

There will be moments where we just wish that we could be lazy and cherry-pick everything all at once. Well there are some scenarios where you can actually do that, but remember it is not perfect. Lets imagine we are using the source above, and CyanogenMod uploaded like 20 different commits, your source is not exactly like CM source but you wish to keep your changes and also have all changes from CM what to do?

1)	You can cherry-pick one on one. And waste a lot of time…

2)	You can make a new source based on cm and include your changes there

3)	You can git pull

Git pull basically will update your source with CM source by merging all the commits. You do that with:

git pull cm cm-13.0 (in case your rom is Marshmallow)

The git pull should only be used when the sources are very similar and you are using as a base. Basically it’ll warn you that’ll merge the code and voila done! 20 commits added to the source.

Now when on the situation that the sources are slight different or very different it’ll create a conflict. Again the git will do comparisons between the entire sources, check which commits are missing, compare the files just like with cherry-picking. And when you do git status, it’ll appear on red all files where you need to review your code.

After you finish, same as before

[B]git add .
git commit[/B]

Done.

I hope you are understanding everything until here.

git reset HEAD~n

This command is rarely used but is super usefull. Basically you are saying to git that you with to remove n commits you've reated/cherry-picked. For you guys to understand how it works, basically imagin you have just written/cherry-picked 10 commits, and you do git reset HEAD~3 , he'll basically remove from records 3 commits. But... That doesn't mean he reverted the code! In case you wish reset the code you need to do git reset --hard.

Also it is very usefull to help rewrite the history. Lets think the example I given before. When you do a git reset HEAD~3 , basically on a history of A B C D E F G H I J commits, the git will remove only the recorded commits H I J. If you do after a git status, you'll understand that the code is still there, which means the changes you done on H I J will still be there. So you can create a new commit with H I J all together . Simple!

Another way to create a unique commit with H I J is by using git rebase -i HEAD~3 , where you basically are instructing the git that you wish to rebase the source with an interactive selection. You'll see that it'll appear a commit list where you can remove other commits from history. But since you wish to create a unique commit you'll wish to change the instruction from pick with the letter s (without capslock term for squash) . Example:

[B]pick d0c49f4 UsbDeviceManager: Remove charging from persisted function composition
s 2c755a6 SystemUI: fix NFC tile sometimes hiding
s e39d1d7 Revert "base: start nfc service prior to systemui"
pick 9164274 DocumentsUI : Hide advanced menu option when in forced mode
pick 5f8d3f8 Themes: Expose Power Dialogs
pick 9153396 SysUI: Don't let rogue themes ruin notifications
pick 226797c Revert "Only show keyguard panel if on lockscreen + no activity on top"
pick e843aaf Fix NPE in DocumentsUI when rotate UI before format as internal
pick 26079d3 SystemUI: detect rotation and resize mKeyguardBlur accordingly
pick 80b060d Automatic translation import
pick 4282864 Automatic translation import
pick d442df6 wifi: Enable WiFi IpReachabilityMonitor by default
pick 9cf937f SystemUI: Add margin in qs_tile_top between instruction text and add button[/B]

Cool right? :)

And I believe I've covered with some examples the git. As you can see, once you get used to git you'll be able to use it easily.

Toolchains

There has been a big debate from many developers on which toolchains to use. Many prefer sabermod, others prefer uber, others google toolchain, etc. It is in fact debatable. If you ask me which one to use, I would say google or uber (very similer to google) and the gcc used by stock on the source. The reason is simple, the code is optimized for a certain gcc version, if you start using a gcc version which is not prepared to be used on the code, it won't translate correctly your instruction.

The source

When you start building roms, you should at least know a little about how the source of android is organized, what language it is used, etc. On android it is used C, C++, Java, Assembly. Withing the following folders inside the source it is used:

Java

Frameworks

Packages

external (some packages)

C++

ART

bionic

hardware

C

kernel

hardware

Assembly

bionic

some of the kernel libs

This is important for you to navigate easily within it. Some of you ask yourselfs what is one thing or another. I'll represent that to the following image which is published on the google developers page.

View attachment 3800925

Now I believe you wish to know which parts should be used in order to put a rom working for a certain device. Those parts are:

Hardware

Kernel

device tree

Vendor blobs

These parts are the "base" of your build to put a certain device in a working, since on these parts is where lies the instructions for the device to understand what to do.

The build folder is where you specify instructions of the building process. For example : Certain optimization, which GCC to use, where will the compiler find the apps to compile.

frameworks, system, packages and some of the folders within the external folder are the packages of what you will have inside the rom which is general and can be present on any rom.

external folder has present different things, compilers such as proguard, jemalloc. external software used by the rom such as sqlite.

prebuilts folder has present everything related to stuff which is already compiled, in this case clang, gcc toolchains.

I've done a very summarize version of the source in order for you to understand how that google built things, so I hope it is simple to understand these basics.

(to continue)