[Android] theimpulson:
Introduction

I am writing this guide as I was unable to find a well-documented thread over XDA (there are just a lot of them). In case there is anything that concerns this guide, feel free to ask in this thread. I expect you to know the basics of Linux, GIT, Android. Please look over some guides in the forum in case you are not familiar with them. In case you want to contribute to this, feel free to do so. I welcome everyone's contribution to be it fixing my derp, pointing a mistake or adding a new section.

Links to some nice documentations:

Git Documentation

Linux Kernel Newbies

How to take logs in Android

How to get an Android kernel up to date with linux-stable

Checking Current CAF Tag of the Kernel

We will be using best_kernel for this purpose, whose aim is to compare your current kernel source code with different CAF tags and select the best possible match using git diff.

Clone/Download your kernel repo and commit your changes(if not already committed) as best_kernel depends upon git diff to compare changes.

Depending upon your kernel version, add required CAF kernel repo as remote and fetch it. best_kernel will use this fetched data to compare changes.

After fetch is done, run best_kernel. It should run automatically if you have placed dotfiles in the $BIN directory and have the necessary dependencies installed.

best_kernel will compare changes and will provide results based on that. However, depending on the number of tags, it will take time. A lot of time, so go and grab a cup of coffee till then.

A Dummy Example:

Downloading stock kernel source code from OEM’s website

wget http://nokiaphones-opensource.azureedge.net/download/phones/Nokia6.1_V2.22J.tar.bz2 && tar xvjf Nokia6.1_V2.22J.tar.bz2 && cd kernel

Adding and committing all files due to lack of a .git dir.

git init && git add --all && git commit -m "Initial Commit"

As the kernel source code version is 4.4, I will add 4.4 remotes from CAF and fetch it.

git remote add caf https://source.codeaurora.org/quic/la/kernel/msm-4.4/ && git fetch caf

After fetch, I will check for the current CAF tag using best_kernel. As my device platform is sdm660 and CAF has been releasing sdm660 tags as sdm660.0, I will be using it as an argument in best_kernel. This will ensure that best_kernel doesn’t waste time checking CAF Tags of other platforms and will speed up the process considerably.

best_kernel "*sdm660.0"

RESULT will be something like this:

Best match TAG: LA.UM.6.2.r1-06900-sdm660.0 Lines changed: 177924

What is CAF/Code Aurora | CodeAurora Android Releases Page | CAF Kernel Sources​

Creating/Importing GIT History for Kernel Source

A lot of OEMs release kernel sources in a compressed format excluding the .git dir which results in no git history. Creating or it's better to say to import one is simple and easy.

Download and check the current CAF tag of your Android kernel source.

Note: If your chipset is not Qualcomm, you should visit Linux Kernel Source or AOSP's Kernel Common and use that as a base while checking out to the specific version of your kernel source you are going to copy-paste. Qualcomm devices can also use that as a ref but I will recommend them to use CAF source.

Based on that CAF tag, download the respective kernel repository from CAF and checkout for that specific tag.

Copy and paste your downloaded kernel source (with no history) over the CAF one (which has the history of that CAF tag).

Commit the changes.

You have accomplished creating/importing history.

A Dummy Example:

My current CAF tag is LA.UM.6.2.r1-06900-sdm660.0.

Therefore, cloning kernel source from CAF and checking out for this tag while creating a new branch named rebase.

git clone https://source.codeaurora.org/quic/la/kernel/msm-4.4/
cd msm-4.4/
git checkout tags/LA.UM.6.2.r1-06900-sdm660.0 -b rebase

Now, I will copy-paste the kernel source that I downloaded from OEM's website over the CAF one. Assuming the downloaded kernel is in ~/Downloads/kernel and CAF kernel source is in ~/msm-4.4 directories.

#Renaming the dir so, that it will get merged while copying due to similar names
mv kernel/ msm-4.4/
cp -avr msm-4.4/ ~/msm-4.4
cd ~/msm-4.4
git status
git commit -m "Import Nokia Kernel Source"

Keep in mind that a .gitignore is generated which can and will ignore files during commit which were not getting ignored earlier due to its absence. In case during compilation you encounter any error stating about missing file, add it forcefully and commit again.

Remember, its always good to split that one big commit into small parts. It's helpful when you start rebasing your kernel usually when new CAF Tag for new android versions come out. Solving a few conflicts ina lot of commit is much better than solving a load of them in one. It also gives you a chance to easily diff on websites like GitHub so that you can figure what's a particular line is doing.

Doing that is also easy. Just reset head for a particular folder like driver/touchscreen and commit them separately. Check git rebase documentation which will help you with this.

Upstreaming your Kernel

Merging upstream versions is a really good thing. It brings in security patches, stability, new features and much more. The usual and best way is to visit Linux Kernel Source and check if there are upstream updates available. If yes, start merging them in your kernel source one by one.

Note: If your chipset is not Qualcomm, you should visit Linux Kernel Source or AOSP's Kernel Common and use that as a base. Qualcomm devices can also use that as a ref but I will recommend them to use CAF source.

A Dummy Example:

Assuming my current kernel version is v4.4.205, I will fetch v4.4.206 from Linux Kernel Source and merge that. I can also fetch and merge different branches from CAF Source which are usually upstreamed with Linux-stable (for example, aosp/upstream-linux-4.4.y branch)

git fetch https://git.kernel.org/pub/scm/linux/kernel/git/stable/linux.git v4.4.206
git merge FETCH_HEAD

Fix conflicts if you get some. What to keep and what not is a question that depends upon the device to device. Usually, you should compare local changes and upstream and try to figure out what could be the best way to fix it. I will advise to check the history of that file in upstream and see which commit caused that conflict and why. You can also check similar merges by other same kernel versions to see if they had the same conflict and how they fixed it.

I will also advise merging AOSP's kernel/common and new CAF upstream tags (Only QCOM devices should merge new CAF tags) regularly when available. The reason is pretty simple. There are commits that get merged in AOSP like which improves performance or other stuff which doesn't get merged in Linux Kernel Source. This way you can get the benefit from all sources. This, however, is not pretty easy and might result in a huge number of conflicts considering if OEM has modified the source a lot from upstream.

CAF also hosts f2fs and other file system changes on their servers. Merge it if you want and use it.

Compiling Standalone Kernel

Compiling a kernel standalone requires you to have a compatible toolchain like GCC or Clang. The best way to figure out which one suits your need is testing and debugging. The standard AOSP GCC toolchains are recommended for devices with old kernels, to begin with, while newer ones can use clang with backports or upstream merges. In case you want to compile with clang, checking out android-kernel-clang is a good way, to begin with.

Compiling standalone kernels can give a lot of missing headers errors/warnings. This is due to not compiling it in a ROM environment. This can be easily fixed either by correcting paths of those header files being included or creating an out dir manually and using it as output. You can also use this commit to do that automatically.

Clone your kernel source and desired toolchain(s). Export arch and cross compiler arguments. Choose the desired defconfig to make and compile.

A Dummy Example:

I will clone the kernel source and toolchain in the same dir and export the following arguments in kernel source root dir.

export ARCH=arm64
export CROSS_COMPILE=/media/hdd/aayush/kernel/aarch64-elf-gcc/bin/aarch64-elf-
make nokia_defconfig
make -j$(nproc --all)

Kernel conflicts might occur. You can search on GitHub to check if someone has fixed that already, otherwise fix the issues yourself. Comparing code with upstream repositories is also a way to figure out the differences. Choosing defconfig can be a bit tricky if you are doing bringups. I recommend pulling the current defconfig from your device itself. Its usually by the name of config.gz in /proc/ of your device.

Links to some cross compiler

GNU Toolchain by ARM

DragonTC

GNU and Linaro