[Android Software Development] yuweng:
AIO-TWRP-Compiler on BBQLinux

First step

Refer to here on how to setup BBQLinux either native boot, dual-boot or on VBox... :good:

Download the script & extract it to your HOME or if you are 100% sure that you have already setup a fully working Build Environment on Ubuntu or any other linux distro then remark these lines.

Start Terminal => Edit => Profile Preferences => Colors => unchecked => Use colors from system theme => Built-in schemes => White on Black => Close

BBQLinux => Menu => Administration => BBQLinux Python Switcher => Python 2

BBQLinux => Menu => Administration => BBQLinux Java Switcher => OpenJDK 7

Start the script with below command

sh twrp

Menu 1

You can use either OmniROM or Cyanogenmod repo to build TWRP

Choose either to use TWRP branch android-5.1 or android-6.0. Typically, for legacy devices with ICS or JB doesn't support TTF so if TWRP android-6.0 doesn't boot then you'll have to use android-5.1 branch.

You can either choose to use Full-Source or Part-Source. Compiled out recovery.img will have exactly the same size & files on ramdisk.

Yes, Part-Source is only using 41 of the projects to build a fully working TWRP. Syncing in progress. After syncing, you can refer to ~/AIO-TWRP-Compiler/sync.log for any error

Typically, when you have already downloaded the base repo, re-syncing it will only take a few seconds

Menu 2

Copy either your boot or recovery.img to ~/AIO-TWRP-Compiler folder

Key-in your vendor name

Key-in your device name

This step is extracting the ro.product.device & ro.product.model from your device directly which is typically require for identification when flashing update.zip from your vendor

Credits to RD osm0sis, the script will auto download osm0sis's unpackbootimg for mkbootimg arguments required for repacking recovery.img . . . :good:

mkbootimg arguments will be display & auto updated to the BoardConfig.mk

Part of the original mkvendor.sh script is adapted for the extraction of boot or recovery.img which also does the git initialization. If the ramdisk can be unpack then the recovery.fstab will be copied to the device tree folder, if not then it will create a basic MTK recovery.fstab, boot.mk & also BOARD_CUSTOM_BOOTIMG_MK. If you're not building for MTK then remove that line from BoardConfig.mk

You will need to add/ remove TWRP flags accordingly & manual edit on the TARGET_CPU_VARIANT either cortex-a7 or cortex-a9 for older MTK, refer to your device spec. Possible values are cortex-a5, cortex-a7, cortex-a8, cortex-a9, cortex-a15, krait, scorpion. Use generic for devices that do not have a CPU similar to any of the supported cpu variants

You will need to manual edit on the recovery.fstab to suit your device or TWRP-ify the extracted stock recovery.fstab

Menu 3

omni.dependencies is only a warning, will continue to build when you choose OmniROM repo

Make 4.1 is only a warning, will continue to build

Java 1.7 is only a warning, will continue to build

When you have build earlier, subsequent build will only take few minutes bcos of CCACHE... :good:

Menu 4

Updated Menu 4 support for Others. Or you can use Flashify or Rashr or above Others to install compiled recovery.img to your device, do check if they are compatible or it will brick your recovery or worse, whole device !

Key-in your device recovery partition path. Above is typical for LG

Done, test TWRP on your device to make sure it is working properly if not then add or remove flags accordingly & use Menu 3 to build again

Menu 5

Choose either Option 1 or 2, Option 2 is recommended & it won't take long if you are using Part-Source.

Menu 6

The script will auto open up a new browser for you to register a new account on github

Your device tree on github

AIO-TWRP-Compiler

bash script itself is self-explanatory. Thanks to the OP, he has offer to take over the maintenance of this script so hit your Thanks button to him... :good: If you found any error/ bug or have any new ideas/ features for this script then pm the OP or do a git pull request... :good:

Above is the detail explanation on the script GUI front-end, here i'll explained what the script actually does.

Menu 1

This menu will create a local_manifest.xml to remove unwanted projects based on your selection. When select Full-Source, it will also create local_manifest.xml to remove existing recovery & replace it with TWRP based on your selection.

<?xml version="1.0" encoding="UTF-8"?>
<manifest>
<remove-project name="android_bootable_recovery" />
<project path="bootable/recovery" name="omnirom/android_bootable_recovery" revision="[COLOR="Blue"]android-6.0[/COLOR]" />
</manifest>

Part-Source

It is normal that OmniROM & CM will consistently add new projects to their repo when they introduce new features or functions to it. The beauty of this Part-Source is that even when projects grows, this script will grep -v only the needed projects for building TWRP on any base repo.

This Part-Source is the extreme minimal for building TWRP therefore some of the additional projects are needed when you use particular TWRP flags. As of today ( March 2016 ), it will build a fully working TWRP with only 44 projects on both OmniROM 4.4 & CM 11 base repo. In the future, Teamwin may also add a new feature or function then you will also need to add additional projects to it too or OP will update it based on your feedback. Refer to compile.log for details.

# Optional => TARGET_USERIMAGES_USE_F2FS := true
[COLOR="Red"]#[/COLOR]grep -v f2fs-tools ~/x > ~/x.tmp;mv ~/x.tmp ~/x

If you use TARGET_USERIMAGES_USE_F2FS := true then remove the remark #

# Optional => TW_INCLUDE_NTFS_3G := true
[COLOR="Red"]#[/COLOR]grep -v external/ntfs-3g ~/x > ~/x.tmp;mv ~/x.tmp ~/x

if you use TW_INCLUDE_NTFS_3G := true then remove the remark #

# Optional => TW_USE_TOOLBOX := true
[COLOR="red"][B]#[/B][/COLOR]grep -v external/toybox ~/x > ~/x.tmp;mv ~/x.tmp ~/x
[COLOR="Red"]#[/COLOR]grep -v external/mksh ~/x > ~/x.tmp;mv ~/x.tmp ~/x

If you use TW_USE_TOOLBOX := true then remove the remark #. toybox available on android-6.0/ cm-13.0 only, don't have such projects on android-4.4 but it'll work, refer to the end for more info.

# un-remark the below required for x64
[COLOR="Red"]#[/COLOR]grep -v linux-x86/aarch64/aarch64-linux-android-4.8 ~/x > ~/x.tmp;mv ~/x.tmp ~/x
[COLOR="red"]#[/COLOR]grep -v linux-x86/aarch64/aarch64-linux-android-4.9 ~/x > ~/x.tmp;mv ~/x.tmp ~/x

TARGET_ARCH := arm64
TARGET_CPU_ABI := arm64-v8a
TARGET_CPU_ABI2 := armeabi-v7a
TARGET_ARCH_VARIANT := armv8-a
TARGET_CPU_VARIANT := generic
TARGET_2ND_ARCH := arm
TARGET_2ND_ARCH_VARIANT := armv7-a-neon
TARGET_2ND_CPU_ABI := armeabi-v7a
TARGET_2ND_CPU_ABI2 := armeabi
TARGET_2ND_CPU_VARIANT := cortex-a7

If you are building x64 TWRP then remove the remark # & update BoardConfig.mk accordingly. Take note, not available on android-4.4/ cm-11.0 base repo, use later base repo.

# un-remark the below required for kernel compiling if not using TARGET_PREBUILT_KERNEL
[COLOR="red"]#[/COLOR]grep -v linux-x86/arm/arm-eabi-4.6 ~/x > ~/x.tmp;mv ~/x.tmp ~/x
[COLOR="red"]#[/COLOR]grep -v linux-x86/arm/arm-eabi-4.7 ~/x > ~/x.tmp;mv ~/x.tmp ~/x
[COLOR="red"]#[/COLOR]grep -v linux-x86/arm/arm-eabi-4.8 ~/x > ~/x.tmp;mv ~/x.tmp ~/x

if you are building your own kernel then remove the remark #

# Optional for x86 SoC
[COLOR="Red"]#[/COLOR]grep -v linux-x86/x86/i686-linux-android-4.7 ~/x > ~/x.tmp;mv ~/x.tmp ~/x
[COLOR="red"]#[/COLOR]grep -v linux-x86/x86/x86_64-linux-android-4.8 ~/x > ~/x.tmp;mv ~/x.tmp ~/x
[COLOR="red"]#[/COLOR]grep -v linux-x86/x86/x86_64-linux-android-4.9 ~/x > ~/x.tmp;mv ~/x.tmp ~/x

TARGET_ARCH := [COLOR="blue"]x86[/COLOR]
TARGET_CPU_ABI := armeabi
TARGET_ARCH_VARIANT := [COLOR="blue"]x86[/COLOR]
TARGET_CPU_VARIANT := [COLOR="blue"]x86[/COLOR]

if you are building for Intel SoC then remove the remark # & edit the BoardConfig.mk accordingly

This script has been tested & it works on any base repo from OmniROM android-4.4, 5.0, 5.1 & 6.0 & Cyanogenmod cm-11.0, cm-12.0, cm12.1 & cm13.0 using Full-Source or Part-Source but why use Full-Source to build TWRP when Part-Source works, most importantly, you are not building OmniROM or Cyanogenmod but TWRP.... :cowboy: Using android-4.4/ cm-11.0 base repo has its advantage, compiled TWRP size will be smaller & it can build all version.

If you use android-6.0/ cm-13.0 then obviously you can only build TWRP android-6.0 only, just edit the line 62, 63 for OmniROM or 72, 73 for CM accordingly. Building TWRP android-4.4 on cm-13.0 or on OmniROM android-6.0 will result in a build error ! Later base repo won't build TWRP older version !

Many newbies doesn't know that OmniROM/ CM repo are incremental updates & they create a separate cm11, cm12, cm12.1, cm13 each & sync individually to each of these folders.

Above syncing cm-13.0 & as you can see, it is also using source from cm-10.1, 10.2 & 11. Syncing all to the same ~/omni or ~/cm folder will do.

Sometimes on certain devices, OmniROM android-4.4 or Cyanogenmod cm-11.0 base repo just won't boot then you can try by editing this line to android-5.0 or cm-12.0 & updating it will only take a very short time if you use Part-Source.

Menu 2

Device tree

This script will create a barebone minimal device tree for building TWRP. Anything less will result in a build error. It will create five files which is just sufficient to build a fully working TWRP using prebuilt kernel. It will build kernels too, just update the correct kernel config to BoardConfig.mk & remove the remark # needed projects for building kernels. However, i don't see the needs to build the same kernel with every TWRP new build, it will take a longer time to build, most importantly, you are not building a ROM.

PRODUCT_MAKEFILES := \
$(LOCAL_DIR)/cm_[COLOR="blue"]device name that you have key-in[/COLOR].mk

AndroidProduct.mk

TARGET_ARCH := arm
TARGET_CPU_ABI := armeabi
TARGET_ARCH_VARIANT := armv7-a-neon
TARGET_CPU_VARIANT := [COLOR="blue"]cortex-a9[/COLOR] [COLOR="Red"]<= You only need to change this to suit your device[/COLOR]

BOARD_RECOVERYIMAGE_PARTITION_SIZE := [COLOR="Red"]0x105c0000[/COLOR] [COLOR="red"]<= You need to update this so that build will error out when the size exceed your recovery partition size[/COLOR]
BOARD_FLASH_BLOCK_SIZE := 131072

TARGET_PREBUILT_KERNEL := device/TD2/9300/kernel

BOARD_MKBOOTIMG_ARGS := --base 10000000 --pagesize 2048 --kernel_offset 00008000 --ramdisk_offset 01000000 --tags_offset 00000100

[COLOR="Blue"]#MTK common twrp flags[/COLOR] [COLOR="Red"]<= You only need to change the below flags to suit your device[/COLOR]
TW_NO_EXFAT := true
TWHAVE_SELINUX := true
TW_DISABLE_TTF := true [COLOR="red"]<= Will error out on TWRP android-6.0[/COLOR]
[COLOR="red"]#[/COLOR]TW_USE_TOOLBOX := true [COLOR="red"]<= You'll need extra projects for this flag, edit the script accordingly[/COLOR]
TW_NO_EXFAT_FUSE := true
TW_THEME := [URL="https://github.com/omnirom/android_bootable_recovery/blob/android-6.0/gui/Android.mk#l96-L100"]portrait_mdpi[/URL]
[COLOR="red"]#[/COLOR]RECOVERY_SDCARD_ON_DATA := true [COLOR="red"]<= For [URL="https://twrp.me/faq/datamedia.html"]data/media[/URL] device[/COLOR]
TW_USE_MODEL_HARDWARE_ID_FOR_DEVICE_ID := true
TARGET_USE_CUSTOM_LUN_FILE_PATH := "/sys/devices/platform/mt_usb/gadget/lun%d/file"

BoardConfig.mk

# Inherit device configuration
$(call inherit-product, device/[COLOR="blue"]TD2[/COLOR]/[COLOR="blue"]9300[/COLOR]/device_[COLOR="blue"]9300[/COLOR].mk)

## Device identifier. This must come after all inclusions
PRODUCT_DEVICE := [COLOR="blue"]9300[/COLOR]
PRODUCT_NAME := cm_[COLOR="blue"]9300[/COLOR]
PRODUCT_BRAND := [COLOR="blue"]TD2[/COLOR]
PRODUCT_MODEL := [COLOR="blue"]9300[/COLOR]
PRODUCT_MANUFACTURER := TD2

$(call inherit-product, build/target/product/[URL="https://github.com/CyanogenMod/android_build/blob/cm-11.0/target/product/full_base_telephony.mk"]full_base_telephony.mk[/URL])

# Time Zone data [COLOR="red"]<= This have been disable as some device will not boot, to enable it just remove the remark [B]#[/B] below[/COLOR]
[COLOR="red"]#[/COLOR]PRODUCT_COPY_FILES += \
[COLOR="Red"]#[/COLOR]bionic/libc/zoneinfo/tzdata:recovery/root/system/usr/share/zoneinfo/tzdata

PRODUCT_BUILD_PROP_OVERRIDES += \
PRODUCT_MODEL="i9300" \ [COLOR="Red"]<= This is where Menu 2 will get the ro.product.device & ro.product.model info[/COLOR]
TARGET_DEVICE="bird77_a_cu_ics2"

PRODUCT_NAME := full_[COLOR="Blue"]9300[/COLOR]
PRODUCT_DEVICE := [COLOR="blue"]9300[/COLOR]

device_(device-name).mk & cm.mk will be auto-created based on what you have key-in for the vendor & device name at Menu 2

For building recovery, you don't need to use full.mk, using it will result in need of extra projects on the Part-Source & it will also copy the extra fstab.goldfish & ueventd.goldfish.rc to the root of the ramdisk

# mount point	fstype device	[device2]

/boot emmc /dev/bootimg
/recovery emmc /dev/recovery
/system ext4 /dev/block/mmcblk0p3
/cache ext4 /dev/block/mmcblk0p4
/data ext4 /dev/block/mmcblk0p5
/emmc	vfat /dev/block/mmcblk0p6 flags=display="Phone Storage";settingsstorage
/sdcard vfat /dev/block/mmcblk1p1 flags=display="SD Card";storage;removable
/sd-ext auto /dev/block/mmcblk1p2
/nvram emmc /dev/nvram flags=display="Nvram";backup=1
/uboot emmc /dev/uboot flags=display="Uboot";backup=1

#/boot	emmc	/dev/block/platform/mtk-msdc.0/by-name/boot	flags=display="Boot";backup=1
#/recovery	emmc	/dev/block/platform/mtk-msdc.0/by-name/recovery	flags=display="Recovery";backup=1
#/system	ext4	/dev/block/platform/mtk-msdc.0/by-name/system	flags=display="System";backup=1
#/cache	ext4	/dev/block/platform/mtk-msdc.0/by-name/cache	flags=display="Cache"
#/data	ext4	/dev/block/platform/mtk-msdc.0/by-name/userdata	flags=encryptable=/dev/block/platform/mtk-msdc.0/by-name/metadata
#/nvram	emmc	/dev/block/platform/mtk-msdc.0/by-name/nvram	flags=display="Nvram";backup=1;flashimg=1
#/external_sd	auto	/dev/block/mmcblk1p1	flags=display="External SDcard";storage;wipeingui;removable;backup=0
#/usbotg	auto	/dev/block/sda1	flags=display="Usb-Otg";storage;removable;backup=0

#/boot mtd boot
#/recovery mtd recovery
#/cache yaffs2 cache
#/data yaffs2 userdata
#/misc mtd misc
#/system yaffs2 system
#/sdcard vfat /dev/block/mmcblk0p1
#/sd-ext ext4 /dev/block/mmcblk0p2

recovery.fstab - Edit or delete whatever that is not relevant to your device. Use osm0sis AIK to unpack your stock recovery.img & TWRP-ify the recovery.fstab... :good:

#
# Copyright (C) 2013 Intel Mobile Communications GmbH
#
# Sec Class: Intel Confidential (IC)
#
# Android fstab file.
#<src> <mnt_point> <type> <mnt_flags and options> <fs_mgr_flags>
# The filesystem that contains the filesystem checker binary (typically /system) cannot
# specify MF_CHECK, and must come before any filesystems that do specify MF_CHECK
#

/dev/block/platform/soc0/e0000000.noc/by-name/ImcPartID068	/system	ext4	defaults defaults
/dev/block/platform/soc0/e0000000.noc/by-name/ImcPartID069	/data	ext4	defaults defaults
/dev/block/platform/soc0/e0000000.noc/by-name/ImcPartID070	/cache	ext4	defaults defaults
/dev/block/mmcblk1p1	/sdcard	vfat	defaults defaults
/dev/block/platform/soc0/e0000000.noc/by-name/ImcPartID076	/nvm_fs_partition	ext4 defaults defaults
/dev/block/platform/soc0/e0000000.noc/by-name/ImcPartID074	/misc	emmc	defaults defaults
/dev/block/platform/soc0/e0000000.noc/by-name/ImcPartID071	/boot	emmc	defaults defaults
/dev/block/platform/soc0/e0000000.noc/by-name/ImcPartID119	/recovery	emmc	defaults defaults
/dev/block/platform/soc0/e0000000.noc/by-name/ImcPartID120	/recoverym	emmc	defaults defaults
/dev/block/platform/soc0/e0000000.noc/by-name/ImcPartID001	/mobilevisor	emmc	defaults defaults
/dev/block/platform/soc0/e0000000.noc/by-name/ImcPartID013	/splash_screen	emmc	defaults defaults
/dev/block/platform/soc0/e0000000.noc/by-name/ImcPartID115	/mvconfig	emmc	defaults defaults
/dev/block/platform/soc0/e0000000.noc/by-name/ImcPartID118	/secvm	emmc	defaults defaults

Most latest Android devices are using fstab version 2 AKA fstab 2 while TWRP is using fstab 1. Eg. Above is my Intel X3 stock recovery.fstab

# mount point	fstype device	[device2]

/boot	emmc	/dev/block/platform/soc0/e0000000.noc/by-name/ImcPartID071
/recovery	emmc	/dev/block/platform/soc0/e0000000.noc/by-name/ImcPartID119	flags=display=recovery;backup=1
/imei	emmc	/dev/block/platform/soc0/e0000000.noc/by-name/ImcPartID022	flags=display=imei;backup=1
/mobilevisor	emmc	/dev/block/platform/soc0/e0000000.noc/by-name/ImcPartID001	flags=display=mobilevisor;backup=1
/mvconfig	emmc	/dev/block/platform/soc0/e0000000.noc/by-name/ImcPartID115	flags=display=mvconfig;backup=1
/secvm	emmc	/dev/block/platform/soc0/e0000000.noc/by-name/ImcPartID118	flags=display=secvm;backup=1
/system	ext4	/dev/block/platform/soc0/e0000000.noc/by-name/ImcPartID068
/data	ext4	/dev/block/platform/soc0/e0000000.noc/by-name/ImcPartID069
/cache	ext4	/dev/block/platform/soc0/e0000000.noc/by-name/ImcPartID070
/nvm_fs_partition	ext4	/dev/block/platform/soc0/e0000000.noc/by-name/ImcPartID076

/external_sd	vfat	/dev/block/mmcblk1p1	flags=display="Micro SDCard";storage;removable
/usbotg	auto	/dev/block/sda1	flags=display=USB-OTG;storage;removable

My TWRP-ify recovery.fstab. Typically, for ext4 partition, you don't need to specify flags=display= & backup=1, it will be taken care of by TWRP, for more info regarding fstab flags then refer to Main Thread for *RECOVERY.FSTAB*... :good:

RD osm0sis's unpackbootimg is able to extract --second <2ndbootloader-filename> & also --second_offset <address>

The repack arguments will be auto-updated to the BoardConfig.mk Thanks to RD osm0sis. Some TWRP flags ref taken from here... :good:

osm0sis's unpackbootimg also support --dt <filename> :good:

Some devices require additional info for factory update.zip then add it accordingly to the device tree, typically is your device_(device-name).mk. Look at other device tree here & here & here for reference. On CM base repo, you can also use mkvendor.sh... :good: However, you will need to manually edit certain files as no DEV has been updating it, currently it will result in a build error even when using Full-Source !

Do take note that even an extra space on some of the lines at BoardConfig.mk will result in a build error !

Menu 3

Below are the commands to build TWRP, thats it !

cd ~/omni
[COLOR="blue"]OR[/COLOR]
cd ~/cm
prebuilts/misc/linux-x86/ccache/ccache -M 25G
export USE_CCACHE=1
export LC_ALL=C
source build/envsetup.sh
lunch [COLOR="blue"]omni[/COLOR]_S939-eng [COLOR="blue"]<= S939 is the device name[/COLOR]
[COLOR="Blue"]OR[/COLOR]
lunch [COLOR="blue"]cm[/COLOR]_S939-eng
make -j4 recoveryimage

Menu 4

Below command generically will work on most Android devices. Updates : Has updated this menu to support this, just key-in /path-to-your-device-recovery-partition

adb push [COLOR="blue"]$OUT[/COLOR]/recovery.img /data/local/tmp
adb shell su -c "dd if=/data/local/tmp/recovery.img of=/[COLOR="blue"]path-to-your-device-recovery-partition[/COLOR]"

To use the $OUT path, you must first source it then lunch your device name, refer to previous Menu 3 for info.

Menu 5

Option 1 is make clean

Option 2 is wipe the whole repo base folder except the hidden .repo folder. As mention on the script GUI front-end, after that, use Menu 1 will have all the files extracted out.

Menu 6

This menu is using curl to create github repo.

It is for you to publish your device tree either for diagnostic & debugging when you ask for help. It is also for you to share a working build so that others can refer to it when they make a build of their own or git clone yours & build it themselves.