[Sony Xperia XZ1 Compact ROMs, Kernels, Recoveries,] j4nn:
building LineageOS 16.0 including TWRP for pie

I am sharing my steps used to build los16 with repos provided by [MENTION=4297219]modpunk[/MENTION] in the OP, in the hope that more devs could participate in the development.

Disclaimer: the result of this does not boot, so I am not sure at all if my howto and modifications are good or wrong.

Still hoping it might be useful to someone or maybe [MENTION=4297219]modpunk[/MENTION] or [MENTION=4063475]derf elot[/MENTION] could provide some hints to reach current status of this project?

Big thanks to [MENTION=4297219]modpunk[/MENTION] and [MENTION=4063475]derf elot[/MENTION] for working on this project.

Basically I tried to follow los15 howto as provided in the README.md of the modpunk's android_device_sony_lilac repository - just adapting it for los16 instead of los15:

initialize repo: used branch "lineage-16.0", first sync was done on 2019-03-18 with resync on 2019-03-19 (for the record of the los16, twrp and modpunk's repos source tree state)

for local manifest I've used this:

<?xml version="1.0" encoding="UTF-8"?>
<manifest>
<!-- SONY -->
<project name="cryptomilk/android_kernel_sony_msm8998" path="kernel/sony/msm8998" remote="github" revision="lineage-16.0-LA.UM.7.4.r1" />
<project name="cryptomilk/android_device_sony_common-treble" path="device/sony/common-treble" remote="github" revision="TESTING" />
<project name="cryptomilk/android_device_sony_yoshino" path="device/sony/yoshino" remote="github" revision="TESTING" />
<project name="cryptomilk/android_device_sony_lilac" path="device/sony/lilac" remote="github" revision="TESTING" />

<!-- Pinned blobs for lilac -->
<project name="cryptomilk/android_vendor_sony_lilac" path="vendor/sony/lilac" remote="github" revision="lineage-16.0" />

<!-- TWRP recovery -->
<remove-project path="bootable/recovery" name="LineageOS/android_bootable_recovery" groups="pdk" />
<project name="omnirom/android_bootable_recovery" path="bootable/recovery" remote="github" revision="android-9.0" />

<remove-project path="vendor/qcom/opensource/data-ipa-cfg-mgr" name="LineageOS/android_vendor_qcom_opensource_data-ipa-cfg-mgr" groups="qcom,pdk-qcom" />
</manifest>

Please note, I've used TESTING branch instead of lineage-16.0 as it seems that the newest changes happen there for los16 development. Similarly I tried to guess the right kernel branch based on the most recent commits: lineage-16.0-LA.UM.7.4.r1 - hopefully that is right.

Also I tried to build TWRP in the process, so replacing los recovery with upstream twrp for pie one in the local manifest.

There has been a conflict in los source tree with data-ipa-cfg-mgr that has been pulled also from modpunk's local manifest, so that is why I am removing the los one in my local manifest here.

to speed up the sync and reduce the disk space, I am using this command:

repo sync --no-tags --no-clone-bundle -c -j8

in the Extract vendor blobs step, I am using

SRC=/path/to/unpacked-fw/xz1c-pie-8.24 ./extract-files.sh

command, having full fw unpacked into plain files all readable for normal user including fixes for symlinks to be relative instead of pointing to / directory. As I was not sure about the right fw version, I've tried pie 4.45 too for the blobs extraction. During the build, some blob was missing, so I've added proprietary-files-rootfs.txt file into device/sony/lilac directory with following content:

# ROOTFS_TRIMAREA
-sbin/tad_static;rootfs

for Setup the environment, I am using following:

unset JDK_HOME
unset JAVA_HOME
unset JAVAC
export OUT_DIR_COMMON_BASE=/path/to/lineageos/output
export WITH_TWRP=true

source build/envsetup.sh
lunch lineage_lilac-userdebug

Those unset's I had to use to avoid build problem complaining about javac different versions for .mk and soong - quite difficult to find that workaround.

The OUT_DIR_COMMON_BASE is to make compiled stuff to go outside the source tree to speed up greping there.

The WITH_TWRP option seems to be needed to let some makefiles know we like to build twrp recovery.

I've tried lineage_lilac-userdebug first, after lots of fixes during the build, the result did not boot, so I tried lineage_lilac-eng lunch target too with basically the same result unfortunately.

before Build LineageOS step, you may try to apply my changes/fixes that I had to use to avoid build errors (some were quite tricky actually) - please see the attachment here for the combined patch.

This is my repo status after the changes:

project bootable/recovery/ (*** NO BRANCH ***)
-m applypatch/Android.bp
-m applypatch/applypatch.cpp
-m crypto/lollipop/cryptfs.c
-m minui/events.cpp
-- mtdutils/Android.bp
-m otautil/Android.bp
-m otautil/Android.mk
-m updater/Android.mk
-m updater/install.cpp
project build/make/ (*** NO BRANCH ***)
-m tools/releasetools/common.py
project device/sony/common-treble/ (*** NO BRANCH ***)
-m sepolicy/vendor/file.te
project device/sony/lilac/ (*** NO BRANCH ***)
-- proprietary-files-rootfs.txt
project device/sony/yoshino/ (*** NO BRANCH ***)
-m recovery/twrp.fstab
-m sepolicy/vendor/file.te
-m sepolicy/vendor/hal_livedisplay_default.te
-m sepolicy/vendor/startup-logger.te
-m sepolicy/vendor/toolbox.te
project vendor/sony/lilac/ (*** NO BRANCH ***)
-m proprietary/framework/qti-telephony-common.jar

following command would build los16 including twrp recovery:

make -j5 bacon

The built images and installation zip can be found under ${OUT_DIR_COMMON_BASE}/lineage-16.0/target/product/lilac directory:

twrp recovery image: recovery.img

kernel boot image: boot.img

los16 installation zip: lineage-16.0-20190321-UNOFFICIAL-lilac.zip

Here are the results for lineage_lilac-eng build:

tried to boot recovery.img first, got hanging on twrp splash screen.

Installed the zip file via modpunk's twrp, tried to boot it, got a hanging black screen after SONY splash screen.

Flashed userdata from stock fw to do an erase: that made built twrp bootable and actually working - installed the built los16 zip just fine.

So this twrp is the only working thing for me so far. But it seems my built twrp is not that flexible as modpunk's as mine needed userdata erase to boot while modpunk's did not.

I tried to get some logs from main los16 boot, but logcat is not reachable. Have seen also the commit to redirect logcat into /cache for charging boot mode - tried that, but the log file was not created there.

The lineage_lilac-userdebug behaved similarly if I remember correctly (tried that build first), only twrp I could not boot as the userdata erase thing has been discovered with the -eng build. But re-tested that, it helped also with -userdebug twrp build, so both variants seem to behave the same.

The blobs from pie 8.24 vs pie 4.45 did not make a difference (after a quick rebuild followed the extraction).

In order to check my build environment, I tried similarly to build sony's aosp-9.0 - it also needed some fixes, but the result has been working (needed the special oem partition image flashed as described in sony's howto to make it boot - it would hang without it).

Attached result of following command (git added the new files first):

repo diff -u >my-changes.diff

Hopefully this helps anybody who likes to participate.

Thanks for any hints.