yashsingh1/xdaforums huggingface dataset:
[Sony Xperia XZ1 Compact ROMs, Kernels, Recoveries,] modpunk:
There is something which can be compiled. I dunno if it boots yet.
[Sony Xperia XZ1 Compact ROMs, Kernels, Recoveries,] andacro:
Read on Reddit that what's holding back LOS16 is "LineageHW deprecation in favor of new "treble-ready" HALs".

From the thread "What's holding back the LOS16 release ?" posted 9 days ago in r/LineageOS by u/FishTheFish152.
[Sony Xperia XZ1 Compact ROMs, Kernels, Recoveries,] cRaZy-bisCuiT:
Any news here? How to participate?
[Sony Xperia XZ1 Compact ROMs, Kernels, Recoveries,] derf elot:
hi there,

yes those are the right branches you are using there. I was planning to PR some of the changes needed to build to the testing branches this weekend. I'll also look more closely at your patches, but from what I can tell you fixed the neverallow selinux issues - great!

the main issue right now is still the init, I know of at least a couple of crucial things missing from stock (there is more than one fstab now, for example - this is not implemented yet), but I was waiting for modpunk to push his recent changes before working on it again. he also told me that he has some uncommitted stuff, but hasn't had the time yet to sort them out.

also, feel free to PR your patches to modpunk's git. any help is much appreciated!

edit: instead if re-adding the proprietary-files-rootfs.txt, you can just remove the references to it from extract-files.sh and setup-makefiles.sh
[Sony Xperia XZ1 Compact ROMs, Kernels, Recoveries,] j4nn:
[MENTION=4063475]derf elot[/MENTION], tried my best to fix/workaround the errors during the build, but as mentioned, I am not 100% sure about them being correct, like for example the selinux stuff changes.

Similarly I was not sure about that if tad_static is needed or not, that's why I rather added the proprietary-files-rootfs.txt.

I am wondering about the behaviour of twrp build - any idea why I needed to erase userdata to allow built twrp to boot?
[Sony Xperia XZ1 Compact ROMs, Kernels, Recoveries,] derf elot:
Not entirely sure what's going on with the twrp (some decryption problems come to mind first), but I am assuming you are using his latest twrp pie build, right (https://androidfilehost.com/?fid=11410963190603861821)?

As for building with twrp, I would recommend you clone it into bootable/recovery-twrp. this way you can build with or without twrp. also, you need to pick this topic from omnirom: https://gerrit.omnirom.org/#/q/status:open+project:android_bootable_recovery+branch:android-9.0+topic:pie-lineage
[Sony Xperia XZ1 Compact ROMs, Kernels, Recoveries,] j4nn:
Thanks for the tip with twrp omnirom patches, I will check it.

I had to replace stock recovery with the twrp one using 'recovery' directory as there was some build conflict and that was the easiest way to resolve it.

Yes, the latest pie twrp from modpunk works nice. Just wanted to check if my build of it would work too - it did not, booted and worked only after userdata erase.

Maybe with the topic patch set it would be better.
[Sony Xperia XZ1 Compact ROMs, Kernels, Recoveries,] modpunk:
You wont believe it, but [MENTION=4063475]derf elot[/MENTION] made great progress with Lineage 16.0. We have a boot animation loop but it is just a matter of time till we get that fixed.
[Sony Xperia XZ1 Compact ROMs, Kernels, Recoveries,] derf elot:
:highfive:

My goal for the weekend.
[Sony Xperia XZ1 Compact ROMs, Kernels, Recoveries,] derf elot:
Weekend is still early...but...

look at this

:good:
[Sony Xperia XZ1 Compact ROMs, Kernels, Recoveries,] HoiFung:
This day is finally here:good:.
[Sony Xperia XZ1 Compact ROMs, Kernels, Recoveries,] Pickx:
Fantastic. What other major issues are left before this build is (roughly) usable?
[Sony Xperia XZ1 Compact ROMs, Kernels, Recoveries,] derf elot:
a lot is working (like radio, sound, even camera (!)) but some aren't (like gps, nfc or wifi). haven't tested thoroughly yet, this will come today. it still needs some under the hood changes, like a rework of selinux and ironing out some other issues/bugs.

But [MENTION=4297219]modpunk[/MENTION] and I are working on it, we made some major progress in the last 5-6 days :)
[Sony Xperia XZ1 Compact ROMs, Kernels, Recoveries,] danish0175:
Finally:good::good:
[Sony Xperia XZ1 Compact ROMs, Kernels, Recoveries,] derf elot:
yes, a lot more has changed in stock moving from O to P than what we expected, so it wasn't very trivial.
[Sony Xperia XZ1 Compact ROMs, Kernels, Recoveries,] danish0175:
Thank you very much for all the work
[Sony Xperia XZ1 Compact ROMs, Kernels, Recoveries,] Myself5:
I'm pretty sure I have fixes for GPS, NFC and WiFi. Did you push everything to cryptomilks repos yet? If so I'll give it another try. Kinda lost my motivation after I had everything except for the cam working with Oreo blobs on Pie and Pie blobs (based on your early patches) wouldn't even throw an UART log.

In case you beat me to it and to save everyones time, I have everything needed for the Oreo blob Pie bringup to the Carbon Gerrit.
[Sony Xperia XZ1 Compact ROMs, Kernels, Recoveries,] .:B:.:
Whoop!
[Sony Xperia XZ1 Compact ROMs, Kernels, Recoveries,] derf elot:
I'm about to open the PR's (and do another test build with these cleaned up branches). Would be great if you could help us out, I have temporarily disabled GPS and NFC (as [WIP] commits) to allow it to boot. This is all based on stock P blobs now.
[Sony Xperia XZ1 Compact ROMs, Kernels, Recoveries,] Myself5:
alright, I'll keep an eye open. How do modpunk and you communicate? I always only ever reach him on the Lineage Slack.
[Sony Xperia XZ1 Compact ROMs, Kernels, Recoveries,] derf elot:
Messed sth. up in yoshino, still need a second. I'll PM you our freenode nicks.

---------- Post added at 01:37 PM ---------- Previous post was at 01:23 PM ----------

PR's are opened, should be all good, will test again soon

edit: made a build, so i didn't mess up the clean-up and boots fine with these branches
[Sony Xperia XZ1 Compact ROMs, Kernels, Recoveries,] modpunk:
I will work on selinux next.
[Sony Xperia XZ1 Compact ROMs, Kernels, Recoveries,] modpunk:
I've built a new TWRP today using https://github.com/minimal-manifest-twrp/platform_manifest_twrp_omni/tree/twrp-9.0

After some fixes it mostly works and also decrypts the ROM is based on Android-9.0. LineageOS next ...
[Sony Xperia XZ1 Compact ROMs, Kernels, Recoveries,] modpunk:
WORKING

RIL data

Wifi

Camera

Audio

Vibrator

Notification LED

Torch

Fingerprint sensor

Sensory (gyro, accel, etc.)

NOT WORKING

Bluetooth

GPS (extremly slow to get a fix)

Phone calls (UNTESTED)

Headphone calls (UNTESTED)

NFC
[Sony Xperia XZ1 Compact ROMs, Kernels, Recoveries,] modpunk:
Bluetooth and GPS seems to be working now.
[Sony Xperia XZ1 Compact ROMs, Kernels, Recoveries,] modpunk:
WORKING

RIL data

Wifi

Camera

Audio

Vibrator

Notification LED

Torch

Fingerprint sensor

Sensory (gyro, accel, etc.)

Bluetooth

GPS

Phone call

Headphone calls

TODO

Stock Camera App

IMS

NFC

I think I can do LineageOS 16.0 RC1 next week.
[Sony Xperia XZ1 Compact ROMs, Kernels, Recoveries,] Wyzco:
Hey just curious, when you cay "Camera works" does this utilize Raw Capture Camera2 API?
[Sony Xperia XZ1 Compact ROMs, Kernels, Recoveries,] Namnodorel:
Is there a problem with the stock camera app in P or have you just not tried to install it so far?
[Sony Xperia XZ1 Compact ROMs, Kernels, Recoveries,] modpunk:
If the camera blobs from sony support RAW then you get RAW. Till now they never supported it on the XZ1c ...
[Sony Xperia XZ1 Compact ROMs, Kernels, Recoveries,] derf elot:
Not tired yet, it's on the list of things to do. Probably will happen this weekend.
[Sony Xperia XZ1 Compact ROMs, Kernels, Recoveries,] Namnodorel:
Okay, sounds good. If any problems arise, I'd be happy to look into it again as soon as I can get my hands on a LOS16 build.
[Sony Xperia XZ1 Compact ROMs, Kernels, Recoveries,] danish0175:
To flash this when RC1 is released ,can we do that on Oreo or it is a must to upgrade to pie first?
[Sony Xperia XZ1 Compact ROMs, Kernels, Recoveries,] derf elot:
Thanks, I'll let you know how it goes. I'll also look into the possibility of removing google stuff from it if possible (and if it still does contain any).

Both are an option. modpunk opted for an in-place upgrade from 15.1 (flashing only the P fw files), I did a clean install of stock first, booted once and then started fresh with 16.0.
[Sony Xperia XZ1 Compact ROMs, Kernels, Recoveries,] tramtrist:
2 questions:

Is signature spoofing enabled for things like microg?

Do you have any idea if the work done by j4nn with TA partition can be used here? I have my TA partition and it would be nice to use it.
[Sony Xperia XZ1 Compact ROMs, Kernels, Recoveries,] modpunk:
I don't patch it to enable signature spoofing. Also if microg would be more active in providing up to date binaries I would consider it. But e.g. there hasen't been a a build of UnifiedNlp in two or three years. Also I think there is no build with the Pie API level yet.

There are no hacks needed for the camera, it just works.
[Sony Xperia XZ1 Compact ROMs, Kernels, Recoveries,] tramtrist:
I didn't realize UnifiedNlp was so far behind... Oh well
[Sony Xperia XZ1 Compact ROMs, Kernels, Recoveries,] danish0175:
Pls upload the P fw files also
[Sony Xperia XZ1 Compact ROMs, Kernels, Recoveries,] modpunk:
Wow, took me the whole day to find out that:

persist.vendor.radio.apm_sim_not_pwdn=1

breaks initializing the SIM. I already thought that my phone is broken. This normally tells to not power down the SIM in airplane mode so that it forgets the PIN unlock ...
[Sony Xperia XZ1 Compact ROMs, Kernels, Recoveries,] Myself5:
So thats why my XZ1 refused to detect a SIM...
[Sony Xperia XZ1 Compact ROMs, Kernels, Recoveries,] modpunk:
Did you do a build based on our (my and derf elots) branches?
[Sony Xperia XZ1 Compact ROMs, Kernels, Recoveries,] derf elot:
the prop was only added a couple of days ago, iirc. so if you are referring to the issue you mentioned to me a week ago or so, that shouldn't be the reason. interestingly, the prop didn't break the sim initialization on either of my devices. still a bit mysterious.
[Sony Xperia XZ1 Compact ROMs, Kernels, Recoveries,] Hrelativity:
Can I update to 16 with your lineageos15.1?

Support you and modpunk!
[Sony Xperia XZ1 Compact ROMs, Kernels, Recoveries,] mcgi5sr2:
Are there any non device commits I would need from the lineage gerrit to get this working? Happy to try and get a build up and do some QA work. Could also work on a guide to help get others up and testing if thats something that would help
[Sony Xperia XZ1 Compact ROMs, Kernels, Recoveries,] Myself5:
Yup, that's what I'm using.

Ah damn, guess I need to keep on looking then.
[Sony Xperia XZ1 Compact ROMs, Kernels, Recoveries,] j4nn:
There are some doubts about camera quality to be the same on pie after unlock vs still locked oreo - please see the the discussion since post#25 in the "[XZp] rooted kernel hiding bootloader unlock with working fota" thread. Some even tried to post comparison photos.

For this reason (and to be on the safe side) it may be useful to include my kernel patch that hides bootloader unlock state, available here on my github.

With that kernel patch you would not need to patch secd and phones with restored TA could benefit some drm features if some sony blobs were included in los build (if some drm stuff is still present in them).
[Sony Xperia XZ1 Compact ROMs, Kernels, Recoveries,] derf elot:
thanks, I will try this. I presume the dmverity patch is not needed in our case? in that case, would you mind if I picked your commit and just keep the cmdline and qseecom changes?
[Sony Xperia XZ1 Compact ROMs, Kernels, Recoveries,] j4nn:
I would recommend to cherry-pick the whole commit as is.

The change in the drivers/md/dm-android-verity.c just compensates the command line fix ups - having the change there allows the dm verity driver to behave the same way as the fix ups of command line would not be done, that is accepting not matching verity metadata.

The commit as a whole has been used already for quite some time in [MENTION=7836278]LazerL0rd[/MENTION]'s xzp kernel:

https://gitlab.com/ZKP/maple/commit/1b5ba79228147cd564f6d510dd8ef13333709c02

It has also been verified to work with system/vendor changes if dm verity is disabled in device tree, as documented in my "rooted kernel hiding bootloader unlock with working fota" threads "alternate use" (dm verity in device tree can be disabled by use of magisk manager but you probably have that already in your kernel source tree).
[Sony Xperia XZ1 Compact ROMs, Kernels, Recoveries,] derf elot:
No need to pull anything from gerrit, it should compile and work out of the box.
[Sony Xperia XZ1 Compact ROMs, Kernels, Recoveries,] mcgi5sr2:
running the extract-files.sh on the extracted ramdisk, system and vendor images I found a couple of files that seem to be in the wrong place.

actually in /lib/firmware/ufs/

my Arch system seems to think its a segaCD image /shrug

actually in /sbin/ffu

also, I am unable to get envsetup.sh to find the device files, going to look at this more tomorrow, but for now getting the issue below when trying to lunch:

and this output from envsetup.sh which I was sure used to list the added devices from roomservice.xml
[Sony Xperia XZ1 Compact ROMs, Kernels, Recoveries,] derf elot:
yes,. we moved them, they are pinned and should be in vendor_sony_lilac already. did you forget to sync that?
[Sony Xperia XZ1 Compact ROMs, Kernels, Recoveries,] mcgi5sr2:
I noticed them in there, should files that are pinned be extracted, ie should we remove them from the list of files being extracted as they are provided in repo?
[Sony Xperia XZ1 Compact ROMs, Kernels, Recoveries,] derf elot:
no, they should be in the list. but with a sha1sum of the one in the vendor repo.
[Sony Xperia XZ1 Compact ROMs, Kernels, Recoveries,] mcgi5sr2:
Thanks for explaining appreciate it.

The build issues are exclusive to Arch linux, I suspect its a jdk issue.
[Sony Xperia XZ1 Compact ROMs, Kernels, Recoveries,] j4nn:
I can confirm this. Built it today (or now I should say yesterday ;) on the first trial and could boot it no problem (did not test further yet).

Thank you [MENTION=4297219]modpunk[/MENTION] and [MENTION=4063475]derf elot[/MENTION].

Unfortunately it seems not to be that simple with twrp build. Tried to use the omni minimal manifest as hinted here, adding local manifest with modpunk's repositories using twrp-9.0 branch where available (lineage-16.0 otherwise), but that failed.

I did not have the time to try to analyse and fix the problems, just wondering, maybe you have some quick hint, like should I checkout something from somewhere in addition to normal repo sync?
[Sony Xperia XZ1 Compact ROMs, Kernels, Recoveries,] derf elot:
for twrp, doing what you did should be fine. in addition, you need to delete Android.bp from the kernel root dir and pick these two commits:

https://gerrit.omnirom.org/#/c/android_bootable_recovery/+/32819/

https://gerrit.omnirom.org/#/c/android_bootable_recovery/+/32957/

then it should work.
[Sony Xperia XZ1 Compact ROMs, Kernels, Recoveries,] j4nn:
[MENTION=4063475]derf elot[/MENTION], thanks for the hint with those two gerrit commits.

However I cannot confirm that those two commits together with kernel's Android.bp removal would allow to build twrp.

Here is what I needed to do in addition to succeed with the build:

- added device/qcom/sepolicy* from los16 into local manifest

- added vendor/qcom/opensource/* from los16 into the local manifest

- actually needed (modified) kernel's Android.bp in order to be able to install qseecom kernel headers needed during the build

- cannot use OUT_DIR_COMMON_BASE to move build out directory out of the repo tree due to kernel's Android.bp bug

Attached you can find a diff with changes I had to use.

With that twrp could be built in omni minimal tree and seems to work, including decryption, adb, mtp and zip installation of los16.
[Sony Xperia XZ1 Compact ROMs, Kernels, Recoveries,] derf elot:
Well, I am not using the minimal but full Omni trees, and for me it works fine. But maybe I should have posted my local manifest as well :p

<?xml version="1.0" encoding="UTF-8"?>
<manifest>
<!-- SONY -->
<project name="cryptomilk/android_kernel_sony_msm8998" path="kernel/sony/msm8998" remote="github" revision="lineage-16.0" />

<project name="cryptomilk/android_device_sony_common-treble" path="device/sony/common-treble" remote="github" revision="twrp-9.0" />
<project name="cryptomilk/android_device_sony_yoshino" path="device/sony/yoshino" remote="github" revision="twrp-9.0" />
<project name="cryptomilk/android_device_sony_lilac" path="device/sony/lilac" remote="github" revision="twrp-9.0" />
<project name="cryptomilk/android_vendor_sony_lilac" path="vendor/sony/lilac" remote="github" revision="lineage-16.0" />

<!-- QCOM -->
<project name="omnirom/android_device_qcom_caf-sepolicy" path="device/qcom/sepolicy" remote="github" revision="android-9.0" />
<project name="omnirom/android_vendor_qcom_opensource_commonsys" path="vendor/qcom/opensource/commonsys" remote="github" revision="android-9.0" />

<!-- TWRP -->
<remove-project name="platform/bootable/recovery" />
<project name="omnirom/android_bootable_recovery" path="bootable/recovery" remote="github" revision="android-9.0" />
</manifest>

With that, the two commits and deleting the kernel .bp works for me. android_vendor_qcom_opensource_commonsys contains both the qseecom headers and cryptfs_hw.
[Sony Xperia XZ1 Compact ROMs, Kernels, Recoveries,] j4nn:
[MENTION=4063475]derf elot[/MENTION], thanks - that probably would have helped.

I tried to use the omni minimal twrp tree due to modpunk mentioning it:

Anyway, I wonder - can we build it within los16 tree instead?

I assume there is some reason for it's move from los16 (or rather los15?) to omni (full or minimal).

Another question: I see quite a lot of libs and other stuff in los to be pinned - are they patched in some way, being different than what can be retrieved from the fw?

Or is there another reason for the pinning and committing them in the android_vendor_sony_lilac?
[Sony Xperia XZ1 Compact ROMs, Kernels, Recoveries,] derf elot:
building in los is getting quite problematic, we didn't manage to get it to work.

as for the pinned stuff, there are different reasons for it. for ffu, there was an issue with the keeping it in ramdisk, so I modified it to load the lib from /vendor and also put the binary there. it doesn't hurt to keep blobs out of ramdisk and in /vendor anyway (because treble (tm)). for the SDM stuff, those are updated binaries which are needed for los livedisplay [MENTION=4413955]2.0[/MENTION] and not available in the stock ROM. the camera libs were extracted from the stock apk's and pinned to the appropriate folders.
[Sony Xperia XZ1 Compact ROMs, Kernels, Recoveries,] modpunk:
a) You need pathmap support and this doesn't work with Android.bp files, so this is not going to happen in LineageOS

b) Because of a) there are always conflicts with the LineageOS recovery

b) the keyring patch for init is not in LineageOS

So I just started to use the minimal TWRP tree ...
[Sony Xperia XZ1 Compact ROMs, Kernels, Recoveries,] modpunk:
[MENTION=4063475]derf elot[/MENTION]

My phone rebooted out of the blue yesterday. Looking at the kernel logs, it was a modem crash ...

[20725.884954] modem subsystem failure reason: dog_hb.c:266:DOG_HB detects starvation of task 0x31bfd7e0, triage with its own.
[20725.885015] subsys-restart: subsystem_restart_dev(): Restart sequence requested for modem, restart_level = SYSTEM.
[20725.895241] last_subsyslog last_subsyslog: Could not read SMEM ID. NULL
[20725.895273] last_subsyslog last_subsyslog: Failed to extract modem error log
[20726.006432] Kernel panic - not syncing: subsys-restart: Resetting the SoC - modem crashed.
[Sony Xperia XZ1 Compact ROMs, Kernels, Recoveries,] j4nn:
[MENTION=4063475]derf elot[/MENTION], I can confirm that with your local manifest (together with your previous hint) twrp can be built easily within omni minimal tree.

Thank you.
[Sony Xperia XZ1 Compact ROMs, Kernels, Recoveries,] .:B:.:
Trying to build the 16.0 tree myself at the moment, but getting stuck at this. Can I just copy over the 15.1 one or is this a newer APK on 16.0?

ninja: error: 'vendor/sony/lilac/proprietary/app/CameraAddonPermission-release/CameraAddonPermission-release.apk', needed by '/var/media/data/code/lineage/16.0/out/target/product/lilac/obj/APPS/CameraAddonPermission-release_intermediates/package.apk', missing and no known rule to make it

Already pulled the blobs from the RC0 but I checked the system.img contents, the APK is not in there.
[Sony Xperia XZ1 Compact ROMs, Kernels, Recoveries,] j4nn:
[MENTION=7927349].:B:.[/MENTION], I guess you need to pull it from phone running the needed stock fw version or extract the sin files of that stock fw to pull everything from a directory tree (using the SRC environment variable with the extract-files.sh script).
[Sony Xperia XZ1 Compact ROMs, Kernels, Recoveries,] j4nn:
Please check my post here comparing photo quality between still locked xz1c oreo fw with unlocked lost drm keys running los16 with stock camera blobs.
[Sony Xperia XZ1 Compact ROMs, Kernels, Recoveries,] beggar23:
Ewww, that's a bit disappointing, tbh.

(I am blaming Sony, just for the record.)
[Sony Xperia XZ1 Compact ROMs, Kernels, Recoveries,] tramtrist:
Thanks [MENTION=4418980]j4nn[/MENTION] ... I think your work should be included if possible
[Sony Xperia XZ1 Compact ROMs, Kernels, Recoveries,] russel5:
Hi bro, do you find a fix?

Im made some research and find this commit --> here

How do you think if we a change ssr script to restart all subsystem to RELATED, im think this not restart our phones. Seems problem in sources, because on AEX some people have similar bug. AEX used lineage sources.
[Sony Xperia XZ1 Compact ROMs, Kernels, Recoveries,] modpunk:
This is handled by ssr_setup service now. It was missing in our init. I've added that now, however I don't find anything which actually starts this as it is trigged by a property. Nothing seems to set this property.
[Sony Xperia XZ1 Compact ROMs, Kernels, Recoveries,] modpunk:
Awesome, ssr_setup is there but not configured at all. It looks like /system/etc/init.ssr_level.sh is setting those. I think SELinux prevents to set those. So the best thing would be to put them in an init file.
[Sony Xperia XZ1 Compact ROMs, Kernels, Recoveries,] modpunk:
After more digging, I think the right thing would be to set the property:

persist.vendor.ssr.restart_level=venus,adsp,slpi,modem,cdsp

Then ssr_setup will configure it.
[Sony Xperia XZ1 Compact ROMs, Kernels, Recoveries,] russel5:
Anyone can tell me, what need to change in sources of stock kernel to enable reboot into recovery from power menu.

Find it in Sony's aosp kernel sources

[MENTION=7034101]Namnodorel[/MENTION] developer settings? :laugh:
[Sony Xperia XZ1 Compact ROMs, Kernels, Recoveries,] Namnodorel:
Afaik that isn't something to set in the source, but a setting you need to toggle in the developer settings.
[Sony Xperia XZ1 Compact ROMs, Kernels, Recoveries,] Namnodorel:
Yeah. There is a setting there called "Advanced Restart", which enables the buttons for restarting into both recovery and bootloader.
[Sony Xperia XZ1 Compact ROMs, Kernels, Recoveries,] j4nn:
[MENTION=4063475]derf elot[/MENTION], could you please briefly explain what does it mean to LOS'ify a (sony stock or stock based) kernel?

Thank you.