yashsingh1/xdaforums huggingface dataset:
[Sony Xperia XZ1 Compact ROMs, Kernels, Recoveries,] j4nn:
[MENTION=4297219]modpunk[/MENTION], well libsuntory is statically linked into several binaries, so it's not that easy to override it's functions - as far as I understood, your "secd extension" served as a drmfix, containing lib injected in order to provide faked suntory credentials (possibly coming already from Tobias) and intercepted suntory functions making those (phone specific) credentials look valid. Having libsuntory compiled in binaries statically as it's now the case with pie fw, you would need to patch all that stuff into the binaries, instead of simply preloading a lib.

Also secd is not the only binary that checks such stuff on pie. There are more directly accessing tzxflattest and tzsuntory tz app as far as it seems to me...
[Sony Xperia XZ1 Compact ROMs, Kernels, Recoveries,] j4nn:
A donation was received on Mon Apr 29 13:05:23 UTC 2019 from somebody I cannot associate a xda user with. Thank you.
[Sony Xperia XZ1 Compact ROMs, Kernels, Recoveries,] j4nn:
xz1c locked oreo vs unlocked pie comparison

I tried to compare still locked XZ1c oreo (47.1.A.8.49) with unlocked-and-lost-drm-keys xz1c stock camera (LOS16 with camera blobs from 47.2.A.10.45 as in the current los testing).

Please see attached screenshots for spots to check and sample showing some differences.

Please see here for the original photo files for comparison - tried it two times (you need to avoid zoom out to see true pixels):

190504-110836-talost-dsc_0011.jpg vs 190504-110838-locked-dsc_0062.jpg

190504-111842-locked-dsc_0068.jpg vs 190504-111844-talost-dsc_0018.jpg

The two xz1c phones have been hold still as seen on two-xz1c-cmp-placement.jpg picture attached (shown for illustration) - but on a bench of a playground, holding it still on the bench with one finger touching capture button on the front phone and then on the second phone, while both have been set to 10 seconds timer to avoid phone movements during the capture.

In the first trial the front xz1c has been the one running los16 with stock blobs camera, the second one still locked oreo xz1c.

In the second trial the order has changed - front one being the locked oreo and the second one lost drm keys los16.

Exif metadata are practically the same at least for the first trial, differing only in the software version (and timestamps, - being los16, + being locked oreo):

-Software |PQ2A.190405.003_0_a600
-Date and Time |2019:05:04 11:08:36
+Software |47.1.A.8.49_0_a600
+Date and Time |2019:05:04 11:08:38

The second trial differs a bit with these properties in addition (- being the locked, + being the talost):

-Exposure Time |1/2500 sec.
+Exposure Time |1/3200 sec.
...
-Shutter Speed |11.28 EV (1/2486 sec.)
+Shutter Speed |11.64 EV (1/3191 sec.)
...
-Metering Mode |Pattern
+Metering Mode |Unknown

Did not have much time to do more comparisons, these results still may be a coincidental even though we can see similar quality differences in both of the trials.

But I tried my best to eliminate both xz1c movements (I have no tripod and would need two anyway) so I believe they were really still and to take comparison photos at practically the same time having two phones
[Sony Xperia XZ1 Compact ROMs, Kernels, Recoveries,] j4nn:
With the above camera quality comparison test, camera settings as displayed in the attached screenshots have been used.
[Sony Xperia XZ1 Compact ROMs, Kernels, Recoveries,] mad0701:
I've make a short compare between 0011 & 0062:

The angel was not the same: in pic 0062 you can see the house on the right side over the fence. This could be also another focus point and in this case also another sharpness in another area of the pic. Also the antenna is not visible in Pic 0011

I have look on my phone and the only possibility to change the focus to MF is in the settings direct before the Shot.

I'll believe, a compare must be really make from the same position, same angle, same parameter and manual Focus.
[Sony Xperia XZ1 Compact ROMs, Kernels, Recoveries,] j4nn:
[MENTION=7055162]mad0701[/MENTION], you are right, it is not exactly the same, but it is very similar - the two phones were in the position as shown on the illustration next to each other without any space between them, so the angel should have been practically the same.

With the second trial the order of the two phones has been switched and we get similar differences - not only in quality but also in the angle as you have mentioned (but this time switched).

The problem is that when you might achieve the same angle with a tripod, you cannot get the same conditions if you do not capture it at the same time - sun light can drastically change in a moment.

If you try to get same conditions capturing at the same time you cannot have exactly the same setup/angle with two phones at that same time obviously.

Just compare both trials, even cross compare all the 4 photos (originals on androidfilehost).
[Sony Xperia XZ1 Compact ROMs, Kernels, Recoveries,] modpunk:
When I load them in GIMP and put one over the other they look equally bad. That's simply a smart phone camera sensor and the sensor isn't the best.

So I can't see any degrade in camera quality with LOS16.
[Sony Xperia XZ1 Compact ROMs, Kernels, Recoveries,] j4nn:
This is not about LOS16 - I just use it because it has been the last thing I tested.

This is about stock camera, possibly not having the same quality with unlocked (and not masked) phone with lost drm keys.

Please, just check the window spot marked in the thumbnail picture - that spot is present in all 4 images. With locked oreo, you can see more details, i.e. there are visible horizontal interior window curtains (jalousie).

The quality of the TV antenna on the house roof is also clearly different, but present only in two of the 4 images.
[Sony Xperia XZ1 Compact ROMs, Kernels, Recoveries,] Cluey:
Do I need to use the newflasher tool or can I just use Flashtool?
[Sony Xperia XZ1 Compact ROMs, Kernels, Recoveries,] nussbaum:
Use newflasher only.
[Sony Xperia XZ1 Compact ROMs, Kernels, Recoveries,] j4nn:
A generous donation was received at 15:39 UTC today from somebody I cannot associate a xda user with. Thank you very much for your support.
[Sony Xperia XZ1 Compact ROMs, Kernels, Recoveries,] gomifromparis:
Hi,

I rooted my Z1C thanks to your awesome work and [MENTION=4976984]alagga[/MENTION] pdf. Thanks very much to you both ! I'm running the 1.16.20 version.

I am encountering some Magisk trouble though.

I want to play with/use a couple modules , but I get mixed result.

The High Power LED module loads and works (https://xdaforums.com/showpost.php?p=78677693&postcount=32), but the others I tried (Greenify3Magisk, sysconfig-patcher201904060, GMS Doze) won't.

They install OK but don't appear in the list of installed modules, and obviously don't load nor do anything.

So I tried to upgrade to Magisk 19.1 from Magisk Manager but got a bootloop, so I patched the stock boot I had backed up thanks to [MENTION=4976984]alagga[/MENTION] :)

With this patched Magisk 19.1 kernel, the modules that weren't appearing in the modules list now appear in it (without having reinstalled them), but the High Power LED module now breaks the torchlight (the toggle button won't work).

And of course all the Sony apps are now crashing , so it seems I'd better stay with your FOTA kernel for now !

Any idea ? Maybe I need to repackage the not working modules to the same format as the one that is working (the High Power LED one ?)

Thanks again
[Sony Xperia XZ1 Compact ROMs, Kernels, Recoveries,] alagga:
I'd suggest to not update Magisk Version.

Better wait till [MENTION=4418980]j4nn[/MENTION] updates his kernel with the newest magisk Version.

(If you use stock boot and patch it with magisk you don't get the hideunlock features)

For the not showing modules I can't really help, sorry. (I guess it could be caused if the modules are designed for the newer magisk Version)
[Sony Xperia XZ1 Compact ROMs, Kernels, Recoveries,] gomifromparis:
ok thanks
[Sony Xperia XZ1 Compact ROMs, Kernels, Recoveries,] gomifromparis:
Actually I don't think the High LED module is working. I see the files in /data/adb but they don't seem to get merged in /system. And the led output is still low ...

Did anyone manage to install and use a magisk module ? I see everyone uses is for root management but didn't find things on using modules, or I missed it ?

Thanks
[Sony Xperia XZ1 Compact ROMs, Kernels, Recoveries,] gomifromparis:
Another strange phenomenon, when you connect logcat through adb, the phone seems to be in a crashloop of some process getting respawned and recrashing. Dunno if it's supposed to happen but there is no such thing when you reflash the stock kernel.

I attach 2 iternations of the crash, seems to be the ric ??

05-09 23:48:36.925 8310 8310 W ric : type=1400 audit(0.0:10659): avc: denied { noatsecure } for scontext=u:r:init:s0 tcontext=u:r:ric:s0 tclass=process permissive=0 ppid=1 pcomm="init" pgid=1 pgcomm="init"

05-09 23:48:36.935 8312 8312 W crash_dump64: type=1400 audit(0.0:10661): avc: denied { noatsecure } for scontext=u:r:ric:s0 tcontext=u:r:crash_dump:s0 tclass=process permissive=0 ppid=8311 pcomm="ric" pgid=8310 pgcomm="ric"

05-09 23:48:36.942 8310 8310 F libc : unable to stat "/proc/self/exe": Permission denied

05-09 23:48:36.942 8310 8310 F libc : Fatal signal 6 (SIGABRT), code -6 in tid 8310 (ric)

05-09 23:48:36.964 8313 8313 I crash_dump64: obtaining output fd from tombstoned

05-09 23:48:36.965 1176 1176 I /system/bin/tombstoned: received crash request for pid 8310

05-09 23:48:36.965 8313 8313 I crash_dump64: performing dump of process 8310 (target tid = 8310)

05-09 23:48:36.966 8313 8313 F DEBUG : *** *** *** *** *** *** *** *** *** *** *** *** *** *** *** ***

05-09 23:48:36.966 8313 8313 F DEBUG : Build fingerprint: 'Sony/G8441/G8441:8.0.0/47.1.A.16.20/623594567:user/release-keys'

05-09 23:48:36.966 8313 8313 F DEBUG : Revision: '0'

05-09 23:48:36.966 8313 8313 F DEBUG : ABI: 'arm64'

05-09 23:48:36.966 8313 8313 F DEBUG : pid: 8310, tid: 8310, name: ric >>> /system/bin/sh <<<

05-09 23:48:36.966 8313 8313 F DEBUG : signal 6 (SIGABRT), code -6 (SI_TKILL), fault addr --------

05-09 23:48:36.967 8313 8313 F DEBUG : Abort message: 'unable to stat "/proc/self/exe": Permission denied'

Sorry for the annoyance if all this is expected
[Sony Xperia XZ1 Compact ROMs, Kernels, Recoveries,] Gaaat:
I'm using an Xz Premium with j4nn latest patched kernel and magisk 19.1, I've not encountered any issue updating from magisk 18.1 neither using my modules, I even created some modules to "push" APKs to system with no problem, maybe the module you're trying to install is for an outdated version of Magisk or simply is not compatible with android pie
[Sony Xperia XZ1 Compact ROMs, Kernels, Recoveries,] j4nn:
[MENTION=727558]gomifromparis[/MENTION], [MENTION=4976984]alagga[/MENTION], try this please:

Use this to patch with your magisk version:

boot-*-47.2.A.10.45-hideunlock.img
[Sony Xperia XZ1 Compact ROMs, Kernels, Recoveries,] gomifromparis:
you mean the magisk V19 is only available on Pie ?
[Sony Xperia XZ1 Compact ROMs, Kernels, Recoveries,] danish0175:
Can I use this on los16?
[Sony Xperia XZ1 Compact ROMs, Kernels, Recoveries,] Gaaat:
No, magisk is more or less compatible with every Android version.

I mean that your High Led module can be incompatible with the latest firmware version, but who knows, you should probably ask on that module's thread
[Sony Xperia XZ1 Compact ROMs, Kernels, Recoveries,] gomifromparis:
oups sorry I was commenting on [MENTION=4418980]j4nn[/MENTION] " use boot-*-47.2.A.10.45-hideunlock.img ", but I am on Oreo so 47.1.A.16.20
[Sony Xperia XZ1 Compact ROMs, Kernels, Recoveries,] gomifromparis:
never mind, I couldn't manage to get over the logcat crashloop with the fota kernel, and couldn't update magisk version, so I flashed the 16.20 firmware back again and magisked it through TWRP . Now everything OK (except Sony stuff of course) and don't get the crash loop any more in logcat.

and I keep my restored DRM keys !
[Sony Xperia XZ1 Compact ROMs, Kernels, Recoveries,] j4nn:
Restored TA (drm keys) is useful on any firmware, but some stuff may not be enabled if bootloader unlock is detected - depends on the rom and/or used kernel.

I am not sure what you mean by "this"?

From your posts I did not get you have been using oreo 16.20 firmware.

Unfortunately oreo needs specially patched magisk to make proper bootloader unlock hide working - there is the sony ric daemon which detects "root" aka magisk components in case it thinks bootloader is still locked (which it does in case kernel hides that).

So for oreo you have to use the pre-rooted kernel, which has patched magisk init integrated. For that you may be limited to use magisk modules compatible with the integrated magisk version.

I tried to get the patch for magisk integrated into official magisk release, but the patch is still pending in the pull request here:

https://github.com/topjohnwu/Magisk/pull/1077
[Sony Xperia XZ1 Compact ROMs, Kernels, Recoveries,] gomifromparis:
OK thanks for the detailed answer. Don't you think the ric process crashing every 3 seconds (I saw it using adb logcat) might be an issue (see https://xdaforums.com/showpost.php?p=79496418&postcount=927 ) ?

I was frightened this would eat up some power, that's why I went with full magisk on original kernal.

Might try Pie , but I don't like upgrading major OS versions on phones, my Z3C was useless (too much power use) after the 5.1 upgrade from 4.4

Thanks again
[Sony Xperia XZ1 Compact ROMs, Kernels, Recoveries,] j4nn:
[MENTION=727558]gomifromparis[/MENTION], I agree, crashing ric every 3 seconds is not normal. But I wonder under what circumstances does it happen.

The boot-*-47.1.A.16.20-hideunlock-rooted.img kernel is pre-rooted by patched magisk v18.1. Could you please test it together with MagiskManager-v7.0.0?

That means having not modified stock 16.20 firmware used with the boot-*-47.1.A.16.20-hideunlock-rooted.img kernel (do not flash magisk from twrp, kernel is pre-rooted so there is no need), just install manually MagiskManager-v7.0.0.apk. Do not install any magisk modules. And check if ric crashes or not.
[Sony Xperia XZ1 Compact ROMs, Kernels, Recoveries,] wyt18:
I used the temp root process while on Oreo to backup and restore DRM keys; then I upgraded to the latest .45 stock rom via newflasher. What is the best way now to implement bootloader unlock hide?
[Sony Xperia XZ1 Compact ROMs, Kernels, Recoveries,] gomifromparis:
[MENTION=500933]wyt18[/MENTION] here is what I do

in fastboot mode, do

fastboot reboot bootloader

fastboot flash boot thekernelfile.img

---------- Post added at 10:11 AM ---------- Previous post was at 09:14 AM ----------

OK reinstalled the stock firmware and the boot-*-47.1.A.16.20-hideunlock-rooted.img and Magisk 7.0.0 and the ric is in crashloop in adb logcat
[Sony Xperia XZ1 Compact ROMs, Kernels, Recoveries,] Geneaux:
Can this be adapted for a G8343 Xperia XZ1 with Android 8(build: 47.1.A.2.313)? The settings say the security patch level is August 2017. My thought process is that this is likely as vulnerable as it can get for the exploit, but it still failed to get the TA partition.
[Sony Xperia XZ1 Compact ROMs, Kernels, Recoveries,] j4nn:
[MENTION=727558]gomifromparis[/MENTION], I will test it. It worked for me, but I am not exactly sure which versions of magisk had been used.

[MENTION=5531545]Geneaux[/MENTION], just upgrade to the 47.1.A.2.324_CE1 version and it should work. You may skip flashing the customization files if that's what you worry about (oem.sin).
[Sony Xperia XZ1 Compact ROMs, Kernels, Recoveries,] gomifromparis:
[MENTION=4418980]j4nn[/MENTION], please don't use valuable time for this, I moved on to stock kernel + magisk and it works ok for me.
[Sony Xperia XZ1 Compact ROMs, Kernels, Recoveries,] ojtsss:
I messed up.

I tried to follow the guide from user 'munted'. I started with software version 47.2.A.10.28. and used newflasher.exe to flash 47.1.A.8.49_CE1 with persist und*.ta files deleted.

When I checked the Security info in the Service Menu there were none.

Is there anything I can do?
[Sony Xperia XZ1 Compact ROMs, Kernels, Recoveries,] j4nn:
[MENTION=9926279]ojtsss[/MENTION], you have not flashed the bootloader - you can't delete .ta under boot subdirectory.

Please make sure you have complete 'boot' subdirectory there while flashing a firmware.

Incompatible bootloader results with single "None" word in service security screen.
[Sony Xperia XZ1 Compact ROMs, Kernels, Recoveries,] j4nn:
[MENTION=727558]gomifromparis[/MENTION],

I've ported my patch of magisk for sony ric daemon to magisk-19.1 and repacked oreo kernels with it - updated in post#2 and post#3 of "rooted kernel hiding bootloader unlock with working fota" threads for XZ1c, XZ1 and XZp:

boot-*-47.1.A.16.20-hideunlock-magisk-19.1.img

boot-*-47.1.A.16.20-hideunlock-magisk-19.1-noverity.img

Link to source code updated in blu hiding kernels threads OPs and created an updated pull request to upstream magisk:

https://github.com/topjohnwu/Magisk/pull/1454
[Sony Xperia XZ1 Compact ROMs, Kernels, Recoveries,] j4nn:
Thank you [MENTION=4800473]ecksdeeay[/MENTION], donation received on Tue May 14 20:54:54 UTC 2019.

Another donation received on Tue May 14 18:10:06 UTC 2019. Thank you.
[Sony Xperia XZ1 Compact ROMs, Kernels, Recoveries,] ecksdeeay:
XZP (G8141)

Downgrade and backup TA very quick with no reboots

...now on my G8142 I did UL when on the downgraded fw then used the kernel hiding bootloader for the latest piewith magisk as it was already to tap into from the system.

As I'm in no hurry I am now trolling the threads to figure out twrp install when I get round to the UL...or should I just roll with magisk?

Edit

; so after a short read (After UL and booting hide kernel) I go through enabling magisk then install twrp (using dd install Method, I guess)?
[Sony Xperia XZ1 Compact ROMs, Kernels, Recoveries,] j4nn:
[MENTION=4800473]ecksdeeay[/MENTION], very nice renoroot success time!

But I guess you still cannot beat mine here (go to the post#428 for screenshots):

:cool:
[Sony Xperia XZ1 Compact ROMs, Kernels, Recoveries,] ecksdeeay:
UL BL after install of latest pie instead of ULocking, when downgraded to the exploited oreo,

(offered j4nn my Locked TA from oreo and Unlocked TA.img from pie)

The screenshots are all from *10.45 XZP pie fw

First 4 are pre-restore of Locked TA

Then the other 4 are when the Locked TA was restored with the flashed hide kernel

Had some grief with twrp and magisk but now have them both installed so it was worth it in the end I guess!
[Sony Xperia XZ1 Compact ROMs, Kernels, Recoveries,] j4nn:
[MENTION=1987749]Aqq123[/MENTION], I believe your TA backup is all right - if camera worked on stock oreo fw any version, it indicates device master key had been restored in TA. Just make sure you flash boot/ with all files under it including .ta there with any fw.

If you want oreo, flash 16.20 and use boot-G8141-47.1.A.16.20-hideunlock-magisk-19.1.img according to the guide - alternate or not.
[Sony Xperia XZ1 Compact ROMs, Kernels, Recoveries,] Aqq123:
Thank you for your answer. You helped me a lot by ruling out the TA backup corruption.

The thing is, I tried two different .16.20 versions earlier (CE1 and DE) and the issue showed with both (while with 12.75 it didn't). In the end, I found out what the problem was (in fact the reason was trivial), and it's all working as expected now. Details follow.

* * *​

The cause was bootloader version mismatch due to the fact that it wasn't being flashed properly. The file boot_delivery.xml must be present in the ./boot/ directory (relative to newflasher.exe and the root of the unpacked firmware). In the old exploitable version 47.1.3.254 provided by [MENTION=4418980]j4nn[/MENTION], this file is properly located and, as a result, the bootloader flashes correctly. However, with this 47.1.16.20 CE1 firmware, or XperiaSite's .16.20 DE and .12.75 CE1 versions, the boot_delivery.xml file is in ./, the

root

directory of the unpacked firmware, and as a result gets

ignored

by Newflasher (using version 13, which is the latest at the moment).

It did not show up as an issue going from .3.254 and .12.75 because both share the same bootloader version (1_O_79) so it did not matter if it was being reflashed or not. However, .16.20 comes with a newer version of the bootloader (1_O_82), which is why it's essential that the bootloader is flashed.

To be clear, I am absolutely

not

blaming Newflasher for this. In fact, it logs everything wonderfully: every time flashing starts, it will query the bootloader version:

Bootloader version: 1306-5035_X_Boot_MSM8998_LA1.1_O_79

And when the bootloader wasn't flashed at the end, it said:

Processing boot delivery...

boot_delivery.xml not exist in boot folder or no boot folder.

Resolving this would have been faster if I paid attention to these messages earlier. But in fact (and it's nothing to be proud of) I came up with all these conclusions completely independently and only then did I start looking at the logs to confirm my suspicions. And the messages were there all the time, in plain sight. The problem was between the keyboard and the chair. The takeaway is that it's good to look at log messages even if the process is supposedly successful.

In fact, for people familiar with this device (unlike me), it would have been even easier to spot what was going on, as the "old" _79 bootloader, when unlocked, shows an orange triangle with an exclamation mark saying that "your device can't be trusted," and also pauses for a while. On the other hand, the "new" _82 bootloader displays an open padlock asking the user to relock the bootloader again (which is idiotic, as there is no way to do this, at least not officially) but at least it doesn't seem to pause for as long. So there is a clear, visible difference between the two in the booting process.

Hopefully the above explanation helps if anyone runs into the same issue in the future. And once again thank you for all your work [MENTION=4418980]j4nn[/MENTION], what you've accomplished here is quite amazing.
[Sony Xperia XZ1 Compact ROMs, Kernels, Recoveries,] ecksdeeay:
Here we go? new fw!! (G8142 haven't checked/seen on just seen the same with my G8141 yet)

Its only a 344mb package

[MENTION=4418980]j4nn[/MENTION] just to add to the personal messages I've included the screen from when I had to install magisk in twrp
[Sony Xperia XZ1 Compact ROMs, Kernels, Recoveries,] j4nn:
unlock on pie 47.2.A.10.45 has been verified

In case anybody missed the post#949 from [MENTION=4800473]ecksdeeay[/MENTION], unlocking from 47.2.A.10.45 fw has been verified on his G8141 phone.

That means one may backup TA on exploitable oreo fw, postpone unlock for later while using current sony stock fw and when decided to unlock, just do it from any fw up to the tested 47.2.A.10.45 version (and possibly also newer but that obviously has not been verified).

TA-locked which has been backed up on exploitable oreo fw seems to work ok when restored within twrp for use with pie 47.2.A.10.45 fw.

Thanks again ecksdeeay for the test.
[Sony Xperia XZ1 Compact ROMs, Kernels, Recoveries,] j4nn:
created a poll about releases of future kernels hiding BL unlock

Please vote in your kernel thread for your phone model:

[XZ1c] rooted kernel hiding bootloader unlock with working fota

[XZ1] rooted kernel hiding bootloader unlock with working fota

[XZp] rooted kernel hiding bootloader unlock with working fota

(at the top of each forum thread page - browser needs to be used, poll seems not to appear in the labs app)
[Sony Xperia XZ1 Compact ROMs, Kernels, Recoveries,] lilbrat:
Question, how did he push it in twrp really courios?
[Sony Xperia XZ1 Compact ROMs, Kernels, Recoveries,] ecksdeeay:
Open terminal and root shell is ready for Adb commands via computer