yashsingh1/xdaforums huggingface dataset:
[Sony Xperia XZ1 Compact ROMs, Kernels, Recoveries,] j4nn:
[MENTION=9830462]Homer-J[/MENTION], things like this should be done when you have enough time not getting disturbed and after good sleep - I have a bad experience myself.

That said - if you had not backed up persist partition, your Android Attest Key would have been lost, regardless TA backup.

You should never flash persist partition. But no worries there - it rather seems to be an advantage not to have the Android Attest Key after bootloader unlock - see here for more details.

You can install 47.2.A.10.28 via fota system update if you follow the howto in kernel hiding bootloader unlock thread - you need to have full untouched stock fw in order to fota system update to work (i.e. to install actually - not only to be offered).

To revert drmfix, you need to flash both - vendor and system. There are no files to be written into system partition in the drmfix zip file, but the system partition gets mounted in rw mode and that causes changes to it already (not mentioning the empty directory which would be copied there too).

Let's see if you get widevine L1 instead of L3 when you clean up to untouched stock fw with running BLU hiding kernel.
[Sony Xperia XZ1 Compact ROMs, Kernels, Recoveries,] yamquad:
[MENTION=4418980]j4nn[/MENTION], thank you, with booting your actual hideunlock image, magisk is working. I patched it to the stock boot image, after backup according to your describtion here.

Just wondered, if TWRP has to be adapted each time the FW changes?
[Sony Xperia XZ1 Compact ROMs, Kernels, Recoveries,] Klaus N.:
[MENTION=4418980]j4nn[/MENTION]

I have a full system backup of the previous 8.24 fw via TWRP.

For some reason, I forgot to remove the persist.sin before flashing the current 10.28 fw, and therefore it's now not provisioned anymore. If I know restore the backup, will this bring the Android Attest key back? I guess, it should be part of "system" (although it is not part of/system).

Do you think, this will work?
[Sony Xperia XZ1 Compact ROMs, Kernels, Recoveries,] russel5:
If you don't have backup of persist partition, then nope
[Sony Xperia XZ1 Compact ROMs, Kernels, Recoveries,] j4nn:
[MENTION=4898998]Klaus N.[/MENTION], it's like [MENTION=4688377]russel5[/MENTION] says - system backup from twrp does not save persist partition. Unless you saved it manually before flashing it, there is no way to restore the android attest key.

But as mentioned already and deeply tested in Android Attest Key lost thread - that key might only be useful while bootloader is still locked. From the moment BL is unlocked, that key only provides information that BL has been unlocked to play store (or can be used for that purpose).

Even my BL unlock hide kernel patch would not help - as tested with modified Auditor app in the linked thread - using Android Attest Key you get signed info from Trust Zone that BL has been unlocked.

[MENTION=4688377]russel5[/MENTION], how went your testing with regard to google pay - did you get it working with magisk beta?
[Sony Xperia XZ1 Compact ROMs, Kernels, Recoveries,] russel5:
It works if I'm delete gms folder in /data/data and reboot. Problem with gpay persist only on gms 15.0.90 and upper. All other things not helping and i dont know why.
[Sony Xperia XZ1 Compact ROMs, Kernels, Recoveries,] Cillu:
Went through all the steps and did a lot of flipping through different threads to get to get back down to 16.20 while getting root, Magisk and camera still functioning. It's been more than 2 years since I've rooted any android device and it feels refreshing to use Viper4Android, Adaway, and the likes again. Thank you j4nn et al for the taking the time and resources to make this all work. One thing I forgot to migrate over properly was Google Authenticator so I thought I had lost my 2FA keys, but thankfully I still had a Ti backup from early 2017 so not all was lost.
[Sony Xperia XZ1 Compact ROMs, Kernels, Recoveries,] j4nn:
[MENTION=4902785]Cillu[/MENTION], it's good to hear that it works. I see it is quite useful for some that 16.20 kernel hiding bootloader unlock is supported.

Care to share the reasons you are staying with oreo?
[Sony Xperia XZ1 Compact ROMs, Kernels, Recoveries,] Cillu:
After upgrading to Pie my phone's performance dropped, got introduced to a huge battery drain, and I wasn't particularly fond of the UI changes. Not to mention that it was a good opportunity for me to clean up my phone after 2 years of files everywhere. Maybe my phone just needed to be cleaned up and Pie would have ran perfectly, but I still love Oreo. Thinking back, I've noticed that I've inadvertently only used the even-numbered versions of Android starting from Gingerbread, ICS, KK, Marshmallow, and now Oreo, heh.
[Sony Xperia XZ1 Compact ROMs, Kernels, Recoveries,] kuuiu:
dd if=TA-locked.img of=/dev/block/bootdevice/by-name/TA

dd: /dev/block/bootdevice/by-name/TA: Permission denied

It turns out that ./renoroot extracts the TA (because I exported the unlocked TA to the computer and restarted the phone), regardless of whether the TA was extracted before, after the extraction is completed, this command is also the command to enter the temporary root, but first After the TA is extracted, it can be restored. Restarting the video image of the mobile phone is enhanced.
[Sony Xperia XZ1 Compact ROMs, Kernels, Recoveries,] kuuiu:
Thank the author
[Sony Xperia XZ1 Compact ROMs, Kernels, Recoveries,] cyanagi:
Doesn't work on XZs，since the different CPU？

Is it possible any other way get temp root for XZs？

Or i have to unlock my phone first

thx！
[Sony Xperia XZ1 Compact ROMs, Kernels, Recoveries,] j4nn:
[MENTION=8180954]kuuiu[/MENTION], I am sorry, but I do not understand you and most likely I am not alone.

Please note, it is against this forum rules to post in other language than in English - see the rules here, point 4.

[MENTION=9838540]cyanagi[/MENTION], that's right, the exploit is designed only for the listed phones with the listed firmware versions. As already mentioned here, XZs has different cpu, different number of cores. The exploit is very sensitive to timing, so it's not possible to adapt without testing on real phone, sorry.

[MENTION=4898998]Klaus N.[/MENTION], thank you very much for your yet another donation. Your support is exceptional and really appreciated.
[Sony Xperia XZ1 Compact ROMs, Kernels, Recoveries,] j4nn:
Tested also build without parallel make (removed the -j7 option). It did not help (stock ramdisk, not modified sony source).

Patched final linking of kernel with dtb images in order to produce dtb variants in the same order as it was present in the stock kernel, to have the dtb blob binary identical - no luck:

--- edit ---

Tried with ramdisk rebuilt (unpacked stock ramdisk and re-packed it again) - to avoid possible garbage that could have been there from stock boot image split process.

According to [MENTION=5697586]moofesr[/MENTION], it still failed to boot the same as the trials before.

[MENTION=490110]janjan[/MENTION], could you please let me know which toolchain and which mkbootimg do you use for build of your kernel?

Do you use some different option with mkbootimg?

I am already out of ideas what can be wrong with this:(
[Sony Xperia XZ1 Compact ROMs, Kernels, Recoveries,] nailyk:
Hi all,

just got my xz1c. Still locked for now (am reading xda threads about it, but almost really don't care of keeping it locked)

How can I help you on this?

(right now am on .179)

BTW hi [MENTION=4120742]munjeni[/MENTION] ! Funny to see you here, with the same device this time :highfive:
[Sony Xperia XZ1 Compact ROMs, Kernels, Recoveries,] mirhl:
Munjeni still just has z1c.

He just has a big helpful heart.
[Sony Xperia XZ1 Compact ROMs, Kernels, Recoveries,] golden-guy:
I won't be able to get my hands on it before mid of next week, but has anybody tried to make an arm64 build for that POC: https://github.com/xairy/kernel-exploits/blob/master/CVE-2017-7308/poc.c

PS: Will have received my XZ1c by then as well and join the fun :)
[Sony Xperia XZ1 Compact ROMs, Kernels, Recoveries,] j4nn:
[MENTION=5473651]golden-guy[/MENTION], please check the the opening post, the CVE-2017-7308 had been already checked. You need to find userspace exploit to get NET_RAW capability first in order to be able to exploit via that CVE. Only very few processes have the needed capability. The oreo selinux setup is not nice with that regard.
[Sony Xperia XZ1 Compact ROMs, Kernels, Recoveries,] golden-guy:
I read it, but wasn't sure about the progress on this one. So something like a ping is not supported on the shell? Since these cmds are normally symlinked with the shell binary (toybox/busybox) I would have assumed this cap is given when the exploit is executed via shell.

Either way, please let me know how to best support this effort. I will continue to do some research for now.
[Sony Xperia XZ1 Compact ROMs, Kernels, Recoveries,] j4nn:
[MENTION=5473651]golden-guy[/MENTION], I am afraid that even though ping might have that cap, it does not mean the shell has it nor commands started from it.

If inotify/rename vulnerability would not go well, I see blueborne followed by CVE-2017-1000112 (memory corruption in UDP fragmentation offload) exploit as the next target.

You may check post#54 from [MENTION=3258505]4rz0[/MENTION] that seems very interesting with regard to blueborne.

There is a problem to compile a kernel that would boot stock firmware on an unlocked phone - that would be useful in general for any kernel exploit debugging/testing.

Still waiting if [MENTION=490110]janjan[/MENTION] has some idea what is wrong as he knows the right way. Or anybody else?
[Sony Xperia XZ1 Compact ROMs, Kernels, Recoveries,] nailyk:
Just unlocked mine. Feels free to send me some kernels to test on stock (with the according version to flash please)
[Sony Xperia XZ1 Compact ROMs, Kernels, Recoveries,] janjan:
Unfortunately I am away from home and my PC..( summer holidays) I can't remember which one I used. But i use to compile my own. I will post my script, steps etc when I get home.
[Sony Xperia XZ1 Compact ROMs, Kernels, Recoveries,] j4nn:
[MENTION=6979267]nailyk[/MENTION], could you please test following kernel?

boot-12.179-osvs.img

It is a build of not modified sony kernel source released for 47.1.A.12.179 fw with ramdisk extracted from that fw's kernel.sin.

It is another attempt to test kernel build procedure.

This time I've added os version info and signed it with a test key.

Thanks.
[Sony Xperia XZ1 Compact ROMs, Kernels, Recoveries,] j4nn:
Anybody here with unlocked xz1c can test a kernel to boot for me please?

It seems that everybody previously willing to test is on summer vacation.

I can build a kernel for your stock (or stock based) fw version (to avoid need of reflash/downgrade) if that makes it easier for you.

I believe the test could be done even without flashing of the kernel - using 'fastboot boot boot.img' command - to just boot the image from usb.

Let me know please. Thanks.
[Sony Xperia XZ1 Compact ROMs, Kernels, Recoveries,] blue-spiderfire:
I'm still on 47.1.A.2.2.324 and I would like to test, if your kernel wouldn't hurt my daily driver.
[Sony Xperia XZ1 Compact ROMs, Kernels, Recoveries,] Raz0Rfail:
I would help you. I have an unlocked bootloader and I'm on build .235 so my test will fail?
[Sony Xperia XZ1 Compact ROMs, Kernels, Recoveries,] j4nn:
So we have a working build procedure now (reinvented the wheel after all; -)

With Raz0Rfail we could finally boot a kernel built from 12.235 source.

The cause of bootloop with all previous build attempts (that were tested by [MENTION=5697586]moofesr[/MENTION]) was missing os version setting in kernel boot image (what a shame that sony did not include that option in their kernel's readme).

So big thanks go to both, [MENTION=5697586]moofesr[/MENTION] and [MENTION=5505196]Raz0Rfail[/MENTION] for help with the testing.

Working build procedure allowed to get first insight into inotify/rename vulnerability time relations - I've reverted the fix for the vulnerability and built a kernel patched with important points timing recording:

boot-12.235-timing.img

It can boot the 47.1.A.12.235 stock firmware.

That allowed to finally proceed with post#53 to get some timing measurements (test result from Raz0Rfail):

...
status = 0x03 (#1) [0]
rename_start = 8614972865
inotify_start + -4
fsnot_start + 214
fsnot_strlen + 249
copy_name + 5349
kfrcu_start + 5355
rename_end + 5688
kfrcu_kfree + 690549
addk_wake + 5775
fsnot_strcpy + 273
inotify_end + 1332
...

This shows that there is only 273 - 249 = 24 timer cycles long the racy/vulnerable part of the inotify handler code, the part between the strlen and strcpy there.

While the actual kfree (marked as kfrcu_kfree) was invoked many orders later (690549 - 5355 = 685194) after the kfree_rcu call, confirming the problem with delayed kfree from kfree_rcu described in post#33, post#44 and post#48. The max delay of kfree in the results from one test attempt was even 1 316 601 timer cycles, while the min delay was 182 715, so that is quite a big range (done 10 trials).

The log from qemu showed in post#73 indicates that it may eventually work, still it will be very difficult (in case of qemu test I've put artificial delay of 20000 timer cycles in between the strlen and strcpy in order to increase probability to trigger the bug) - and try to "fix" the delay/timing problem of the kfree-add_key to fill the slub entry in time (i.e. to achieve the use after free exploit). That is we need to fit the delayed kfree followed by add_key spray into the 24 cycles long vulnerable part - seems impossible right?
[Sony Xperia XZ1 Compact ROMs, Kernels, Recoveries,] j4nn:
Could someone with unlocked and rooted XZ1c post a kernel boot log here please?

I assume you could do it with something like this:

adb shell
su
cd /data/local/tmp
dmesg > dmesg.txt
chmod 644 dmesg.txt
exit
exit
adb pull /data/local/tmp/dmesg.txt

Stock based fw/kernel would be preferred, but possibly any would do.

Thanks.
[Sony Xperia XZ1 Compact ROMs, Kernels, Recoveries,] Tobi@s:
Hope it helps.

Since I'm rooted the boot image isn't stock but magisk patched
[Sony Xperia XZ1 Compact ROMs, Kernels, Recoveries,] j4nn:
[MENTION=5697586]moofesr[/MENTION] or [MENTION=5505196]Raz0Rfail[/MENTION] or anybody else with _unlocked_ bootloader on XZ1c, could you please do a test with following kernel:

boot-12.270-mvuaf.img - sources patched for another timing test, ramdisk extracted from 47.1.A.12.270 (so it needs to be used with this stock fw)

You may either flash it with fastboot, or possibly just boot it with fastboot without flashing (in such case you may verify the kernel build date in "about phone -> kernel" that it was built today to be sure the kernel actually booted via fastboot from usb instead of the stock one in flash) - see post#53 for details.

After the kernel boots, copy the attached utility to the phone via adb and start the test as bellow:

View attachment mvuaf.zip

adb push mvuaf /data/local/tmp
adb shell
cd /data/local/tmp
chmod 755 mvuaf
./mvuaf

If this gets tested by multiple persons it would be great so we have timing from multiple devices.

Thanks.
[Sony Xperia XZ1 Compact ROMs, Kernels, Recoveries,] moofesr:
Here you go
[Sony Xperia XZ1 Compact ROMs, Kernels, Recoveries,] j4nn:
Thanks to both [MENTION=5697586]moofesr[/MENTION] and [MENTION=5505196]Raz0Rfail[/MENTION] for testing.

Guys, could you please do one more test with following updated mvuaf?

Unfortunately the offset did not advance as expected.

It would be great if you could get results where ts_range at the beginning is bigger than 100 000. Please skip those when "ts_range 0 (count=10)" is present.
[Sony Xperia XZ1 Compact ROMs, Kernels, Recoveries,] Homer-J:
After flashing the current pie version G8441_47.1.A.2.324_CE1 again and flashing the kernel from here, I get again the existence of a key indicated (the second is as mentioned before negligible and the third would probably be indicated again after a downgrade to oreo) my widevine level has reached level 1 again.

Thanks to j4nn for his support.
[Sony Xperia XZ1 Compact ROMs, Kernels, Recoveries,] j4nn:
[MENTION=9830462]Homer-J[/MENTION], I believe we need to mention (after long PM discussions trying to make it work) that your use of oreo drmfix from janjan's kernel thread with pie firmware resulted in a corrupted TA so that widevine key was listed as L3 instead of L1. This corrupted state had even been recorded in the TA as retrieved again:

the 0x7e5 unit in locked original said something like this

.....p.OKq.OKt.n

while in the TA-locked_presumed.img that unit contained

.....p.CORRUPTq.CORRUPTs.CORRUPT...

(dots are unprintable characters).

Luckily the TA corruption by oreo drmfix running on pie was not fatal to the phone so it could still boot but with less drm keys even when compared to not restored TA...

Restoring the TA-locked and flashing full stock followed by use of my kernel BL unlock hiding kernel fixed everything (except obviously the Android Attest Key lost by flash of persist partition).

So anybody please be advised: follow my howtos very carefully and do not try to invent something if you do not know for sure what you are doing. It can be dangerous and you may end up with a not recoverable brick (if for example you flash garbage to TA)...

Edit: there has been a misunderstanding in this matter - it is unclear what exactly caused the TA corruption here.

Even though Homer-J mentioned flashing janjan's kernel's "drmfix" in following post:

he did not actually flash the 2-Patching.zip file that is the oreo drmfix - he flashed only the 8.24 janjan's pie kernel.
[Sony Xperia XZ1 Compact ROMs, Kernels, Recoveries,] Homer-J:
That's right, I just have to correct that I used the janjan kernel for pie after I upgraded to pie. I also forgot to exclude persist*.sin from the pie upgrade. But the problem will have been mainly caused by the double patch of the kernel.

It should be added to the description that when flashing any firmware the attest key will be lost if you don't exclude the persist*.sin.
[Sony Xperia XZ1 Compact ROMs, Kernels, Recoveries,] j4nn:
[MENTION=9830462]Homer-J[/MENTION], double flashing of any kernel has no other effect than like flashing the last kernel.

The problem in my opinion has been caused by flashing oreo drmfix (the "2-Patching.zip" file) into pie firmware and booting that.

It resulted in a TA drm keys corruption (mainly visible in widevine L3 instead of L1).

To avoid persist in flashing is explicitly mentioned in the howto in this thread even with links confirming the consequences!
[Sony Xperia XZ1 Compact ROMs, Kernels, Recoveries,] Homer-J:
By doublepatch I meant flashing first these kernel XPERIA-XZ1 Compact(G8441)-LATEST-47.2.A.8.24-(11/March/2019) what I had done under pie and then these kernel what I had done a short time later.

It is mentioned that before TA backup the persist*.sin should not be flashed, my logic told me that I have the backup of the keys for afterwards and I would be safe... the sentence in the manual should be changed so that it is clear that it always leads to the irreversible loss of the attest key flashing persist*.sin, not only before TA backup.
[Sony Xperia XZ1 Compact ROMs, Kernels, Recoveries,] j4nn:
[MENTION=9830462]Homer-J[/MENTION], I undestand your "doublepatch" of kernels - as said, it's the same as flashing the last one.

The problem you are not mentioning here is that you have flashed also the oreo's drmfix (the "2-Patching.zip" file).

Ok now I understand your point with persist, I will add some clarification in the howto.

Nevertheless it is still useless to care about that Android Attest Key on unlocked bootloader as already many times explained in detail.
[Sony Xperia XZ1 Compact ROMs, Kernels, Recoveries,] ecksdeeay:
I received a used XZPdual earlier and got up to page 36 of this thread and continuing to read through...before I get to the end is there anything I should be doing with my device (see attachments) before proceeding with the pg1 instructions. (still nervous on trashing the keys and whatnot with this my most expensive phone to date)

May need tuition/tips? if anyone cares to add before I can finish reading!

Wanting root then pie

Happy to donate if all goes well:fingers-crossed:
[Sony Xperia XZ1 Compact ROMs, Kernels, Recoveries,] j4nn:
After many PMs exchanged with [MENTION=9830462]Homer-J[/MENTION] we have come to a conclusion that there has been a misunderstanding in this TA corruption matter - it is unclear what exactly caused the TA corruption here.

Even though Homer-J mentioned flashing janjan's kernel's "drmfix" in following post:

he did not actually flash the 2-Patching.zip file that is the oreo drmfix as I've originally assumed - he flashed only the 8.24 janjan's pie kernel.

I believe that [MENTION=490110]janjan[/MENTION]'s pie kernel alone should not cause any such behaviour.

But since his kernel releases are not provided with updated kernel sources containing his changes, it is not possible to check/verify the changes he has patched into his kernel.
[Sony Xperia XZ1 Compact ROMs, Kernels, Recoveries,] j4nn:
[MENTION=4800473]ecksdeeay[/MENTION], it is nice that you read this whole thread - but you may also use a shortcut, the post#2 contains the list of the most important posts/achievements.

That may rather be useful from the exploit development point of view.

If you are only interested into backing up your drm keys with aim to unlock bootloader and restore drm - you may just follow the howtos and you should be ok, if you follow them carefully.

There is even a pdf guide linked here containing windows related hints.

Watching my videos linked in the guide may help you too.
[Sony Xperia XZ1 Compact ROMs, Kernels, Recoveries,] janjan:
What do u mean by TA corruption? It is cleared posted in my thread that the pie doesn't need any kind of drmfix. And for your kind information I didn't modified anything to edit the TA. My source are already posted with commits and with my changes, so don't blame others when things go wrong [MENTION=4418980]j4nn[/MENTION]. BTW you can take the bounty and don't need to complain and report others thread..Take it easy and enjoy your weekend.
[Sony Xperia XZ1 Compact ROMs, Kernels, Recoveries,] ecksdeeay:
I find it fascinating coming from a hardware perspective so I've read through to pg59 now and like to know how others have got on with this but most haven't mentioned which device has been used. Anyways it's late here and will have to wait to do the script part, flashing and potentially hosing/BORKING my XZPdual.

There's also another thread I may need to read through rooted-kernel-hiding-bootloader-unlock
[Sony Xperia XZ1 Compact ROMs, Kernels, Recoveries,] j4nn:
I am sorry janjan - but did you read my post?

It says:

Concerning your kernel sources - they have not been released when checked some days ago (and also discussed in PMs).

But if you released them finally, ok, that would be very kind of you to eventually comply with GPL license and the rules of this forum.

Concerning TA corruption reported by [MENTION=9830462]Homer-J[/MENTION] - his postings are unfortunately written not in very clear way. But as his screenshots show, there was certainly some problem with his TA.

We could resolve it by full restore of backed up TA-locked image together with full stock fw flash, but it is unclear what was the cause of the problem.

From his posts I've assumed (incorrectly) that he has flashed your oreo kernel's drmfix (I could imagine a TA influence in that case).

But he did not, mentioning only your pie 8.24 kernel. And as written, I do not think that your kernel alone could cause a TA problem like that.

What do you mean [MENTION=490110]janjan[/MENTION] - me to report other's threads? I did not report any thread here at all!