yashsingh1/xdaforums huggingface dataset:
[Sony Xperia XZ1 Compact ROMs, Kernels, Recoveries,] j4nn:
If something is missing (not listed) in security test menu it does not mean it had been lost.

I guess you mean SOMC Attest Key - that seems not to be listed with pie, but if you downgraded back to oreo, it would get listed again. SOMC FIDO Key should be listed with pie, may not be provisioned if unlock is detected by fw. If unlock state is ignored, it would get provisioned and working again.

You may possibly test for the fido key with google account 2nd factor auth by your phone as documented here:

Android Attest Key post#68. Check it since post#61 and possibly re-test please. Thanks.
[Sony Xperia XZ1 Compact ROMs, Kernels, Recoveries,] mad0701:
It doesn't matter what is listed or not (from my personal point of view) if all things are working so as with a stock rom [emoji16]
[Sony Xperia XZ1 Compact ROMs, Kernels, Recoveries,] tramtrist:
As far as I can tell all things are working. It's listed as not provisioned but my Google 2F authentication from another PC. It all worked well... So I guess there's no problem after all
[Sony Xperia XZ1 Compact ROMs, Kernels, Recoveries,] Premium Unleaded:
I factory reset my device, then downgraded to 47.1.A.2.324_CE1 to restore my TA-locked backup via your method. Downloading a fresh copy of the 47.1.A.16.20_R18B and using that to update seems to have done the trick. Installed TWRP, flashed Magisk & the secd-ignore in recovery, the security test showed both blobs [ACTIVE], Android Key & both SOMC keys PROVISIONED and both cameras functioning normally as far as I can tell.

Looks like whatever went wrong the first time round was either during the update flashing or when I was preparing the 47.1.A.16.20 firmware for flashing. Anyway, thank you for your help & work.
[Sony Xperia XZ1 Compact ROMs, Kernels, Recoveries,] TempleZhou:
Awesome!! Thinks a lot. I flashed it to my rooted XZ1 with FW 2.33(Android Pie) and everything worked as unlocked BL including video image enhancement.
[Sony Xperia XZ1 Compact ROMs, Kernels, Recoveries,] ParkourPaul:
Must have missed this post before. Installed this patched secd and all keys are now provisioned and OK. Video enhancement has been restored. Thank you good sir.

Edit: OTA came, downloaded but it fails after the update. Any idea what kind of fcukery would be involved to be able to update via OTA?
[Sony Xperia XZ1 Compact ROMs, Kernels, Recoveries,] mad0701:
It's already known, that's a rooted device fails during OTA update. Wait that the version is ready in xperia firm, DL, delete ta and specific sin files and flash them via newflasher
[Sony Xperia XZ1 Compact ROMs, Kernels, Recoveries,] ParkourPaul:
Yes but removing magisk and root still does not allow OTA. Does the patched secd stop OTA?
[Sony Xperia XZ1 Compact ROMs, Kernels, Recoveries,] vofferz:
My guess is that the unlocked bootloader itself stops OTA from working. So probably nothing you can do but to update manually.
[Sony Xperia XZ1 Compact ROMs, Kernels, Recoveries,] ParkourPaul:
I can't decide between 1080p 960fps + latest security Pie or tried and tested Oreo + xposed. Mmmm.
[Sony Xperia XZ1 Compact ROMs, Kernels, Recoveries,] vofferz:
Pie seems to work fine, at least for me. Of course if you need some modules only xposed has, it's a different story. I decided I can go without as adblocking seems to work and magisk brings enough features. On the plus side I can now pass safetynet and use banking apps without uninstalling xposed and rebooting...
[Sony Xperia XZ1 Compact ROMs, Kernels, Recoveries,] arslancn:
i think 1080p 960fps not a big deal because of low slow motion time, but latest update is slightly better than oreo
[Sony Xperia XZ1 Compact ROMs, Kernels, Recoveries,] Pixel Shader™:
Relocking the bootloader after backing it up

Hi, I was wondering if I can relock the bootloader after backing it up with this temp_root method, so when I relock and unroot the warning unlocked bootleader message won't appear.
[Sony Xperia XZ1 Compact ROMs, Kernels, Recoveries,] camaro322hp:
as far as i know, there is currently no known method of relocking bootloader for the xz1c.
[Sony Xperia XZ1 Compact ROMs, Kernels, Recoveries,] lilbrat:
Well, in theory I would say yes, reason being flashtool can relock your bootloader, as I've done it a lot on my xz, and the warning does disappear.. As you now have the ability to backup your keys and restore them. Now does it work all I can say is give it a try...
[Sony Xperia XZ1 Compact ROMs, Kernels, Recoveries,] Pixel Shader™:
Exactly, now we can backup our TA which means we can relock the bootloader. I have done this on my previous Xperia X Performance and when I restored the TA my bootloader relocked and thus the warning message didn't appear. If this new TA backup method doesn't do the same, then it is not a backup. It is more like just a drm features fix, tbh.
[Sony Xperia XZ1 Compact ROMs, Kernels, Recoveries,] lilbrat:
Well, I will find out a bit more this weekend as this will be the first time I've decided to back up my TA, and this will be on my XZ1 so I will be able to test part of your original post as I try out the temp root first, once If I can get a TA backup, I will tinker a bit as I'm curious as I want of I know if the backup taken from the firmware in the Op will work with my current Pie firmware but I need a bit more info..
[Sony Xperia XZ1 Compact ROMs, Kernels, Recoveries,] Pixel Shader™:
I managed to backup the TA, however, I am unsure whether I can relock the bootloader (unroot) pr not when I want git ride of the warning screen (most likely when I want to sold it one day). So if you find anything please le me know asap, and thank you.
[Sony Xperia XZ1 Compact ROMs, Kernels, Recoveries,] mad0701:
What does you expect to get for an old phone? Just to compare: the phone XZP has cost 1y ago 799€. The prise of an used one (~6 month) is around 200€ (tendency falling). I didn't think about nor make the work for me to reverse all and have maybe trouble with a buyer, too.

Gesendet von meinem viennalte mit Tapatalk
[Sony Xperia XZ1 Compact ROMs, Kernels, Recoveries,] Sexy scientist:
Jenn, if you are still looking for one, I have 47.1.A.5.51 , never downgraded, have bootloader still locked, all 3 keys still provisioned. Happy to help.
[Sony Xperia XZ1 Compact ROMs, Kernels, Recoveries,] j4nn:
Thanks for the offer. That thing has already been tested, so there is no need any more.
[Sony Xperia XZ1 Compact ROMs, Kernels, Recoveries,] j4nn:
The restore of TA backup from locked state does not re-lock the boot loader with XZp/XZ1/XZ1c phones - that has been many times tested.

And think about it: it is actually a good think - we are lucky, that the bootloader does not get re-locked with restore of TA from locked state.

If it re-locked the bootloader, what use it would have, to restore DRM keys and get back to locked bootloader again, no way for custom firmwares, rooting and so on?

We are lucky that with TA restore we get device master key restored and still have the bootloader unlocked, restoring stock camera functionality without firmware cracks/patches.

From technical point of view - it seems to me, that the locked/unlocked bootloader state is not part of TA partition. It is most likely a fuse or other restricted area accessible only from Trust Zone.

I guess it might be possible to re-lock (or even unlock without use of sony website) if we could get full control of Trust Zone code. But that is another level of exploits, definitely not easy at all. I am afraid it would be even lot more difficult than this kernel exploit. In other words I doubt that is going to happen ever.

You say that this TA backup method is not a real TA backup - what is it then?

You can restore identical content of whole TA partition from locked backup including all drm keys.
[Sony Xperia XZ1 Compact ROMs, Kernels, Recoveries,] wyt18:
thanks j4nn for all your great work. can you confirm if one is able to upgrade to the latest stock firmwares (via newflasher) after executing this fix, while keeping bootloader unlocked, root, and TA partition intact?
[Sony Xperia XZ1 Compact ROMs, Kernels, Recoveries,] j4nn:
[MENTION=500933]wyt18[/MENTION], you can obviously flash any stock fw via newflasher.

You can get root via magisk. If you flash kernel (boot image partition) from stock, you would need to flash (patch it) with magisk again. If you skip fotakernel sin file when flashing stock fw with newflasher, your custom recovery would be kept - fotakernel resides on the recovery partition.

The TA partition changes with every boot - stock firmware writes many units there with each boot and on runtime. The good thing is that the fw does not remove repeatedly the keys if it detects unlocked state.

That may change in future if sony implements it differently, but it is not very probable.

There is a very interesting post from [MENTION=3665957]the_laser[/MENTION] describing the unlocking (and re-locking) internals here:

https://xdaforums.com/showpost.php?p=76291096&postcount=19
[Sony Xperia XZ1 Compact ROMs, Kernels, Recoveries,] Pixel Shader™:
You didn't get my point. As I said, I am fine with a rooted and unlocked BL Xperia XZ1, however, I don't want that unlocked bootloader screen to appear in every reboot that's when I decide to unroot and lock the bootloader. Also, I doubt this TA backup will remove that BT warning screen. I need someone to confirm it like the avoid loss of Abdroid attest keys...
[Sony Xperia XZ1 Compact ROMs, Kernels, Recoveries,] j4nn:
[MENTION=9652322]Pixel Shader™[/MENTION], I get your point, but I am not sure you get mine. TA can obviously be fully backed up and restored, giving back camera functionality with unlocked bootloader by restore of device master key.

Restore of locked TA will not remove unlocked bootloader warning screen on boot, because bootloader will not be re-locked by the TA restore. And as explained, that is actually behaviour we benefit from.

If you cannot accept the warning screen that is a result of unlocked bootloader forever, simply do not unlock it, no way around that in my opinion.

But let's see if anybody can find a way how to relock it - I would like that too at least for testing.
[Sony Xperia XZ1 Compact ROMs, Kernels, Recoveries,] Pixel Shader™:
Thanks now I understand that there is no way to unroot and lock your bootloader back as it was. I get the wrong idea when i head TA backup as on my previous Xperia X Performance the warning screen disappeared when i retored back my TA and flashed a stock firmware. I thought; it would be the same case since both of these TA are 2MB...
[Sony Xperia XZ1 Compact ROMs, Kernels, Recoveries,] lilbrat:
I still am going to disagree, till I get home and verify this.. as I've relocked the bootloader on my X and XZ a whole lot and have watched the "warning" disappear each time when using flashtool reason being it's easier for in my case to update when the new security/updates come out as I debloat the hell out my phone each time I updated and granted flash tool would be the easier way to go (before we had Newsflasher) but Sony Companion will not repair your software with an unblocked bootloader.. but once you relocked with flashtool all is good. Now the XZ1 series may be a different beast but till someone's proves this to be wrong I see no reason to think otherwise.. So now as there is a way to backup the the TA. all should be good.. Even if you root and want to go back, back up everything, push your TA. back, relock you bootloader, and flash a new ftf or hook it you to Sony companion and see what happens..
[Sony Xperia XZ1 Compact ROMs, Kernels, Recoveries,] j4nn:
Anybody who is going to unlock their phone, please contact me.

With my second unlock, I've managed to execute custom code

before

the android service splash screen with droid and progress bar.

As the unlock is a one way thing, with only one test opportunity, I cannot test this further myself any more.

The only requirement is a still locked bootloader XZp/XZ1/XZ1c phone which is wanted to be unlocked in following days.

Thanks.
[Sony Xperia XZ1 Compact ROMs, Kernels, Recoveries,] Pixel Shader™:
I guess I'll help. I was about to unlock my BL, yesterday lol! I did the right thing to check this thread back. So what do you want? Did you find anything that has to deal with that warning screen?
[Sony Xperia XZ1 Compact ROMs, Kernels, Recoveries,] j4nn:
[MENTION=9652322]Pixel Shader™[/MENTION], I like to test what might be possible to backup in the middle of the unlock procedure.

I was able to start twrp right after entering unlock code with fastboot command, before the droid progress screen, which probably shows progress of erase.

Unfortunately the started twrp showed errors and for yet unknown reason rebooted automatically, so I could not do any examination with it.

I need to analyse boot scripts of twrp, hopefully to find reason for the auto reboot, fix that and rebuild twrp for another test.

This is not a simple task.

Another problem may be targetting different phone than XZ1 Compact, as I can test only with xz1c.

So even though testing with XZp or the big XZ1 would be also useful, it is lot more difficult for me, because it is not easy to rebuild twrp for different target.

In my opinion this possibility could be very very interesting, depending on what is possible to backup in that time, it might be useful for other xperia phones for which exploit is not possible...

Luckily I recorded my unlock trial, so I have a video documenting all of it - some analysis (of twrp reboot reason) is therefore possible.
[Sony Xperia XZ1 Compact ROMs, Kernels, Recoveries,] Pixel Shader™:
I am not sure what I am supposed to do. Do you want me to do the same thing you did while unlocking the boorloader of my Xperia XZ1? And if we supposed that i did exactly what you have done, how can I prevent the twrp reboot? Also, I would link to see the your recorded trial if that possible.
[Sony Xperia XZ1 Compact ROMs, Kernels, Recoveries,] j4nn:
[MENTION=9652322]Pixel Shader™[/MENTION], right now, just wait please with your unlocking. Reproducing my attempt without any changes would obviously make no sense.

Please do not full quote, particularly if answering the last post, it is not useful.

I need to build patched twrp for another trial, that and the analysis mentioned above will take some time.

It would also be great (i.e. easier) if anybody with still locked XZ1 Compact would like to test this during unlock (after backup of TA).

Thanks.
[Sony Xperia XZ1 Compact ROMs, Kernels, Recoveries,] Pixel Shader™:
Alright I will wait I have managed to backup my TA, so far and didn't attempt to unlock the bootloader, yet. Now, I am going to return back to Android Pie while waiting. Thanks.
[Sony Xperia XZ1 Compact ROMs, Kernels, Recoveries,] seXneo:
That sounds interesting. Just checking in to let you know that I'm also planning to unlock my XZ1c bootloader soon. I'm currently on 47.1.A.16.20 in case that matters

For the exploit to work I'd have to downgrade
[Sony Xperia XZ1 Compact ROMs, Kernels, Recoveries,] Tarch1:
Unlocking Bootloader

Anyone know a method for unlocking the bootloader if the rooting status in Service menu>Configuration> Rooting Status: Unown ?
[Sony Xperia XZ1 Compact ROMs, Kernels, Recoveries,] kountry83:
Just did the whole thing with backup TA and unlock/restore ta image. Upgrade to latest 47.2.A.4.41 pie for my Sony XZP dualsim. Still shows this, any ideas.

Thank you
[Sony Xperia XZ1 Compact ROMs, Kernels, Recoveries,] Pixel Shader™:
Does unlocked bootloader warning screen still appears when you boot up your Xperia XZP?
[Sony Xperia XZ1 Compact ROMs, Kernels, Recoveries,] kountry83:
Yes

Also another thing I wanted to mention is that I deleted the .ta files and persist sin before I flashed 47.2.A.4.41 pie.
[Sony Xperia XZ1 Compact ROMs, Kernels, Recoveries,] Pixel Shader™:
Just to tell you if it is useful. I managed to downgrade my XZ1 to Oreo using Flashtool and I deleted the persist file and not marking the TA during flashing which result of all my keys still exist even the Android attest ones. I managed to upgrade back to Pie using same method and my keys are still there. I didn't use the new_flasher at all as it didn't work for me.
[Sony Xperia XZ1 Compact ROMs, Kernels, Recoveries,] seXneo:
I received the OTA update to 47.2.A.4.45 today. Shall I postpone it until after the backup?
[Sony Xperia XZ1 Compact ROMs, Kernels, Recoveries,] M1P:
I'll wait as well, so just give us a notice when you're ready for some actions.

My current state is: XZ1c (rev. 3), 47.2.A.4.41
[Sony Xperia XZ1 Compact ROMs, Kernels, Recoveries,] janbockelson:
Hey j4nn,

I'm one of this lucky (or not) guys who owns a new XZ1 ( G8341 ), already not rooted and with closed bootloader, FW is 47.1.A.16.20 DE, and I want to root it and UB for custom roms. :)

It would be great if you could help me with your announced method. The shown rutine in the first post seems bit tricky for me. I'm more rookie, had custom roms on my Z and XZ, but root and backing up TA wasn't that difficult as with XZ1. And to be honest, I didn't really knew what I have done while rooting and UB, but it worked. :D

So I'm bit scared of the posted procedure and would like to try your second experience. If you are interested feel free and contact me. You are welcome! :o

Thanks in advance
[Sony Xperia XZ1 Compact ROMs, Kernels, Recoveries,] j4nn:
unlocking xz1c with in the middle test

Please downgrade and do the TA-locked backup as described in the howto.

Just backup your stuff before downgrade as with downgrade comes a factory reset.

In fact I would recommend to do a factory reset just before the downgrade in order to remove the binding to your google account. This way you can avoid going online after the downgrade if used without sim and skipping wifi configuration.

So the steps should be:

- backup

- factory reset, do not do the initial setup, just power off

- enter flash mode, downgrade the fw

- boot up without sim and do not connect to wifi

- use the howto to do the TA-locked backup (do not do the unlock yet!)

In addition, backup also the misc and persist partitions, similarly as the TA-locked extending the step "6. backup your TA partition" in the howto.

If you have TA-locked.img already and did not yet reboot the phone with done temp root, you may get a new root shell with ./renoshell command.

Otherwise do the temproot with renoroot again as described in the howto and in the step 6. do following in addition:

cd /data/local/tmp
dd if=/dev/block/bootdevice/by-name/persist of=persist-locked.img
dd if=/dev/block/bootdevice/by-name/misc of=misc-locked.img
chown shell:shell *.img
sync
sync

And then read the partitions images out from the phone to your PC:

adb pull /data/local/tmp/TA-locked.img
adb pull /data/local/tmp/misc-locked.img
adb pull /data/local/tmp/persist-locked.img

Then use your phone keeping the exploitable firmware (disable auto updates) or at least do not upgrade to pie. Oreo 16.20 fw may be ok.

OTA update to 47.2.A.4.45 is update to pie latest version - it just got released for customized DE (so DE is updated from oreo 16.20 right to pie 4.45).

I am not sure if twrp that I am preparing would not have further problems when run on phone with pie.

I believe that the special test could allow at least to avoid full erase that comes with unlock, so you may put some test data or test apps into your phone to see, if they are kept after the unlock...

After you have done the above, please wait further instructions which will detail the unlock procedure with in the middle twrp test.
[Sony Xperia XZ1 Compact ROMs, Kernels, Recoveries,] j4nn:
[MENTION=6074348]janbockelson[/MENTION], just posted few videos on youtube:

- xperia xz1c temp root preview (renoroot)

- xz1c factory reset and initial setup to prepare for renoroot

- renoroot example getting one reboot with hit on 2nd try

- enable oem unlocking and reboot to fastboot for unlock (for oem unlocking not to be grayed out you need to go online at least once)

Hopefully they can be helpful for you.
[Sony Xperia XZ1 Compact ROMs, Kernels, Recoveries,] j4nn:
unlocking with twrp booted in the middle

Youtube video with my unlocking that includes test attempt to boot twrp in the middle of unlock procedure:

- unlocking with twrp booted in the middle (enable subtitles to view!)

Following commands were used - for documentation purposes.

$ time ./fastboot -i 0x0fce oem unlock <the unlock code>
...
OKAY [ 0.347s]
finished. total time: 0.347s

real 0m0.397s
user 0m0.000s
sys 0m0.000s

$ ./fastboot reboot bootloader
rebooting into bootloader...
OKAY [ 0.000s]
finished. total time: 0.251s

$ ./fastboot boot twrp-modpunk/android-8.1/twrp-3.2.3-0-lilac-1.img
downloading 'boot.img'...
OKAY [ 0.957s]
booting...
OKAY [ 5.740s]
finished. total time: 6.696s

$ ./fastboot reboot bootloader
rebooting into bootloader...
OKAY [ 0.000s]
finished. total time: 0.150s

$ ./fastboot boot twrp-modpunk/android-8.1/twrp-3.2.3-0-lilac-1.img
downloading 'boot.img'...
OKAY [ 0.950s]
booting...
OKAY [ 5.740s]
finished. total time: 6.689s

This is not the howto for the test. I will post a modded twrp for xz1c for the test. Please wait.

A big thanks goes to [MENTION=4403504]tonsofquestions[/MENTION] who did testing of some other stuff for me until I unlocked my xz1c and who discovered a method to make 'fastboot boot kernel.img' working on xz1c (it does not work if fastboot is entered by hand, but it works after 'fastboot reboot bootloader' is done or after 'adb reboot bootloader').
[Sony Xperia XZ1 Compact ROMs, Kernels, Recoveries,] Pixel Shader™:
So [MENTION=4418980]j4nn[/MENTION] what do you suggest? Should we wait and not unlock our bootloader, yet? All I have now is the "TA-locked.img" and "misc-locked.img".
[Sony Xperia XZ1 Compact ROMs, Kernels, Recoveries,] j4nn:
[MENTION=9652322]Pixel Shader™[/MENTION], please do not unlock yet, in few hours I will post howto unlock with the in the middle test.

Did not you forget to backup the persist-locked too? I guess it would not be needed, but thought better to backup it, as we do not know, what state the phone is in the middle of the unlocking procedure.
[Sony Xperia XZ1 Compact ROMs, Kernels, Recoveries,] tonsofquestions:
This is fantastic, [MENTION=4418980]j4nn[/MENTION]! If I hadn't unlocked my phone earlier in a previous test with you, I would absolutely help you with this one. I'm dying to know the results, and hear how the TA files locked/mid/after stack up compare.

It's very exciting, and if it works, it's a huge step forward for unlocking phones.

I was traveling for a bit, so didn't get a chance to post here sooner. But thank you for the credit!

For anyone interested in the history - I was helping j4nn test out some kernel modifications post-unlock, specifically around booting an image without modifying the flashed phone. I had different boot experiences than he did - sometimes booting into Andriod properly and sometimes only booting to the charging screen. After some extensive experimentation, I discovered that the previous state of the phone mattered (entering fastboot on reboot vs from a powered-off state), which kind of makes sense upon reflection.

After realizing that there was a "fastboot reboot bootloader" command, I found that that it changed the boot state - thus allowing me to boot a kernel (or recovery) image when coming from a fully-off phone state, without running firmware/images on the phone, but *also* without having to flash anything to the phone!

It seems almost obvious in retrospect, but I'm not aware of anyone else who's come across about this, though I haven't done extensive searching. Incredible!
[Sony Xperia XZ1 Compact ROMs, Kernels, Recoveries,] j4nn:
Please see attached git diff of the twrp sources.

Recovery script is ignored, but I assumed that the rest of twrp would ack/remove it from misc partition but it seems not to do so for some reason.

That is why there is that additional step which restores misc-locked.img to misc partition to avoid executing the wiping script on next boot.

It is not a security hole as long as user does not enable OEM unlocking in the phone's developer options - unlock via sony's code depends on it. Without it unlock would not be possible.

It could be a threat only for someone who enabled oem unlocking, did not yet unlock and lost his phone full of data.