yashsingh1/xdaforums huggingface dataset:
[Sony Xperia XZ1 Compact ROMs, Kernels, Recoveries,] j4nn:
[MENTION=5853845]Meloferz[/MENTION], yes, that's what I also have thought (similar link had posted already in the Android Attest Key lost thread).

Concerning FIDO - it might not be only about phishing protection, but possibly also about Fast Identity Online authentication, i.e. using fingerprint for auth, for example for paypal payments confirmed by fingerprint:

paypal-fido-authentication-and-fingerprint-payments

Implementing-Android-based-fingerprint-authentication-for-online-payments

I think those two SOMC keys means this:

SOny MObile Computing FIDO key

SOny MObile Computing Attest key

while FIDO means Fast Identity Online

https://fidoalliance.org/about/what-is-fido/

https://fidoalliance.org/adoption/overview/

https://searchsecurity.techtarget.com/definition/FIDO-Fast-Identity-Online

FIDO supports the Universal Authentication Framework (UAF) and the Universal Second Factor (U2F) protocols. With UAF, the client device creates a new key pair during registration with an online service and retains the private key; the public key is registered with the online service. During authentication, the client device proves possession of the private key to the service by signing a challenge, which involves a user–friendly action such as providing a fingerprint, entering a PIN, taking a selfie or speaking into a microphone.
[Sony Xperia XZ1 Compact ROMs, Kernels, Recoveries,] meowthink:
My guess is that Android attest key contains SecurityLevel and Customization info.

After boot on different userdata flashed, the provision daemon (in user space) tries provisioning the key then it destroyed(by some security feature of SoC).

Unfortunately there seems no way to get around this circle, as the userdata is signed and will be checked by kernel, kernel is signed and will be checked by bootloader, once bootloader unclocked those keys will gone...
[Sony Xperia XZ1 Compact ROMs, Kernels, Recoveries,] tabbed:
I've been following this thread for a while and it's convinced me to go out and buy a XZ1c, since it's the most modern compact phone that seems like it may have a fair shot at linux support.

Off ebay, as new as one can expect. Shrink wrap doesn't mean that it's brand new I know, but it sure looked brand new to me. I haven't inserted a sim card, nor have I connected it to any local network.

Checking Security in the service menu gives:

> SOMC FIDO Key PROVISIONED, ver. 4

> SOMC Attest Key PROVISIONED, ver. 2

> Android Attest Key PROVISIONED

Under the software version, I have:

1307-7511_47.1.A.12.145 (GLOBAL-LTE4A-user) :(

Customization Version:

1310-9560_R6B

So I guess my phone shipped with newer firmware.

Anyway, let me know if I can help. I plan to unlock my bootloader anyway and don't mind being a test subject/losing drm keys.
[Sony Xperia XZ1 Compact ROMs, Kernels, Recoveries,] blue-spiderfire:
I think I can help here and I like to help. Sorry, I didn't take part in this topic for month - have been busy with study. Now have a B.Sc. grade and some time for testing.

As I wanted to have all options for TA backup and customization I didn't did any update to my XZ1C and phone is locked.

I'm still on

> Generic_47.1.A.2.324 (I think it's the second available firmware)

> Global-Gdialer

> SOMC FIDO Key PROVISIONED, ver. 4

> SOMC Attest Key PROVISIONED, ver. 2

> Android Attest Key PROVISIONED

[MENTION=4418980]j4nn[/MENTION] do you still need a tester?
[Sony Xperia XZ1 Compact ROMs, Kernels, Recoveries,] tramtrist:
[MENTION=4418980]j4nn[/MENTION] Take this guy up on his offer :fingers-crossed:
[Sony Xperia XZ1 Compact ROMs, Kernels, Recoveries,] notaz:
If you decide to downgrade, don't flash persist_X-FLASH-ALL-C93B.sin and see if your attest key survives.

More information here:

https://xdaforums.com/xperia-xz1-compact/help/android-attest-key-lost-bootloader-t3829945
[Sony Xperia XZ1 Compact ROMs, Kernels, Recoveries,] j4nn:
[MENTION=4503201]blue-spiderfire[/MENTION], please backup everything you need. Even though we will not do a downgrade as you have old enough fw, I am afraid unlock would erase everything like a factory reset.
[Sony Xperia XZ1 Compact ROMs, Kernels, Recoveries,] j4nn:
Tools to backup TA partition (drm keys) are released - see the first post of this thread!
[Sony Xperia XZ1 Compact ROMs, Kernels, Recoveries,] yacloo:
Bootloader is based on uefi

maybe unlock status stored in efi nvram rather than in TA
[Sony Xperia XZ1 Compact ROMs, Kernels, Recoveries,] DocLM:
Awesome work [MENTION=4418980]j4nn[/MENTION].

Do you know if is possible to restore the partition from a more recent firmware? Assuming I have root (like on a custom rom). On older Xperia devices I remember that restore from a different firmware will lock boot loader and temporary brick the device (until I restore an official firmware)
[Sony Xperia XZ1 Compact ROMs, Kernels, Recoveries,] tramtrist:
:good::highfive::victory:
[Sony Xperia XZ1 Compact ROMs, Kernels, Recoveries,] j4nn:
In my opinion it does not matter where the unlock flag is stored. We cannot toggle it. Even if we could, having custom kernel and re-locking bootloader would not allow to boot.

So we need to keep it unlocked. Now the only question remaining is on what point we filter the flag value out - we could maybe do it in kernel already instead of secd. But that would need custom kernel then.

I am not sure I understand your question.

In my opinion it is best to restore TA when running the same fw which was used to back it up. That way you can be sure there are no incompatibilities.
[Sony Xperia XZ1 Compact ROMs, Kernels, Recoveries,] j4nn:
Guys, post your results, if the released exploit and TA backup/restore worked with your phone, please.

Particularly other models except XZ1c are not tested at all. But I guess any experience with any of the models should be useful.

Thanks.
[Sony Xperia XZ1 Compact ROMs, Kernels, Recoveries,] tanapoom1234:
Thank you so much [MENTION=4418980]j4nn[/MENTION] for your work. I just finished testing the process. (xz1 compact)

1. Started on FW 47.1.A.16.20 Thailand, all 3 keys provisioned

2. flashed 47.1.A.2.324_CE1 EXCLUDING THE persist_X-FLASH-ALL-42E5.sin, all 3 keys remained provisioned

3. first attempt at exploit caused reboot. second attempt successful and got temp root and backed up TA

4. after unlocking, SOMC FIDO and SOMC Attest keys NOT PROVISIONED, camera doesnt work (green pics)

5. got temp root again and restored TA partition. also backed up unlocked TA partition for reference

6. After restoring TA, camera works normally but OMC FIDO and SOMC Attest keys still NOT PROVISIONED, but Android Attest Key IS Provisioned

if you need my backed up TA partitions and screenshots i can send it to you somehow. Again, thanks so much for spending so much time on this!
[Sony Xperia XZ1 Compact ROMs, Kernels, Recoveries,] j4nn:
[MENTION=8010702]tanapoom1234[/MENTION], please try to flash the optional patched secd. I guess the two SOMC keys will became provisioned too.
[Sony Xperia XZ1 Compact ROMs, Kernels, Recoveries,] j4nn:
[MENTION=4503201]blue-spiderfire[/MENTION], I would still like to use your offer to test with me to try to backup the android attest key.

Even though [MENTION=8010702]tanapoom1234[/MENTION] already managed to keep the android attest key by not flashing persist partition, I would like to test if android attest key could be recovered from one device and used with another device. A small chance, but still, better to try than not.
[Sony Xperia XZ1 Compact ROMs, Kernels, Recoveries,] tanapoom1234:
[MENTION=4418980]j4nn[/MENTION] Flashing the secd zip does not change anything. SOMC FIDO and SOMC Attest still both NOT PROVISIONED. Also, I noticed something else. the Suntory blobs are shown as [Error]. I messed up a bit and forgot to screenshot right after TA partition restore, and i flashed back to the latest firmware, so im not sure if that messed it up. But, the Suntory Blobs were also [ERROR] right after bootloader unlock.
[Sony Xperia XZ1 Compact ROMs, Kernels, Recoveries,] j4nn:
Flashing the patched secd must be done as the last action - if you flashed latest fw, flash after that the patched secd and then check the security screen please. The provided secd is compatible with all xz1c firmwares up to 47.1.A.16.20 (the last one I've compared the binaries in).
[Sony Xperia XZ1 Compact ROMs, Kernels, Recoveries,] tanapoom1234:
I flashed the patched secd last, after I flashed 47.1.A.16.20 fw. the two keys are still unprovisioned and suntory blob still error.
[Sony Xperia XZ1 Compact ROMs, Kernels, Recoveries,] j4nn:
As mentioned, this optional secd step is still experimental.

But when testing with [MENTION=5677791]tramtrist[/MENTION], he got the two SOMC keys provisioned, suntory blobs ready and rooting status not allowed - I have screenshots from him proving that. And there is also a screenshot with updates coming.

But then it somehow stopped working - so there is probably still some issue, but it seems the keys can be re-enabled. It seems to be only about that "unlocked" state flag checked somewhere and disabling the keys.

Most likely the SOMC Attest Key enables sony updates.
[Sony Xperia XZ1 Compact ROMs, Kernels, Recoveries,] pbarrette:
I'm virtually certain that all of the Attest and FIDO keys are being derived from TrustZone.

Additionally, I believe the Suntory blobs are being decrypted by the TrustZone.

It's most likely that once the bootloader is unlocked, the TZ won't attest to the device anymore.

You'll note that the

libta_drm_ext.so

also has exported functions for SuntoryTee_VerifyBlob[/u] and other SuntoryTee bypass functions.

So it's quite likely that the Suntory provisioning will be broken as well.
[Sony Xperia XZ1 Compact ROMs, Kernels, Recoveries,] j4nn:
I know about that libta_drm_ext.so and have seen the functions there (just disassembly, not sure if there is a source code available). Still it seems to me that the lib fixes primarily the missing device master key and from that failing other stuff.

I hope/believe, that having device master key enables all the other keys in trust zone, that those keys in trust zone are actually not erased with unlock there, instead device master key is removed from TA. Without device master key, the trust zone keys would not be accessible.

But having the device master key is not sufficient by itself as secd (and possibly other components) detects the unlocked flag and switches to "reduced functionality mode" disabling access to the trust zone keys, even though it could be possible to access them when device master key is available.

This is just my theory, I have no prove or reverse engineering work to back it up. Maybe except that screenshot from [MENTION=5677791]tramtrist[/MENTION] where updates were coming and the security test service screen showing all ok except the android attest key.
[Sony Xperia XZ1 Compact ROMs, Kernels, Recoveries,] Vildanoff:
[MENTION=4418980]j4nn[/MENTION]

I have XZ1 fw *16.20 and I downgrade it to * 2.281 excluding persist_X-FLASH-ALL-C93B.sin and now I have all 3 PROVISIONED keys.

Then I stoped. I do not understand how to do?

Can you help me?
[Sony Xperia XZ1 Compact ROMs, Kernels, Recoveries,] Daniel#T:
adb shell <enter>

then

cd ............
[Sony Xperia XZ1 Compact ROMs, Kernels, Recoveries,] notaz:
Many thanks for the release!

For doing this from 47.1.A.8.49, it appears there is a lot of luck involved, it took me ~5 hours (with crash-resets in between) to finally get the root shell. At the start it ran like for an hour without "overwrites" even increasing, I thought my kernel is not compatible with the exploit, but with persistence it worked. I'm still on DE customization, and later confirmed the kernel is exactly the same as in CE1.

Looking at the partition table, it appears there might be other things besides TA worth backing up. Many partitions like "keymaster" seem to contain ELF binaries, intended to run in UEFI-compatible bootloader perhaps? Or maybe in some "secure" environment? There are also /dev/block/sdb and /dev/block/sdc devices containing gpt partitions named xbl and xblbak respectively, which are probably the x-boot bootloader binaries themselves.

31240192 sda
2048 sda1 TA # tad_static
16384 sda2 LTALabel ro ext4 /lta-label
512 sda3 rpm
512 sda4 rpmbak
2048 sda5 tz
2048 sda6 tzbak
512 sda7 hyp
512 sda8 hypbak
512 sda9 pmic
512 sda10 pmicbak
1536 sda11 abl
1536 sda12 ablbak
4 sda13 devinfo
512 sda14 keymaster
512 sda15 keymasterbak
512 sda16 cmnlib
512 sda17 cmnlibbak
512 sda18 cmnlib64
512 sda19 cmnlib64bak
2048 sda20 sti
128 sda21 devcfg
128 sda22 devcfgbak
128 sda23 storsec
128 sda24 storsecbak
30720 sda25 xfl
30720 sda26 xflbak
512 sda27 tzxflattest
512 sda28 tzxflattestbak
512 sda29 tzxfl
512 sda30 tzxflbak
128 sda31 xflkeystore
128 sda32 xflkeystorebak
32768 sda33 persist rw ext4 /persist
1536 sda34 fsg # rmt_storage
16 sda35 sec
112640 sda36 modem ro vfat /firmware
16384 sda37 dsp ro ext4 /dsp
65536 sda38 boot
1024 sda39 bluetooth ro vfat /bt_firmware
256 sda40 apdp
256 sda41 msadp
4 sda42 dpo
33424 sda43 splash
4 sda44 limits
1024 sda45 toolsfv
8192 sda46 logfs
65536 sda47 FOTAKernel
1024 sda48 ddr
2048 sda49 modemst1 # rmt_storage
2048 sda50 modemst2 # rmt_storage
4 sda51 fsc # rmt_storage
8 sda52 ssd # qseecomd
389120 sda53 cache rw ext4 /cache
1024 sda54 misc
512 sda55 keystore
512 sda56 frp
64 sda57 vbmeta
16384 sda58 appslog rw ext4 /rca
32768 sda59 diag rw ext4 /idd
24576 sda60 Qnovo rw ext4 /qns
1028 sda61 fsmetadata # init
32768 sda62 rdimage
409600 sda63 oem ro ext4 /oem
1572864 sda64 vendor ro ext4 /vendor # dm-1
5120000 sda65 system ro ext4 /system # dm-0
23197144 sda66 userdata rw ext4 /data
4096 sdb
4052 sdb1 xbl
4096 sdc
4052 sdc1 xblbak
[Sony Xperia XZ1 Compact ROMs, Kernels, Recoveries,] dcrin3:
man...

man.... this is nuts and the code is absolutely insane. I have an unlocked bootloader, but I appreciate the hard work and time you've put in.

Thank you [MENTION=4418980]j4nn[/MENTION] and take care of yourself.
[Sony Xperia XZ1 Compact ROMs, Kernels, Recoveries,] j4nn:
I guess the 47.1.A.2.324 is more stable with regard to timing with the exploit. But it happens with both that it sometimes takes long time to get successful overwrite.

In any case, it seems to be good not to run anything that does not need to be run during the exploiting time.

So it may help to remove sim card, disable wifi, stop all background stuff that might be there and is stoppable. If you did a factory reset before it may be better than having long time used 47.1.A.8.49 with many apps installed (but that's probably not the case here). As mentioned, even the animated background eats some cpu and makes differences with timing, so static background should be preferred.

Also any kind of power saving is not desired, so it's good to turn on the always on option when connected on usb cable for adb.

That is right. If you check the backup-tools.zip, the script backs up all sda partitions except system, vendor and userdata (userdata is backed up to a .tar archive). The partitions from sdb and sdc are backed up too.

The thing is that [MENTION=4503201]blue-spiderfire[/MENTION] who offered to do testing with these backups did not get to it yet, so I do not know, if the backups could be useful.

I hope that backing up all partitions except the big (or encrypted) ones would be sufficient.

I checked that backing up system can be binary compared with the original from fw package and they matched, so it's ok to skip them.

[MENTION=9191353]notaz[/MENTION], did you mean to back up anything else than it is done with these scripts?

There are obviously regions not backed up, like the beginning of sda and not sure if there are any holes or space at the end of sdX devices, if such space could contain anything useful?

Partition layout is backed up into a text file too.
[Sony Xperia XZ1 Compact ROMs, Kernels, Recoveries,] DocLM:
[MENTION=4418980]j4nn[/MENTION] tested with success on XZ Premium. Like notaz post, it takes some hours to work with unexpected reboots sometime.

EDIT: single sim variant
[Sony Xperia XZ1 Compact ROMs, Kernels, Recoveries,] HandyMenny:
I can confirm that this temp root exploit works on xz1 g8341 with firmware 47.1.A.2.324_CE1. I restarted the phone after the first setup and then I tried the exploit. It took about ten minutes. I used temp root to make a backup of all partitions, I don't want to unlock the bootloader yet.

Edit: Note that I downgraded without persist.sin and I still have android attest key provisioned
[Sony Xperia XZ1 Compact ROMs, Kernels, Recoveries,] Unbounded:
I have successfully downgraded firmware and my all keys are PROVISIONED.

Well, everything till now looks good, but when I try to run "adb push renoroot /data/local/tmp" it shows like this :

E:\XZ1 compact\renroot>adb push renoroot /data/local/tmp

error: device '(null)' not found

USB debugging is on and OEM unlocking is also allowed.

Please help me to go further.

Thanks in advance.
[Sony Xperia XZ1 Compact ROMs, Kernels, Recoveries,] Vildanoff:
[MENTION=4418980]j4nn[/MENTION]

After the TA partition was restored, the bootloader remained open. The camera is working (No green fill).

But only

Suntory Blobs [ERROR]

CKB OK [ACTIVE]

WIDEVINE OK [ACTIVE]

SOMC FIDO Key NOT PROVISIONED

SOMC Attest Key NOT PROVISIONED

Android Attest Key PROVISIONED

I flashed secd-ignore-unlock.zip, but everything remains unchanged
[Sony Xperia XZ1 Compact ROMs, Kernels, Recoveries,] notaz:
Not really, I just neglected to look at your backup tools before posting, sorry.
[Sony Xperia XZ1 Compact ROMs, Kernels, Recoveries,] orsonmmz:
Great job j4nn! Another XZ1c rooted thanks to your efforts. More details below, though they do not diverge from what already has been said:

- Started with 47.1.A.12.34 (Customized MY).

- Flashed using newflasher: the same firmware version with another customization (PLUSPL) with *.ta and persist.sin deleted. All keys still provisioned.

- Downgraded to 47.1.A.8.49 CE1 customization (no *.ta, no persist.sin) - all keys still provisioned.

- It took renoroot several minutes get a root shell. No problems saving the TA partition.

- Unlocking bootloader via the official Sony method results in Suntory blob error, CK and WIDEVINE are not even mentioned, both SOMC keys not provisioned, but the Attest key is still there.

- Restoring the TA partition brings back CK and WIDEVINE.

Now I plan to upgrade 47.1.A.16.20 and then Lineage. Perhaps I will play with secd fix later.

In your face, Sony!
[Sony Xperia XZ1 Compact ROMs, Kernels, Recoveries,] j4nn:
I guess you need to install sony driver in order to be able to use adb from windows.

https://developer.sony.com/file/download/xperia-xz1-compact-driver/
[Sony Xperia XZ1 Compact ROMs, Kernels, Recoveries,] j4nn:
Could you please be specific - was it a xzp or xzp dual?
[Sony Xperia XZ1 Compact ROMs, Kernels, Recoveries,] j4nn:
Updated the first post:

- added links to confirmations of tests for phone models

- added link to renoshell source code provided on github

- added a note about not working on other targets and details about primary vulnerability required

Please keep up posting your test results.

You may include info about how long it took to get a root shell, how many reboots, how many events in the last trial which succeeded with how many overwrites (just one with success is the best, more means previous overwrites did not hit wanted object in shaped heap resulting with possibly unstable system). This info is interesting for statistics, so we all know, how fast can we get a temp root on each device/firmware.

Thanks.
[Sony Xperia XZ1 Compact ROMs, Kernels, Recoveries,] LinFan:
It's a normal xzp not dual.

Sorry, I meant the fw you provided 254?

I'll do some testing later today, and provide screens hots. I did get TA-locked.img yesterday when I tried using temp root.

I haven't unlocked my bootloader yet.
[Sony Xperia XZ1 Compact ROMs, Kernels, Recoveries,] 4rz0:
Would it be possible to use the temp root to properly back up the phone trough means like Titanium?
[Sony Xperia XZ1 Compact ROMs, Kernels, Recoveries,] WheelchairArtist:
First of all big thanks [MENTION=4418980]j4nn[/MENTION] !!

First tried renoroot with build 47.1.A.5.51 with DE customization, had it running several times (whole day) but it didn't succeed.

Had event counts in the hundred-thousands and usually 3/4 overwrites and also 2 reboots.

Screen was always on, tried with airplane mode and removing sim / sdcard but nothing.

Maybe i was to impatient, however updated to build 47.1.A.8.49_CE1 without any ta or persistent.sin and after 3 tries and a phone call (around 3 hours): success! root shell with all keys :)

Haven't unlocked at the moment until we understand better what the other keys do and how we back them up. atleast thats the plan for the moment^^

I will donate you something in the course of the day! :)
[Sony Xperia XZ1 Compact ROMs, Kernels, Recoveries,] DocLM:
It is single sim variant
[Sony Xperia XZ1 Compact ROMs, Kernels, Recoveries,] Blu-eyes:
[MENTION=4418980]j4nn[/MENTION] could you dump your persists partion and compare with the partition of another phone with attest key present?
[Sony Xperia XZ1 Compact ROMs, Kernels, Recoveries,] LinFan:
I am on newest OTA update xz premium G8141 47.1.a.16.20

All keys seems to be normal on Security test.

gonna downgrade to fw you provided to try temp root

currently downgrading to CE1 47.1.a.3.254 using NewFlasher v13 removed the persist.sin file

Security screen seems fine on downgraded firmware.

Testing exploits now.

activated exploit, waiting for temp root.

got root after 3m49.75s

successfully pulled TA-locked.img

unlocking phone now with Androxydes flashtool.

unlocked bootloader first time.

gonna get temp root again with untouched stock fw.

security screen NOT PROVISIONED both keys SOMC FIDO SOMC ATTEST KEY

Suntory BLOBS ERROR.

trying exploit on unlocked bootloader. phone reboots on first try, after around 4k events. trying exploit again second time just after reboot.

got temp root 8.8k events and 3 overwrites.

got camera working no green pictures. suntory blobs says ERROR though

i am going to relock my bootloader and see if i can get OTAs

sorry for no pictures :/
[Sony Xperia XZ1 Compact ROMs, Kernels, Recoveries,] j4nn:
I guess it could be possible (with some additional linux commands after getting temp root). But you need to be already on an exploitable kernel or be able to _upgrade_ to it. Otherwise with downgrade, you get everything erased, so no point to do any such backup then, I guess.

[MENTION=7556135]WheelchairArtist[/MENTION] (and others), why do you first try the exploit on a not supported kernel (fw)? I tried to clarify a bit more the note about supported targets, hope that helps.

That is something I would very much like to do. The best candidate is anybody with exploitable firmware on the phone who did not do any downgrade whatsoever: for example [MENTION=4503201]blue-spiderfire[/MENTION], who volunteered for it few days ago. But I guess he must be busy or something because I cannot reach him.
[Sony Xperia XZ1 Compact ROMs, Kernels, Recoveries,] WheelchairArtist:
Just thought it's one firmware version before 8.49 with patchlevel november so it should be vulnerable too. So i gave it a try before flashing firmwares around, no problem as i hadn't used my phone anyway.

If it had worked there would be another firmware on your list. One could call it pure curiosity ;)
[Sony Xperia XZ1 Compact ROMs, Kernels, Recoveries,] jrbmed08:
Thank you so much for your work on this.

Edit: Deleted...I did something stupid with Newflasher and figured out my problem. Wanted to leave my post thanking you though.