yashsingh1/xdaforums huggingface dataset:
[Sony Xperia XZ1 Compact ROMs, Kernels, Recoveries,] tonsofquestions:
As you say, there are levels and levels. What's hardbricked for one user might be fixable by someone at Sony by just flashing the right thing. They were offering a suggestion of something more to try, that you might not have thought of, since the Emma tool is a useful one for some of those slightly more confusing states.

However, you're right that it's useless without flashmode, but that can sometimes be missed. They were trying to be helpful, at least the way I read it.

I'd guess that it would be possible to do this, but it would require with the right tools and some luck (I suspect).

Because the image/text changed across some firmware versions, we know that it's something that's coming from fastboot. That means that it should be possible to make a bootloader that skips the check, displays a blank screen, or something else.

That said, finding where in the binary code that change needs to happen would be tricky to track down, because we don't have access to what's going on. Even if we could make a change, I'm not sure how we'd flash the file to the phone - flashmode only writes signed files (we'd have made a modification), and I don't think we can flash fastboot from fastboot, at least not on the XZ1. (Though I have done it on other phones before.)

And even if we could get past both of those issues, testing would be a nightmare, probably with a high risk of bricking the phone.

So yes, in theory it should be possible, now that we've unlocked the bootloader. But actually managing to do it is a whole other feat.
[Sony Xperia XZ1 Compact ROMs, Kernels, Recoveries,] Pixel Shader™:
@ j4nn No the annoying warning message only disappeared when I restored back the TA partition and flashed a stock ftf firmware. It is still fine for me but in the XZ1's case I wasn't able to remove that annoying warning screen at all. I spent a night trying to remove it. I restored the TA and flashed the stock clean firmware but still nothing. It seems Sony has expected that one day there will be possibility to restore the TA, so it moved the reason for that message to a different partition or something like that my guessing. That's actually a very smart move from Sony and yet annoying.
[Sony Xperia XZ1 Compact ROMs, Kernels, Recoveries,] vofferz:
My X Compact also displays the warning about unlocked bootloader even with stock rom and TA restored. And why wouldn't it as the bootloader HAS been unlocked... Nothing to do with stock firmware or TA partition.
[Sony Xperia XZ1 Compact ROMs, Kernels, Recoveries,] Pixel Shader™:
It is weird, maybe you didn't restore the TA in Marshmallow as in my case I returned back to Marshmallow and made sure it is restored perfectly. Also, did u relock the bootloader using flashtool?
[Sony Xperia XZ1 Compact ROMs, Kernels, Recoveries,] SilverGamer_YT:
Any chance to fix a unlocked bootloader with no ta backup?
[Sony Xperia XZ1 Compact ROMs, Kernels, Recoveries,] vofferz:
TA is restored just fine, using the dirtycow method, but of course I didn't relock the bootloader as I think it would prevent me from using root, which was the whole point in unlocking in the first place.

But yeah, it might be that relocking isn't possible on XZ1C at all. Ibdon't know. I'm just curious why did you want to unlock if you want a locked bootloader after all?
[Sony Xperia XZ1 Compact ROMs, Kernels, Recoveries,] mirhl:
TA is drm and bootloader. So you can't restore one without the other, at least for phones before XZP

There's a reason if tobias fix mounted it, instead of flashing it back

And yes, AFAIK that was also what could remove the bootloader unlock warning
[Sony Xperia XZ1 Compact ROMs, Kernels, Recoveries,] vofferz:
Ok. By TA I was referring only to the drm keys that affect camera functionality etc.

Anyway, I don't feel it matters if the phone tells me that it's unlocked when I boot it as long as everything works perfectly (except OTA) and I get root access, which is why I did the unlocking.
[Sony Xperia XZ1 Compact ROMs, Kernels, Recoveries,] Pixel Shader™:
Dude [MENTION=7803472]vofferz[/MENTION], I am not sure if u read carefully what I said. I said I am fine with unlocked bootloader as long as the device with me, however, I don't want it to be permanent. I need to relock the bootloader and unroot to remove the annoying bootloader screen so I can save warranty, resell it or exchange it without being questioned about it. So this is the whole point why I need to be able to return back and relock whenever I want to. This is for Xperia XZ anyway. For Xperia XZ1 I already decided to live with that warning screen even if it is permanent.
[Sony Xperia XZ1 Compact ROMs, Kernels, Recoveries,] mirhl:
So.. While I was trying to investigate WarrantyService I noticed a hell of references to:

com.sonymobile.permission.TRIMAREA, com.sonymobile.miscta.api, MiscTAUtil and getSecureTimeAndWriteToMiscTA.

Rcahandler, bootinfo and many more are likely on the same line.

Has anybody ever looked into simply accessing TA using the plain APIs the system already lies down?

And even if their use was in turn somehow restricted to just system apps.. Why couldn't you still try to exploit them (or the reminder of the userspace)?

I mean, android security bulletins are full of these kinds of vulnerabilities (versus whatever unholy kernel privilege escalation thing)
[Sony Xperia XZ1 Compact ROMs, Kernels, Recoveries,] j4nn:
[MENTION=4900593]mirhl[/MENTION], actually I've tried something like you've described:

from hijacked secd I tried to write device master key unit - 0x1046b (66667) - using libmiscta api, injected my write call before another unit write call - it was denied (or ended with some error, not remember exactly now).

I did not try to call that directly as I was not sure if some init api does not need to be called first, but there might be different results if called from temp root context instead of secd's context.

It is also denied to write the device master key unit with prepared devkey.ta file in flash mode (unlocked bootloader).

Not exactly sure what use case do you have in mind while we already have temp root?

In my opinion bootloader unlock state is not stored in TA. It is most likely controlled by some soc's fuse, which is accessible only from Trust Zone.

I think unlocking prepares unlock code in TA and bootloader (part of which is executed in TZ too) uses that and if it verifies right with some crypto function it toggles the unlock fuse in TZ context.

Similarly as the Android Attestation api calls an applet in Trust Zone, which uses Android Attest Key to sign verified boot info including bootloader unlock state, which is known by that TZ code regardless what kernel or userspace might try to provide.
[Sony Xperia XZ1 Compact ROMs, Kernels, Recoveries,] mirhl:
It wanted to be an alternative to that?

As I said, it's not every day that we would be blessed with it.
[Sony Xperia XZ1 Compact ROMs, Kernels, Recoveries,] Shahnewaz:
So I've been waiting with my XZ1 Compact for some time, and finally decided to bite the bullet.

Had to downgrade my firmware, I downgraded to 47.1.A.2.374 from 47.1.A.16.20 using only system, kernel, userdata and cache. All keys intact, bootloader untouched.

I've been trying to get a temp root using your tool for a while now... no success. Tried multiple times, letting it run over 10k events, getting a couple overwrites and resulting in a reboot.

Current run is going at 38k events with 10 overwrites and going. How long did it take for you guys to get the temp root?
[Sony Xperia XZ1 Compact ROMs, Kernels, Recoveries,] j4nn:
[MENTION=4849927]Shahnewaz[/MENTION], you guys should really read the howto very carefully.

For xz1c there is supposed to be used 47.1.A.2.324 (or 47.1.A.8.49) and not 47.1.A.2.374 fw version. There are even links for easy download of the firmwares...
[Sony Xperia XZ1 Compact ROMs, Kernels, Recoveries,] j4nn:
[MENTION=4900593]mirhl[/MENTION], in my opinion chances for full temp root are always better if kernel exploit can be used. Leveraging via userspace usually means to find multiple vulnerabilities and in the end you may not still be able to do what we need (ta backup). Even the kernel exploit nowadays needs multiple vulnerabilities due to all mitigations in place - security of android since oreo has become really good.

Anyway, my post about using libmiscta from secd obviously needed temp root first, so it was not exactly what you had in mind (exploiting userspace from not privileged app), still it was rather interesting, that secd could not write device master key.
[Sony Xperia XZ1 Compact ROMs, Kernels, Recoveries,] Shahnewaz:
Well that was a huge waste of time. :eek:

Flashed the proper firmware (47.1.A.2.324_CE1, system, kernel, userdata and cache), gave it another shot. Got root and pulled TA, misc and persist. :D

Will come in handy in the future if I decide to unlock my bootloader. Although I feel like I should pull every oddball partition in the device in case some other important stuff is hiding in there. For now I'll stay un-rooted.

Looking forward to see what you can learn from post#520!

Many thanks for your awesome work! :victory:
[Sony Xperia XZ1 Compact ROMs, Kernels, Recoveries,] mirhl:
Why would.. writing master key matter?

We are just interested into reading it, isn't it?

Also, yes, I know you may even need multiple vulnerabilities. But as I said, if nuking the kernel is almost starting to become a once-in-a-phone-lifetime event, userspace seems waaaay more prone to holes instead.

Then sure, I never really looked into the technicalities of whatever all those dozens of "execute arbitrary code within the context of a privileged process" actually mean. But that certainly does sound enticing.
[Sony Xperia XZ1 Compact ROMs, Kernels, Recoveries,] SGH-i200:
I backup'd the TA partition and will unlock the phone in the next days, if you don't have new testing ideas! ;)
[Sony Xperia XZ1 Compact ROMs, Kernels, Recoveries,] VBBoston:
How can we do secd fix to enable video enhancement for XZ premium? .zip file offered for XZ1 does not work, maybe there is a better TWRP for XZP that provides vendor folder modification and perhaps oem folder too?
[Sony Xperia XZ1 Compact ROMs, Kernels, Recoveries,] SGH-i200:
Are you sure the secd fix is the enabling thing? I was thinking it is the TA partition...
[Sony Xperia XZ1 Compact ROMs, Kernels, Recoveries,] HandyMenny:
See here: https://xdaforums.com/showpost.php?p=78366552&postcount=67

[MENTION=4418980]j4nn[/MENTION] I'm going to unlock the bootloader of my xz1, do you have any tests in mind that I can do before unlocking the bootloader?
[Sony Xperia XZ1 Compact ROMs, Kernels, Recoveries,] wyt18:
deleted
[Sony Xperia XZ1 Compact ROMs, Kernels, Recoveries,] Ryhorse:
Why are so many keys not provisioned? I did everything that was written in the OP.

Suntory Blobs [ERROR]

CKB OK [ACTIVE]

WIDEVINE OK [ACTIVE]

SOMC FIDO Key NOT PROVISIONED

SOMC Attest Key NOT PROVISIONED

Android Attest Key NOT PROVISIONED

[MENTION=4418980]j4nn[/MENTION] Please answer this post. Please don't ignore it like you have with other similar posts. Thank you.
[Sony Xperia XZ1 Compact ROMs, Kernels, Recoveries,] Shahnewaz:
Did you flash everything or the whole FTF at any point in the process?
[Sony Xperia XZ1 Compact ROMs, Kernels, Recoveries,] Ryhorse:
Not everything. I deleted the simlock.ta and persist_xxxx.sin file before flashing to one of the supported firmwares.
[Sony Xperia XZ1 Compact ROMs, Kernels, Recoveries,] j4nn:
If you did everything as written, you certainly have also the screenshot of service menu taken before doing fw downgrade in step 2, haven't you?

That screenshot would answer your Android Attest Key.

For more details you may check Android Attest Key lost thread.

The other stuff (Suntory Blobs and the two SOMC keys) do not look right, because the stock firmware knows, your bootloader has been unlocked and for that reason are simply disabled.

There are few patches trying to make the fw to ignore the unlocked state, but all of them are experimental just for testing.

I am still working on a proper solution for that.

You may check following kernel which hides the unlocked state (it is only for the 47.1.A.2.324 fw):

boot-2.324-uh.img

That will enable everything except Android Attest Key, which you've lost for some reason possibly even before you had started with instructions here.

With that kernel the phone will work as never unlocked. But it is not compatible with magisk - do not flash magisk over it, it would not boot.
[Sony Xperia XZ1 Compact ROMs, Kernels, Recoveries,] mad0701:
I've buy the XZP used. It was unrooted and with FW Version Pie.

I've never seen the Assistant key on my phone. After downgrade, root and upgrade to Pie 2.31 the key was also not available. The patch Secd was also flashed. Also with missing the assistant key, I received an update notification.

In your case: does you have flashed the Secd patch after upgrade?
[Sony Xperia XZ1 Compact ROMs, Kernels, Recoveries,] Ryhorse:
Ok, when I bought my XZ1c, I debranded it with generic firmware. This is probably why I lost my Android Attest Key.

I attached the state before I unlocked bootloader. It shows that everything was there except my Android Attest key. After I unlocked bootloader and after I restored my TA key backup I got the NOT PROVISIONED state on a number of keys.

What’s the point of restoring the TA key backup if it seemingly does nothing? Camera still works but the X-Reality engine and Clearaudio do not work. I don’t believe the Android Attest Key is needed for those features to work.

I also did root an Xperia X Performance some time ago using basically the same process (temp root, backup TA keys, unlock bootloader, restore TA keys) and there everything was fine after I restored the TA backup. Why isn’t that the case with XZ1 compact?

I did.
[Sony Xperia XZ1 Compact ROMs, Kernels, Recoveries,] j4nn:
[MENTION=6995680]Ryhorse[/MENTION], the Android Attest Key (as already proved with post#98 up to post#103 in it's thread) is not useful at all from the point when you unlock your bootloader.

But you are most likely right, that no other functions depend on it.

Do you think that restore of TA did nothing?

That's very simple - just restore your TA-unlocked.img backup and see for yourself - the camera will get you only solid green pictures (at least with oreo firmwares).

Another thing to test may be widevine level 1.

The restore of TA recovers device master key.

My opinion (which is still to be proved or disproved yet) is that device master key is necessary to enable (decrypt) the other keys, like SOMC FIDO, SOMC Attest, CKB and most importantly WIDEVINE key.

The service tests security screen does not provide info if a key really works or not - as had already been discovered with the Android Attest Key by [MENTION=4782500]pbarrette[/MENTION] (see post#13) - it can be made provisioned by simply creating of 5 empty files in keymaster64 directory, but that obviously will not make it work.

It is very possible that other keys may not work even when the status looks all right, particularly when using any kind of drmfix without restore of locked TA state.

You can hardly compare xz1c to previous phone generations - as software security is improved, the similar improvements are done in hw.

That is why "rooting and keeping all stock functionality" is very hard for any recent phone.

Try to do the same thing (TA restore) with any newer xperia - you would not get it with XZ2 compact already, not to mention anything newer than that.

It is quite possible that XZ1c will be last xperia compact phone where you are able to restore functionality that sony designed to be lost with bootloader unlock.
[Sony Xperia XZ1 Compact ROMs, Kernels, Recoveries,] mirhl:
Can you put a big disclaimer/FAQ underlining >SD835 security is different?

I'm tired of people missing this basic fact.

(or >2018, anybody knows anything about XA2?)
[Sony Xperia XZ1 Compact ROMs, Kernels, Recoveries,] pbarrette:
Well..

One of my original ideas was to try exploiting the Sony Companion backups.

In theory, if you could decrypt and re-encrypt a backup, then you should also be able to inject a malicious payload into it.

If that payload were data that is used by a privileged app, then you could potentially get an already privileged app to execute your commands.

Google and Qualcomm have certainly hardened Android, but has Sony hardened its own privileged software to the same degree?

The problem that I ran into is that I couldn't figure out how the backups were being encrypted.
[Sony Xperia XZ1 Compact ROMs, Kernels, Recoveries,] j4nn:
preview of FOTA done with rooted XZ1c

Here is a preview video showing my current work - I just managed to patch xz1c kernel in a way that bootloader unlock is properly masked so it appears as still locked and at the same time it is patched to support magisk so that sony stock firmware does not detect it - links to preview video uploaded to youtube:

xz1c unlocked & rooted doing FOTA upgrade part 1

xz1c unlocked & rooted doing FOTA upgrade part 2

This allows (together with TA-locked restore recovering device master key after BL unlock) to successfully install a FOTA system upgrade while having root before and after the upgrade.

Also sony specific apps are offered to be installed/updated. All that even without having a google account signed in.

The patched kernel is not flashed, it is only booted via fastboot in order to keep stock fw untouched - this way FOTA can be verified and installed the same way as on a still locked phone.

Thanks to [MENTION=4403504]tonsofquestions[/MENTION] for a lot of testing of initial BLU-hiding kernel several months ago, when we first verified the concept to install FOTA this way (I did not have an unlocked xz1c that time). The initial BLU-hiding kernel was not compatible with magisk (it would not boot if used with magisk).

The problem of this concept is unfortunately the usability for normal users.

Currently I need to recompile the xz1c kernel to integrate the BLU-hiding patch together with magisk components.

And this is needed for each firmware version from which one should be able to do FOTA and have root there including all the drm keys (for sony proprietary features).

This is what I had in mind.

I would say this pretty much confirms all functionality of a locked phone have been restored (camera, video enhancements and now even fota upgrade of the whole system).

The kernel replaces the BL unlocked state with "Rooting status ALLOWED_SINCE_SL_IS_UNLOCKED" aka "Bootloader unlock allowed: Yes (simlock unlocked)" in order to differentiate from a still locked phone.

The phone's safetynet check can be fully verified including cts, magisk is working, still with the patched Auditor app true state about bootloader unlock can be seen coming from TEE (Trust Zone) due to working Android Attest Key.
[Sony Xperia XZ1 Compact ROMs, Kernels, Recoveries,] Unbounded:
[MENTION=4418980]j4nn[/MENTION], [MENTION=4403504]tonsofquestions[/MENTION]

Great work once again my friends.

With this work of yours, even though with unlocked bootloader, we can literally use our XZ1C like it was never unlocked.

Thanks a lot.

Cheers!
[Sony Xperia XZ1 Compact ROMs, Kernels, Recoveries,] MxdaF:
XZ1 compact G8441

It is there any workaround?

Bootloader unlock allowed: No

thanks
[Sony Xperia XZ1 Compact ROMs, Kernels, Recoveries,] SGH-i200:
You are a genius! :good: Great that this possible, but for now I'm not sure why I would want to use the patched kernel, since I can upgrade the firmware via newflasher.
[Sony Xperia XZ1 Compact ROMs, Kernels, Recoveries,] tonsofquestions:
Bravo, congrats! And you got it with Magisk, too? Amazing! Now for one with Microg! :P

I know you worked very hard on this for a long time, so I'm glad to see it come to light.

I also (personally) consider it unideal that it's basically a tethered root, even as an advanced user - there's no way to reboot the device without a computer and maintain the root, magisk, or FOTA, though rebooting again afterwards to boot the alternative kernel would work. That said, it's definitely better than nothing. :)

I personally don't think building a kernel for every firmware version would be necessary - unless you think there are major differences between kernels across network operators. Even then, I suspect getting some major milestones would be sufficient for historical purposes, an image for the (older) exploitable firmware, one for the last Oreo Rom (16.20), and maybe one or two in between, along with the most recent (and widely released) PIE firmware. I don't think it's unreasonable to have someone use newflasher to upgrade to a more recent ROM immediately after unlocking, which would cut down on all the minor steps along the way.

That said, I'm not sure how much work it is to do, and/or if there's a way for it to be automated with a script by the end user, to add the bits to a kernel pulled directly from the phone/downloaded image.

Either way, exciting times lie ahead!
[Sony Xperia XZ1 Compact ROMs, Kernels, Recoveries,] j4nn:
[MENTION=1995555]SGH-i200[/MENTION], obviously the way to upgrade via FOTA is not straightforward, so flashing a downloaded fw via newflasher is probably easier.

The thing is that you may just flash the kernel and not use FOTA or flash stock kernel just before trying FOTA to revert the change.

The benefit of using my kernel (flashed or just fastboot-ed) would be the proper hiding of unlock status allowing full cts safety net pass with stock firmware having working magisk, without need of any other patches (like those variants of secd to ignore unlock).

[MENTION=4403504]tonsofquestions[/MENTION], unfortunately each fw has it's specific kernel image which contains patch level explicitly set (different customizations do not matter though).

That basically does not allow to switch just kernel without the rest of the fw and does not allow downgrade when you boot newer patch level.

I guess there may be some way - if we could stop the stock fw to upgrade keys when newer patch level is booted - probably [MENTION=4297219]modpunk[/MENTION] knows more about this, as his twrp build does exactly that (twrp kernel boot.img contains patch level set to 2025-12-31 with comment "As TWRP doesn't do key upgrades it should just be fine using it for decryption").

The completely other thing is api incompatibility of few kernel builds - for example pie's kernel is definitely not compatible with oreo fw and vice versa, already because bootloaders (and from there loaded Trust Zone applets) are not compatible (tested myself).

I would like to convert my kernel patch to a loadable kernel module - that might allow better compatibility with multiple kernel builds.

But it is very difficult, because the functions I need to hook cannot be hooked easily without kernel rebuild.

It might be solvable if I could somehow remap read only kernel space memory regions for write access and make the patching on runtime from the kernel module.

That means a lot of work is still ahead.
[Sony Xperia XZ1 Compact ROMs, Kernels, Recoveries,] SGH-i200:
That sounds promising! :good:
[Sony Xperia XZ1 Compact ROMs, Kernels, Recoveries,] j4nn:
Here are few most important screenshots from the FOTA update video posted in post#623 with descriptions.

The rest of the screenshots in next post.

Point your mouse cursor over the screenshots in order to see the file name to read the "description".
[Sony Xperia XZ1 Compact ROMs, Kernels, Recoveries,] j4nn:
The rest of the screenshots from FOTA update video posted in post#623 (number of screenshots limited to 20 in single post - see the first 20 in previous post).

Point your mouse cursor over the screenshots in order to see the file name to read the "description".
[Sony Xperia XZ1 Compact ROMs, Kernels, Recoveries,] serajr:
[MENTION=4418980]j4nn[/MENTION] what a great achievement! This ^^ is absolutely amazing... Congrats! Respect!!

I own a XZ2 (XZ1 as well), so in case you want to test your hack on a A/B (seamless) device, just let me know.