yashsingh1/xdaforums huggingface dataset:
[Sony Xperia XZ1 Compact ROMs, Kernels, Recoveries,] shoey63:
[MENTION=4418980]j4nn[/MENTION]

- Fastboot boot boot-H8216-52.0.A.8.50-hideunlock.img on pure stock XZ2 firmware works as previously reported.

- On modified firmware it does not.

- Fastboot flash boot boot-H8216-52.0.A.8.50-hideunlock.img also does not work on modified stock xz2 firmware (kernel panic/bootloop)

Apparently this can be circumvented by some Magisk voodoo as reported by [MENTION=4898998]Klaus N.[/MENTION] albeit at the loss of camera preview.

- This must have something to do with the disabling of verification in vbmeta (the equivalent of disabling dm-verity in appended dtb for Slot A only devices)

Can you think of a Magisk-less workaround? We already know OTA will work with your current commits on pure stock, but how about simply hiding unlock status in user-space on modified systems to pass playstore verification while waiting for a firmware update? I have already tried unpacking the stock boot image and adding androidboot.verifiedbootstate=green to the command line, then repacking which also bootloops on Tama. This simple trick actually works on Yoshino though.

Looking forward to your thoughts on the matter.
[Sony Xperia XZ1 Compact ROMs, Kernels, Recoveries,] j4nn:
[MENTION=4898998]Klaus N.[/MENTION], [MENTION=4675598]shoey63[/MENTION], that is quite interesting behaviour - I mean the fact that camera works properly on unmodified stock fw running hideunlock kernel without use of any camera fix (for the cases that unlock has been done with fw older than 52.0.A.3.126)?

In my opinion, if disabling verity in vbmeta and the kernel (or flashing magisk) looses the camera preview - that's getting your phone to normal state, where you needed black preview fix because of the unlock on oreo, is not that right?

Can you please try this:

disable verity via that fastboot vbmeta command

flash the hideunlock kernel image that does not contain magisk (either from fastboot or from twrp)

flash magisk 18.1 from twrp (over the hideunlock kernel)

install serajr's camera fix as a magisk module (available in xz2 twrp package)

reboot

test camera

I've analysed the camera fix, it is about using tzsuntory TrustZone applet from 52.0.A.3.84 firmware, which is the last one, that had working camera for phones unlocked on any fw up to this version. So it replaces that tzsuntory applet with older version on newer firmwares. So I would assume if this fix worked for you before, it should work also with unlockhiding kernel.

Of course it is possible that the behaviour may be different as was the case with sony ric daemon on oreo, which allowed to boot with unlocked bootloader and magisk integrated, but bootlooped in case of kernel hiding bootloader unlock as detection of magisk was active assuming still locked phone.

I would still find it quite odd as the fix is in use of old tzsuntory TrustZone applet, which runs in TrustZone, where unlock cannot be hidden (it is above linux kernel)
[Sony Xperia XZ1 Compact ROMs, Kernels, Recoveries,] Klaus N.:
[MENTION=4418980]j4nn[/MENTION]

Yes, I already tried also this, but [MENTION=3989065]serajr[/MENTION]'s camera fix does only the trick with the pure stock kernel for some reason...

Btw, I've sent you a pm.

Sorry for the noob question, but why is your hideunlock kernel less than half of the size compared to the pure Sony stock kernel?

Only an idea: Maybe we've to flash your hideunlock kernel on both slots (A and B), switch back to the slot of the fw, and then flash Magisk?
[Sony Xperia XZ1 Compact ROMs, Kernels, Recoveries,] j4nn:
[MENTION=224920]Klaus[/MENTION] N, so not to misunderstand: the camera (preview or what ever else) now works with pure stock firmware running the original stock fw kernel?

And if you boot any of those hideunlock kernels, you get black preview? While previously working fix from MartinX3/serajr now does not help (in case of hideunlock kernels)?

If camera works now with stock fw (including stock kernel), what happens if you use the camera fix?

The kernel size is what you get by compiling the kernel according to sony's howto.

I guess the stock kernel is bigger because they flash the whole partition size.

I am not sure if it might be a problem if only the relevant size of the kernel is flashed into that 'boot' partition.

I have no idea how those slot A / B stuff is supposed to work.

You should always check if my kernel runs or not by examining the build date - that should be different vs the one in stock kernel.
[Sony Xperia XZ1 Compact ROMs, Kernels, Recoveries,] Klaus N.:
- The camera preview

a) does not work

aa) with pure Sony stock kernel

ab) with your hideunlock kernel - regardless of installing camera fix (serajr's or Martin's) or not

b) works

ba) with pure Sony stock kernel + e.g. serajr's Magisk camera fix (I didn't try Martin's fix for now, as it changes /system/vendor permanentely - removing serajr's Magisk fix is by far easier/faster, ofc ;))

bb) -/- (nothing else!!!) :p

Btw, I've checked and tried with both builds of your hideunlock kernels (I mean the ones repacked without Magisk - different date, minimal different size [I guess, the 1st is with dm-verity on, the 2nd with dm-verity set to off]): Both of them are booting into system, but for some reason the camera fix does not work. Maybe [MENTION=3989065]serajr[/MENTION] has an idea?

[MENTION=4418980]j4nn[/MENTION]

The XZ2 has no recovery partition anymore. That means, if you want to have TWRP - e.g. by flashing a bootimage you will lose it -, it's always neccessary to do the whole installation procedure of the op of Martin's thread again (otherwise you cannot install Magisk...).
[Sony Xperia XZ1 Compact ROMs, Kernels, Recoveries,] j4nn:
[MENTION=4898998]Klaus N.[/MENTION], in my opinion, the reason that camera fix does not work with hideunlock kernel is most likely that the fw thinks that bootloader is still locked and therefore tries to use drm for camera, calling different code paths than in case of unlocked bootloader. The keys had been lost though, drm and therefore camera with (locked appearing BL) does not work.

The thing is that you did not need any camera fix on unlocked bootloader with fw up to 52.0.A.3.84 including.

I believe that if tried to test any of those older firmwares with a (to be built) kerne hiding unlocked bootloader, you would get the same behaviour (not working camera), while there would be no camera fix (as it has not be needed there).

Now there are phones that have been unlocked on 52.0.A.3.126 fw version (or any newer).

These phones do not need any camera fix for any recent fw - if I am not mistaken.

I wonder what would happen if kernel hiding bootloader unlock was tested there - if camera stopped working too or not.
[Sony Xperia XZ1 Compact ROMs, Kernels, Recoveries,] Klaus N.:
I believe you're right. But I don't want to throw it into the waste. Shame on Sony that there's not fix for that; I have to say, it's unbelievable how they treat their customers! Maybe they want to lose them?
[Sony Xperia XZ1 Compact ROMs, Kernels, Recoveries,] j4nn:
[MENTION=4898998]Klaus N.[/MENTION], exactly, shame on sony for that.

Even though not tested, from the above behaviour I think it is more probable that camera would not work with kernel hiding bootloader unlock with the phones unlocked on 52.0.A.3.126 fw (or any newer) either.

I wonder what they intended to do with that change between 52.0.A.3.84 and 52.0.A.3.126.

Checked some time ago the differences of that tzsuntory blob only to discover that the change is so minimal as one byte value parameter has been flipped - screenshot attached.
[Sony Xperia XZ1 Compact ROMs, Kernels, Recoveries,] Klaus N.:
Their intention is to lose customers, ofc.

I really wonder if I ever buy a Sony device again (I've already bought 7 Sony smartphone devices until now) - probably the XZ2 will be the last one. I will write to Sony tomorrow, because I'm that angry!!!

Good night
[Sony Xperia XZ1 Compact ROMs, Kernels, Recoveries,] Sungfive:
Is it possibile to restore TA with android Pie on Xz1 or have I to downgrade?

Thanks

Inviato dal mio G8341 utilizzando Tapatalk
[Sony Xperia XZ1 Compact ROMs, Kernels, Recoveries,] j4nn:
[MENTION=6417612]Sungfive[/MENTION], restore TA possible on pie, backup only on oero with exploitable fw version.

If you already have TA backup, it has been tested to unlock on pie and restore TA there too - see post#954.
[Sony Xperia XZ1 Compact ROMs, Kernels, Recoveries,] Sungfive:
Really thanks. I've already the TA backup but probably I messed up something with the restore.

Have I to restore it through TWRP?

In case how do I do it?

Simply flash the img right (in which partition)?

Thanks
[Sony Xperia XZ1 Compact ROMs, Kernels, Recoveries,] j4nn:
[MENTION=6417612]Sungfive[/MENTION], just boot twrp and use adb shell - there you have root by default, use the same commands as in the howto. Just be sure not to make any mistake - flashing garbage into TA results with a hard brick.
[Sony Xperia XZ1 Compact ROMs, Kernels, Recoveries,] Sungfive:
Sure. Thanks

Inviato dal mio G8341 utilizzando Tapatalk
[Sony Xperia XZ1 Compact ROMs, Kernels, Recoveries,] Sungfive:
Finally I solved my problems with DRM.

I'm on 47.2.A.10.45. Previously I followed the entire tutorial resulting in no DRM keys restored. Probably I made something wrong when restoring the TA-locked (with android oreo FW) and with the temp root shell.

I didn't know which steps I made wrong so I passed to Android Pie without DRM features.

To restore them:

1) I went to TWRP and flashed a new stock firmware with full reset with newflasher

2) I flashed aroma twrp for xz1 https://xdaforums.com/xperia-xz1/themes/twrp-3-3-0-aroma-compatible-ultra-low-t3922527 which was the only one I found that let me decrypt data. Came back to TWRP in order to use ADB shell to restore the TA-locked.img as explained in post #954 URL="https://xdaforums.com/showpost.php?p=79587510&postcount=954"]https://xdaforums.com/showpost.php?p=79587510&postcount=954[/URL]. I couldn't use ADB shell (device unauthorized), I flashed magisk and it seemed to work.

3) Restored the TA-locked.img with adb shell (with command prompt on PC).

adb push TA-locked.img /data/local/tmp

and then

cd /data/local/tmp
dd if=TA-locked.img of=/dev/block/bootdevice/by-name/TA
sync
sync

4) Tried to boot in order to see if it could work, unfortunately I had still no DRM features enabled, they were not recognized by the system

5) Tried to use https://xdaforums.com/xperia-xz1/development/rooted-kernel-hiding-bootloader-unlock-t3898833.

Flashed it with

fastboot reboot bootloader

fastboot boot boot-G8441-47.2.A.10.45-hideunlock-rooted.img

The XZ1 booted, DRM features were enabled, so I flashed (wrongly) Magisk. I restarted the phone but he had no more DRM features. So i tried to flash the boot.img used previously (I had only booted it before). Phone went in bootloop.

6) I flashed a new firmware, booted the phone and the boot.img as in point 5, checked DRM (OK), restarted in bootloader and flashed the boot.img

fastboot flash boot boot-G8341-47.2.A.10.45-hideunlock-rooted.img

TA restore has been verified on 47.2.A.10.45

Root, DRM and FOTA perfectly working
[Sony Xperia XZ1 Compact ROMs, Kernels, Recoveries,] :
i read a while ago, that unlocking the bootloader without backing up the ta-partition doesen't break the camera anymore.

So... is it really needed to back them up? what are the disadvantages?
[Sony Xperia XZ1 Compact ROMs, Kernels, Recoveries,] Trigger1112:
Hi, I've read through the OP and munted's guide and have the TA backup file but before I unlock the bootloader I need to know, is there a way to check the integrity of the TA backup before unlocking?

P.S. Thanks to [MENTION=4418980]j4nn[/MENTION], [MENTION=3463150]munted[/MENTION], [MENTION=4120742]munjeni[/MENTION] and everyone else who has contributed.
[Sony Xperia XZ1 Compact ROMs, Kernels, Recoveries,] j4nn:
[MENTION=7835385]TheVan28[/MENTION], camera seems to somehow work on pie even if drm keys have not been backed up & restored.

But there are some doubts about the quality vs still locked oreo - see post#914 and also the other threads referenced in that post, which compares camera quality on xz1c locked oreo vs unlocked lost drm pie.

There may be other features conditioned by present device master key - see post#967 with specification details about widevine L1 being encrypted by device unique key.
[Sony Xperia XZ1 Compact ROMs, Kernels, Recoveries,] vofferz:
[MENTION=4418980]j4nn[/MENTION] What's your opinion on Android attest key and flashing pesist...sin?

If I understand correctly, Google sees bootloader unlocking even with your patched kernel if the attest keys are present and that disables the ability to use nfc payment on GPay.

Are the attest keys needed anywhere? Can they be backed up somehow?
[Sony Xperia XZ1 Compact ROMs, Kernels, Recoveries,] j4nn:
[MENTION=7803472]vofferz[/MENTION], no time right now, do not flash persist, we can test if backup possible
[Sony Xperia XZ1 Compact ROMs, Kernels, Recoveries,] :
I take very few photos, and as long as they look better than those from my current z5c, i am happy with it. I'm sure that both pictures will look the same when you send or post them on social media thanks to compression.

For widevine, i never use services that require it (i would still be ok with 480p netflix on such a tiny screen).

Apart from that, if there is nothing else to lose will i be fine without keys?
[Sony Xperia XZ1 Compact ROMs, Kernels, Recoveries,] j4nn:
[MENTION=7835385]TheVan28[/MENTION], I guess you may be right, it may not be that bad if the drm keys were lost.

In my opinion, having the opportunity (to backup TA via exploit) is quite rare (already since nougat not possible with sony phones /correct me if I am wrong please/) as android security (including hw security features) improves a lot every year.

When you unlock, all your data will be erased the same way as full factory reset. That means the same result as you would go through when downgrading to exploitable fw to be able to get the TA backup.

There are obviously other features that get disabled when you lose drm keys / unlock, I am not sure about all of them, but just to remember few other: sony backup tool, wifi display mirroring (miracast), video image enhancements, I am not sure about some audio features (some mentioned them if I am not mistaken)...

So my question would rather be - why not to do the backup when you can?

Why not to be on the safe side that the key is present and any feature not verified to work without the key would be more likely to work when the key is present on an unlocked phone?

It is not that difficult as it may appear from the (longer) howto...
[Sony Xperia XZ1 Compact ROMs, Kernels, Recoveries,] j4nn:
[MENTION=7803472]vofferz[/MENTION], your understanding is correct (at least it has been like that recently).

The thing is that it is not that straightforward - having Android Attest Key provisioned and working (verifiable via the patched Auditor app) allows a third party to see the truth about bootloader unlock status.

Not having the key results with an exception from the attestation api - but that can be considered by a third party either as that the phone is too old to have the security hw features for the attestation, so it may be let to pass by default or it may be required to have the api working letting only phones with those security features working and providing "locked" results.

There even may be a list (and google has it for sure) which phone models do support those security hw features and therefore should have the attestation key working.

So it may _currently_ help not to have the attestation key working in order to pass google pay checks as _currently_ implemented.

But that may change later - google may simply insist to get the signed attestation data from a phone that is listed as having those features - very easy from google side not even influencing the number of compatible phones (if not considering the modded/unlocked ones - those they actually like to ban).

The thing is that having bootloader unlocked makes the Attestation key useless, as it will only attest the unlocked state.

The only hope here is that somebody comes with a TrustZone exploit which would allow to fake the unlock status in TrustZone so that the attestation key could sign the data with "locked" state.

Such exploit might even make re-lock possible (or even an unlock without sony key).

Unfortunately that is another level of exploitation, above linux kernel and it is not very probable that it would ever happen.

To sum it up, when you flash persist, your lose the Android Attest Key forever, it seems not to be possible to restore it from 'persist' partition backup (as tested with [MENTION=4688377]russel5[/MENTION]).

So you would close your option to wait for a trustzone exploit, after which the key could eventually get some use.

There may be some ways to simulate losing the key to allow current google pay checks to pass - without flashing (i.e. erasing) persist.

That might possibly avoid losing the key forever - but it is untested and there is still some risk to lose the (otherwise useless) key forever.

Let me know please, if you like to do some testing in this area.
[Sony Xperia XZ1 Compact ROMs, Kernels, Recoveries,] :
Widevine L1 is working fine on my friends unlocked xz premium without drm
[Sony Xperia XZ1 Compact ROMs, Kernels, Recoveries,] Gaaat:
Not sure if useful, but on XZP with android attest key lost (doesn't appear in security test menu) , fw 10.45, 10.45 bootloader with j4ann unlock hiding patch and latest magisk with magisk hide GPay nfc payment is working.
[Sony Xperia XZ1 Compact ROMs, Kernels, Recoveries,] j4nn:
[MENTION=7835385]TheVan28[/MENTION], [MENTION=9165204]Gaaat[/MENTION], XZ Premium is different comparing to XZ1 and XZ1 Compact - xzp started with nougat where android attestation has not been required to be implemented and therefore it never had Android Attest Key (so that is why you do not see it in security menu - it never has been there). XZ1 line started with oreo, having the Android Attest Key there.

Google is the one who certifies devices that comply with their set of requirements, so google has a list of devices, to which Android Attest Key has been provisioned. This is the reason I think that it may be very easy for google to require the signed attestation data that include locked state from devices, that google knows, that have the hw attestation implemented.

So xzp has currently some advantage, that it did not come with that hw attestation, so google will never insist on it on that phone - unless it is decided not to support devices without hw attestation anymore (those days might come sooner or later bringing very bad news to all custom roms).
[Sony Xperia XZ1 Compact ROMs, Kernels, Recoveries,] thanktuzg:
my device is xz2 8.0 Oreo, in step 5 it running forever, please help me
[Sony Xperia XZ1 Compact ROMs, Kernels, Recoveries,] j4nn:
[MENTION=6778066]thanktuzg[/MENTION], I am sorry - but have you read the "SUPPORTED TARGETS" paragraph of the howto?
[Sony Xperia XZ1 Compact ROMs, Kernels, Recoveries,] KpKi:
DRM restored but not sure if it makes any difference

Greetings,

So i am using an XZ1 with the latest software 47.A.10.62. I backed and restored my TA keys. Lost the useless attest keys (NVM them).

Without the patched kernel none of the Xperia exclusive features actually works ! Besides the camera which doesn't need DRM anyway on pie.

Any idea what I did wrong?
[Sony Xperia XZ1 Compact ROMs, Kernels, Recoveries,] lilbrat:
Your ok.. now go here... https://xdaforums.com/xperia-xz1/development/rooted-kernel-hiding-bootloader-unlock-t3898833/page9 , download the kernel patch it with the latest magisk ,then flash it.. check page 8 if your have the single sim..
[Sony Xperia XZ1 Compact ROMs, Kernels, Recoveries,] j4nn:
[MENTION=2349099]honz[/MENTION], if you need to stay on oreo, there is no point to be able to install system fota, so you can "fastboot flash boot kernel.img" to use the kernel with normal power/boot up. You can flash either version, the noverity one allows you to make modifications in system/vendor partitions. Otherwise all customizations would need to be done via system less magisk modules-a benefit might be better hiding level for Google pay or some other apps. You can disable the notification in the menu afaik.

Magisk 19.3 has finally merged in my patch for Sony ric daemon of oreo FW, so it's the first version which can work with upstream magisk binaries. Still the kernel needs to be magisk integrated with a special script-your attempts to switch magisk version will fail because of not properly hiding from Sony ric daemon.
[Sony Xperia XZ1 Compact ROMs, Kernels, Recoveries,] j4nn:
As some may noticed already I have built 10.62 blu hiding kernel using just recently released Sony kernel sources: the kernels are available on my androidfilehost:

https://androidfilehost.com/?w=profile&uid=5862345805528050190

It is a shame Sony released sources for a may patch level while releasing fw update with June(?) patch level (10.80 version) without sources.

The kernels have magisk 19.3 integrated with verity on and are not tested, just assumed to work.
[Sony Xperia XZ1 Compact ROMs, Kernels, Recoveries,] blue-spiderfire:
How can I restore my TA.img, which I backed up priviously?

Situation

- lineago 15.1

- I think, I didn't followed the instruction (flashing lineage OS) correctly and missed the point:

Remove the follwing from the folder so they won't get flashed:

all *.ta files (keep the one in the boot/ directory)

cache*.sin

fotakernel*.sin

kernel*.sin

system*.sin

userdata*.sin

vendor*.sin​
[Sony Xperia XZ1 Compact ROMs, Kernels, Recoveries,] PeppeBove:
can i flash this on 47.2.A.10.80?
[Sony Xperia XZ1 Compact ROMs, Kernels, Recoveries,] ecksdeeay:
Is not built with/for that kernel release.
[Sony Xperia XZ1 Compact ROMs, Kernels, Recoveries,] PeppeBove:
sorry, i'm OP, but...

how can i have a working twrp with latest firmware?

i just downgrade, backup TA, temporary root, restore TA, upgrade to latest release .80...

but now how can i have root and twrp working? i lost last step i think

PS:

i tryed "fastboot flash recovery twrp-3.3.1-0-lilac-pie-2.img" but i'm still not able to run recovery
[Sony Xperia XZ1 Compact ROMs, Kernels, Recoveries,] ecksdeeay:
The latest kernel is 62.

80 kernel source has not been released from Sony.

It's all been mentioned by j4nn many times in his threads here, just a few posts back actually
[Sony Xperia XZ1 Compact ROMs, Kernels, Recoveries,] PeppeBove:
so... sorry to insist, but now only way to got root is to downgrade again to 10.62?right?only way?
[Sony Xperia XZ1 Compact ROMs, Kernels, Recoveries,] lilbrat:
First help folks out by telling us which device you have..
[Sony Xperia XZ1 Compact ROMs, Kernels, Recoveries,] ecksdeeay:
Xperia ZP yes 10.62.

twrp is best to try 3.2

I mentioned it back here somewhere

https://dl.twrp.me/maple/twrp-3.2.3-0-maple.img.html

That works for G8141.

The latest didn't mount /data
[Sony Xperia XZ1 Compact ROMs, Kernels, Recoveries,] j4nn:
You can restore your TA even if running LOS. Just get a root shell and use the dd command exactly the same as in the howto.

You may get a root shell either via magisk or installing su los addon or just booting twrp (there via either adb or built-in Terminal).

Not removing those files does not matter (if losing your data due to userdata flashing had been ok for you). That instruction is there to save some time (and flash erase cycles), because for example flashing stock system and vendor partition just before flashing los system and vendor partitions is really useless - it gets all overwritten.
[Sony Xperia XZ1 Compact ROMs, Kernels, Recoveries,] PeppeBove:
Obviously I have a xz1c, isn't this my phone forum's section?

Inviato dal mio G8441 utilizzando Tapatalk
[Sony Xperia XZ1 Compact ROMs, Kernels, Recoveries,] j4nn:
You could just flash upstream magisk zip over your 10.80 stock fw (i.e. without kernel hiding bootloader unlock) to get root, couldn't you?

edit: or see the next post.