yashsingh1/xdaforums huggingface dataset:
[Sony Xperia XZ1 Compact ROMs, Kernels, Recoveries,] Blu-eyes:
I think that with IDA trace replayer and good knowledge of arm Assembly should be possible to backport, but this will be time-consuming, this would be a good chance to learn new know-how, but I think that this time can be spent on better things.

semiOT:

If you like learning Arm Assembly, starting with some Cortext Microcontroller is really fun and stimulating and gdb here is very usefull; if you need some hint, I'm here.

What :eek::eek: ? In my option is a lot of time spent on a "stupid" fota update process, but at least you learned gdb :p :highfive:.

At least I like this reversing stuff for leaning think, but I won't give all my day to it, There are a lot of good things out of there.

("Time is a valuable thing. Watch it fly by as the pendulum swings" L.P.)

Not misunderstand me, I really really appreciate your work, what I say early is only my opinion.

See you,Blu-eyes
[Sony Xperia XZ1 Compact ROMs, Kernels, Recoveries,] munjeni:
Thats wasting of time cracking android stufs, I know that, I was inactive on xda almost for year, didn't touched android in that time, have no idea why I got back. It just was something which I wanted to do some years ago, to analyse fota package, I had some free time for that now, but I have some better hobby than cracking android

https://xdaforums.com/showpost.php?p=80235988&postcount=1140 (done)

https://xdaforums.com/showpost.php?p=80521515&postcount=1272 (need to finish and start small production)

I'm not interesting in asm stufs, only c, and php, php is my first language I have learned, c learning :)

Edit:

About fota update, I found something, LzmaDec_DecodeReal and LzmaDec_DecodeReal_2 functions, one of it I'm believing obfuscate compression, thats why we see "strange" bytes after uncompresing with regular lzma tool, whole package is streamlined by blocks, I'm believing most of the blocks is twice compressed. Reverse enginering this might be nightmare.
[Sony Xperia XZ1 Compact ROMs, Kernels, Recoveries,] Blu-eyes:
I agree with you about the difficult to reverse, but now I have to make a question

What is the point/(adavantages) of reversing fota_ua unpack function?

OT:

I really like https://xdaforums.com/sho...postcount=1140, this will be only commercial or you will open-source it?

It seems very interesting, The output is Filtered against high-frequency switching noise?
[Sony Xperia XZ1 Compact ROMs, Kernels, Recoveries,] munjeni:
Just not like when I start something and stuck :)

Yes, constant current and constant volt, I'm based it on manufacturer datashet http://www.xlsemi.com/datasheet/XL4015 datasheet.pdf , it have prety well regulated output, 3mA noise but if you add some good condenseur at out it going very well, generaly xl4015 is good for money. Whole thing is similar to https://hacktronics.co.in/step-down...ger-power-step-down-module-with-usb-voltmeter , but I have done it with oled, adc, dac...etc and it have much better precission than comercial one, and it is fully digital. l I had plan to start selling it but its not worth for money since I need to spend two hours on it and half of hour to programm it and measure, probably kid from my brother will do small production :) I will try to make 3 in 1 revork station, it will be worth for money, I'm just lazy to finish it and create plan :) No I have no plan curently to open source it, only if kid is not interested for it I will put on github
[Sony Xperia XZ1 Compact ROMs, Kernels, Recoveries,] j4nn:
[MENTION=8662566]kaleyaki[/MENTION], you can disable updates, just enter the settings->Software update and with upper right popup menu, you can disable auto download of system (and/or apps) updates. There is another option (Automatic system updates) in developers menu too.

When you start google play app, there is upper right popup setting: Auto-update google apps, you might turn that off too (I am not sure if this is always there or only when not signed in google account).

And you may also disable the notification, but that should not be needed, if you apply the above.
[Sony Xperia XZ1 Compact ROMs, Kernels, Recoveries,] xtess3ractx:
Just seen this was away for work thanks for replying, it says bootloader unlock allowed but I'm unsure about simlock
[Sony Xperia XZ1 Compact ROMs, Kernels, Recoveries,] mirhl:
https://research.checkpoint.com/the-road-to-qualcomm-trustzone-apps-fuzzing/

*tips*
[Sony Xperia XZ1 Compact ROMs, Kernels, Recoveries,] j4nn:
rooting still locked XZ1c, including permanent twrp and magisk, possible to run LOS16

Please see my thread started in the Security Discussion forum subsection:

Android PIE VerifiedBoot Bypass: sony xperia XZ1 locked bootloader permanently rooted

Showing video recordings of xz1c still locked bootloader rooting, getting permanent twrp recovery and permanent magisk, installing Lineage OS 16.0 as a secondary OS to SD card with dual-boot option via recovery boot or just replacing sony stock fw fully with LOS16, having bootloader still locked (as shown, no bootloader unlocked warning screen) keeping Android Key and ID Attestation still working (the android attest key).
[Sony Xperia XZ1 Compact ROMs, Kernels, Recoveries,] munjeni:
I'd like to see bypass on abl signature check and possibility for abl replace with simple uefi bootloader :D In that case xz1c is my next phone. I thinked to buy https://puri.sm/products/librem-5/ but its like building brick :(
[Sony Xperia XZ1 Compact ROMs, Kernels, Recoveries,] YamiYukiSenpai:
Is TA backup necessary if we don't have plans on going back to stock Xperia?
[Sony Xperia XZ1 Compact ROMs, Kernels, Recoveries,] j4nn:
[MENTION=6071751]YamiYukiSenpai[/MENTION], that is up to you. I would always backup if there is a way.

Consider this: LOS16 contains 481MB of proprietary sony binary blobs retrieved from stock rom (as of current version based on 47.2.A.11.228 fw). Are you sure that there is no code in those blobs that could check drm keys?

And for example provide different stock camera quality in case they are present vs not present?
[Sony Xperia XZ1 Compact ROMs, Kernels, Recoveries,] tonsofquestions:
Bravo, [MENTION=4418980]j4nn[/MENTION] truly remarkable!

I also really enjoyed the videos and silent-movie-style subtitles. :)

I'm guessing (but could be wrong) that it involved some exploit to flash the bootloader, allowing flashing of other unsigned data.

Is it possible to restore back to a stock state afterwards, if one wanted to sell the phone and have it be locked down again?

When the phone is still running the stock ROM, but has magisk and/or twrp, how does it deal with system updates? It may be a moot point by now, with no Android 10 (we can hope they'll change their mind), but I know you spent a lot of time working on the hideunlock kernels, so I was curious how that interacted with this exploit/setup.

I'm looking forward to the (hopefully soon) release and/or more detail! :)
[Sony Xperia XZ1 Compact ROMs, Kernels, Recoveries,] j4nn:
[MENTION=4403504]tonsofquestions[/MENTION], it is possible to revert the exploit.

Kernel hiding bootloader unlock is obviously not needed as bootloader does not need to be unlocked.
[Sony Xperia XZ1 Compact ROMs, Kernels, Recoveries,] mcmanuf:
I was about to unlock my bootloader, and then I saw your magnificient update, of being able to root without unlocking bootloader. WOW.

I understand you need to think about how to progress, but do you advise me to wait? Your best guess, are we talking hours or days or weeks?

My finger are tingling to get started, but I would hate myself for not waiting a few days when you have just made a breakthough, and have to live with the annoying unlocked boot message when unlocking bootloader, if I can avoid this by being patient.

What to do. ohh what to do.
[Sony Xperia XZ1 Compact ROMs, Kernels, Recoveries,] mcmanuf:
Discrepency in downgrading guide, regarding *.ta files?

------------- deleted -----------
[Sony Xperia XZ1 Compact ROMs, Kernels, Recoveries,] j4nn:
[MENTION=1958376]mcmanuf[/MENTION], I am not sure when the new exploit is ready for release (and there is also the question if it should go the responsible disclosure way or not, considering the misuse risks).

Certainly it has many advantages but it also has limitations. It is also more difficult and lot more dangerous to apply than renoroot (which is piece of cake in comparison).

It might look very comfortable from the preview videos, but they show only the final usage phase - the new exploit should be used only by advanced users who understand what they are doing.
[Sony Xperia XZ1 Compact ROMs, Kernels, Recoveries,] mcmanuf:
Thank you for the explanation. I hope you will have good luck on your further endeavors.

Maybe one day we will be able to relock bootloader. Otherwise it's also perfectly fine, and this phone is nearing EOL as Sony has abandoned it. Maybe we will still have some security updates in near future, but im not convinced based on Sony behavior and financial situation.

I have pulled the trigger and unlocked my bootloader. Hope it will not haunt me later ?

Thanks
[Sony Xperia XZ1 Compact ROMs, Kernels, Recoveries,] tonsofquestions:
Ahh, sorry. I was unclear.

Yes, I know that the patch is unnecessary to hide the unlock state.

However, we did discover that the updates would fail if, say, a new kernel had been flashed to the phone (as opposed to fastbooting off of the image directly).

So my question was whether this exploit prevented system updates by itself, or if the could still be applied from the ROM - though I realize that if you patch magisk on the phone, that will probably block it, too.
[Sony Xperia XZ1 Compact ROMs, Kernels, Recoveries,] munjeni:
Number 5 almost done :) Implemented overlay menu and screen control, menu is overlayed over every screen (must implement hide menu, camera menu...etc) start-stop camera, start-stop jbc, temerature setup ...etc, so powerfull tool, a lot of work. Whole overlay is managed with gd library in combination with raspberry pi libs, wow! :)

https://ibb.co/gMMpjhT

https://ibb.co/HN7jsRY
[Sony Xperia XZ1 Compact ROMs, Kernels, Recoveries,] papin97:
Hello [MENTION=4418980]j4nn[/MENTION]

First, I want to say thank you for your dedication to this phone model.

I just got this XZ1 SOV36 and this phone is currently using G8341 firmware (that wasn't me) (software info screenshot is provided below).

I want to root this phone (and possibly reverting to stock SOV36 because of the battery drain issue while keeping the benefit of unlocked SIM that is currently present in this firmware), and DRM still intact. Either unlock BL or keep locked BL does not matter. I am using MacOS (Windows is okay too).

Can you make a step-by-step guide for me? Thank you in advance. I hope you can help me.

Screenshot of software info attached below
[Sony Xperia XZ1 Compact ROMs, Kernels, Recoveries,] j4nn:
[MENTION=5125085]papin97[/MENTION], if your xz1 sov36 phone is currently running G8341 fw, than you may try to downgrade to 47.1.A.2.324_CE1 G8341 fw in order to get temp root via renoroot as described in the OP to get locked TA partition backup. Probably better to skip flashing bootloader (the 'boot' subdirectory). After you have your TA backup, you may flash a sov36 fw.

Does your rooting status say "Bootloader unlock allowed: Yes" in service menu?

If so you may proceed with bootloader unlock as described in the guide. If it says No, having TA backup is handy in any case.

The new exploit for permanent root without bootloader unlock is not yet finished, not ready for release. Also there is the question about eventual responsible disclosure...
[Sony Xperia XZ1 Compact ROMs, Kernels, Recoveries,] papin97:
Should I check the bootloader unlock status before or after flash a sov36 fw? Currently it says "No".
[Sony Xperia XZ1 Compact ROMs, Kernels, Recoveries,] j4nn:
[MENTION=5125085]papin97[/MENTION], the bootloader unlock status will not change with flashing of different fw. You can backup your TA just in case, it may get handy eventually. But you will not be able to unlock by entering sony unlock code. You may try going to your gsm operator to get the bootloader unlock-able (if you succeed with that, make a new TA backup before unlocking).

Please do not full quote a previous post while posting a single line. Thanks.
[Sony Xperia XZ1 Compact ROMs, Kernels, Recoveries,] sabotage154:
quick question here. i already backuped my TA using your method here. Thank you for that. I was thinking of trying other roms like lineage or resurrection remix. when i flash those i beleive my TA backup only works on stock and will not work on custom roms. the custom roms will either have drm fix or not.

If i flash custom rom do i risk losing my ability to keep my TA backup if i decide to restore back stock rom with DRM backup?

In other words will i permanently lose ability to restore stock rom with my DRM backup if i try a custom rom?

what do i do to make sure i dont lose my ability to revert back to stock rom with DRM backup?

thank you.
[Sony Xperia XZ1 Compact ROMs, Kernels, Recoveries,] j4nn:
[MENTION=4002550]sabotage154[/MENTION], that is a difficult question. I would guess that in 99% you should always be able to go to stock rom and use your locked TA backup and get all drm functions working if using the kernel patched to hide bootloader unlock.

But nobody can ever know if some kind of drm fix would not mess up security settings of your phone that might be stored somewhere else than TA partition, like for example keystore or some rpmb stuff.

In my opinion there is no PIE based rom that would use any kind of drm fix, so using those, should be safe.

With oreo, there has been a drm fix, that uses fake credentials, that might in theory has some unknown influence, but it is very unlikely - I did not encounter anything like that during my testings...
[Sony Xperia XZ1 Compact ROMs, Kernels, Recoveries,] JimKatsanos:
quick question: Am i losing something by not backing up the DRM Keys? Using android 9(renoroot etc dont work, probably due to exploit patch). I mean anything (image processing, night photos performance, all this screen enchancement stuff or even HDR)

Also, is it possible to flash an older stock rom without unlocked bl, so that i can use the exploit?
[Sony Xperia XZ1 Compact ROMs, Kernels, Recoveries,] vofferz:
Yes. Read the first posts of this thread.
[Sony Xperia XZ1 Compact ROMs, Kernels, Recoveries,] sohrab1985:
I just backed up my XZ1 TA image and unlocked the bootloader and then restored the TA image. Everything just work fine but in the service menu as you can see in the screenshot it says "bootloader Unlock allowed: Yes" and also SOMC Fido Key and Android Attest Key say "NOT PROVISIONED" (second screenshot attached) . Is it supposed to be like that? or it should be "bootloader Unlock: Yes"

And another question, now that I have root, if I want to flash the locked TA image just to be sure, what should I do? I mean what command should I use?

Thanks.
[Sony Xperia XZ1 Compact ROMs, Kernels, Recoveries,] SXUsr:
Re read the OP, which you should have done a few times already before getting this far.
[Sony Xperia XZ1 Compact ROMs, Kernels, Recoveries,] sohrab1985:
Believe me I've read the OP, not just the OP but also [MENTION=3463150]munted[/MENTION] pdf guide, if I haven't, I wouldn't be able to backup my device TA and get root, so that's not the problem and as I said before everything works just fine. My question was simple, I just want to know if I want to flash the locked TA just to be sure what command should I use? Because I have root now and I don't need to use renoroot and all those commands.

So the question is if this command is what I look for or not:

adb push TA-locked.img /data/local/tmp

BTW, [MENTION=4418980]j4nn[/MENTION], you did a great job making this tool, thank you so much. And thanks to [MENTION=3463150]munted[/MENTION], for his great guide. It's really helpful.
[Sony Xperia XZ1 Compact ROMs, Kernels, Recoveries,] SXUsr:
The answer you're looking for should be simple if you've read the OP. You're looking to be spoon fed instead of just applying what you already know.
[Sony Xperia XZ1 Compact ROMs, Kernels, Recoveries,] sohrab1985:
I don't know why you insist I haven't read the OP? You're right, I haven't. My device did all the work itself. Did I insult you or anyone else in any way? What king of talking is that? looking for to be spoon fed!! You think I haven't saw that part myself? So how did I restored the TA in the first place?

I apologise to everyone, I really didn't intend to spam the thread, so I won't continue.
[Sony Xperia XZ1 Compact ROMs, Kernels, Recoveries,] j4nn:
[MENTION=3723095]sohrab1985[/MENTION], your questions are valid - I also wonder, why are you getting FIDO key not provisioned.

I can only suggest to check if you are running the right bootloader version that comes with the firmware version which has been flashed last to the phone.

That is important for drm stuff to work as mentioned several times already (and you need all files under boot/ directory in order to get bootloader flashed while using newflasher, i.e. there need to be a .xml, .ta and 3 .sin files).

As for TA restore, you may use any other root shell to restore TA-locked.img, using the commands quoted by SXUsr above, i.e. it is not necessary to use renoroot again, you may simply use a root shell as provided by magisk or adb shell from twrp.

It is just that the first time restore is recommended / guaranteed to work if you use the same fw and renoroot again without flashing anything else since the unlock.

It is sufficient to restore the TA once, should not be needed to repeat.

-- edit --

Basically you need to restore TA-locked, have proper bootloader version and be running a kernel hiding bootloader unlock in order to have all drm functions working with otherwise unmodified stock firmware. Your attest key may not be provisioned regardless TA backup/restore - it gets lost with flashing persist.sin. But I cannot recall a situation that would make fido key not provisioned.
[Sony Xperia XZ1 Compact ROMs, Kernels, Recoveries,] SXUsr:
Straight talking! You've asked for answer you already know and have done. Does your camera work? If you've unlocked your BL why do you think it would say anything other than "yes"? Anything else is irrelevant.
[Sony Xperia XZ1 Compact ROMs, Kernels, Recoveries,] sohrab1985:
Thanks for the reply.

I flash the latest firmware using newflasher again to see what happens.

Although the FIDO Key is not provisioned, everything (camera, cast and ...) on my phone work as expected, so my question is since I don't have any problem should I be looking for a fix or I should leave it as it is?
[Sony Xperia XZ1 Compact ROMs, Kernels, Recoveries,] j4nn:
[MENTION=3723095]sohrab1985[/MENTION], check my previous post please - edited it probably the same time as you just posted.

Anyway, in my opinion, fido key should be provisioned. Be sure you are running the right bootloader version.

Had the fido key been provisioned before your BL unlock?

You will hardly notice fido key not working. It might be used with biometrics authentication to have strong auth without a need for password (if I understand it right).

It is hard to distinguish the difference, when an app just allows something by fingerprint or if it used fido key with fingerprint to cryptographically authenticate something.
[Sony Xperia XZ1 Compact ROMs, Kernels, Recoveries,] sohrab1985:
I checked your previous post as well, thanks for the help.

Yes the FIDO key was provisioned with locked bootloader but the attest key got lost because of the persist.sin (as you mentioned) , of course I didn't know the reason at the time. I read about it later in one of your posts.

I'm going to flash the latest firmware again and start fresh to see what happens.

It's not in any way a problem for me and your tool works just fine and I'm grateful for it. I'm just a bit curious about it ;)

Again, thank you for your help
[Sony Xperia XZ1 Compact ROMs, Kernels, Recoveries,] SXUsr:
Are you staying on 47.1.A.16.20? If so just flash the boot.img from this POST to get your FIDO, SOMC Attest, and Suntory blobs fixed. It never bothered me before but looked to fix it this morning.

EDIT: Just updated Magisk and got a bootloop, so restored my backed up boot.img and flashed the secd file from the OP and that shows the above as still being fixed.
[Sony Xperia XZ1 Compact ROMs, Kernels, Recoveries,] sohrab1985:
I would ask a question if you don't say "just read the OP". ? Just kidding.

I'm one 47.2.A.11.228 XZ1 dual.

I also flashed that boot.img and I also got bootloop so I had to restore my previous kernel.

Just one thing, by that secd, you mean the step 11?

Because that step was optional so it wasn't included in [MENTION=3463150]munted[/MENTION] guide, so I skipped that part. Could it be because of that?

Off topic, I had my phone locked for the last two years, so I've been away from doing this stuff for quite some time. Lots of things have changed and to be honest its a little bit confusing for me, but it'll come back to me. So sorry if I sometime ask obvious questions.
[Sony Xperia XZ1 Compact ROMs, Kernels, Recoveries,] SXUsr:
The boot loop, possibly because of incompatibly as it's for the XZ1C, and yes, step 11, but download and flash the one attached to the OP, and make sure you have a backup of your rom just in case. Flashing that on my XZ1C fixed the provision issues.

I'm sorry for coming across as an hard ass, it's just my way, trying to get you to help yourself ultimately. :) I flashed that modded kernel 3 or 4 times and boot looped 3 or 4 times this morning trying to get it to work with an updated Magisk before thinking to try the secd file.

Try it and report back. :good:
[Sony Xperia XZ1 Compact ROMs, Kernels, Recoveries,] j4nn:
[MENTION=3723095]sohrab1985[/MENTION], [MENTION=2735402]SXUsr[/MENTION], when using 47.2.A.11.228 fw with XZ1 dual, the kernel-G8142-47.2.A.11.228-hideunlock.zip should be sufficient to enable _all_ drm features without use of step 11. You can flash any version of magisk (19.3+) while using the kernel in flashable .zip format (ak3). If some magisk version makes the phone bootloop, you need to use a different one.

Using the patched secd is not recommended, the kernel hiding the bootloader unlock does a better job for all firmware components, including wifi display for example.
[Sony Xperia XZ1 Compact ROMs, Kernels, Recoveries,] SXUsr:
Just flashed the kernel-G8441-47.1.A.16.20-hideunlock.zip and Magisk-v20.1.zip using the commands in #1029, restored my original secd and all is working fine now.

I was just happy to see the keys provisioned but did want the actual kernel to work.

Thanks.