yashsingh1/xdaforums huggingface dataset:
[Sony Xperia XZ1 Compact ROMs, Kernels, Recoveries,] lilbrat:
[MENTION=490110]janjan[/MENTION], [MENTION=4418980]j4nn[/MENTION], having read through both your post/threads completely, both need to be tweek a bit..

[MENTION=490110]janjan[/MENTION], your kernels in my case have worked fine as I have used firmware. 24 version till I updated to. 28 without any issues as have others so if there was a TA. corruption issue with said kernel I am sure there would be more folks crying... Though you do need to post a warning as some folks have a tendency not to read everything or use a bit of common sense.. So post a warning in your thread not to flash the "oreo drm fix" with the pie kernels..

[MENTION=4418980]j4nn[/MENTION], while I am still playing with your work and in awe of it, I for one still find it a bit confusing when reading the information on the backup procedure of the TA. partition as the having the XZ1, one starts reading about it in one forum then you need to jump to the main thread in the compact side of the house, and re_read the process again but it I did find your process a bit confusing even after reading it numerous times.. (I may be a bit slow) but the pdf guide by one of the members (I apologize as I don't remember their name) was a great help and quite clear .. I have kept a copy just in case..

But in short I have used both kernels janjan's. 24 and j4nn latest . 28 pre release kernel with my backed up ta. and both have work flawlessly..
[Sony Xperia XZ1 Compact ROMs, Kernels, Recoveries,] russel5:
[MENTION=4418980]j4nn[/MENTION], continue of this https://xdaforums.com/showpost.php?p=79228463&postcount=746 i'm find something new.

All problem is one file in /data/data/com.google.android.gms/databases/dg.db

This is sqlite database and if i'm delete this file after update my google services, gpay will works again.

After successfully adding my banking card this file recreated and have something like this
[Sony Xperia XZ1 Compact ROMs, Kernels, Recoveries,] ecksdeeay:
Newb using the newflasher

Excuse the making double sure Q. (I Usually use FlashTool)

About to downgrade XZP(dual)

removed all .ta along with the persist and extracted the partition zip

do the attachments (the firmware folder with newflasher and the deleted files in the bin) look right to just hit the terminal command now?

Getting bored of reading I need to get this done! 1 step at a time and slowly but surely!:o Edit; If I'd kept reading I would've read the importance of keeping the boot folder as it is

erm! that windows pdf tut':laugh: threw me right off

EDIT; I should've left the Maple_XBootConfig_MiscTA.ta in the /boot folder

see #799
[Sony Xperia XZ1 Compact ROMs, Kernels, Recoveries,] j4nn:
[MENTION=4800473]ecksdeeay[/MENTION], nice:)

I guess, that should be ok.

Maybe just this: I've never flashed (unpacked the partition zip) - I would assume it should be ok, but I've never done it personally.

It's good you unzipped the boot zip - flashing bootloader is important.

You might skip appslog, diag, Qnovo and ssd too.

You might keep cust-reset.ta if you are switching customization (flashing different oem).
[Sony Xperia XZ1 Compact ROMs, Kernels, Recoveries,] ecksdeeay:
Appreciate the reassurance [MENTION=4418980]j4nn[/MENTION] and I'll do as suggested.

Boot zip? the only zip in the archive was partition... with the boot as a dir'
[Sony Xperia XZ1 Compact ROMs, Kernels, Recoveries,] j4nn:
[MENTION=4800473]ecksdeeay[/MENTION], you are right, the G8142-47.1.A.3.254_RU.rar contains boot already as directory.

When you download a fw with xperifirm, it may depend on the extract options - I do not use the extract feature.

So xperifirm downloads just 3 files (so original file names are kept, like FILE_20057635 for example) - that may be easily extracted in linux manually.

The biggest file (more than 2.5GB) contains boot.zip and partition.zip.

Some guys already forgot to flash boot/ so that's why I've mentioned it like that...

The advantage of not extracting in xperifirm is that you see how fw variants are stored on sony servers - the biggest file is the same across customizations within single fw version (with exception of US which has it different). And you get a file with customizations specifics. And a third file with some certifications or whatever.

This also allows to make disk space optimizations when trying to archive multiple fw versions...
[Sony Xperia XZ1 Compact ROMs, Kernels, Recoveries,] ecksdeeay:
Downgrade went well

(I should've put the contents of G8142 with newflasher (I 1st ran it out of that dir' doh!)

Too late again to do anymore and shall have to wait to complete the other steps.

:good:

Edit; the guide helped me out when newflasher asked (1 question in linux) Y or n at the start of flashing.

EDIT; I didn't flash the /boot/Maple_XBootConfig_MiscTA.ta and ended up with the Pie bootloader still there

Now flashed Oreo bootloader with the RU fw and all looks as it should (unless anyone would care to point out anything?) last 3 pic attachments
[Sony Xperia XZ1 Compact ROMs, Kernels, Recoveries,] j4nn:
[MENTION=4133204]lilbrat[/MENTION], there is this thread containing the guide for backup and restore of TA and it is universal for XZ1/XZ1c/XZp phones.

It has been linked to and announced in other threads of XZ1 and XZp forums (asking if there had been interest to implement renoroot compatibility too).

I do not see any benefit duplicating it in those two forums.

As the kernels patched to hide bootloader unlock are specific for each phone model, I have started separate threads for them for each phone model and linked to TA backup thread as the first step if a phone is still locked. I believe this threads-split is therefore quite reasonable.

Concerning the guide itself - yes, it requires certain level of computer (and phone handling) knowledge and does not contain windows specific stuff - like howto install win drivers and so on.

In my opinion, that is out of scope and kind of expected that a user wanting to unlock a phone knows a bit why he does it and can read about the tools used/linked if not knowing them already.

Still you have the pdf guide from [MENTION=3463150]munted[/MENTION] there too to help with windows specifics - it could have been updated as step 11. has changed with release of kernel patched to hide bootloader unlock allowing to actually install system fota update. But it serves the purpose even like it is in my opinion (you should use it as supporting if something is not clear and follow the official guide mainly).

[MENTION=4688377]russel5[/MENTION], that is very interesting find. Thank you.
[Sony Xperia XZ1 Compact ROMs, Kernels, Recoveries,] ecksdeeay:
Took 2 reboots.

I think both were running for about 10mins.

2nd run was with no overwrites,

3rd attempt hit the target:victory:

Now going to get Pie fw ready as I forgot to download it for flashing after UL of the BL and the other tools needed for root!
[Sony Xperia XZ1 Compact ROMs, Kernels, Recoveries,] Baltojikale:
bootloader unlocking not allowed

Thanks for this exploit. Fallowing j4nn and munted excellent guides I was able to backup TA partition on XZ1. As was recommended flashed appropriate ofw to to my XZ1 then started exploit. It took 4 restarts (event count 6827; overwrites 1) . And finaly backed up TA partition. And only before unlocking bootloader I was greeted with "bootloader unlock allowed: NO". So i understand that all that effort to backup phone's files and TA partition was useless. Thus my question would be if it is possible using temporary renoroot to change /system mounting from ro to rw? My only goal is to change services.jar file located in system/framework/ folder.
[Sony Xperia XZ1 Compact ROMs, Kernels, Recoveries,] j4nn:
[MENTION=4800473]ecksdeeay[/MENTION], as pointed already in my PMs (and also several times in this thread), it is important to flash bootloader together with the fw, because bootloader contains TrustZone applets loaded before android kernel. The tz applets are used for drm stuff and they may not be compatible across different fw version. Pie's bootloader is not compatible with oreo one, so running oreo fw with pie bootloader or vice versa is not really good idea. It can for example result with not working camera (it hangs on capture instead of taking /green or whatever/ photos) and the service security screen says single word "None"... Can be fixed easily by flashing the right bootloader version though.

Just verified that if boot/Lilac_XBootConfig_MiscTA.ta file is removed, bootloader is not flashed:

Processing boot delivery...

Found boot_delivery.xml in boot folder.
- Boot delivery version: 1306-5035
- Verifying if boot delivery match with device...
searching for: PLATFORM_ID="0005E0E1";PLF_ROOT_HASH="C30DEC2471CEA311E6918657367B51068A39583BBF89FD68B379BCD5A709AB1B"
Found bootdelivery match: COMMERCIAL_0005E0E1
TA file: Lilac_XBootConfig_MiscTA.ta

Processing ./boot/Lilac_XBootConfig_MiscTA.ta
- Unable to open ./boot/Lilac_XBootConfig_MiscTA.ta!

Device is put now out of flash mode.

End. You can disconnect your device when you close newflasher

That means it is needed to keep boot/*.ta file.

The guide said "The (newflasher) tool should skip dangerous .ta files automatically" since beginning, so just following it closely, everything would be ok.

The post#456 contains a link to newflasher source code checking specific TA units to skip flashing them.

So in my opinion removing all .ta files is an old precaution not really necessary...

[MENTION=4800473]ecksdeeay[/MENTION], I have updated the guide explicitly mentioning not to remove .ta inside the boot/ directory in order to avoid any confusion possibility, even though the .pdf guide also mentions "keep boot subdirectory (including the one .ta there)".
[Sony Xperia XZ1 Compact ROMs, Kernels, Recoveries,] j4nn:
[MENTION=9035490]Baltojikale[/MENTION], if you have "bootloader unlock allowed: NO", I am sorry, I am not aware of any way to unlock it. Your TA backup may be useless as you say, but if you eventually find a way to unlock BL, it may get useful.

I always wonder where such phones come from - did you get it within some promotion sale or as locked to a gsm operator?

You might be able to get it bootloader-unlocked within customer service of your phone's seller, depending on conditions you bought the phone with.

Concerning your second question: yes, the temp root is fully capable root shell. You can remount /system or /vendor partitions to be writable. Then you can modify anything there.

But since bootloader is locked, verified boot is active and kernel is checked against signatures then verified kernel checks hashes of data read from /system and /vendor partitions - the subsystem taking care of it is called dm-verity. The checksums are even a bit redundant, so in certain small modifications, dm-verity may be capable to "revert the errors" (aka your changes). If not, it would not boot, resulting in a "boot loop".

In other words, you may use renoroot temproot for testing "modifications" on runtime, but you cannot expect to create persistent modifications.

In fact if you like to do some runtime modifications for testing, I would not recommend remounting /system and /vendor partitions to be writable, instead you may use bind-mount to simulate a change similarly as magisk system-less modules work.
[Sony Xperia XZ1 Compact ROMs, Kernels, Recoveries,] j4nn:
Just came across this - https://source.android.com/security/overview/acknowledgements#2018:

So it seems my guess above about the presentation's topic CVE may have been right.

That CVE has been fixed in upstream linux kernel with following commit:

net: Set sk_prot_creator when cloning sockets to the right proto

containing even a C source code to trigger the bug (not an exploit, but a nice start to better understand the presentation to come in few days)...
[Sony Xperia XZ1 Compact ROMs, Kernels, Recoveries,] Baltojikale:
Ok first the reason why i want to access system/framework folder is to change services.jar with smali/backsmali pached service.jar file which enables signature spoofing. So according to what you wrote it is possible to achieve one runtime signature spoofing via bind mounting. But as i understand every time user would like to achieve to bind mount it would require to perform renoexploit to enable root or it could be done without temporal root ? Ether way if bind mounting would work it would open new spoofing ways without rooting and downgrading google play services.
[Sony Xperia XZ1 Compact ROMs, Kernels, Recoveries,] nAµ'izat:
Dear j4nn (or anyone who may advise): i went through with your TA image pulling / pushing method according to munted's detailed walkthru (only the "-i 0x0fce" switch of his unlock command appeared redundant / not working for me), which seemed a success at first glance (camera working with unlock BL), but upon checking the security section of service menu i somehow ended up with "Suntory blobs [ERROR]" & both SOMC keys NOT PROVISIONED (android key however remains PROVISIONED thanks to the backup)…

Will this affect performance at all, are these 2 fixable?

Due to the scarce options for custom ROMs (RR & Carbon abandoned, LOS15.1 & AEX in beta stage) for the xz1c, it seemed the cleanest option to continue with your hide unlocked BL kernel (not very clear for me just how far janjan's solution is from yours, results-wise).

i went with your 47.1.A.2.324_CE1 (after downgrading from latest v.Pie — but just after i had tried and failed with 8.49: just as you pointed out yesterday, because i over-cautiously excluded the boot .ta too). Graining temp root only worked for the 2nd instance on the pulling way, and only for the 4th attempt for the push-back.

Also might worth mentioning that i only excluded the persist .sin and *.ta files. i excluded the partition folder too for the first go (when i failed with the 8.49 build), however i included the folder for the attempts after that, as well as left all the qnovo, appslog, ssd, diag .sin files (and also all the amss .sin files too, even though munted's pdf mentions them as possibly skippable files)

Of course, i wasn't forward-thinking enough to stick with fw 8.49, as the hide unlock kernel only comes for that version of the 2 targets for temp root, not the 2.234, so in hope to get the keys maybe to provisioned-state again, i choose to flash fw 16.20 (the customized NO/BA, provided by vofferz in another thread). This is where i decided to make this helpless effort of screaming for your assistance here, before flashing the hide unlock kernel.

Sorry if the reason (what i screwed up) for this had been explained in detail already, i did not succeed with the search function :|

Thanks a million!
[Sony Xperia XZ1 Compact ROMs, Kernels, Recoveries,] shoey63:
I'd love to know where this obsession with deleting TA files comes from when using newflasher. Developer [MENTION=4120742]munjeni[/MENTION] is no fool and his newflasher tool automatically prevents anything dangerous from being flashed to the device.
[Sony Xperia XZ1 Compact ROMs, Kernels, Recoveries,] ecksdeeay:
Androxyde's Flashtool

Edit; unless I've been doing it wrong for quite a while now?

I'll take the opportunity to link to my other related post on this OP.#83
[Sony Xperia XZ1 Compact ROMs, Kernels, Recoveries,] j4nn:
[MENTION=9035490]Baltojikale[/MENTION], just wondering why do you need to do signature spoofing on bootloader locked phone with stock fw?

Do you like to replace google stuff with microg or something?

I am afraid that would be hard to achieve with just temp root - or when you succeed, you would need to redo it after each phone's reboot.

As discovered and posted about it in this thread, it is possible to make some persistent changes using 'oem' partition - see post#372. But persistent fully fledged root is not possible due to selinux restrictions.

You may use './renoshell' repeatedly to get temp root shell once you succeed with renoroot until the phone is rebooted. After reboot you would need to 'renoroot' again.

Still wondering about some details who sells (and under what conditions) not unlockable xperias?

[MENTION=9851474]nAµ'izat[/MENTION], just follow the rest of the guide - follow the link in updated point 10. to use the kernel hiding bootloader unlock. That will enable you the rest of the drm keys and features.

I would recommend to ignore all the "secd-ignore" patches;) - the mentioned kernel does the job properly.
[Sony Xperia XZ1 Compact ROMs, Kernels, Recoveries,] nAµ'izat:
Thank you j4nn!!

Completing the kernel booting fixed all my above issues, but Magisk manager app wasn't offered to install, and upon manual installation i get "magisk is not installed" (v18.1), also get "ctsProfile: false" when checking safetynet.

Do i still miss something, please?
[Sony Xperia XZ1 Compact ROMs, Kernels, Recoveries,] j4nn:
[MENTION=9851474]nAµ'izat[/MENTION], you most likely booted the kernel from usb as you followed the guide - that time everything worked as my kernel was active.

But 'fastboot boot kernel.img' command just loads the kernel from usb into memory, not flashing it, just running it (and you need to reboot to fastboot by a command to get the command working).

When you boot my kernel, your installed magisk manager will detect magisk installed.

If you reboot after and do not use the 'fastboot boot kernel.img' command to boot my kernel, you are most likely booting your stock fw kernel, which does not have magisk and does not hide bootloader unlock.

Either boot my kernel from usb or follow the guide for alternate use where you can flash the kernel to avoid need of usb booting (but if wanted fota system update then, you need to revert the changes first).
[Sony Xperia XZ1 Compact ROMs, Kernels, Recoveries,] nAµ'izat:
Thank you again!

Although i'm not totally new to utilizing adb and flashing, i'm still a complete fool to this.

So then, while running the rooted kernel, i'm safe from getting OTA updates, right? if one wants them tho, would need to do it with the stock kernel running - however after such an upgrade, the previously used rooted kernel would not necessarily work, now, would it (for instance from my current Oreo, 16.20, the next upgrade offered is already for Pie)?
[Sony Xperia XZ1 Compact ROMs, Kernels, Recoveries,] j4nn:
[MENTION=9851474]nAµ'izat[/MENTION], when running my patched kernel, you are able to get fota system update, but you need to run as booted from usb via fastboot having untouched stock fw including kernel in the phone's flash in order to be able to actually fully install it.

When you install fota, you can boot what has been installed, the installation is finished anyway, even though the updated fw detects unlock then. There are kernels built here for most of the fw versions, so most likely you can download a new one just before accepting system fota update in order to use it with new fw version.

In case of oreo 16.20 with DE customization, next fota is pie 4.45, so both kernels are available here. It is also demonstrated in screenshots and in the video linked in the kernels thread just bellow downloads in post#2.

Similarly you may test the kernel having still the exploitable fw after backing up the TA, unlocking and restoring, so can immediately see the effect and even accept oreo to oreo system fota, as the next version is oreo 12.34 also provided here.
[Sony Xperia XZ1 Compact ROMs, Kernels, Recoveries,] Baltojikale:
long story short spoofing purposes for gaming... recently succeeded rooting redmi 6a (after waiting for 2 weeks for xiaomi to allow me to do that - some stupid waiting police...) and arranged everything for spoofing in the end it was only one file changed (already mentioned services.jar) so i was wondering if it was possible to achieve that with my as it seem unrootable sony xz1... About seller cant say much... I bought phone one year ago at that moment in my country gsmarena.lt was selling xz1's cheapest and had in their warehouses ocean blue model.... thought that phone was not designed for my country couse I had to wait for it to be shipped to shop about 1 week and it had UK type socket adapter not EU type. Checked the original box no additional markings were there. if there is a way to check some more details i could do that though before backing up TA it was flashed with firmware this thread suggested.
[Sony Xperia XZ1 Compact ROMs, Kernels, Recoveries,] ecksdeeay:
Apologies j4nn for getting sidetracked (shown in the hidden content) reading through this (being directed all over the forum) thread as I read all through albeit the last several pages where others pointed out the boot importance with newflasher tool.

Okay cut the crap! Now if I UL BL and restore TA all I have to do is this (for my XZPdual) step 10

Instead of flashing twrp, you may just 'fastboot boot' it if you need it.

Instead of the steps 10. to 13., you may use patched and rooted kernel hiding bootloader unlock available in following forum threads in order to be able to even install FOTA system update

[XZp] rooted kernel hiding bootloader unlock with working fota

giving you back sony drm functionality that fw disables when it detects unlocked bootloader status. For more details see also post#645 of this thread.

I am not in any rush to hose/BORK my shiny New (2nd hand but MINT) device...many thanks for the support so far:good:

These steps are confusing after the TA.img is restored and not sure which to follow? (I'm ready to UL BL with Sony code but what's the sure way to get #root, after that?)
[Sony Xperia XZ1 Compact ROMs, Kernels, Recoveries,] kuuiu:
Events count 77805 Did not obtain temporary ROOT permissions, restarting the phone to 2715 obtain temporary ROOT permissions after, so if you get temporary ROOT permissions for too long to try to restart the phone, Sometimes you will get temporary ROOT permissions soon. The time to obtain temporary ROOT permission after multiple tests is random.

I don't know English, I translate it with Google.
[Sony Xperia XZ1 Compact ROMs, Kernels, Recoveries,] niaboc79:
[MENTION=4418980]j4nn[/MENTION] did you try to compare a few TA backup with and without drm to check if there's a way to add drm back if you didn't save them?
[Sony Xperia XZ1 Compact ROMs, Kernels, Recoveries,] j4nn:
[MENTION=599722]niaboc79[/MENTION], in my opinion, there is no way. See this post here.

I believe, it works like it is described there.

I can confirm that TA-unlocked is missing 66667 (0x1046b) unit, which is just 16 binary bytes present in TA-locked looking pretty "random".

Tried to even find a code in system that would somehow work with that TA unit (to see where the key goes or how it is used), but I could not find it anywhere in firmware userspace libs or binaries.

I guess that the device master key unit is used by bootloader when it is setting up Trust Zone before loading the kernel - but as said, it is just my guess, did not look into it...
[Sony Xperia XZ1 Compact ROMs, Kernels, Recoveries,] j4nn:
[MENTION=4800473]ecksdeeay[/MENTION], I would recommend to use the kernel patched to hide unlocked status - it is the only solution that does it properly for all firmware binaries and libraries.

With any secd patch, just one binary is fixed. But there are many others that check the bootloader unlock status via kernel api directly/independently and therefore with only secd patch, some components of the fw may think "still locked" while other would see "already unlocked" with unpredictable results.

Read the kernel thread that corresponds to your phone and either use the booting from usb without flashing or use any of the alternate ways. Everything is described, just read it.

One think: just do not be surprised by the warning screen after power on that your bootloader is unlocked. There is nothing we can do with that screen.

And restoring TA does not "re-lock" the bootloader (which is a benefit for us as we can restore the device key and still be able to modify kernel or system or get root).
[Sony Xperia XZ1 Compact ROMs, Kernels, Recoveries,] j4nn:
[MENTION=4675598]shoey63[/MENTION], seeing you here I am wondering how is going the thing with TAMA standalone kernel to be bootable from fastboot you were testing with [MENTION=6031898]MartinX3[/MENTION]?