yashsingh1/xdaforums huggingface dataset:
[Sony Xperia XZ1 Compact ROMs, Kernels, Recoveries,] j4nn:
[MENTION=6566897]Likesoft[/MENTION], by upgrade to pie, there is no loss of keys. It's only the service screen that got modifications, displaying slightly different info.

If you then downgrade back to oreo (skipping the persist.sin), you get the same screen with all keys active as you are used to (assuming still locked boot loader).

Concerning the stuff displayed in the service security screen - it is overrated. The screen can show a key as working while in fact it may not be.

And it can show error or something for a key which may still be usable and working.

While still locked, there is verified boot active, checking signatures and checksums of kernel/boot image, which in turn has dm-verity active for system and vendor partitions - i.e. every read of those partitions during boot is checked in a tree of checksums. That means even with temp root, it is not possible to modify anything in system, vendor nor boot partitions.

But I wonder if temp root could be used to setup a higher privileged app or service or some kind of suid binary in /data or /oem partitions?

(suid probably not as it is disabled in mount options and most likely denied with selinux)

As data may not get erased with upgrades, such thing might even survive a switch to next fw version.

How system apps get their updates? The updates do not get flashed into the system partition - because if you do a factory reset, the updates are reverted too getting not app updated fw version that was on a phone. So I would assume that system apps updates get installed somewhere in data partition and some of them may have higher privileges - for example the play store app and it's related services.

Can we install a custom system service app in data partition by use of temp root? Possibly opening a door for other easier root on demand method?

Any thoughts on that?
[Sony Xperia XZ1 Compact ROMs, Kernels, Recoveries,] tonsofquestions:
Just so I correct my knowledge: what would happen if you try? I don't think it's storing a backup image (otherwise we could modify that, too), so does it fail to boot with an error, and you have to reload system? Or is it something else?

That's a cool idea. I suspect it's likely not possible - otherwise someone else would have come up with a way to do it by now. Magisk is close - that's kind of what it's doing with not modifying system and mounting on top of them (or, at least that's my understanding), but even that requires some modifications first.

App updates definitely don't get installed to system. They install like any other app and just mask the system one. That's why if you try to remove them, you get a "uninstall updates" screen (https://fscl01.fonpit.de/userfiles/...uninstall-apps-app-uninstall-updates-w782.png) rather than to merely disable them (http://cdn.androidbeat.com/wp-content/uploads/2014/02/disable_apps.jpg). That said, I'm actually not convinced other apps have elevated permissions. The play store gets permissions to install apps/track you/etc. but I don't think it's root-level permissions. It's just calling normal system functionality, and probably comes with some permissions other apps can't ask for.

It would be really awesome if there were a way to do it - and have the best of all worlds. It could be worth asking in a different forum, where folks are more savvy about that part of the way things work.
[Sony Xperia XZ1 Compact ROMs, Kernels, Recoveries,] c_tho:
note that i've got

Suntory blobs [ERROR]

SOMC FIDO Key NOT PROVISIONED

and the only thing i can find that isn't working is the video enhancement. bank apps and camera stuff all fine.
[Sony Xperia XZ1 Compact ROMs, Kernels, Recoveries,] munted:
You're on 9.0 Pie right? I wonder if 8.0 Oreo would have had the same issues after rooting? :confused:
[Sony Xperia XZ1 Compact ROMs, Kernels, Recoveries,] j4nn:
I believe kernel would panic resulting with reboot so you would experience a boot loop. Likely possible to fix with stock re-flashing. There is no backup image. If not mistaken, there is a signed tree of checksums, that is traversed while accessing blocks of the partition.

If unlocked, this is disabled (found it in kernel source) and you can modify system and vendor partitions even when running unmodified stock kernel.
[Sony Xperia XZ1 Compact ROMs, Kernels, Recoveries,] Likesoft:
thanks for your help, I successfuly got root and backed up the drm key.

after that, as I planed, I tried to use the temp root for other apps but all other apps says my device have no root.

I expected I can use the root permission for every app in the device before reboot but it was not.

for example, I tried to use titanium backup but it told me that there's no root permission on my device even after I got the temp root. so I tried to install su app(supersu) but it also said that i have no root permission.

Is the root permission from temp root only for the adb shell? Is there any other way to use the permission to other apps even though it's unavailable after reboot?

If there's no way, I think I have to give up to use temp root and try to unlock bootloader.
[Sony Xperia XZ1 Compact ROMs, Kernels, Recoveries,] j4nn:
[MENTION=6566897]Likesoft[/MENTION], this has nothing to do with "root" permissions for apps. This is an exploit to get a root shell (adb only) in order to backup TA (drm keys) before bootloader unlock.

That said: just discovered an interesting fact - you can get a persistent root from temp root with our xperia devices!

get a temp root shell with renoroot and then

G8441:/data/local/tmp # mount -o rw,remount /oem
G8441:/data/local/tmp # mount | grep oem
/dev/block/sda63 on /oem type ext4 (rw,seclabel,relatime,data=ordered)
G8441:/data/local/tmp # cp /system/bin/sh /oem
G8441:/data/local/tmp # chmod 6755 /oem/sh
G8441:/data/local/tmp # ls -lZ /oem/sh
-rwsr-sr-x 1 root root u:object_r:oemfs:s0 298840 2018-11-22 19:52 /oem/sh
G8441:/data/local/tmp # mount -o ro,remount /oem
G8441:/data/local/tmp # sync

reboot the phone and start adb shell, then:

G8441:/data/local/tmp $ /oem/sh
#

and you have a root shell! Tested on a still locked xz1c!

just try it and see : -)

but an important question: what to do with selinux security context?
[Sony Xperia XZ1 Compact ROMs, Kernels, Recoveries,] Likesoft:
Wow, you've found out new way!

I just want to use some kinds of titanium backup or root explorer. Backup and restore...?

I didn't know the SELinux, now I know although I have root permission, I cannot use root permission for apps while SELinux is enforcing.

And there's must be no way to bypass SELinux on XZ1, that's why you ask me the 'important question' :crying:

I planned to use stock rom without unlocking boot loader as long as I can apply root permission to some apps(like titanium backup or root explorer).

but it looks like using stock rom is not hopeful although you've found out the way applying persistant root. :crying:

It's an extra question but the way to have persistan root looks like the method of copying the 'sh' from '/oem/' to '/system/bin/'.

I, until now, thought 'su' binary is essential for root permission, but in this case, 'su' is not essential?

or do I also have to apply su binary(like supersu, phh's su) for superuser?
[Sony Xperia XZ1 Compact ROMs, Kernels, Recoveries,] munted:
Hi all is anyone able to help me please? I'm up to step 9 but something is wrong.

After restoring the DRM keys it still says "this device has been unlocked and cannot be trusted" when you turn it on

In the Security Menu all 3x keys say NOT PROVISIONED.

The camera does work, video image enhancement does not.

I know the DRM keys backup worked because when I open TA-unlocked.img and TA-locked.img in notepad++ they look very different.

I've attached a screenshot with what I've done.

I've taken screenshots of everything I've done to help make a guide but obviously it is no good at the moment because my guide doesn't work!

Edit: The attachment is a bit blurry, you can see the screenshot here: https://i.imgur.com/GeUSAjg.png
[Sony Xperia XZ1 Compact ROMs, Kernels, Recoveries,] j4nn:
You did all right. If camera works with the unmodified stock firmware - that confirms TA (with device master key) was restored properly!

The other things (security screen) are normal - it is due to the software knows, the phone has been unlocked. Similarly the message on boot.

The two somc keys seem to be disabled due to unlock - but they are not useful, so you do not need to care about that.

The main point is that if you now flash the latest oreo fw (16.20) camera would still work without any modifications (that would not be the case if you did not backup and restore the TA).
[Sony Xperia XZ1 Compact ROMs, Kernels, Recoveries,] munted:
Thank you very much for getting back to me, ok sorry I read the instructions right but not the goal correctly!

Would you recommend Oreo over Pie? I flashed 9.0 --> G8441_Customized SG_1311-1763_47.2.A.0.306_R3C

But it doesn't seem to work correctly, it doesn't get a mobile signal, I can't make or receive phone calls, only wifi works.

Do you know a rom that works? I'll download what you recommend using XperiFirm (unless I should download it from somewhere else) and give it a go.

P.S. Just sent you a donation as well, thank you for all your hard work and replying to questions. Today or tomorrow I should finish the instructions and maybe you can check they're correct and if so hopefully the instructions with screenshots will help others :)
[Sony Xperia XZ1 Compact ROMs, Kernels, Recoveries,] tonsofquestions:
It's unfortunately always going to say that. It's at the bootloader level, and not specific to the TA backup.

This is expected, as j4nn says, though there is a zip you could flash that might fix it.

I've heard reports of this. Strangely enough, on my phone (with the TA restored), the image enhancement does work. I'm not sure what's different for folks.

They'll look very different no matter what. Even just rebooting the phone and backing up the TA again will display large differences.
[Sony Xperia XZ1 Compact ROMs, Kernels, Recoveries,] j4nn:
Not sure, but it might be specific to pie vs oreo?

I would recommend latest oreo firmware with customization that fits your region. Using pie from Singapore in other regions might work or might have problems.

[MENTION=3463150]munted[/MENTION], thank you for the donation. Really appreciated.

It looks like most consider this as granted - download number gets higher and higher while nobody donates for more than a week - it seems like there are few donations only when this is discussed in the thread...

Well, it would be ok if most at least hit the thanks button in the first post (if not using a full post), but not even that is a case:

You know guys, I am still not done with this, still working on it - analysing how bootloader unlock status is checked and what influence it has in the firmware, in order to implement a way to hide this fact.

That could fix the remaining issues - possibly not so important as the camera, but still...
[Sony Xperia XZ1 Compact ROMs, Kernels, Recoveries,] munted:
I flash G8441_Customized DE_1310-4373_47.1.A.16.20_R6B which seemed to work fine, mobile phone reception worked, camera worked, couldn't see any problems. Ok cool.

Flashed back to 47.1.A.2.324_CE1 and ran the renoroot exploit then successfully installed TWRP, but then had problems with Magisk.

I tried to install Magisk Magisk-v17.3.zip in TWRP while still running 47.1.A.2.324_CE1 but it would just bootloop.

I installed Magisk-uninstaller-20181022.zip then tried to install Magisk 17.3 then DM-VERITY AND FORCED ENCRYPTION DISABLER then did a factory reset and then finally a reboot but still bootlooped.

Ok I thought I'd just install a new flash but no luck, I can't boot into TWRP any more. I'm so sorry are you able to help me out a bit? I can flash back to 47.1.A.2.324_CE1 and run renoroot and reinstall flash but I don't know what I should do then to

a) Keep TWRP

and b) Get Magisk to work.

Thank you, sorry to be so much bother :(
[Sony Xperia XZ1 Compact ROMs, Kernels, Recoveries,] j4nn:
Try to flash full 47.1.A.16.20 fw with newflasher - remove *.ta, keep boot subdirectory (including the one .ta there), remove persist (and optionally Qnovo, amss*, ssd) sin files.

Try to boot it, check camera and video enhancements. At least camera should work if TA was previously restored.

Flash the latest twrp. Then magisk. As far as reported by other guys, these two steps were straightforward...
[Sony Xperia XZ1 Compact ROMs, Kernels, Recoveries,] nahsha:
Hi, would there be anything to try on xzp (nougat) to try and obtain the DRM files?

Thanks
[Sony Xperia XZ1 Compact ROMs, Kernels, Recoveries,] mirhl:
https://xdaforums.com/crossdevice-dev/sony/progress-newflasher-xperia-command-line-t3619426/page39

https://xdaforums.com/xz-premium/how-to/devs-xz-premium-drm-fix-solution-t3659940

https://xdaforums.com/xz-premium/how-to/ta-backup-g814x-t3691603

https://xdaforums.com/crossdevice-dev/sony/stack-corruption-fastboot-mode-ta-backup-t3777538

The stuff I know.

Besides, exploiting trustzone should be pretty doable too.
[Sony Xperia XZ1 Compact ROMs, Kernels, Recoveries,] nahsha:
I recall the good old days where we could exploit and root with locked bootloader :)

I have a potential of getting a pretty good priced xzp (barely used) but I'm not sure of the possibilities of ta backup and treble :(
[Sony Xperia XZ1 Compact ROMs, Kernels, Recoveries,] mirhl:
Treble is not there simply almost no OEM feels like repartitioning already-shipped phones.

But the phone is identical to XZ1, display aside, and that has treble.

Then, it is alleged TA is not storing DRM anymore.

But it must exist somewhere nonetheless, and somehow its information must be passed to the system.

EDIT: in fact, I wonder if this info is stored somewhere in kernel memory. In this case root might even not be needed, considering you could use one of the tens of speculative execution flaws that have been uncovered in these last months
[Sony Xperia XZ1 Compact ROMs, Kernels, Recoveries,] j4nn:
It seems that exploiting CVE-2017-7533 (race between inotify_handle_event and vfs_rename) shall be very difficult. I've spent a lot of time with it only to discover that the out of bounds overwrite that results from the simplest race win as demonstrated in the available PoC cannot do any harm.

The problem is that the minimal slub size for armv8 64bit architecture is 128 bytes while the filename to grow in the rename can reach max of 32 bytes to still get an in place overwrite directly resulted from the race win.

That is the reason why longer filenames in the rename have no effect in the notify handler (the available PoC does not work with longer target filenames).

The sizeof struct inotify_event_info is 48 bytes, that makes 80 with the max directly overflowing filename, having no effect in the slub cache since the allocation unit of 128 bytes does not overflow at all (the remaining space after the object in a used slub is not used for anything).

I guess that the only way with this cve might be exploiting use after free.
[Sony Xperia XZ1 Compact ROMs, Kernels, Recoveries,] serajr:
I can confirm what the_laser stated here (and below):

Also, restoring the TA partition will no longer relock the BL.

How can I confirm this? A friend of mine works at a company that tests network hardware and software for almost every oem/brand phones. His sony devices (including xa2, xz2/compact/premium) come with an userdebug build, which runs as root as default (due the low level firmware testings the company needs to perform/report back). I've guided him on how to back up the TA, unlock his xa2/xz2 and push their respective TA back again onto devices! And voila... everything working as expected!

He can not share userdebug builds (for obvious reasons, ofc), and he has already tried to flash such builds with newflasher and flashtool, unsuccessfully! None of them even recognized them as sony regular builds!!
[Sony Xperia XZ1 Compact ROMs, Kernels, Recoveries,] mirhl:
Uh? So that's an even better situation than that of >1yo devices?

I wonder what was even that catastrophic forecasting all about then?

EDIT: munjeni in particular seemed to be all about "unit 8B2 not being in TA anymore"

p.s. funny days those of leaks from brown devices
[Sony Xperia XZ1 Compact ROMs, Kernels, Recoveries,] Inerent:
At least maybe he can provide us the ta backup. Munjeni then can reconstruct the TA maybe with some sort of keygen for every device. He really can help reconstructing deleted ta part.
[Sony Xperia XZ1 Compact ROMs, Kernels, Recoveries,] serajr:
No way mate, TA partition backup has personal data like imei, etc.

Munjeni is a pure genius, also my blood brother, but unfortunately neither he nor anyone else succeeded on that, even in older xperia models! Sad, but true.
[Sony Xperia XZ1 Compact ROMs, Kernels, Recoveries,] Inerent:
What I don't understand is why people are so determined to backup the TA partition as long as there is a DRM fix!

The warranty with or without TA backup is still gone, because when we unlock bootloader, Sony provide us with a code based on our IMEi, so they know!

Devs should look into editing camera libs and apk, also should look into Qualcomm libs. In my opinion, for example, SD845 has 4k60fps enabled in libraries and drivers, I really think the SD845 is jut a SD835 with higher clock speed and cpu FLAG renamed to 845, but has new capabilities enabled in libraries.
[Sony Xperia XZ1 Compact ROMs, Kernels, Recoveries,] j4nn:
Why some functions should be lost with unlock if I paid full price for it?

Why I should not be allowed to modify open source components? In my opinion it is unfair leverage from Sony to prevent experimental third party firmwares on their devices.

My goal is to fully own the device, being able to remove / replace / add anything in it, with full original functionality.

So called "drm fix" is as far as I understand a fw specific crack, needed to be updated for each newer sony fw release, right?

Restoring TA to put the device in the state before unlock is useful not for the warranty claim (that's obviously lost with unlock via sony's website), but for the option to get full stock fw functionality including ota updates without needing any special changes. That allows easier sell for example to anybody, who may not be interested in modded firmwares.
[Sony Xperia XZ1 Compact ROMs, Kernels, Recoveries,] mirhl:
Warranty being lost, is all to be seen (and I don't know which part of cameras has, ever, been open)

Anyway I think a "generic" patcher working on patterns (be them even asm, duh) should still be in the realm of feasibility.

It's just subpar because there might be a handful of additional services depending on it aside of camera, and.. idk, nobody ever gives feedback about x-reality or noise cancellation...

But nevertheless, can we please keep this DEVONLY?
[Sony Xperia XZ1 Compact ROMs, Kernels, Recoveries,] j4nn:
right, let's develop - now I really could use some help as I am getting out of ideas:

I can imagine how CVE-2017-7533 (race between inotify_handle_event and vfs_rename) could be exploited via use after free. It is after all outlined in the already mentioned slides - see page 14, please, or a screenshot here:

My understanding of that:

Thread-1 uses old filename (before rename), which is longer then 32 bytes, i.e. stored as external_name in dentry - for the purpose of the exploit, original length should be 128 < len < 256.

When Thread-2 invokes the rename, it results with kfree_rcu of the old_name, after the Thread-1 already calculated the file name length from the old name.

Because the dcache copy_name replaces just the pointers to external_name, the new file name from the rename does not get into the inotify handler code (which used the old_name pointer).

The new name could get there only when both file names would be shorter than 32 bytes, i.e. they would be stored directly in dentry without use of external_name pointers, but as explained above, this cannot be exploited.

So the new file name in form of pointer to external_name cannot be processed by kernel's inotify handler, but as outlined in the slide, we may "spray" the heap to overwrite already freed slub unit, where the old_name was stored, with our payload using the add_key syscall.

If the old_name's freed slub is overwritten before the strcpy in Thread-1 is called in the inotify handler, next slub allocation unit may be overwritten via strcpy which would use our payload stored in freed old_name place - the use after free method.

I tried to implement it, but it seems to me, that it is not possible to match the timing outlined in the slide to get the use after free race win.

Please correct me if I am wrong, but as far as I could understand the kernel RCU subsystem, the kfree_rcu does not immediately call the kfree callback, instead it stores the entry to be freed into a list, which is handled in background, causing very big delay between the kfree_rcu call and the time when the old_name is actually freed with kfree.

When I tried to debug & trace it with qemu and gdb, I had to insert a delay of about 15ms(!) between the rename and add_key calls in the test app in order to get a successful overwrite of the freed old_name with the new kmalloc in add_key.

In that time the Thread-1 working in inotify handler must be already finished using still the old_name which was not kfreed yet, because of the kfree_rcu/rcu_reclaim background delay.

Am I missing here something?

Is there any way I can force kfree_rcu called from vfs_rename to call kfree immediately?

Or is it possible to somehow stop Thread-1 in order to wait until the old name is freed with kfree and updated with add_key?

Please note, I can test only with qemu and that is significantly different than with a real device. Testing with not rooted real device is not possible because you mostly cannot implement any feedback if some steps were successful or not.

Is here anybody who could help debugging this on an unlocked device?
[Sony Xperia XZ1 Compact ROMs, Kernels, Recoveries,] munjeni:
removed
[Sony Xperia XZ1 Compact ROMs, Kernels, Recoveries,] munjeni:
That's a realy good news! If I decide to buy 2018 model first step will be dumping trim area hardware tool way :)

[MENTION=4418980]j4nn[/MENTION] did you see s1boot boot image? Curently I reasearching for about one hour getting my post where I stated about sin file which contain s1 bootloader elf file, can't get that post right now, but that elf file (lilo bootloader?) is inside standard android img format and one good thing which I found on that image file after unpacking is that kernel on that image is selinux permissive! Maybe it can be temporary boot by fastboot? Also I have found that there is probably serial console enabled! Don't call me wrong if some things is not right, that was almost 3 month ago, researched only hour on it so things might be wrong! ... just for idea :)
[Sony Xperia XZ1 Compact ROMs, Kernels, Recoveries,] moofesr:
I was reading a story about a new RAMpage exploit that uses Rowhammer to gain root. Is this something that could be used for the purposes of this backing up DRM keys via temp root?

If you have an un-rooted XZ1C, it might be worth visiting the webpage for the RAMpage attack and downloading the test app to see if the device is potentially vulnerable to this attack or not.
[Sony Xperia XZ1 Compact ROMs, Kernels, Recoveries,] Inerent:
I can confirm that we can downgrade the bootloader, I just did with XZP in process reverting to android Nougat.

Here is the proof: https://xdaforums.com/xz-premium/how-to/upgrade-oreo-to-nougat-guide-t3775147/page3

However I have upgraded again to last oreo and cannot confirm that rampage exploit works in Nougat :( and don't have the time to downgrade again ...