yashsingh1/xdaforums huggingface dataset:
[Sony Xperia XZ1 Compact ROMs, Kernels, Recoveries,] j4nn:
[MENTION=4120742]munjeni[/MENTION], not sure if that could be the reason, but check the .rc scripts - sony starts fota-ua with a parameter on it's command line:

service fota /sbin/fota-ua c

see the 'c' there...

That is in fotakernel, now this is in the main system - /system/etc/init/fota.rc:

exec u:r:fota-ua:s0 root root system cache drmrpc -- /sbin/fota-ua c

Again there is 'c' in the cmdline of the fota-ua.

While the autostart of fota-ua service in fotakernel can be controlled to get traces, inputs and outputs, the invocation in the main system is very tricky - if you skip it, main system boots fully, while it needs to be started in the first reboot after clicking install button of the fota update (as the main system boots to run fota-ua in the /system/etc/init/fota.rc to do the first verification phase).

The very first verification phase does not have /cache/recovery/fota/ua_status present. When fota-ua starts, it would write "\1\0\0\0\0\0\0\0" into it. With ua_status set to '03 00 00 00 00 00 00 00' we are going straight to install phase (which however also checks everything it needs to update). It's question what is done exactly in the very first verification phase.

To see the reboots, you may have a look at this (a bit older) video:

xz1c unlocked & rooted doing fota update from oreo to pie and from pie to pie

Concerning userdata / cache partition - if you need to set up empty partition you may use following command:

mkfs.ext4 -b 4096 -e continue -O ^64bit,^huge_file cache.img

Concerning restarts - it may try to switch between main and fotakernel boots.

It is hard to recognize, because fota-ua in the main system may not allow full boot and in splash animation there is that green android the same as with fotakernel running and it may then reboot again.

There may be a 'warmboot' setting set somewhere to force certain kernel boot.

Or a parameter to 'reboot' system call.
[Sony Xperia XZ1 Compact ROMs, Kernels, Recoveries,] munjeni:
I saw this in your strace but I forgot it completely

execve("./fota-ua", ["./fota-ua", "c"],

Edit:

the same thing happen.

Edit:

I think this is now the less impotant, having fotakernel flash succed and strace log I think is enought step for looking into how thing is extracted from update_package_renamed to fw_backup and flashed to block device, I just wanted to succedd in all but seems it can't be done. I think better idea is having more time for analysing update_package_renamed right?

---------- Post added at 09:45 PM ---------- Previous post was at 09:28 PM ----------

At 4:48 its clear that recovery doing update, thats probably reason why I not getting system update, seems fota-ua waiting for recovery action which is not happening? I must go now, see you next day

Edit:

/data/update_package_renamed
420897111 4096
1
0 102759

I don't use uncrypt tool, fota-ua read it directly :) Problem might be my block.map e.g. it have one block more that size of the update_package_renamed and probably (in case bin diff tool based package) compare empty block with system partition and gets confused. Don't know. :)

Edit:

Compared now system.ext4 with system.img from sinunpack, my tool miss 80mb data at the end of ext4 partition! In ext4 header its clear about size of the partition, but that 80mb more for what is that? Probably fota ua fail here from that reason, I will try next day to reupload cache and data partition! Great tool! :)
[Sony Xperia XZ1 Compact ROMs, Kernels, Recoveries,] mrpinq:
Stuck in restart loop

Shortly: I screwed up...

I got new phone (sony xz1c) and I wanted to install lineage on it. First I noticed that I need to save drm keys and I followed this instruction ( also [MENTION=3463150]munted[/MENTION] pdf guide). I got the the step 10 and then it started to go strange. There was many "updates" on the guide and the pdf guide when to different path ( I just wanted to install lineage). Then I followed one of the updated in here and I installed twrp, flash magisk and then install lineage 15.1(because original system was Android 8) and then updated that to Lineage 16.1. Everything looked good, but I couldn't install firefox browser from google play and camera app(sony and lineage camera) gives error that "can't find any storage". I wiped the phone with twrp, but I still had same problem. Then I installed the hideunlock kernel in zip and then the phone could not start a OS and went to restart loop. I can get to twrp and I tired to reinstall Lineage 16, but it just prints one line and then restarts the twrp. I tried to restore backup, but it just gives "Failed" message. Nothing else. I also tried to newflash G8441_Customized DE_1310-4373_47.1.A.16.20_R6B ( like in pdf), but newflash says that I don't have space in disk. So was thinking ask from here what to do, before I make more mistakes...

Update: Ok, I Manage to install Lineage 16 again, but the same storage issue was there. So I went to TRWP and wiped the Internal Storage and it worked! And the newflash no space problem.. yeah, I didn't have sapce on my computer, not phone...
[Sony Xperia XZ1 Compact ROMs, Kernels, Recoveries,] munjeni:
Anybody explain this?

code:

unsigned long long x = (unsigned long long) -1;
printf("0x%llx\n", x);
printf("0x%llx\n", (unsigned long long)((unsigned long)0x00134985 * (unsigned long)0x1000));
unsigned long i;
unsigned long long test = 0;
for(i=0; i < 0x1000; ++i)
test += 0x00134985;
printf("0x%llx\n", test);

output:

root@ubuntu:~/qemu_aarch64/anyxperia_dumper# ./sony_dump.x86_64
0xffffffffffffffff
0x134985000
0x134985000
root@ubuntu:~/qemu_aarch64/anyxperia_dumper# ./sony_dump.i386
0xffffffffffffffff
[COLOR="Red"]0x34985000[/COLOR]
0x134985000

Red!!! What...???

Whole day I have wanted to figure out why my tool anysindumper fail to add the rest from the sparse to the end of ext4 partition and figured out folowing calculation fail:

case CHUNK_TYPE_DONT_CARE:
printf("DONTCARE NOW=%llX\n", curr_out_poss);
curr_out_poss += sparse_data_out_sz;
fseeko64(out, curr_out_poss, SEEK_SET);
printf("DONTCARE END=%llX %llX blocks=0x%X blocksz=0x%X\n\n", curr_out_poss, (unsigned long long)(chunkHeader.ChunkBlocks * sparseHeader.BlockSize), chunkHeader.ChunkBlocks, sparseHeader.BlockSize);
printf("0%llx\n", (unsigned long long)((unsigned long)0x00134985 * (unsigned long)0x1000));
break;

it fail to calculate long long excatly at this : 0x00134985 * 0x1000 , instead of writing last block to offset 0x134985000 it write to offset 0x34985000, omg, I spent whole day on this, it might be I am realy crazy or something is wrong there :(
[Sony Xperia XZ1 Compact ROMs, Kernels, Recoveries,] j4nn:
[MENTION=4120742]munjeni[/MENTION], that is correct behaviour. When compiled as 64bit code, native long type is 64bit (no difference to 'long long').

While when compiled as .i386 (i.e. 32bit), native 'long' is 32bit wide. When multiplying two types, you get the result in type of the largest types of the two operands.

Since both operands are 'long' you get the result as 'long' (32bit if compiled as i386), regardless you cast it later to 'long long'.

To have the result "correct / normally expected", you would need to cast at least one of the operands to 'long long' (and then you do not need to cast the result).
[Sony Xperia XZ1 Compact ROMs, Kernels, Recoveries,] j4nn:
[MENTION=4120742]munjeni[/MENTION], getting back to fota-ua, I've found few details previously overlooked:

warmboot=0x77665502: the first boot after clicking install button is main android system running fota-ua (also mentioned before),

but it is not only doing verification, it _installs_ fotakernel from the fota update package in order to reboot into it

(there is no ua_status file present in this boot - that probably means it is starting from scratch)

warmboot=0x6f656d46: the second boot is the fotakernel, but already updated (11.228), that's the ua_status '03 00 00 00 00 00 00 00' fota_ua run

warmboot=0x6f656d50: after all is updated, it reboots again into fotakernel to do some finalization - there is no ua_status file anymore, but there are other files, like 'executed' and 'status'

warmboot=0x77665501: finally it reboots to the updated main android system

I've patched installation splash screen pictures to display which kernel is running and captured video of all the fota installation from pie 10.107 to pie 11.228 - you can check it here:

https://youtu.be/ItvCUsxzReg

You can also find /cache/recovery folder content when each kernel has booted, just before starting 'fota-ua', defining the start state of each fota-ua phase attached here.

This means the ua_status '03 00 00 00 00 00 00 00' phase may also fail some checking because it may expect fotakernel to be already updated.
[Sony Xperia XZ1 Compact ROMs, Kernels, Recoveries,] Blu-eyes:
I found this "https://googleprojectzero.blogspot.com/2018/09/" where is explained how to use CVE-2018-9445 to make privilege escalation (zygote->vold-> init->kernel), hope that this will be helpful, I can't try this because I don't have my S5( for USB host).
[Sony Xperia XZ1 Compact ROMs, Kernels, Recoveries,] j4nn:
[MENTION=7556135]WheelchairArtist[/MENTION] & [MENTION=7543898]Blu-eyes[/MENTION]: thanks, that usb storage vulnerability looks very interesting.

You know, I can't test everything.

I feel that I am close to fully exploit via inotify/rename (CVE-2017-7533) in combination with binder data leak, which I am working on currently. So I like to continue, considering how much work I've done already.

It would be great if you could try to get root via the usb storage bug on xperia phones - I would need to lose all hopes with my current work to be forced to switch to something else.
[Sony Xperia XZ1 Compact ROMs, Kernels, Recoveries,] Inerent:
Just wanted to tell you J4NN that total bounty for temp roon on snapdragon 835 845 devices is more than 20000 USD just bere on xda. You should protect your rooting exploit and collect some Cash. Just Take a look AT Verizon pixel 2 xl bere on hda there is a nice bounty.

Good luck and hopeing you Will do IT.
[Sony Xperia XZ1 Compact ROMs, Kernels, Recoveries,] WheelchairArtist:
@Blu-eyes

Very nice finding. I ordered some equipment the last days to fully replicate the exploit, i think i will receive my otg adapter in 2 days and then i will try to replicate the attack you linked as far as i can.

@j4nn

I was totally wrong with the wchan field, i looked at the wrong value, thanks for correcting me and yes you would need to bypass kaslr first to bypass pan and pxn with the mirroring attack at least, which you luckily have more or less. Now i was asking myself maybe it would be enough to write some shellcode into kernel space which just dumps the ta partition, which is what we want in the first place? So by writing into kernel space and pointing the kernel pointer to the shellcode you would bypass pan and pxn and maybe the shellcode would also be small enough to fit in your overwrite? pan and pxn only prevent accessing and executing user data but the shellcode would be accessed and executed from kernel space and then you wouldn't even need to bypass selinux, although this would be easier then bypassing pan and pxn, right? Because the kernel space mirroring attack is used to bypass pan and pxn and therefore access user data which we wouldn't need if we can fit code into the kernel space.

Please forgive me for writing bogus, i am a little tired.

Also i second what @Inerent told you ;) Therefore you would need a root shell with selinux disabled but maybe you could do this also with some shellcode in kernel space.

I totally undertand that you stick with your inotfy exploit since you put a lot of time and effort in it and made some respectable progress! ;)

seems like there is even null free shellcode which you would need and would also fit from size.
[Sony Xperia XZ1 Compact ROMs, Kernels, Recoveries,] Blu-eyes:
[MENTION=4418980]j4nn[/MENTION]

I appreciate the effort that you put in this exploit, I hope that you find a method to bypass kaslr, pan and pxn, I can't help you in this because I don't know anything about how these work.

I'll continue my research about CVE-2018-9488 and CVE-2018-9445. I posted here my "research" to let you know that you aren't alone in this research and to provide a starting point for other dev that wants to research about that.

[MENTION=7556135]WheelchairArtist[/MENTION]

Take a look also at "exploit-db.com/exploits/45379/" and "bugs.chromium.org/p/project-zero/issues/detail?id=1590&desc=3" they contain the PoC of the previous article.

I'll try this exploit on an emulated android, feel free to ask me anything.
[Sony Xperia XZ1 Compact ROMs, Kernels, Recoveries,] j4nn:
[MENTION=8615083]Inerent[/MENTION], I am afraid that what I have implemented (if it is eventually finished) would not be a universal temp root solution at all - there will be a strong dependency on the to be exploited kernel.
[MENTION=7556135]WheelchairArtist[/MENTION], there is most likely no way to write any code into kernel space - at least not without the mirroring attack. Executable memory regions are read only and writable locations are not executable. Those PXN and PAN mitigations are very effective. The linked shell code is only userspace executable and only for Intel platform (uses intel assembly to invoke a system call).

Guys, it's good to see that I am not alone here with this temp root task. It would be great if you could eventually test it also on a phone.
[Sony Xperia XZ1 Compact ROMs, Kernels, Recoveries,] Blu-eyes:
Sure, I'll try on my Xperia once it work on the emulator, I need to find some time to downgrade, this is my daily driver and I'm afraid of loosing all data.
[Sony Xperia XZ1 Compact ROMs, Kernels, Recoveries,] j4nn:
Right, be sure to back up. Lost some data myself. Upgrade seems quite safe, skipping few partitions with newflasher to avoid data erase. But it did not work this way with downgrade. Had to flash everything (i.e. erasing all) to recover from a boot loop that appeared after downgrade with data/ssd partitions skipped.

As updated in the first post, beware to dowgrade to the 47.1.A.2.281 Journalists firmware - you might lose Android attest key.

I hope 47.1.A.2.324_CE1 fw is better - please check the first post for more details.
[Sony Xperia XZ1 Compact ROMs, Kernels, Recoveries,] Blu-eyes:
Thank for the advice, but I already lost my attest key when I debranded my phone, so nothing to worry about that( I haven't understand what this key is for, without it the drm is ok and the safety net test pass).

I think that this key is part of Android rollback protection, but I'm not sure

OT: when I debranded I skipped some ta file, the phone booted fine but the drm key were missing, after that,a restore with Xperia companion fixed all, except android attest key.
[Sony Xperia XZ1 Compact ROMs, Kernels, Recoveries,] pbarrette:
If you need a copy of the US .281 firmware for exploit testing, send me a PM.

I've got an archive of firmware for the XZ1c that includes:

G8441_Customized US_1310-4371_47.1.A.2.281_R8B.zip

G8441_Customized US_1310-4371_47.1.A.2.324_R8B.zip

G8441_Customized UK_1310-6856_47.1.A.2.324_R8B.zip

G8441_Customized UK_1310-6856_47.1.A.2.374_R12B.zip

G8441_Customized UK_1310-6856_47.1.A.5.51_R12B.zip

G8441_Customized UK_1310-6856_47.1.A.8.49_R14B.zip

G8441_Customized US_1310-4371_47.1.A.12.34_R8B.zip

G8441_Customized DE_1310-4373_47.1.A.12.75_R6B.zip

G8441_Customized DE_1310-4373_47.1.A.12.119_R6B.zip

G8441_Customized US_1310-4371_47.1.A.12.119_R8B.zip

G8441_Customized DE_1310-4373_47.1.A.12.145_R6B.zip

G8441_Customized US_1310-4371_47.1.A.12.145_R8B.zip

G8441_Customized DE_1310-4373_47.1.A.12.179_R6B.zip

G8441_Customized US_1310-4371_47.1.A.12.179_R8B.zip

G8441_Customized US_1310-4371_47.1.A.12.205_R8B.zip

G8441_Customized UK_1310-6856_47.1.A.12.205_R16B.zip

G8441_Customized DE_1310-4373_47.1.A.12.235_R6B.zip

G8441_Customized US_1310-4371_47.1.A.12.235_R8B.zip

G8441_Customized DE_1310-4373_47.1.A.12.270_R6B.zip

G8441_Customized US_1310-4371_47.1.A.12.270_R8B.zip
[Sony Xperia XZ1 Compact ROMs, Kernels, Recoveries,] j4nn:
Just wondering - is here anybody with an experience in Jump Oriented Programming?

I believe I have everything to start a JOP attack to perform the get root patches in kernel, i.e. I can bypass KASLR, can patch a function pointer in kernel and most importantly can plant a block of arbitrary data into kernel space knowing it's kernel address (I've just implemented this but it is not tested/verified yet).

Now the question is, how to find a suitable JOP dispatcher gadget?

I have no experience with this stuff, just started looking at it with few tools.

In my opinion JOP based attack is the only way to bypass PXN and PAN (if not considering the mirroring attack).

Anybody?
[Sony Xperia XZ1 Compact ROMs, Kernels, Recoveries,] Blu-eyes:
I can be totally wrong,

What about loading a kernel module?

from Oreo Google impose the support for loading kernel module(I don't now if they must be signed)

With your exploit can you force to load a userspace module in the kernel?

and from here do anything you want...

Excuse me if I'm totally wrong, but I don't have,yet, the knowledge to understand any sort of complex exploit..
[Sony Xperia XZ1 Compact ROMs, Kernels, Recoveries,] j4nn:
[MENTION=7543898]Blu-eyes[/MENTION], such kind of thing is not possible in my opinion. It would be the same as trying to execute any other shell code.
[Sony Xperia XZ1 Compact ROMs, Kernels, Recoveries,] Blu-eyes:
Correct me if I'm wrong, again

On a standard Linux machine, when you load a kernel module (for example ath9k.ko, for atheros wireless) with insmod or modprobe, the code in that module will be executed as a part of kernel.

My, maybe wrong idea, Is to force the kernel to load a module so the code in it will be executed in a high privileged context.

--edit--

Or if you can you could replace a module from kernel memory, so your custom code will be executed instead of another.

Thanks a lot for you time spent with me.

I haven't tried yet the exploit that I've linked, I hope to try it soon.
[Sony Xperia XZ1 Compact ROMs, Kernels, Recoveries,] j4nn:
As far as I know, there is no function in kernel to load a module. Loading a module is done with help of userspace tools, like the insmod you've mentioned. The kernel has only limited support for even a filesystem access (like it's nearly not possible to open/read a file in kernel) - even loading a firmware, needed by kernel, is done indirectly with help of userspace tool and events to trigger that. Such tools obviosly need root in order to work. You might be able to generate some hotplug event to possibly trigger load of some kernel module, that again depends on userspace daemon which handles the events and invokes loading of appropriate modules (this handling is most likely different in case of android than in case of linux desktop).

It is not possible to replace (if not considering the mirroring attack) any kernel (or module) code in memory. Code regions are mapped read only, so even kernel code cannot modify them, without mapping them with write permission first.

And it's not possible to execute data areas, like code injected into stack or heap area, because those regions are mapped without execute permission.

With PAN it's not even possible to execute userspace code to be accessed from kernel space, as this access is forbidden unless explicitly allowed.

What is left is to execute in kernel existing code parts that combined in right sequence can result with wanted action - the case of Return/Jump Oriented Programming. But for that an easily modified data block in kernel space is needed too due to PAN mitigation in order to control the invocation of rop/jop gadgets.
[Sony Xperia XZ1 Compact ROMs, Kernels, Recoveries,] mirhl:
https://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2018-17182

https://googleprojectzero.blogspot.com/2018/09/a-cache-invalidation-bug-in-linux.html
[Sony Xperia XZ1 Compact ROMs, Kernels, Recoveries,] Blu-eyes:
If you enable stamina mode or ultra stamina, that underclock the CPU, maybe you will may more successful attempt.
[Sony Xperia XZ1 Compact ROMs, Kernels, Recoveries,] mirhl:
Indeed, I have always wondered where system "keeps track" of ta, considering the original voices surrounding the new "Yoshino drm".

I guess like a lot of heavy tinkerers would still love the whole partition backup then probably.
[Sony Xperia XZ1 Compact ROMs, Kernels, Recoveries,] munjeni:
Ok, but my fotakernel is allready updated! Today I have some free time, I will try to replace wrong userdata and cache partition, hope I finaly get system update

Edit:

solved calculation and now getting right md5sum
[Sony Xperia XZ1 Compact ROMs, Kernels, Recoveries,] j4nn:
[MENTION=4120742]munjeni[/MENTION], that's very nice achievement. Thanks.

btw, running it on phone without strace, the full update (all phases and reboots) takes a bit more than 8 minutes (as can be seen in the linked video).

Running it with strace in all phases on the phone takes a bit more than 20 minutes (strace logging to internal flash).
[Sony Xperia XZ1 Compact ROMs, Kernels, Recoveries,] j4nn:
[MENTION=4120742]munjeni[/MENTION], the logs in xz1c-fota-ua-states.zip - as mentioned, it's the exact content of /cache/recovery before start of fota-ua in the kernel boot as hinted by the parent directory. In other words, it is also in the state that fota-ua stored them in previous boot/run.

Concerning partitions - I am afraid all need to match - I just found that twrp mounted rw /oem and /dsp without a reason, making fota not possible if twrp had been only booted without flashing anything. Posted a patch to fix that.
[Sony Xperia XZ1 Compact ROMs, Kernels, Recoveries,] munjeni:
Vendor partition is also checked by fota-ua, vendor,oem,system.
[Sony Xperia XZ1 Compact ROMs, Kernels, Recoveries,] munjeni:
update_package_renamed is varian of the applypatch from android nootable recovery, in our case patch files is compressed in most case with an lzma?

Offset(h) 00 01 02 03 04 05 06 07 08 09 0A 0B 0C 0D 0E 0F

00000230 00 00 00 00 FF FF FF FF FC FF FF FF FF FF FF 3F ....ÿÿÿÿüÿÿÿÿÿÿ?
00000240 FF FF FF FF FF FF FF FF FF FF FF 0F [COLOR="Red"]5D 00 00 00[/COLOR] ÿÿÿÿÿÿÿÿÿÿÿ.]...
00000250 [COLOR="Blue"]01 00 0A 08[/COLOR] C2 29 93 FD 2F 80 48 A7 65 5E 11 A1 ....Â)“ý/€H§e^.¡
00000260 CB 2A 2C 30 12 27 68 C9 14 A9 25 1E A3 99 AA 80 Ë*,0.'hÉ.©%.£™ª€

Blue part is strange. Its probably an non standard lzma variant? Or is it lzma stream instead of file?
[Sony Xperia XZ1 Compact ROMs, Kernels, Recoveries,] j4nn:
Thank you [MENTION=3539907]apexofservice[/MENTION] for your donation, really appreciated.