yashsingh1/xdaforums huggingface dataset:
[Sony Xperia XZ1 Compact ROMs, Kernels, Recoveries,] j4nn:
[MENTION=3989065]serajr[/MENTION], just tested fota update from oreo to pie - it worked (here is a video).

Missed reboot after installation, so it booted to stock kernel.

But that is good too, as it confirms, that the update can be finished even if you do not have the updated kernel prepared/patched in advance.

Unfortunately when I tried to port my patch to pie kernel, it did not work - fw still detects the unlock.

I will need to look into it to make it pie compatible.

Anybody knows what that N like icon at the top means?

It appeared when I flashed OEM from DE customization while having CE1 there previously (on and from 47.1.A.16.20 fw version).
[Sony Xperia XZ1 Compact ROMs, Kernels, Recoveries,] serajr:
[MENTION=4418980]j4nn[/MENTION] Great news... once again! You don't stop to amaze us.

Regarding android Pie, it's indeed more secure and probably a new research would be necessary (and much more hard work in the horizon)! Count on me if you need!

Isn't the N icon the NFC one?
[Sony Xperia XZ1 Compact ROMs, Kernels, Recoveries,] Kayant:
Amazing work [MENTION=4418980]j4nn[/MENTION]

Your work continues to arouse my interest in reverse engineering, hacking and the like to one day learn a thing or two about how it is done.

As [MENTION=3989065]serajr[/MENTION] mentioned it is indeed the NFC icon. I guess it's enabled by default on that firmware.
[Sony Xperia XZ1 Compact ROMs, Kernels, Recoveries,] j4nn:
I wonder about that message on boot:

Your device software
can't be checked for corruption.

Please lock the bootloader.

Should not be SONY telling us how to do that?

To lock the bootloader?

I can imagine running custom firmware, having bootloader locked with custom signing keys, just for security.

But the missing piece here, is how to insert a custom signing key into the verified boot chain?

Could not be that message claimed with sony's open source program to tell us how to do that?

In my opinion it should be technically possible.
[Sony Xperia XZ1 Compact ROMs, Kernels, Recoveries,] janjan:
It should be possible but not so easy. We can ofc play a bit with bootloader image, but it is too risky. Can be end up with broken device. But yes u r right, it should be technically possible. Personally I need some free time to investigate that.
[Sony Xperia XZ1 Compact ROMs, Kernels, Recoveries,] j4nn:
[MENTION=490110]janjan[/MENTION], what I meant is that boot message above on unlocked phone suggests (i.e. recommended by sony) that we should (re)lock the bootloader - so sony should provide official way to setup custom verified boot signing keys and a way to lock bootloader for them.

I certainly did not mean to find some hack to do that - too risky obviously.
[Sony Xperia XZ1 Compact ROMs, Kernels, Recoveries,] j4nn:
Actually it rather looks to me like sony removed with pie some previously used protections and so it may be easier than it was with oreo.

Please correct me if I am wrong, but with oreo, sony had the bloody ric daemon in the bootimage - for that I had to patch magisk too in order to avoid ric to discover magisk presence.

But with pie - there seems not to be any ric daemon, not in kernel ramdisk, not in vendor nor system partitions.

In fact it rather seems with pie that the fw does not really care if magisk utilities (like su) are present.

But it seems lot of stuff is compiled statically in multiple daemons with pie, that could make those drmfix variants difficult to port.

Hopefully not that case with my kernel patch - let's see...
[Sony Xperia XZ1 Compact ROMs, Kernels, Recoveries,] serajr:
I wasn't clear enough in my statement, I meant about your difficulty in making your patch work in android pie, as you stated.

You'll win this battle :good:
[Sony Xperia XZ1 Compact ROMs, Kernels, Recoveries,] SGH-i200:
I was thinking the same on first bootup of the Pie firmware! ;)
[Sony Xperia XZ1 Compact ROMs, Kernels, Recoveries,] josephnero:
any help with this part for a noob?can't get it to work

cd /data/local/tmp

chmod 755 reno*

./renoroot

Am I supposed to just copy that in adb command or?....
[Sony Xperia XZ1 Compact ROMs, Kernels, Recoveries,] vofferz:
Yes. First type "adb shell" to get command line terminal to the phone and in that terminal type those commands to give needed permissions to the files and start renoroot tool.
[Sony Xperia XZ1 Compact ROMs, Kernels, Recoveries,] josephnero:
[MENTION=7803472]vofferz[/MENTION]

thanks a million times.

how about this part? all of it together?

cd /data/local/tmp

dd if=/dev/block/bootdevice/by-name/TA of=TA-locked.img

chown shell:shell TA-locked.img

sync

sync
[Sony Xperia XZ1 Compact ROMs, Kernels, Recoveries,] j4nn:
kernels hiding bootloader unlock released for XZ1c/XZ1/XZp

Please see here:

[XZ1c]rooted kernel hiding bootloader unlock with working fota

Kernels for XZ1 and XZp plus source branches on github will be released later today...

-- edit --

[XZ1]rooted kernel hiding bootloader unlock with working fota

[XZp]rooted kernel hiding bootloader unlock with working fota

Patched kernel source available here: https://github.com/j4nn/sonyxperiadev-kernel-copyleft/commits/my-bluhide/47.2.A.6.30

Patched magisk source available here: https://github.com/j4nn/Magisk/commits/v18.1-manager-v7.0.0-ric
[Sony Xperia XZ1 Compact ROMs, Kernels, Recoveries,] j4nn:
Updated the howto in the first post with links to forum threads with kernels patched to hide bootloader unlock status.
[Sony Xperia XZ1 Compact ROMs, Kernels, Recoveries,] madclits:
Ummm, hello sir, im new there in xzp forum, just upgrading from z2 to xzp!. I would like to ask, my xzp was currently on 47.1.A.16.20 oreo firmware.

Nah my question is, if i want to root my xzp with the step above, i should use the 47.1.A.3.254_RU firmware right? What tool i could use to flash the RU fw? (in z2 it was flashtool)

and if theres available tutorial on how to use the tool to flash the RU fw to my xzp, please tell me the links.

Thanks a lot sir :)
[Sony Xperia XZ1 Compact ROMs, Kernels, Recoveries,] HandyMenny:
[MENTION=4418980]j4nn[/MENTION] I have performed exactly the above actions. I haven't seen the stock/FOTA recovery. I think I have not lost my data, but neither the TWRP nor the system was able to read them, the system went into bootloop, I solved with "fastboot format userdata". I think the unlock has changed the encryption key or something like that.

I had a strange problem with renoroot. I've used it in the past, but this time it got 10 times up to 30000 events with 5-6 hits without anyone having any success or causing a reboot. I did a strange thing, I changed the date of my system to November 4, 2018, after doing that I had 2 reboots at about 5000 events with 3-4 hit, the third time after 4000 events the exploit was successful. Is it just a coincidence?
[Sony Xperia XZ1 Compact ROMs, Kernels, Recoveries,] j4nn:
[MENTION=3844251]josephnero[/MENTION] and [MENTION=9744549]madclits[/MENTION], just follow the howto closely. There is even a link for a more detailed guide in a pdf with screenshots from [MENTION=3463150]munted[/MENTION] done in windows, that may help you too. Few video tutorials are linked there too. There is a link to newflasher tool to flash the fw, just visit it. Using flashtool may be possible too, but I am not familiar with it and you would need probably to convert the unpacked fw to a .ftf first.

[MENTION=5400898]HandyMenny[/MENTION], thank you for the test. I've checked the TA-mid-twrp.img from your test and it did not contain the device master key. That confirms sony did a good job there, so we are really lucky with the temp root exploit. Concerning your renoroot results - I guess it is a coincidence. But it depends a lot on how you prepare the phone just before starting renoroot. I've also posted a video that shows that - xz1c factory reset and initial setup to prepare for renoroot. It is followed by another example of renoroot in action.
[Sony Xperia XZ1 Compact ROMs, Kernels, Recoveries,] j4nn:
[MENTION=3989065]serajr[/MENTION], I guess it may be interesting to test my unlock hiding kernel patch with XZ2 - do you still like to test it?

I assume your XZ2 is unlocked already, drm keys lost. But that is ok with regard to FOTA system update - it works even in case drm keys had been lost (TA not restored) - verified that myself, it is documented here.

Just let me know please if I should build a test kernel for you and which version.
[Sony Xperia XZ1 Compact ROMs, Kernels, Recoveries,] serajr:
Yeah mate, my XZ2 (H8216) is ready for your tests.

A good test case would be from firmware 52.0.A.3.163 (jan patch) to the 52.0.A.3.202 (feb patch).

Let me know if you need further info and/or files. Pm me if you prefer.

Thanks in advance.
[Sony Xperia XZ1 Compact ROMs, Kernels, Recoveries,] j4nn:
[MENTION=3989065]serajr[/MENTION], I've patched and build the requested kernels for XZ2 - assuming they should at least boot as the unmodified stock kernel rebuilt from source (provided via PM) worked for you.

boot-H8216-52.0.A.3.163-hideunlock-rooted.img

boot-H8216-52.0.A.3.163-hideunlock-rooted.dtbo.img

boot-H8216-52.0.A.3.202-hideunlock-rooted.img

boot-H8216-52.0.A.3.202-hideunlock-rooted.dtbo.img

As mentioned, the kernels have dm verity enabled - you need to be running untouched firmware in order to FOTA not fail, so dm verity is good to be enabled for this purpose.

Please check the unlock status in the security service menu first after you 'fastboot boot' the first kernel.

Thanks.

-- edit --

Source code available here: https://github.com/j4nn/sonyxperiadev-kernel-copyleft

branches my-bluhide/52.0.A.3.163 and my-bluhide/52.0.A.3.202
[Sony Xperia XZ1 Compact ROMs, Kernels, Recoveries,] serajr:
Thank you! :good:

I tell you later if they work, okay!
[Sony Xperia XZ1 Compact ROMs, Kernels, Recoveries,] j4nn:
That is very bad - not working fastboot boot some.img.

I am so glad I've decided for XZ1 Compact instead of (waiting for) XZ2 Compact!
[Sony Xperia XZ1 Compact ROMs, Kernels, Recoveries,] serajr:
It is really bad! Anyway, thank you for your efforts on this! Take some beers ;)

I am so glad I've also decided buying a XZ1 (G8341).
[Sony Xperia XZ1 Compact ROMs, Kernels, Recoveries,] j4nn:
[MENTION=3989065]serajr[/MENTION], thank you for your donation.

I wonder if it was possible to somehow merge the dtbo.img into the kernel boot.img to make it boot from 'fastboot boot'.

I see that the 'fastboot boot' problem is not the only problem with Tama platfrom (xz2, xz2c, xz3).

Do you know more details about that problem with camera if it BL was unlocked on oreo instead of some other fw version?

I see you and [MENTION=6031898]MartinX3[/MENTION] were involved into a workaround for that problem, having some fixes in twrp package.

Is old version of tzsuntory blob needed for certain phones even with pie fw, where camera is supposed to just work?

I am interested into a technical explanation what exactly happened there.
[Sony Xperia XZ1 Compact ROMs, Kernels, Recoveries,] MartinX3:
the fastboot boot problem is a bootloader bug, which may get fixed with android Q, according to alin jerpelea, head of Sony Open Devices Project. :)
[Sony Xperia XZ1 Compact ROMs, Kernels, Recoveries,] j4nn:
Updated post#652 with link to sources for the XZ2 unlock hiding kernels.
[Sony Xperia XZ1 Compact ROMs, Kernels, Recoveries,] j4nn:
[MENTION=3989065]serajr[/MENTION], I have an idea how it may be possible to 'fastboot boot somekernel.img' with XZ2 - if you are still interested, we can try it...

Just to confirm - even when you flashed dtbo.img, it still was not possible to use 'fastboot boot', was it?

You had to flash the kernel together with dtbo.img in order to make it boot, hadn't you?

If so, we can try to merge dtbo and additional device tree properties from bootloader into kernel's dtb to make it boot.

If you want to test it, try following commands in a root shell after a fresh boot of a stock kernel:

cd /data/local/tmp
dmesg > dmesg.txt
cp /sys/firmware/fdt .
chown shell:shell dmesg.txt fdt

and send me the fdt and dmesg.txt files please. I would try to prepare a new test kernel then.
[Sony Xperia XZ1 Compact ROMs, Kernels, Recoveries,] j4nn:
development progress / change log

Updated post#2 - added development progress / change log there with links to corresponding posts.

It may be useful to those interested for example in details about the renoroot exploit and what difficulties had to be overcome or just to find quickly recent achievements...
[Sony Xperia XZ1 Compact ROMs, Kernels, Recoveries,] serajr:
I don't know many details about the hack that fixes only the preview of the camera (which is black without it). I just created a magisk module to avoid pushing the files directly into the vendor partition. But I could check that not all hack files were needed.

I installed android pie on my xz2 only a few days ago, and I think soon after that we started talking about the kernel hack.

MartinX3 is indeed the xz2 kernel/recovery guy that parhaps has the answers related on this for you!

Sure I am, why not? I sent you a pm with the files.

Yes, I had. Only after flashing both kernel and dtbo images device booted up.

Tia!
[Sony Xperia XZ1 Compact ROMs, Kernels, Recoveries,] MartinX3:
TAMA has AvB 2.0

You can deactivate vbmeta or its partition hash values needs to match every partition.

According to Sony it's a bootloader bug probably fixed with Q (fastboot boot boot.img).
[Sony Xperia XZ1 Compact ROMs, Kernels, Recoveries,] j4nn:
[MENTION=3989065]serajr[/MENTION], here is a XZ2 kernel with merged device tree for the boot from fastboot test:

boot-H8216-52.0.A.3.202-hideunlock-rooted-standalone.img

Please make sure your run completely unmodified corresponding firmware, i.e. do not use that fastboot command with options to disable verity/verification, have stock kernel (and everything else) flashed including dtbo and vbmeta partitions. Do not even flash twrp recovery - really try to have everything stock untouched.

First verify, that the untouched fw boots ok as it is.

Then enable adb debugging and use

adb reboot bootloader

to switch to fastboot mode. Please use the command to reboot to fastboot mode.

If you enter fastboot mode by keys combo, use 'fastboot reboot bootloader' command to reboot to fastboot by a command.

Now use

fastboot boot boot-H8216-52.0.A.3.202-hideunlock-rooted-standalone.img

to test the kernel if it boots from usb via fastboot (do not flash it).

This is a first attempt - it may boot already. If not, I may try to add kernel command line options that are added by bootloader when booted from flash.
[Sony Xperia XZ1 Compact ROMs, Kernels, Recoveries,] serajr:
[MENTION=4418980]j4nn[/MENTION] you're insane! It did work!! :good:

I did the procedure twice, in both cases got root:

D:\Android\XZ2_P_9_H8216_52.0.A.3.202\j4nn_unlock_hiding_kernel_patch>fastboot devices
QV702S2G1B fastboot

D:\Android\XZ2_P_9_H8216_52.0.A.3.202\j4nn_unlock_hiding_kernel_patch>fastboot reboot bootloader
rebooting into bootloader OKAY [ 0.003s]
Finished. Total time: 0.004s

D:\Android\XZ2_P_9_H8216_52.0.A.3.202\j4nn_unlock_hiding_kernel_patch>fastboot boot boot-H8216-52.0.A.3.202-hideunlock-rooted-standalone.img
Downloading 'boot.img' OKAY [ 0.603s]
booting OKAY [ 5.641s]
Finished. Total time: 6.279s

D:\Android\XZ2_P_9_H8216_52.0.A.3.202\j4nn_unlock_hiding_kernel_patch>adb shell
* daemon not running; starting now at tcp:5037
* daemon started successfully
H8216:/ $ su
H8216:/ #

When you can, please compile 52.0.A.3.163 kernel, so we can see if the ota installs (I'm pretty sure it will)!

Wow...

Proofs (check the file names):
[Sony Xperia XZ1 Compact ROMs, Kernels, Recoveries,] serajr:
The "head" should see this ^^ then :p
[Sony Xperia XZ1 Compact ROMs, Kernels, Recoveries,] MartinX3:
Wtf

I speak with the Sony open device devs asap :eek:

Maybe i mean that it will the dtbo partition with the fix. xD

But since he said it's a bootloader bug i stopped finding a fix. xD

[MENTION=4418980]j4nn[/MENTION] it would be nice if we could use it to fastboot boot TWRP :o
[Sony Xperia XZ1 Compact ROMs, Kernels, Recoveries,] j4nn:
[MENTION=3989065]serajr[/MENTION], that's great to hear that it worked. Here is the older kernel so you can test fota system update (please make sure all fw is stock, as described above, for proper customization update to be offered also oem partition needs to be flashed).

boot-H8216-52.0.A.3.163-hideunlock-rooted-standalone.img

[MENTION=6031898]MartinX3[/MENTION], I guess it could work also with twrp. I am only not sure about the behaviour of verity setting in vbmeta. My kernel has everything as stock one, so it also works with stock setup of everything else.
[Sony Xperia XZ1 Compact ROMs, Kernels, Recoveries,] MartinX3:
very nice. I am optimistic too, that it will work.

So I can remove the boot.img from the twrp installer.zip

And later also remove the .zip, because twrp will get a function to modify the underlying real boot image if you "fastboot boot".

Can you publish publish a manual so I can reproduce it? :)
[Sony Xperia XZ1 Compact ROMs, Kernels, Recoveries,] j4nn:
[MENTION=6031898]MartinX3[/MENTION], I just pushed my patch for tama 'fastboot boot' dtbo workaround to my kernel source tree on github (my-bluhide/52.0.A.3.163 and my-bluhide/52.0.A.3.202 branches):

https://github.com/j4nn/sonyxperiadev-kernel-copyleft.git

There is no special howto - except the dtbo workaround patch, I just followed instructions from README_Xperia to build the kernel (you do not need the final step 3 to build dtbo obviously). But you need all the external tools as sony describes there.

Magisk is integrated then to allow root when booted via 'fastboot boot' - but you do not need that for twrp.

Just be careful with the unlock hiding patch - it may change behaviour of fw components and therefore cause problems if fw was modified.

For example with xz1c/xz1/xzp and oreo fw it would not boot at all until I patched magisk init to hide magisk's components from sony ric daemon!
[Sony Xperia XZ1 Compact ROMs, Kernels, Recoveries,] serajr:
That'd be great! ;)
[Sony Xperia XZ1 Compact ROMs, Kernels, Recoveries,] j4nn:
[MENTION=3989065]serajr[/MENTION], now that we have kernel that can be 'fastboot boot'-ed, maybe you can test if XZ2's bootloader can 'fastboot boot' it even from fastboot state which was entered by the keys combo when plugging it to usb?

That is without rebooting to fastboot by a command (adb reboot bootloader / fastboot reboot bootloader).

In case of xz1c/xz1/xzp, when you try to 'fastboot boot' from fastboot entered by key combo instead of rebooting to it, the 'fastboot boot' command would not boot the kernel in normal mode, instead it would enter charging mode (not very useful)...

I wonder if sony fixed this with TAMA platform or not.
[Sony Xperia XZ1 Compact ROMs, Kernels, Recoveries,] serajr:
[MENTION=4418980]j4nn[/MENTION] and as we expected, worked flawlessly :highfive:

Just awesome!!

See for yourself (low/crappy vga quality)...
[Sony Xperia XZ1 Compact ROMs, Kernels, Recoveries,] j4nn:
[MENTION=3989065]serajr[/MENTION], nice:)
[Sony Xperia XZ1 Compact ROMs, Kernels, Recoveries,] serajr:
I've already tried that before, with no luck, XZ2 also enters charging mode!
[Sony Xperia XZ1 Compact ROMs, Kernels, Recoveries,] MartinX3:
Holy ****. Good to know!

---------- Post added at 11:23 PM ---------- Previous post was at 11:20 PM ----------

Hmm, I would not need the other modifications. Only the fastboot boot fix :)

The other stuff could be made available as a twrp installer zip to the ones who want to install it. :)

Does this mean you need to release everytime a new kernel or can I extract this fastboot boot patch somehow? :)
[Sony Xperia XZ1 Compact ROMs, Kernels, Recoveries,] j4nn:
[MENTION=6031898]MartinX3[/MENTION], I guess the fix for 'fastboot boot' may be possible with repack of the kernel image and all dtbs in it, without need to rebuild the kernel itself.

The thing with unlock hide is currently so a new build is needed for each new stock kernel, unfortunately.

But just for twrp - you do not need a new release for each new fw or do you?
[Sony Xperia XZ1 Compact ROMs, Kernels, Recoveries,] MartinX3:
I need a new build with each new monthly firmware update (new stock touch kernel drivers and maybe new dtb AvB stuff) :)

And yes. Just to fastboot boot TWRP, execute the TWRP installer and installing a persistent TWRP. :)
[Sony Xperia XZ1 Compact ROMs, Kernels, Recoveries,] j4nn:
[MENTION=6031898]MartinX3[/MENTION], in that case - if you already need to rebuild the kernel for each monthly update - just merging the 'fastboot boot' patch into the twrp kernel source tree should be easiest way to go I guess...
[Sony Xperia XZ1 Compact ROMs, Kernels, Recoveries,] MartinX3:
I feel so blind.

You mentioned this patch all the time but I'm too stupid to find it.
[Sony Xperia XZ1 Compact ROMs, Kernels, Recoveries,] j4nn:
Just select the mentioned branch and click commits on github. Or use this link

merge dtbo into kernel builtin dtb for tama fastboot boot

:)
[Sony Xperia XZ1 Compact ROMs, Kernels, Recoveries,] MartinX3:
Aaargh I looked into the upstream branches.

It's definitely too late for me (1:00 in the morning), thank you very much! :)
[Sony Xperia XZ1 Compact ROMs, Kernels, Recoveries,] shoey63:
[MENTION=4418980]j4nn[/MENTION], many thanks for this. By swapping out the ramdisk with [MENTION=6031898]MartinX3[/MENTION]'s TWRP ramdisk, XZ2 users can finally fastboot boot TWRP!

It takes a little while, and you need to "adb reboot bootloader" first, but eventually TWRP appears, fully functional with working touch.

I'll leave it here for anyone who is interested https://www.androidfilehost.com/?fid=1395089523397900470
[Sony Xperia XZ1 Compact ROMs, Kernels, Recoveries,] j4nn:
[MENTION=4675598]shoey63[/MENTION], that is good to hear...
[Sony Xperia XZ1 Compact ROMs, Kernels, Recoveries,] alagga:
Is this Instuction still up-to-date? :)
[Sony Xperia XZ1 Compact ROMs, Kernels, Recoveries,] j4nn:
[MENTION=4976984]Antiallergiker[/MENTION], the howto in the first post of this thread should always be considered as the primary howto.

I guess the pdf howto can still help, but I am not sure, probably the recent changes regarding patched kernel are not there.

So it may be useful to get started, like setup of all drivers and tools in case of windows use, but the official howto should always be the one that is followed closely.

And there have been added also example videos in the official howto, so that also may be quite helpful.
[Sony Xperia XZ1 Compact ROMs, Kernels, Recoveries,] serajr:
shoey63 thank you so much for this. It worked perfectly! :highfive:
[Sony Xperia XZ1 Compact ROMs, Kernels, Recoveries,] alagga:
So I've followed the guide and after downgrading from stock 47.2.A.6.30 to the here linked 47.1.A.3.254_CE1 Security Screen in Service stays empty and says "none".

After restoring TA (and still on 47.1.A.3.254_CE1), Camera opens and shows picture correctly, but when i press the capture button the Camera app doesn't take a picture (the menu disappers, like you would hold the capture button).

Camera on Pie works regular, but in Security SOMC FIDO Key says "NOT PROVISONED"

Has there sth. gone wrong or is this normal behavior on XZP?
[Sony Xperia XZ1 Compact ROMs, Kernels, Recoveries,] j4nn:
[MENTION=4976984]Antiallergiker[/MENTION], did you downgrade including flash of bootloader? You need to unzip boot.zip in order to get bootloader flashed with newflasher. If you switch between pie and oreo version without switching bootloader too, you get the behaviour you've described.

You need to run the kernel patched to hide bootloader unlock (after TA-locked restored) in order to avoid fw to disable SOMC Fido Key.

Since pie you get camera somehow working even when drm keys had been lost, it is better to verify TA restore on oreo. Downgrade to oreo 16.20 version including flash of bootloader (i.e. unzipped boot.zip) to confirm camera functionality. Then you can 'fastboot boot' the oreo 16.20 kernel hiding bootloader unlock to get root and check about other drm features, possibly even with fota system update to pie.