yashsingh1/xdaforums huggingface dataset:
[Sony Xperia XZ1 Compact ROMs, Kernels, Recoveries,] flashcart:
Just to add to the "working" comments. (XZ1C, 47.1.A.2.324_CE1)

My device was on 47.1.A.2.281, didn't work there (I thought it might because of the success on an XZ1. (Also the message about other versions definitely not working hadn't been added yet.) Had plenty of overwrites and restarts but I suppose the kernel I had would either never work or would require a lot of luck.

Anyway after upgrading it worked on the first try after 22k events/4 overwrites. Current status is working camera, suntory blobs error, CKB OK, widevine OK, SOMC FIDO key not provisioned, SOMC Attest key not provisioned, Android attest key provisioned.

Also got magisk working, flashing the zip caused bootloops but creating a patched boot.img worked fine. Safetynet wasn't passing before this (CTS failure, I assume it detected the unlocked bootloader), but it does now.

Used the backup script as well as manually taking TA just in case.

Have sent a donation, nice work, I wasn't sure if we'd ever see a TA backup for this generation of devices.
[Sony Xperia XZ1 Compact ROMs, Kernels, Recoveries,] tabbed:
Just wanted to report another successful root and TA backup on a XZ1c (G8441). I currently have:

and the camera works, which is very neat. I'm going to flash lineage and see if it's possible to port over any of the sony apps.

Thank you! I'll send a donation shortly. I also ran the backup scripts before and after unlocking (on 47.1.A.8.49, downgraded from 47.1.A.12.145). Is there anything there that you can use? Just doing a simple directory diff, the only files that differ by checksum are:

Thank you again!
[Sony Xperia XZ1 Compact ROMs, Kernels, Recoveries,] Unbounded:
Thank you very much [MENTION=4418980]j4nn[/MENTION] for your quick reply and detailed instruction.

I have successfully backed up my TA.

You are truly a legend bro.
[Sony Xperia XZ1 Compact ROMs, Kernels, Recoveries,] j4nn:
Anybody who has never downgraded, having bootloader still locked and a fw version on his phone is not newer than 47.1.A.8.49 please do not unlock yet, contact me first.

I would like to do the FULL BACKUP and comparison, assuming that you would have all the three keys in security service menu still provisioned.

I see there have been opportunities for testing lost unfortunately.

Or did you [MENTION=6329152]flashcart[/MENTION] by any chance managed to use the AUTOMATED FULL BACKUP scripts twice to have full state before and after unlock, while you've only upgraded, not ever downgraded?

I am afraid this testing opportunity has been lost also with [MENTION=4503201]blue-spiderfire[/MENTION] who PMed me that "TA backup and restoring went well. Camera is working." : -(

Thanks.
[Sony Xperia XZ1 Compact ROMs, Kernels, Recoveries,] jrbmed08:
Unfortunately I have downgraded my XZ1c in the past and I guess unknowingly wiped my attestation key. I did not notice any difference daily usage for what that's worth.

Prior to this current downgrade to back up TA, I could not access the menu to check security settings as my dialer force closed every time. After downgrading (and making sure to remove the persist file), I have the attached results (edit: no longer attached, but only attest key not provisioned, other 2 are fine).

I then went through the TA backup process (manually, not with the batch file), unlocked the bootloader, and proceeded to the second part. Attempting to restore the TA partition did not work and my camera will not save photos (no green pics, just nothing saved on internal storage). See 2nd screenshot, all 3 keys not provisioned. Not sure what I did wrong. Hopefully it isn't a result of my lost attest key.

Edit: Removed screenshots as I realized they show my IMEI. I think people should be able to tell what I'm saying from the descriptions.

Also camera works immediately after going through the restore process but it does not persist past a reboot.
[Sony Xperia XZ1 Compact ROMs, Kernels, Recoveries,] flashcart:
I have the directory from before unlocking the bootloader, after upgrading, while all keys were still provisioned.

And I've just done another backup in my current state, after restoring TA. Apologies if it's not very clean.

Results of using WinMerge to compare the directories at: pastebin(.com)/g2z4HHBh

Again not sure how useful this is because of the other changes I've made. PM me if you want any of the actual files.
[Sony Xperia XZ1 Compact ROMs, Kernels, Recoveries,] russel5:
can someone post here persist.img from locked phone?
[Sony Xperia XZ1 Compact ROMs, Kernels, Recoveries,] greatpatel007:
Restore OTA Update (Tried on XZP and XZP dual )

After TA restore if its shows bootloader is already unlocked in (OEM Unlocking) developer setting just do a factory reset or flash the firmware again.

If your phone shows OEM Unlocking Option same as stock (like you have never enabled it) then you are good to go for OTA update. Just make sure you have stock recovery and kernel.

There is no 100% guarantee that it will work or not for everyone. Tried and tested on XZP/Dual It worked for me.
[Sony Xperia XZ1 Compact ROMs, Kernels, Recoveries,] j4nn:
[MENTION=2904361]greatpatel007[/MENTION], this is interesting.

Just wondering, did come updates for google/system apps only or for sony apps too?

Or only for sony apps?

Anybody with xz1c and restored TA running still the fw used for the exploit - please test, if flashing the secd-ignore-unlock.zip from twrp onto the stock firmware with a factory reset and OEM Unlock option as described by greatpatel007, allows OTA updates and of what components.

Thanks.
[Sony Xperia XZ1 Compact ROMs, Kernels, Recoveries,] OzzyFyshmaster:
I just received a new XZ1c today, so I think it satisfies the above criteria. The problem is I have no idea what I'm doing...the last time I did anything like this was running motochopper to root my droid razr hd many years back.

Regarding the "service menu -> service tests -> security" screenshot, I don't see any option in settings resembling "service menu". Here's a screenshot of About phone though:

Edit: eesh...apparently I don't get posting image urls

Edit2: oh, I guess it's 'cause I'm under 10 posts yet.

here's the straight text url:

imgur.com/a/sgjPNSf

The aforementioned "full backup and comparison" ...does that mean steps 1-6 of the OP?
[Sony Xperia XZ1 Compact ROMs, Kernels, Recoveries,] 4rz0:
You access the service menu with the calling code

*#*#7378423#*#*
[Sony Xperia XZ1 Compact ROMs, Kernels, Recoveries,] arslancn:
I downgraded my XZ1(G8341) from latest firmware to 47.1.A.2.324_CE1 without persist.sin(all keys PROVISIONED),

make TA and full backup with temp root,

unlocked my bootloader and restored TA,

flashed latest firmware and make full setup,

flashed latest twrp, flashed latest magisk and patched secd,

my all keys are PROVISIONED.

Thanks for everything [MENTION=4418980]j4nn[/MENTION] . You are the best.

edit1= after everything, i have take a look at security config, it shows ''Bootloader unlock allowed: No'' . is it normal?

edit2= the screenshots after root https://drive.google.com/folderview?id=12rLjrW-6RX1wuD3BwZEXCmQgiK0cpq1m
[Sony Xperia XZ1 Compact ROMs, Kernels, Recoveries,] gouster3:
Can u please make secd patch for XZ2 (Akari) pie?

Thanx.
[Sony Xperia XZ1 Compact ROMs, Kernels, Recoveries,] OzzyFyshmaster:
Ah, gotcha. Thank you for that. :)

Yes, all three keys are provisioned. It appears to be a virgin phone. Again with the text link to the picture since my post count is now a whopping two:

imgur.com/a/3fpZn6f​

I blanked the IMEI & Keystore hash as it seemed prudent.

---------- Post added at 10:50 PM ---------- Previous post was at 10:03 PM ----------

Okay, I read the bit about the full-backup at the bottom of the OP, as well as the bit about how the auto-full-backup script isn't necessarily grabbing complete images. I'm not in any hurry so I can try to help make the most of it (https://xdaforums.com/showpost.php?p=78075727&postcount=249)... ;)

...So [MENTION=4418980]j4nn[/MENTION]... I have "backup-setup.bat" & adb ready unless there's something you'd like me to do prior to running it.
[Sony Xperia XZ1 Compact ROMs, Kernels, Recoveries,] j4nn:
Please grab few screenshots. Check if any updates come.

This is exactly the situation which I got with [MENTION=5677791]tramtrist[/MENTION] - in security screen it said

Rooting status NOT_ALLOWED

Security conf LIVE

and ok/active/provisioned everything (except the android attest key which was lost before backing up TA).

Then we got a screenshot with available updates.

I am wondering if they were only sony apps related or also google/system too.

You may need to be not on the latest fw though in order to be able to check for updates.

[MENTION=9546098]OzzyFyshmaster[/MENTION], you may try to follow the full backup howto and send persist.img with TA.img to me to check if it got truncated or not. Just do not unlock or upgrade/downgrade or anything. Are you on windows or linux?
[Sony Xperia XZ1 Compact ROMs, Kernels, Recoveries,] OzzyFyshmaster:
I have both windows & linux systems with adb available. I actually just setup the linux system for the express purpose of messing with the phone without the windows hindrance.
[Sony Xperia XZ1 Compact ROMs, Kernels, Recoveries,] j4nn:
Guys, please do not forget about those donations.

Or is it really that only 12 users so far found my tools useful?

While there were 260 downloads already?

You know I am getting now lots of PMs and answering those (and the public posts too) takes a lot of time.

I can do it only instead of my sleep time and still need to do my day job next day.

Already spent on this so massive amount of time that my wife is furious about it wanting me to stop immediately.

And I also feel pity spending more time on this than with my children.

It seems to me that only few of you really understand how difficult this was and how much time it could take.

You have the privilege to backup and restore TA with snapdragon 835 xperia mobiles. I am afraid this will not repeat anytime soon.

Thanks again to all of you who donated so far. And also to those who follow.
[Sony Xperia XZ1 Compact ROMs, Kernels, Recoveries,] arslancn:
screenshots after rooting here

https://drive.google.com/folderview?id=12rLjrW-6RX1wuD3BwZEXCmQgiK0cpq1m
[Sony Xperia XZ1 Compact ROMs, Kernels, Recoveries,] ParkourPaul:
Thanks so much for all the hard work j4nn, I've been waiting, hoping, praying someone would manage this. I'm currently travelling so it's pretty hard for me to help or even implement your work but as soon as I can, I will, and you'll definitely be seeing a donation from me.

[MENTION=4418980]j4nn[/MENTION] Can you not claim the bounty? Either way, if you've benefited from j4nn's work, don't be a leech, pay the man!
[Sony Xperia XZ1 Compact ROMs, Kernels, Recoveries,] tramtrist:
Yes everyone please donate something if youre using this. It’s the right thing to do
[Sony Xperia XZ1 Compact ROMs, Kernels, Recoveries,] russel5:
[MENTION=4418980]j4nn[/MENTION] after backup partitions from locked phone, simg file not open. Try on win and linux, all time

Failed to read sparse file
[Sony Xperia XZ1 Compact ROMs, Kernels, Recoveries,] goofnorf101:
Can someone help me with the downgrade flashing details for xz1c.

So I extract the G8441_47.1.A.2.324_CE1.rar file to a folder and then do I delete all the *.ta files and persist*.sin before flashing?

eg: Delete the files named below:

auto-boot.ta

cust-reset.ta

fota-reset.ta

master-reset.ta

osv-restriction.ta

persist_X-FLASH-ALL-C93B.sin

reset-kernel-cmd-debug.ta

reset-non-secure-adb.ta

reset-wipe-reason.ta

simlock.ta

?
[Sony Xperia XZ1 Compact ROMs, Kernels, Recoveries,] pbarrette:
The partitions on the phone aren't in sparse format.

Some are ext4, while others are raw data.

Still others are in a proprietary format defined by the Partition Type GUID in the partition table header.

You can extract the partition table SIN files and look at them with a hex editor to determine the GUID, but it's not entirely accurate.

For example, the "vendor" and "system" partitions are ext4 formatted, and their Partition Type GUID is: "11B0D797-DA54-3548-B3C4-917AD6E73D74".

But the "oem" partition is also ext4 and has a totally different Partition Type GUID.
[Sony Xperia XZ1 Compact ROMs, Kernels, Recoveries,] Unbounded:
look at this : http://www.xperiablog.net/2018/01/2...-firmware-for-xperia-xz-premium-xz1-and-more/

And you also need to delete Lilac_XBootConfig_MiscTA.ta from "boot" folder
[Sony Xperia XZ1 Compact ROMs, Kernels, Recoveries,] steso90:
Just donated and thank you very much for your time! :)
[Sony Xperia XZ1 Compact ROMs, Kernels, Recoveries,] j4nn:
[MENTION=4782500]pbarrette[/MENTION], you are right, partitions in the phone are not in sparse image format. But the backup-tools script sends partition content over usb via netcat in android sparse image format (I have included img2simg tool for that) in order to speed up the backup time and save backup footprint. Unfortunately there seems to be an issue possibly due to netcat syncing (terminating or not in the right time when the sending party terminates the transfer on end of file).

[MENTION=4688377]russel5[/MENTION], as mentioned also in the first post, the backup-tools are experimental and seem to truncate the files in some cases. I am afraid that is why your simg cannot be read - got exactly that from a user from the locked state and could not extract it myself.

In any case, there could be others who send it to me, but I promised everyone to keep it private, not share it anywhere.

I would send you mine, but my attest key was lost already, so there is no point.

I tried to restore just the content of keymaster64 directory, where stuff for attest key is stored and it made the attest key provisioned in the security screen.

But testing the key for real - and I've found a method to do so, see here please - the key is not working regardless what security screen says. I am afraid the stuff in persist is tied to phone's imei/sn/other id.

And not to forget - the key is not useful at all.

If it is lost, google may see the attestation is not hardware backed.

If it is not lost, it is useful only while the phone is locked and verified boot works. In that case it can provide cryptographic proof that the phone is not modified 100% stock to an external entity and that proof cannot be faked by anything changed in the phone. All the signing and info about verified boot and bootloader lock state comes from trust zone.

It would need complete control over TZ in order to fake the bootloader unlock status and let it be signed by the attest key in TZ.

[MENTION=4782500]pbarrette[/MENTION], thank you for your donation. It was very kind of you considering the exploit cannot help you as you had your phone unlocked before.

[MENTION=5670081]steso90[/MENTION], thank you too.

thanks for a very generous donation received just 30 minutes ago from a person who I cannot link to a xda nickname.
[Sony Xperia XZ1 Compact ROMs, Kernels, Recoveries,] tabbed:
:) (You might consider getting a btc/whatever address for those of us who can't use paypal, I had to go through a friend.)

Just take a look through this thread to see the crazy amount of effort that [MENTION=4418980]j4nn[/MENTION] put into this. If you're an adult who has gotten some value out of this, then you should return some of the value! The same goes with all of the other open source projects that we benefit from.
[Sony Xperia XZ1 Compact ROMs, Kernels, Recoveries,] SGH-i200:
Please don't take it personal. If you want to make sure everyone who downloads will pay you, you will have to let them pay before allowing them to download! :(

Your frustation reminded me of this story:
[Sony Xperia XZ1 Compact ROMs, Kernels, Recoveries,] j4nn:
[MENTION=1995555]SGH-i200[/MENTION], you know, that's not the case here - I am not expecting that everyone who downloads also donates.

You know, I could go the commercial way similar as that drm fix - but I do not like it.

I totally understand that not everyone can donate.

But when you can nearly count the donations on your fingers while seeing that those who donated had been guys who even helped with testing or tried to help during the development, it just does not feel right.

[MENTION=9499586]tabbed[/MENTION], thank you. You are right, probably might be worth it seeing what paypal takes (and will take with currency conversion and withdrawal) for the fees.

Please note, you do not need paypal account - that paypal's page allows credit/debit cards payments too without registering pp account.

Thanks again to all who donated or shall do.
[Sony Xperia XZ1 Compact ROMs, Kernels, Recoveries,] novebis:
[MENTION=4418980]j4nn[/MENTION], Incredible job !. I just received an XZ1 still in its unopened box. I do not know if I'll finally unlock bootlodaer but if I do I'll have to save TA, ... I do not want to lose functionality of the camera ... drm ...

I just made a donation to you right now, I know it is not much for what you have done and even if I am still going to use it (... I think so) but I appreciate the dedication and time you share, just Therefore, in my opinion, it deserves a reward

Greetings and thanks!.
[Sony Xperia XZ1 Compact ROMs, Kernels, Recoveries,] SGH-i200:
You should disable WLAN and insert no data SIM. Maybe you don't have to downgrade at all, because your ROM version is old enough.
[Sony Xperia XZ1 Compact ROMs, Kernels, Recoveries,] novebis:
Do you know if an usb ethernet adapter works on xz1 with current rom stock?
[Sony Xperia XZ1 Compact ROMs, Kernels, Recoveries,] SGH-i200:
I can test this next monday. But why you ask in response to my warning to cut all internet connections to stop automatic system updates, which would make the temp root impossible?
[Sony Xperia XZ1 Compact ROMs, Kernels, Recoveries,] novebis:
I ask it because if the rom stock has support for ethernet usb adapter I do not need root for now root. With ethernet in stock it would be a conforming time.

If on Monday you can try I appreciate it, I wait impatiently!

Thank you!
[Sony Xperia XZ1 Compact ROMs, Kernels, Recoveries,] j4nn:
[MENTION=800675]novebis[/MENTION], I am afraid usb network is not supported in stock kernel - this is from 47.1.A.16.20 fw version:

grep USB_NET_DRIVERS kernel.config
# CONFIG_USB_NET_DRIVERS is not set
[Sony Xperia XZ1 Compact ROMs, Kernels, Recoveries,] novebis:
what would I have to do to connect an ethernet usb adapter with XZ1, I understand that no stock rom is useful and I would have to modify the kernel?

Is there any way to modify the sotock kernel to activate the ethernet option?

sorry for my english I'm using translator
[Sony Xperia XZ1 Compact ROMs, Kernels, Recoveries,] jrbmed08:
Over the past few days, [MENTION=4418980]j4nn[/MENTION] and I have been exchanging PMs to try to solve my issue. After discovering that my camera was acting strange, I had upgraded my firmware and flashed a custom kernel with built-in DRM fix since I thought I had screwed something up. I ended up reflashing my firmware to get rid of the kernel and DRM fix I had flashed, downgrading again, re-running the exploit, and upgrading to the latest firmware once more. And my issue seems to have resolved itself. The problem was not actually that the camera didn't work...for whatever reason, it was still saving photos but not creating a thumbnail or adding them to the gallery after a reboot. But the pictures themselves were still on my internal storage or SD card. It seemed to work only intermittently but now is consistently working well. No idea what caused it, but glad to have a rooted phone with a working camera. Despite DRM fixes being available, I didn't want to unlock my bootloader until a true TA backup/restore became available. Happy to have full control over my device again!
[Sony Xperia XZ1 Compact ROMs, Kernels, Recoveries,] PeppeBove:
New firmware update... Android Pie 9 is come with 47.2.A.0.306!!! [emoji322]

Does Someone test this TA backup?

Ps:this is the right time i join the root
[Sony Xperia XZ1 Compact ROMs, Kernels, Recoveries,] Unbounded:
Just wanted to share quick observations about latest firmware "Customized SG ---> 47.2.A.0.306" that contains android pie.

Before flashing, I deleted all the .ta files and and also deleted the persist_X-FLASH-ALL and then flashed the firmware by newflasher.

Successfully booted, and Android Assets key is PROVISIONED while FIDO and SOMC keys are NOT PROVISIONED.

Camera is working Great, NO any green pictures and all other functions Except video image enhancing are working great.

After that I flashed the TWRP 3.2.3-0 followed by flashing of Magisk 17.1 followed by flashing of secd-ignore-unlock.zip

But, the security test results shows the same as about. (Android Assets key is PROVISIONED and other two keys are NOT PROVISIONED.)

But good things are that root is live, Safetynet Check is Passed Successful and all other things are working properly.
[Sony Xperia XZ1 Compact ROMs, Kernels, Recoveries,] OzzyFyshmaster:
I'm working on getting temp root with renoroot ...building on the suggestion to let the phone sit for a while to finish the boot tasks when it renoroot causes the reboot, I figured I'd throw it in airplane mode, disable everything, deactivate just about every app I can, as well as set background processes in dev options to none. It seems to be taking much longer between reboots, but haven't yet gotten root.

The ratio of Events/Overwrites seems to have gone up considerably (like 30000 events to 5 overwrites) since deactivating/turning everything off....question is...is that working against the goal, toward it, or hard to say?
[Sony Xperia XZ1 Compact ROMs, Kernels, Recoveries,] j4nn:
[MENTION=9540710]Unbounded[/MENTION], could you please check, what is bootloader version with this firmware?

And if it is still possible to downgrade to the exploitable firmware?

I see the fw is available in xperifirm for xz1c with Singapore customization (only).

I guess I can check bootloader version when download is finished -- edit -- there is bootloader_X_BOOT_MSM8998_LA2_0_P_104_X-FLASH-ALL-C93B.sin, so it is new bootloader there!

But if it is newer than the one in 16.20 firmware, it would be great, if you could test, if it is still possible to downgrade.

Thanks.

btw, it's great to hear, that everything works.

[MENTION=9546098]OzzyFyshmaster[/MENTION], it can be like that. Just keep trying (if your are running the exploitable fw).

Try to do as suggested in howto, let it calm down after boot for some minutes, use static background theme, no started apps, always on and so on...

If it goes for long with no Overwrite, like 5000 events, it may help to restart renoroot - just ctrl-c to break it and start it again.

Hope that helps...
[Sony Xperia XZ1 Compact ROMs, Kernels, Recoveries,] SXUsr:
While it would be nice if everyone that downloaded your hard graft would donate, it's important to remember that this community is more about helping for free than making a profit. Making a big deal about not getting paid doesn't paint you in a favourable light regardless of your contribution.

My theming has led to a few donations, and a Z3 Tablet Compact from Sony, which isn't much in the grand scheme of all things contributed, but over the years I've come to learn that it's the contribution that's most important, and not the reward.