yashsingh1/xdaforums huggingface dataset:
[Xperia Z5 Compact General] Cerhio:
I don't understand why you're talking about this. Of course, we understand that. I'm asking you because you're the one who posted this. Do you understand this?

OP, I think the language barrier is too much. I'm not sure how it is in Pakistan or Bangladesh but in Western culture, asking questions is not considered abusive. What you're doing is considered extremely abusive. You might want to spend more time learning English so you stop getting so angry at us for simply being people.

And yes, I want a thesis for each one. We'll wait.

---------- Post added at 10:14 AM ---------- Previous post was at 10:12 AM ----------

Oh and the part I was asking about refers to creating the bootable TA. You don't reference that anywhere. Maybe you need to learn to read?

Also saying that you're not angry is usually what children say when they actually are VERY ANGRY haha
[Xperia Z5 Compact General] :
One post edited for content.

There will be no more disrespectful posts made by anyone.

2.3 Flaming / Lack of respect: XDA is about sharing and this does not involve virtual yelling (flaming) or rudeness. Flaming or posting with a lack of respect is unacceptable. Treat new members in the manner in which you would like to have been treated when you were a new member. When dealing with any member, provide them with guidance, advice and instructions when you can, showing them respect and courtesy. Never post in a demanding, argumentative, disrespectful or self-righteous manner.

I'd like to remind everyone that there is an "Ignore User" feature that works perfectly. Once you add someone to your ignored list, you will not see their posts or PMs. I suggest some of you give it a try.

:good:

The Merovingian

Senior Moderator
[Xperia Z5 Compact General] templar1996:
I may did something wrong because initially the safetynet both are true and good to go. But after rebooting the phone, it went false on both.
[Xperia Z5 Compact General] :
Does magisk reduce battery life and performance like xposed?
[Xperia Z5 Compact General] NeoBeum:
Thats Sony RIC

The mount order and other tidbits for Lollipop are coming in to play.

This was one of the reasons why I started working on a Suzuran specific TWRP/ABootableRecovery.

The Rootkernel fstabs that are floating aroumd only work up to Marshmallow. Any attempt to use it without actually modifying the build folder, as opposed to cut and paste with rename, will work in it's half baked build until you try and do things.

If you didn't take you device down to FTF 32.2.A.0.200 to relock the Bootloader, and then straight to 32.4.A.0.160, any attempt to use Magisk Will fail.

The big bold red headers for the spoilers are crucial reference points the device must be at in order for the process to work.

The built in property resets are overridden by Google if you try upgrading to nougat from anything other than Lollipop with the Bootloader locked.

TWRP 3.1+ has nkk's reset prop functions if you know how to call them for boot or inside recovery.

I will need to quickly see if being online while first boot to Nougat is having an effect with the latest security update in July - but that is of lesser priority as I'm building a CustOEM Recovery and Windows installation partition for the ASUS ROG Community G751 users, and harassing vendors to get driver signing

I wasn't yelling at the other bloke btw, more just laughing at first, then after slamming my head on the table.

Edit: my 2nd device was testing how robust the magisk procedure and recovery can be with tinkering.

See if you notice something odd
[Xperia Z5 Compact General] NeoBeum:
Only if its not working and flooding the device with errors
[Xperia Z5 Compact General] NeoBeum:
I logged in to reply to someone in the XA section, but the problem is identical...
[Xperia Z5 Compact General] nlra:
Hmm. That actually makes some sense. I'll have to see if I can pick apart what's happening during boot and in what order.

You specifically mean any attempt to use Magisk > 13.0 on stock Xperia firmwares older than Nougat will fail, right? And/or possibly any attempt to use any Magisk version whatsoever on Nougat will fail? Because I can run Magisk 12 on Z5 stock Lollipop firmware (+ rootkernel'd boot image) no problem.

As I explained before, I started with 32.2.A.0.200 + locked bootloader, and then I left it there. I am not running Nougat. I am not even running Marshmallow at the moment. Magisk 13 does not work. Magisk 12 does.

The fingerprint string doesn't match the stated firmware version?

Anyway, it looks like Magisk 14 came out today. Perhaps I'll give that a spin and see what happens.

-- Nathan
[Xperia Z5 Compact General] templar1996:
I've noticed that if the safetyNet test fail, i can't open playstore. It would say that i have no connection. Its fine after a flash but after couple of hours, it stop working. Is it because of the SafetyNet?
[Xperia Z5 Compact General] :
How can i install magisk on stock rom?

it always gives me the error "please restore stock boot image"
[Xperia Z5 Compact General] NeoBeum:
Magisk Manager 5.3.0 and Magisk v14.0 is out.

Load for this is different to previous.

Use Magisk 5.1.1 In-App Auto Download and Install zip package v14.0

Take a Backup beforehand.

The intrinsic nature will make future ad-lib/on the go Root/CTS pass more complex.
[Xperia Z5 Compact General] :
Still getting the error after the update
[Xperia Z5 Compact General] NeoBeum:
Which error?

If everything fails, just roll back and stay on 13.3/5.1.1
[Xperia Z5 Compact General] nlra:
Magisk 14.0 initially failed for me in the same way that 13.3 did; HOWEVER...

...you were totally right about this.

I had only a few minutes to play with it this evening, so I started tearing it apart but only got so far (there are umpteen different init.*.rc files that forge a web of dependencies...I will have to figure out what needs to be re-ordered when I've got more time). However, I had recently built a kernel, so I still had a kernel source tree with all of the object files sitting there. So I decided to try and neuter RIC at the kernel level instead of at the initramfs level, which only took a couple of minutes. I packaged my RIC-less kernel up, ran it through Rootkernel for the rest of the mods, flashed it, installed Magisk 14.0, and it worked like a charm!

I now have a completely fully-functioning Magisk 14.0 installed on otherwise stock Sony Lollipop firmware for the Z5c, and I am passing SafetyNet checks now without issue (and Google Play Store says my device is now "certified").

It would of course be best to fix the Magisk installer so that it is able to modify a Sony kernel ramdisk in such a way that RIC is not an issue for it, but in the meantime, I'd imagine that this fix would work for just about anyone, so long as they are willing to take the time to build their own kernel for whatever Sony stock ROM version they want to use.

Thanks for the clue,

-- Nathan
[Xperia Z5 Compact General] NeoBeum:
both TWRP and Magisk are able to modify the RIC, but the Rootkernel process in this guide also disabled it, but it gets renabled for the recovery I built, for the older versions of magisk.

later on it included what TWRP had and it was just doing it all,

and until the latest versions, the scripts werent able to access the post boot or late start stuff to have anything available for us to actually be able to use, so depending on what recovery you use, mount and scripts all played a role in having it fail

I'm just generalizing the whole process here in explaining

once I finish my laptop stuff, I'll look over everything again, and start an install from Lollipop to Nougat.R1/R2.

Nougat is a dead project now, so the security patches should slow and firmware, which will let everyone be able to use some sort of global process for rooting and cts much easier
[Xperia Z5 Compact General] nlra:
Huh. Well I admit that I didn't try to tell Rootkernel to not disable RIC, and then seeing if the Magisk installer itself disables it in the rc files during its install, but your own screenshot of Rootkernel shows you answering 'Y' to the question of disabling RIC.

In any case, if RIC is already set to be disabled by something else (like Rootkernel) at some point in the init scripts, Magisk should properly detect that it's already been done and deal with it appropriately when it applies its own patches.

I diff'd all of the init.*.rc files from before and after Magisk install, and Magisk didn't touch the RIC stuff. Perhaps it did see that Rootkernel already took care of it and so left it alone. And best guess (which I assumed already) is that either RIC disable needs to happen earlier, or Magisk needs to start itself later (after whatever point in boot RIC gets disabled).

My solution was just to get rid of RIC in the kernel entirely...then it doesn't matter if boot scripts toggle the right debugfs bit at exactly the right moment. :)

-- Nathan
[Xperia Z5 Compact General] NeoBeum:
yeh, thats what I was saying

but, magisk can enable and disable RIC too now.

it never did early on, you had to set it yourself

the ric stuff happens for bootloader stuff.

thats when the device detects the colour states, green, orange, red, for tampering, and its when you can also use the scripts for reset prop
[Xperia Z5 Compact General] NeoBeum:
how many apps are you trying to use magisk hide with?

If you try and hide crypto, data stops working
[Xperia Z5 Compact General] NeoBeum:
this is the stock rom
[Xperia Z5 Compact General] :
Then how can i fix the error?

is it because i installed a custom kernel?

where can i get the stock boot.img then?
[Xperia Z5 Compact General] NeoBeum:
por que¿
[Xperia Z5 Compact General] :
just read my old post

My problem is that i'm getting the error "boot image patched by other programs! please restore stock boot image"
[Xperia Z5 Compact General] The_XXI:
Change the colors in your thread to all black ffs. Or something with very high contrast, the selection in XDA is orange, so even if I want a high contrast, I can't with the colors you chose.

Think about the end user. Not your "creativity".
[Xperia Z5 Compact General] krokirkok:
did as written in guid - rom ok

flashed Magisk v14 in recovery - no errors, but magisk manager shows "magisk not installed"

any suggestions?
[Xperia Z5 Compact General] NeoBeum:
Got around to testing Device and SD Card encryption. All OK.
[Xperia Z5 Compact General] NeoBeum:
Grab the 14.0 zip that MManager downloads and then Install it in Recovery
[Xperia Z5 Compact General] NeoBeum:
there's a global dark theme
[Xperia Z5 Compact General] firdyRAY:
Hahahaha you're savage. Anyway i'll keep a look out on this thread. Seems promising. Would love to have it simplified when you have finalised your end product.

Looking forward to try the "stable" ROM from you. Good luck on your first project.
[Xperia Z5 Compact General] NeoBeum:
mmm at least some gets my humour

dont expect much, z5c are legacy, so there's no rush

and I'm currently busy ripping in to Microsoft right now because their API containers for MSFT_Disks are as gumby as the guy in charge of the Department.

I might tweet Donny and see if he can fire him

I did next to no programming for this project so far, I'm just an assembly and debug monkey
[Xperia Z5 Compact General] Janzomaster:
Hi, thanks for your work, just two questions:

- is your twrp version really fully functional with regards to decryption?

- is going back to 5.1.1 and locking the bootloader necessary for devices already rooted? If so, why?
[Xperia Z5 Compact General] NeoBeum:
Recommend to not use encryption on device as decryption to mount may fail at recovery - unsure of cause : possible magisk, or ric interference- temporary work around is to run command while device is turned on, decrypted, and transfer the data directory to an unencrypted sd to regain a backup source, wipe data in Recovery, and transfer

/dev/block/dm-0 (this is where the decrypt system data is located when device is turned on)

/storage/NAMEOFYOURSDCARD/someTempFolderName
[Xperia Z5 Compact General] NeoBeum:
Hi

I have been testing all week, but it failed this morning to decrypt. But i don't know if it's because my test device had a 32.2.A.2.33 fingerprint when it should be 32.4.A.0.160 - I forced an different bootloader to test how robust magisk was last month. So i don't know for sure.

Going back to 5.1 is less of an issue than 6 to 7, or 7 to 7.1

If the device had been rooted previously then this process may or may not work just tagging on at different stages. Going back and locking and restoring keys guarantees a factory build - and catching that with magisk is better than capturing a previously modified boot and device signature, this recovery wont work with 5.1, unless you change the fstab, and link some oem stuff

You'll be able to tell because when you check the Security Test, there'll be a Master Reset TA call that failed. You can overwrite the message after and make it what ever you want,. You can change it back to 'No reason given'... I was bopping around with 'i fkd up so hard' when i was on marshmallow
[Xperia Z5 Compact General] Janzomaster:
So I followed your guide and now have 32.4.A.0160 running with Magisk working. But: the safetynet-Check doesn't go through. Both ctsProfile and basicIntegrity come back "false".

Here's what I did: flashed 32.0.A.6200, restored my devicekey using iovyroot, flashed 32.4.A.160, rebooted to enable oem-unlocking and usb-debugging, restarted to bootloader, unlocked the booloader using the unlockkey from sony, flashed the kernel.elf from 32.4.A160 patched with rootkernel (I answered the questions according to the pic in your guide in the first post), flashed CRC81D43A45-recovery-v1.1.img as recovery, flashed my devicekey and then installed magisk (from here: https://xdaforums.com/apps/magisk/official-magisk-v7-universal-systemless-t3473445) in TWRP.

Where did I go wrong?
[Xperia Z5 Compact General] NeoBeum:
try the procedure with magisk 4.3 v12 and then upgrade it to 5.1 v14
[Xperia Z5 Compact General] kotouc13:
Just wanted to report another successful user (so-far), thanks! <I only own the phone for a few days, so I hope I wont have any undetected battery / overheating problems>

The only glitch is that I booted after unlocking, instead of directly flashing boot, recovery & DA (so i did at lest flashed 7.1.1 & the rest again to be sure). Otherwise, the guide is somewhat "rough" but together with other sources I was able to finish ;)

I have few questions though:

<maybe a noob one> You mentioned Magisk controling the RIC settings -> is it possible to have /system read-only be default, for improved security?

Are there any known bad consequences of using MyXperia on similiary unlocked/"fake-locked" phones, or is it safe to use --- beside the fact, that with bootlocker unlocked && recovery, it does not really prevent thief from wiping the phone...

Is the qualcomm brightness hiding still up-to-date?

Thanks once more!
[Xperia Z5 Compact General] sheraro:
[MENTION=8098250]NeoBeum[/MENTION] Sorry but I think this is the most confusing tutorial I ever saw on xda ! please make it more simple with clear numbered steps in the beginning of the 1st post and keep all informative texts + old versions outside it, I'm hopping to install magisk (especially with the release of xposed for Nougat) but havent found a clear way to do so ...
[Xperia Z5 Compact General] Cerhio:
Please message me if you want how I went about doing it. It's a little confusing but it does work. I have yet to install Magisk but I will do that and install xposed tomorrow morning. Will report how it goes.
[Xperia Z5 Compact General] NeoBeum:
32.4.A.1.54 Update -

Don't upgrade - Recovery partitions mismatched... I might make a script for the next build to make it backward compatible - but for those of you who can't wait -

change "userdata" to "data" in fstab, and you should be able to limp along until i get a proper build going.
[Xperia Z5 Compact General] NeoBeum:
Recovery Build v2.1.54 is complete for FTF 32.4.A.1.54 - currently untested by me, but should be ok - I'll be testing 2 versions, there'll be another coming up in a few days - it's related to Device Encryption & Location for Blocks. This version is basically the updated recovery build, with all the same settings, but correct partitions. The next release will test the Device Encryption Location, from /dev/block/data (what it is currently), to /dev/block/dm-0

I'll be testing Flashing and Installing of Magisk with both of my devices and using different methods to test flexibility in capturing the boot image and hash.

Got other things to do currently and my laptop is still in bits and pieces - this was just done to get something out of the way and working.

There's new partitions you should be able to see in Recovery, in addition to the extra ones I had originally.

The 'Misc' partition from Lollipop is back, and FOTAKernel has been renamed to 'recovery'. Everything else is labelled overthetop style.

Back everything up. Once you get a good recovery and successful restore, then you can skimp out and just backup the 'userdata' partition which is now called 'data'..

Expect another update if Sony are nice, and decide to patch the Bluetooth security flaws... While I was building, the bluetooth directories compiled with no errors - and I was getting most faults with QCOM stuff because I had quad and tri merges happening (just the way I set up my mirrors)- but it means that Bluetooth wasn't really touched . CVE-2017-0781 ... CVE-2017-0785.

I could be wrong, and patches could have been included.

CRCD0FF4662-recovery-v2.1.54.img is attached to the post.
[Xperia Z5 Compact General] NeoBeum:
This was Flash FTF 32.4.A.1.54 - Wipe all, exclude FOTA Kernel Partition Wipe:

Installing Magisk.zip in Recovery without disabling Sony RIC or DM Verity will cause the device to become unstable.

Currently - it's looking like a similar process to lock boot loader, for clean install, and patch of boot image to disable RIC and Verity.

I have access to all functions in TWRP and have done a successful restore and backup.
[Xperia Z5 Compact General] NeoBeum:
Done. If you can't get it working. Google crap.
[Xperia Z5 Compact General] NeoBeum:
you can turn verity and ric back on once you get magisk installed and running. you only need to disable it to get magisk on to the sony devices.

you can actually recover your ta device keys by using rootkernel, I did it this morning on my test device.

the qcom adaptive brightness is a side effect of a Google bug for home screen UI, LockScreen and Sleep, Wake and Deep Sleep.

Nova Launcher can manifest the bug in a different way.

when the device sleeps, battery optimisation kicks in, if the system processes linked with qcom, Google gms or any HAL get cleared from cache, it takes a refresh to get them started again. if you turn the screen off/lock then unlock, problem goes away, or you will see adaptive brightness lag,and display will go bright, then dim.

the sensors are affected too.

somewhere in the garbage, you'll see where I mentioned something about global readwrite that MMan.v13 added, and how it causes some funky stuff to happen, and how to revert it. MMan.v14 leaves the blocks alone, so its functioning much better. it also handles the injection better than earlier versions, which is why going from 7.0.0 to 7.1 would trip root access and cts, and how I extracted the apk first to install instead of zip install, until the last update, and going from 7.1.1(x.160) to 7.1.1(x.54) is so breezy in comparison

all the security stuff is just a time delay deterrent and hindrance for theft.. you only really have a small window to react to be able to render your device unusable. thieves with IT skills mitigates the buffers for security
[Xperia Z5 Compact General] NeoBeum:
NB:14OCT17 - I will be rebuilding the base Recovery Image, as it was using a mix of 7.1.2 for Omnirom and TWRP, and 7.1.1 r17, while I'm still technically ahead of the Sony AOSP, they are now using 7.1.1 r55

It might make debugging Encrypt/Decrypt, Disk/File - a whole lot easier
[Xperia Z5 Compact General] sheraro:
is there a way to do it using magisk or must we reflash a new fresh boot.img (with only drm verify and no verity and ric) ? if we do the later method doesn't it conflict with magisk fonctionnalities ?? I'm asking before flashing everything
[Xperia Z5 Compact General] NeoBeum:
add something like

false 1 > sys/security/sony_ric/enable

to magisk scripts, post-fs should do it, but too early and you'll get lockout, like how you can't install with it.. it'll just be the sony logo

edit: the property file location is,

sys/kernel/security/sony_ric/enable
[Xperia Z5 Compact General] militiaOfDeath:
For your boot.img when using rootkernel, do you add DRM-fix and e.g. busybox, or is the only modification to disable Sony RIC and dm-verity? Asking because I have problems booting back into Android after flashing Magisk in TWRP, so wondering if it doesn't like that I've added DRM-fix and busybox.

edit: flashed your recovery and then it worked right away, so probably something wrong with the TWRP 3.1.1 img I downloaded in another thread.

edit2: have you been able to install systemless Xposed within Magisk Manager and boot Android? When I install it I don't get back into the system again, so something there that's not so happy about Xposed it seems.

edit3: giving up getting Xposed on v7.1.x for now, going back to v6.x that's working until I see people are getting Xposed to work properly.
[Xperia Z5 Compact General] sheraro:
32.4.A.1.54 here, I have xposed systemless api25 installed with magisk manager 5.4 and Magisk 14 with working gravitybox and adaway, the only options I have in rootkernel :

disable ric

disable dm-verity

add drm-fix

make sure You have the correct files flashed (sometimes one can use the wrong kernel in rootkernel, wrong recovery(32.4.A.1.54), wrong ... )

---------- Post added at 10:03 AM ---------- Previous post was at 09:59 AM ----------

is this modification safe for safetynet ?
[Xperia Z5 Compact General] militiaOfDeath:
Thanks for your reply.

So you flash recovery afterwards with fastboot ? And which version of TWRP do you use? I tried both the one rootkernel includes (3.0.2?) and 3.1.1 by manual flashing, but no luck. But I've included busybox in the boot.img every time, so maybe try without that. Since I can't get the fingerprint sensor to work again after restoring my NANDroid backup I can probably give the 7.x another shot.
[Xperia Z5 Compact General] sheraro:
flash CRCD0FF4662-recovery-v2.1.54.img with fastboot (after enabling adb usb in settings) , also verify its cheksum : D0FF4662 before flashing

did you manage to instal other modules succesfully in magisk ? if yes then verify you selected xposed api 25

another point is do you have magisk manager 5.4 or 5.3, because this recovery comes with 5.3, after booting magisk tells you to update its manager, I also suggest to reboot before installing first module

and yes try without busybox because magisk comes with it bundled + there's a module for other apps to use
[Xperia Z5 Compact General] militiaOfDeath:
Will give it a try when I get back home, would really like to be on 7.x, but since I've gotten so used to having Xposed now, running a ROM without it doesn't makes sense to me. Anyways, thanks again for your reply.
[Xperia Z5 Compact General] Mael_RadecFR:
Hi, I want to downgrade to Android 5.1.1 (for the TA backup), I downloaded the .tft file and put it in /flashtool/firmware, but the file is not detected by Flashtool. Any help please?
[Xperia Z5 Compact General] Chollo:
Hi. sorry for my bad English.

My Z5C (5823) has compilación 32.2.A.5.11 (6.0.1), Kernel unlocked, ROOT + TWRP, and I have copy of the TA partition.

What steps should I take to upgrade to 7.1.2 + Root + TWRP.

Thank you very much for that great work.

Regards
[Xperia Z5 Compact General] wihushae:
from where did you get the 5.1.1 ftf? i cant find one :(
[Xperia Z5 Compact General] Mael_RadecFR:
Here it is http://xperiarom.com/android-lollipop-5-1-1-rom-firmware-sony-xperia-z5-compact-e5823/

Btw I gave up with DRM keys, I unlocked the bootloader and went to LineageOS so I don't care.