yashsingh1/xdaforums huggingface dataset:
[Samsung Galaxy Nexus] Ziyan:
It seems like there's some misinformation going on around here. To make things a bit clearer, I'd like to mention first that the hardware encryption/decryption support for OMAP4, so for our device, is completely open source. Secondly, I think the issue is within TWRP. It's designed to support QCOM hardware encryption/decryption; I believe it was never tested on non-QCOM hardware. My WIP commit (I prebuilt the binaries as it's such a hassle to build them automatically for TWRP, but all sources can be found in device/samsung/tuna and hardware/ti/omap4 on any 6.0+ ROM) tries adding OMAP4 hardware encryption/decryption support for TWRP; this is what I used for my build in post #2014.

So yeah, I'm pretty busy nowadays (I'm watching XDA and all, and talk a lot - like every day - with Android-Andi, who's kinda my right hand nowadays...), so personally, I'm not sure when I can take a look at what we are missing to make this completely stable. Until then, there's that, with the random reboot issue, and all source code, waiting for someone to fix it... :)
[Samsung Galaxy Nexus] hga89:
Just tried it, after entering the password it says that it can decrypt the data partition but it gets stuck on "updating partitions" and then spontaneously reboots.
[Samsung Galaxy Nexus] magur0:
I also gave a couple of shots. (~10)

And it can decrypt successfully and also updating the partition details sometimes. But most of the time it's just rebooting randomly. Once I was able to see the mounted /data with the file manager in TWRP and it looked plausible. When I tried to back it all up to usb-otg it rebooted :(

EDIT: Tried again and reached the point to do a backup (this time I did without the data partition!) and it also failed with a reboot...

EDIT2: Just to compare with TWRP 3.1.0-0: Beside that I can't decrypt/mount data the backup without data works here without issues...

This is really the last missing peace in the puzzle... If this would work stable even the nightly OTA's will work in my understatement... And we can do backups (the most important!) :good:

Anyway it's wonderful how the community is participating in giving this old phone a second and third and maybe many more lives!! :fingers-crossed:
[Samsung Galaxy Nexus] bamtan2:
*swoon* ;)
[Samsung Galaxy Nexus] magur0:
No, seriously

! If the problem with decryption would be solved in TWRP this old device will be fresher than never and two of the most important things should then work:

-> OTA Updates with the in build updater (fast and easy security/feature updates!) :cowboy:

-> Finally to be able to do nandroid backups of the whole phone (live a life without fear!) :angel:

Even more that I now know that the hardware encryption in the galaxy nexus is FULLY OPEN SOURCE :victory:

I just had a long and painfull week recovering my phone from a bootloop. Is was much harder because of missing decryption in TWRP: Once because I didn't had and couldn't do a backup - so also fear of loosing everything and second I needed change a file name in the encryption data partition (would have been easy in TWRP when it could decrypt). For the second I got a lot's of help from pro's around (special thanks' to [MENTION=6979267]nailyk[/MENTION] at this point - was a long week, thank's for your countless hours! merci beaucoup!) which customly created boot.img's for LOS and much more...

I really hope that we - one day - can enjoy in peace and harmony with a full working twrp for galaxy nexus! It's worth! :highfive:
[Samsung Galaxy Nexus] Android-Andi:
[MENTION=6979267]nailyk[/MENTION] https://gerrit.omnirom.org/#/c/24760/

Updated Ziyans commit, not tested if it works that way, not sure when I'll find time to test myself.
[Samsung Galaxy Nexus] nailyk:
Thanks :)

Am currently working on it: https://gerrit.nailyk.fr/#/q/projec...maguro+OR+project:android_device_samsung_tuna

Just waiting for build end because right now my device is 14.1
[Samsung Galaxy Nexus] nailyk:
[MENTION=8463548]magur0[/MENTION] Am not able to contact you with PM too... Could you join one of the IRC chan we used before please?
[Samsung Galaxy Nexus] nailyk:
twrp test 2017-08-13

test twrp

Right now am able to decrypt with all version I tried, so probably nothing changed.

Please post last_kmsg after decrypt fail:

adb reboot bootloader
fastboot boot test_twrp.img
try to decrypt, let the device reboot. While into lineage,
adb pull /proc/last_kmsg
[Samsung Galaxy Nexus] _yen_:
hello [MENTION=6979267]nailyk[/MENTION], the second /data entry (f2fs) you add in twrp.fstab makes /data mounted (and showed) twice in recovery.

As I'm not encrypted I can't test further.

bye

yen
[Samsung Galaxy Nexus] magur0:
Sorry, I'm out in the wild right now! Will be back in civilization in the middle of week! I will give feedback then.
[Samsung Galaxy Nexus] Android-Andi:
[MENTION=8463548]magur0[/MENTION] [MENTION=6979267]nailyk[/MENTION] https://gerrit.omnirom.org/#/c/24760/

I have updated [MENTION=2633638]Ziyan[/MENTION] s previous work a littlebit:

use a custombootimg.mk to move the previous prebuilt libs people complained about sourcecompiled inside the recovery ramdisk

This needs TARGET_RECOVERY_DEVICE_MODULES defines!

all recovery stuff is moved to recovery/root incl. the twrp.fstab.

Note: By defining TARGET_RECOVERY_DEVICE_DIRS it won't remove twrp.fstab out of our recovery ramdisk on device specific builds - twrp.fstab don't needs to be in our bootimage ;)

to make sure our recovery.img doesn't have problems because it's oversized (shouldn't, but who knows why there's the reboots) i compiled a test using LZMA compression https://gerrit.omnirom.org/#/c/24806/ (on Android 7 source define "LZMA_RAMDISK_TARGETS := recovery" instead)

Sidenote [MENTION=6979267]nailyk[/MENTION] you can contact me on hangouts ;) i am sure you find out how to find me :)
[Samsung Galaxy Nexus] Android-Andi:
Found some minutes to try it... Mh... Looks like there's still something missing. The libs all exist in recovery ramdisk but they can't be executed... Haven't had enough time to look closer, next days I'll give it another try. The prebuild libs are bigger...

<3>[*** 7.822753] init: cannot execve('/sbin/smc_pa_ctrl'): No such file or directory

<5>[*** 7.824401] init: Service 'smc_pa_wvdrm' (pid 118) exited with status 127

<3>[** 82.213623] init: cannot execve('/sbin/tf_daemon'): No such file or directory

<5>[** 82.213958] init: Service 'tf_daemon' (pid 152) exited with status 127

<5>[** 82.214050] init: Service 'tf_daemon' (pid 152) killing any children in process group
[Samsung Galaxy Nexus] _yen_:
Providing the fact that I'm not encrypted I got this in my test builds which are UA 7 based:

<5>[ 5.041381] init: Service 'smc_pa_wvdrm' (pid 118) exited with status 0
<5>[ 49.905792] init: Starting service 'tf_daemon'...

so it seems that at least init can execute these binaries

I'm using additional sepolicy from [MENTION=6979267]nailyk[/MENTION] plus this commit to go permissive due to numbers of remaining denials I have:

https://gerrit.omnirom.org/#/c/23519/

for what i can see main difference from your approach is also that i use relink_source_files to add all additional files in ramdisk.

Hope this can help... also, some idea how to test this (at least partially) not being encrypted ?

bye

yen
[Samsung Galaxy Nexus] Android-Andi:
TWRP runs in permissive, to be sure I used a permissive kernel too.

I am compiling on Omni6 base using twrp from Omni7.

Maybe it's a BusyBox thing or something ignored on "make recoveryimage" vs recovery.img from rom compile. Most roms now use toybox instead busybox.

I'll investigate further. Our goal should be to not use prebuilt modules and libs if possible, even if source is open.
[Samsung Galaxy Nexus] _yen_:
I rebuild UA with omnirom busybox instead of toybox before building a recovery so it should not.

bye

yen
[Samsung Galaxy Nexus] nailyk:
[MENTION=5153662]Android-Andi[/MENTION] yes probably but I don't have hangouts ;) You know, google, etc... :p

About that cannot exec it is probably because you don't set context to those files.

Yes smc_pa got exited right after the start, that's ok. This daemon is only here to initate the /tee partition and some sockets,

then tf_daemon start and this one should keep running (some times it start twice).

Am building from 14.1 (so with toybox) and yesterday I was able to decrypt, backup, decrypt restore, multiples times.

Here is the (unsecured) recovery if can give a try.

Warning, ring buffer had been resized from 64k to 2MB. It can cause some issues if you try to read last_kmsg from a kernel with smaller buffer size.

All the patchs to build this recovery are here. (i already merged them into 13.0 branch).

Maybe you didn't subscribe 14.1 thread but thoses patches allow 14.1 to get "HW" encryption too and allow me to upgrade the 13.0 to 14.1 on the encrypted maguro.
[Samsung Galaxy Nexus] bamtan2:
Great work everybody!!! Especially nailyk the "new guy"!!! :good: :) :good:
[Samsung Galaxy Nexus] Android-Andi:
recovery.img for maguro attached. works now to decrypt. :)
[Samsung Galaxy Nexus] nailyk:
Great! What are the changes?
[Samsung Galaxy Nexus] Android-Andi:
Just cleaning, I'll push to Omni Gerrit later
[Samsung Galaxy Nexus] Android-Andi:
Another one, maguro isn't different, only some parts now moved to tuna tree which doesn't make a difference here.

https://gerrit.omnirom.org/#/c/24760 (some parts can be moved, discussing now where to place it, else https://gerrit.unlegacy-android.org/2963 is fine too instead the changes in recovery/Android.mk)

https://gerrit.omnirom.org/#/c/24806

https://gerrit.omnirom.org/#/c/24835

Kernel set to permissive,

added CONFIG_RD_LZMA=y in kernel defconfig.
[Samsung Galaxy Nexus] nailyk:
Many thanks! For the comment on gerrit.nailyk too.

Will try to pick your changes this week-end. I need to do some omni work. If I have time I will try to port [MENTION=4695183]zeelog[/MENTION] devices trees on omni too.
[Samsung Galaxy Nexus] Android-Andi:
[MENTION=6979267]nailyk[/MENTION] you are welcome. I started working on Omni7 already, maybe you should wait a little bit before same work is done twice ;)
[Samsung Galaxy Nexus] nailyk:
I will open android-7.1 on gerrit.nailyk right now then. Should be ready in one hour.
[Samsung Galaxy Nexus] Android-Andi:
[MENTION=6979267]nailyk[/MENTION] we can use official Omni Gerrit once I created the branches on Omni
[Samsung Galaxy Nexus] nailyk:
Awesome! Up to you :) branches opened: https://gerrit.nailyk.fr/#/admin/projects/android_kernel_samsung_tuna,branches

Edit: [MENTION=5153662]Android-Andi[/MENTION] vendor branch opened too based on DC 7.1.
[Samsung Galaxy Nexus] magur0:
I tried the "twrp-3.1.1-0-maguro-decrypt-lzma.img" for my maguro and it worked out of the box! :victory:
[Samsung Galaxy Nexus] Android-Andi:
[MENTION=8463548]magur0[/MENTION] please let me know if there's still the reboots from previous version from Ziyan. I can bring it to official TWRP if it works fine.
[Samsung Galaxy Nexus] magur0:
I tried via fastboot boot twrp.img ->

works perfect

Now I installed the recovery via fastboot ->

still works perfectly

I decrypted a couple of times (>5) always worked,

no reboot, no hangs, no troubles

&

data was always mounted

.

I'm going to play little with backing up & restoring and will come back here to report.

GREAT WORK!!! Thank's a lot already (all contributors included)
[Samsung Galaxy Nexus] bamtan2:
That sounds like a confirmation to me! I think some people deserve some donations! :highfive:
[Samsung Galaxy Nexus] Android-Andi:
We are doing this for fun :) as long as you don't act like some user on tab2 everything is fine ;) had to close my LineageOS13 thread 2 days ago because one guy thought XDA or user paying my income every month so I have to publish N builds and stop working on updates for older Android versions [emoji23]

No, user and XDA don't pay my income (but sure, donations accepted if people like to donate) and I do this just for fun and release stuff I like to share or keep it until I think it's ready.

Working together with many people involved makes a lot of fun, finding solutions for issues and improve stuff :)
[Samsung Galaxy Nexus] hga89:
I installed it on my GN and it decrypted the data partition successfully. Thank you!

However, I was not able to use the automatic feature to update Lineage, I had the nightly from July 15th installed and updated to August 12th. I tried twice but TWRP on both tries bypassed the update step and continued to boot into the OS. After two tries I updated the OS manually from within TWRP.

Not sure what could cause the update step to fail?
[Samsung Galaxy Nexus] nailyk:
Weird, both have the /misc partition setup twrp :

/misc	emmc	/dev/block/platform/omap/omap_hsmmc.0/by-name/misc

LineageOS:

/dev/block/platform/omap/omap_hsmmc.0/by-name/misc /misc emmc defaults defaults

I already noticed some bugs into twrp when old & new ways are used at the same time. Can you try upgrading after wiping /cache for the next time please?
[Samsung Galaxy Nexus] nailyk:
Removed.

Am stupid.
[Samsung Galaxy Nexus] Android-Andi:
Yupp, need to move the decryption stuff on Gerrit from 6.0 to 7.1, guess your script only picked the open 7.1 commits on tuna
[Samsung Galaxy Nexus] magur0:
Some more testing and findings with twrp-3.1.1-0-maguro-decrypt-lzma.img from [MENTION=5153662]Android-Andi[/MENTION]:

Booted TWRP: Backup of installed

CM-13

incl. hw encrypted data (System, Cache, Data, Boot, EFS) -> Saved Backup to computer

Booted TWRP: Dirty flashed RR-N via adb sideload

Booted

RR-N*

: Encryption data broken after first boot (Boot dialog says the encryption-pass is right but that the files are destroyed -> System prompts for factory reset -> I dismissed and rebooted to recovery)

Booted back in TWRP -> still able to decrypt files with right pass-> formated /data for further testing

Booted

RR-N

-> works as it should

Booted TWRP -> Restore Backup of

CM-13

incl. encrypted data (System, Cache, Data, Boot, EFS)

Booted

CM-13

-> works but I'm not sure about the encrypted data because the file system is without encryption (I think TWRP backs up the data unencrypted); But Apps+data is present

Booted TWRP -> Restoring a Backup of

LOS-13

from another maguro (System, Cache, Boot)

Booted

LOS-13

-> it's booting and preparing >100 Apps right now.... -> FAILED: System UI has stopped. I try to create a new backup to restore..

*RR-N: ResurrectionRemix Nougat - RR-N-v5.8.4

*LOS-13: [ROM][6.0.1][OFFICIAL] LineageOS 13.0

I never did this intensive backup and restoring because it never worked with my encrypted systems! I think everything works like it should. Somehow I realize that I don't know where the things are stored excactly. Can somebody tell me? Why there is apps in the Backup of

LOS-13

which is just System,Cache&Boot? also TWRP is mentioning when I backup /data that it doesn't backup (exclude) the internal storage??? This should be the area for Downloads,DCIM and everything - right?
[Samsung Galaxy Nexus] Android-Andi:
[MENTION=8463548]magur0[/MENTION] we have a data/media device:

https://twrp.me/faq/datamedia.html

https://twrp.me/faq/backupexclusions.html
[Samsung Galaxy Nexus] bamtan2:
Yeah, it doesn't back up ALL of /data. It backs up everything EXCEPT the virtual sdcard. And it's confusing because sometimes "wipe data" doesn't wipe /data! It will leave the virtual sdcard. But if you go to the partition manager and REALLY wipe /data, then you'd lose everything.
[Samsung Galaxy Nexus] magur0:
Thank's! This gave clarification!

So from the technical part TWRP did everything right! :good: I will try to recover the full backup from the other device (LOS13) after a full wipe and check before restoring if the virtual sdcard is realy gone.
[Samsung Galaxy Nexus] Android-Andi:
[MENTION=8463548]magur0[/MENTION] I have added a note about it here http://andi34.github.io/recoveries_tuna.html since most people seem to not know.
[Samsung Galaxy Nexus] magur0:
And the ultimate proof that this is the one and only recovery for our phones comes here: (using twrp-3.1.1-0-maguro-decrypt-lzma.img)

Booting the new TWRP on a LOS-13 encrypted Galaxy Nexus: decrypt data & do the full monty backup

Booting the new TWRP on a second Galaxy Nexus an do the full monty restore with the newly created backup from the other device

Resulting in having a cloned device. Only difference is that the encryption is gone which can be easily activated again!

Very very nice! I think these images/source-code is ready to go upstream! :good:

Thank's a lot again to all contributors: [MENTION=5153662]Android-Andi[/MENTION] [MENTION=6979267]nailyk[/MENTION] [MENTION=2633638]Ziyan[/MENTION] and all forgotten ones! :fingers-crossed:
[Samsung Galaxy Nexus] efrant:
Seriously?

It is great for backup/restore, but for maximum compatibility with edify scripts, CWM NON-touch is what wins. CWM touch doesn't even cut it.
[Samsung Galaxy Nexus] Vangelis13:
Yep, seriously. I've used it without any issue whatsoever for 5 months.

Flashed 400+ (literally) roms, kernels, radios, mods. No issues, nothing, zit.
[Samsung Galaxy Nexus] efrant:
So because you've had no issues means that it is the most compatible?

Have you flashed a few stock updates on a few stock ROMs using TWRP?
[Samsung Galaxy Nexus] Vangelis13:
Define a few. Define stock roms. In that 400+ of the last few months, certainly.

Mate, I'm an end user. TWRP works, really well. It's not like the thread is flooded with incompatibility issues either here.

You like / find cwm better / more compatible with what you're using it for. Ok, cool.

No offence. :thumbup:
[Samsung Galaxy Nexus] efrant:
No offense taken. :)

The only reason I was asking about stock ROM updates -- and by stock ROMs I mean ROMs that are available on Google's factory image page, and by stock update, I mean the updates available in the "update" link in my signature -- is that those updates have pretty much most edify script commands included, so if you've flashed those updates onto stock ROMs and have them apply without any issues, then sure, I'll agree with you -- twrp would be better.
[Samsung Galaxy Nexus] reference.phone:
Indeed, TWRP backup/restore super fast! But vrtheme flasher does not work with Jelly Bean unlike CWM
[Samsung Galaxy Nexus] codesplice:
After mucking about with a few different kernels on my otherwise stock Jelly Bean, I wanted to go back to stock. I flashed using TWRP in a single go (all added into the same zip queue) without any issues whatsoever:

jb-takju.zip (a CWM-package of the 4.1 IO JRN84D build, repackaged by me to remove /system/etc/install-recovery.sh)

edfff6d328f1.signed-takju-JRO03C-from-JRN84D.edfff6d3.zip (official 4.1.1 OTA)

JB-SuperSU.zip

cerberus_2.0-ICS.zip

TitaniumBackup-5.3.1.zip

I haven't been flashing a lot of mods lately, but I have flashed many many many different ROM and kernel combinations and I don't recall any that required me to regress (as it truly is a regression) back to CWM. TWRP has been rock-solid dependable for me.
[Samsung Galaxy Nexus] FredFS456:
Which version of TWRP have you been using? 2.2.0?
[Samsung Galaxy Nexus] codesplice:
Yep.

Though I might add that I did manually flash the LC03 bootloader (packaged as a CWM-package) via TWRP prior to installing the updates the first time around. Not sure if that matters, but full disclosure and all that.
[Samsung Galaxy Nexus] kobik77:
TWRP is just great.

Personally I can't understand why people do insist on using inferior cwm instead.

TWRP has full touch buttons support.

Batch install.

GooManager support.

Encrypted data support.

File manager

Keyboard!

Terminal!

EFS backup.

And many more.
[Samsung Galaxy Nexus] efrant:
I don't think anyone would disagree about features - twrp has many more. What some people are suggesting is compatibility, NOT features...
[Samsung Galaxy Nexus] tatltael:
Is TWRP 2.2 as compatible as CWM for flashing mods n roms?

I've only used the older version and while it worked for almost all the mods I've tried, it didn't for a few ( zenyth nav keys for example)

I'd be more than happy to go back to TWRP if there was no downside to it.
[Samsung Galaxy Nexus] Vangelis13:
No mod / rom / anything else I've thrown at it has failed to install; give it a try & let us know! :good:
[Samsung Galaxy Nexus] zephiK:
Tried TWRP on my TF101 and liked it so much that I flashed it on my Galaxy Nexus.

This recovery is definitely underestimated and I think that it's on the next level when compared to Clockwork Recovery.

Really fascinated with queued flashing. Makes flashing CM nightlies so much easier because I can set it and it'll flash everything at once :)
[Samsung Galaxy Nexus] kobik77:
+100

Just remember to use GooManager along with it.

I really can't understand why do people stubbornly keep using cwm and not even trying this one.
[Samsung Galaxy Nexus] zephiK:
Yup. I have GooManager as well, works very well. It's like a free version of ROM Manager + Clockwork together.

... because people are close minded and prefer using things that they are comfortable with. I don't mind Clockwork Recovery but it didn't really support the TF101 very well so I switched to TWRP and I was highly impressed. UI is very polished and intuitive... just flashed a kernel? That's fine... just press the "Wipe cache/dalvik" button after you flashed then press on Reboot! So much easier than going through the Clockwork Recovery Menu.

If anyone is reading this and ever thought of making the switch, do it... try it.
[Samsung Galaxy Nexus] creeve4:
Anyone?

I'm also coming from CWM and am having a difficult time finding information on the various TWRP functions. It would be nice if there were a simple guide or explanations for the various features.
[Samsung Galaxy Nexus] aamadorj:
I moved to TWRP recently and so far I have made a couple backups of the full ROM, so I check boot, system and data options. I don't select cache, because it will be rebuilded by the OS, or the other options that are unchecked by default.

Normally a full ROM backup is what you want (boot, system and data), but checking only boot, system or data will produce a backup of that just that specific partition.

Having the possibility to backup the RECOVERY seems interesting, but I think is completely unnecessary to include it in a full ROM backup. I just will make the backup bigger.
[Samsung Galaxy Nexus] MS.:
I can not disagree with this. What ever i flash it just works, while some other users yelling at boot-loops and so on. I'm not trying to blame CWM for that, i'm just saying that if your arms coming out of your a** - nothing will help.

I've tried to switch to latest CWM recoveries but always i came back to TWRP, it's invaluable.
[Samsung Galaxy Nexus] renaud:
This recovery allowed me to encrypt my phone and still be able to flash easily, this is so great.
[Samsung Galaxy Nexus] hga89:
We still have the problem with not being able to install a nightly release in the automated fashion, right? In my case of L13.
[Samsung Galaxy Nexus] nayiZ:
This is a little show stopper. Somebody know's how/what to debug here :confused:

EDIT1:

Actually, I just gave it a try. Using a lineagos-13-nightly from end of july (

without encryption!

) and running the internal updater suggest me to upgrade to version 19th of august. Download, Upgrade prompt warn's: "A compatible twrp is needed - if it is not present the upgrade needs to be done manually in twrp. Lineageos is going to reboot to install update" ... and what happend? Phone booted in TWRP (recent decrypt version from [MENTION=5153662]Android-Andi[/MENTION] , thx btw!) and started to run the update directly - unconditionally and so on ... and phone rebooted again to system... Now I have a brand new lineage nightly upgraded with the internal updater!

So automated fashion works for me for lineageos-13 without device encryption! [MENTION=3443260]hga89[/MENTION]

EDIT2:

Just did a second round. This time the "downgrade" to LOS-nightly-20170812 with the internal updater. And everything worked like the first time.

I'm going to encrypt my phone now to run the test again. Stay tuned for more
[Samsung Galaxy Nexus] hga89:
Yes, my problem is only with the encrypted phone.