yashsingh1/xdaforums huggingface dataset:
[ONE Android Development] Kutep0v:
Same, just different backup methods.

System – files method. Create usual archive by packing all files from a system partition.

System Image – dd method. Create exact partition dump by reading every single block from a system partition.

Of course. As I said, the backups are fully compatible (within my TWRP). You can't make a new "System" backup, but you can restore it from an old backup. In any case, why double-restore the same partition? It doesn't make sense, but if you want, you can restore and "System" and "System Image". :eek:
[ONE Android Development] kewinhw:
Aah thanks for clarifying :)

I guess for me then that change would be fine :) as I only make backups for stock rom and backup the current ROM before I do some change/modding or try another ROM. Only keeping 1-3 backups at most.

i think i have only had a problem restoring on a previous version of your twrp only once, the restoring part worked fine but some thing was behaving weird after booting the rom but I can't remember what it was hehe. Maybe you could make a poll in your thread? :cowboy:
[ONE Android Development] Kutep0v:
I don't think that's necessary.

Perhaps I'll also rename these items to prevent questions:

System -> System Files

System Image -> System
[ONE Android Development] GXGOW:
I take consistency and reliability over file size any day. Backup-size is not really an issue for me either, since I use adb to directly store them on my pc. No problem for me :)
[ONE Android Development] aqilsoomro:
You are the dev and this is your baby so do whatever you like. People who don't like your way may opt for other options. But I believe its your right to change it the way you want.
[ONE Android Development] minecabla:
You could also make it so that system image is checked by default and system files not.
[ONE Android Development] Kutep0v:
TWRP 3.2.1-K2

Took away the ability to backup the "System" and left only the "System Image" (info)

HACK: Prevent "Can't install this package over newer build" error (not tested, but it should work)

Add Portuguese (Portugal) translation.

Use /persist as Qualcomm time fix source during early boot

Use libdrm_platform for minuitwrp in 8.x trees

Add spblob decrypt for secdis method (Pixel 1 non-weaver)

Fix typo in Micro SD to have a consistency

Typo and bug fixing in Dutch(NL) translation, for example where a translated string is too long to fit in button.

Added missing translation in Turkish translation

Fix minor inconsistency in Czech translation

Added 2 translations in Italian language file

Added missing translations in Swedish language file

Added missing translation(s) in Portuguese(BR)

Typo / Uppercase-Lowercase in German translation compared to all other translations

There are a few typos and bugs in the French translation, for example where a string is to long to be displayed in the button (overlapping / cut-offs).

Relink adbd needed in android 8.1

There's a missing translaton in the Greek language file.

Merge up to android-8.1.0_r1 and fix conflicts

Fix build error in AOSP 8.1.0 r1 tree

Fix android.hidle.base@1.0 rule for various 8.0 trees

Decryption not verified. (as always)
[ONE Android Development] Anupam Mahato:
Why i am always getting unable to mount STORAGE error ,everytime i try to wipe(dalvic/system cache/data) ????
[ONE Android Development] Kutep0v:
Partition is damaged, or something else. This issue is not related to recovery.
[ONE Android Development] zertyuiop:
ADB shows that device is unauthorized in K2 version. K1 works well.
[ONE Android Development] Kutep0v:
In K2 buildtype was changed back to user-debug, so first you need to allow debugging in the system. I can return the eng buildtype, but tell me for what purpose you need it.
[ONE Android Development] zertyuiop:
I erased data partition and changed filesystem, so I can't boot using exiting ROM. But it's not nessesary to change buildtype, I do not plan any new experiments in the near future.
[ONE Android Development] Kutep0v:
Still don't see something special. What did you want to do via ADB in this case? MTP and adb sideload should work anyway.
[ONE Android Development] zertyuiop:
Yes, they work, but I tried to push TWRP backup via adb to recover my old ROM. I didn't allow debugging in system.
[ONE Android Development] AlMunt:
Whenever I open TWRP version 3.2.1 K2, TWRP goes to its logo screen where only the word TWRP is showing and spends 30s on that screen every time I load it. In the past it used to load almost instantly, without pausing on that screen for more than a couple seconds. Recently I have done some large ~15gb backups using TWRP. Could those really big backups be slowing down the TWRP loading speed? Also I am using F2FS.
[ONE Android Development] Mr.Ak:
Were you using ext4 in the past when twrp used to load faster?
[ONE Android Development] Kutep0v:
[MENTION=7390294]AlMunt[/MENTION] what file system are you using?
[ONE Android Development] AlMunt:
I think it may have been faster, but I don't remember if there was a noticeable difference in speed when I switched since it was many months ago.

I am on F2FS. It seems like F2FS is probably the reason TWRP is slow because I have less than 2gb free on my phone right now. I'll try switching to ext4 when I have time to do a full backup of my phone.
[ONE Android Development] AlMunt:
Changing the file system from F2FS to ext4 does not work on this recovery(3.2.1 K2). I had to switch to the Official TWRP to go to ext4. In the official recovery I get the message "Formatting Data using make_ext4fs...". In this recovery the message never appears and the partition stays with the f2fs file system.
[ONE Android Development] Kutep0v:
[MENTION=7390294]AlMunt[/MENTION] Really? Just checked the formatting in all available FS.
[ONE Android Development] AlMunt:
I reinstalled TWRP 3.2.1 K2 and tried to switch the cache file system. I got the same results as the ones in your screenshot. I went through the settings and saw the "Use rm -rf instead of formatting" option. Turning it on gave me the results I was getting before where the file system wasn't changing. I think I accidentally turned on that option and forgot about it since it seems to be off by default.

I feel like there should be a warning though saying that using rm -rf prevents file system from being changed.
[ONE Android Development] Kutep0v:
This is quite logical. "Use rm -rf instead of formatting" speaks for itself.
[ONE Android Development] tofu:
Android 8 decryption issue

Cannot decrypt with TWRP android Oreo rom.

Tried both latest Carbon rom and Lineage 15.1.

EDIT: It seems that this issue has been fixed for another phone, maybe this could be applied here as well? See https://github.com/TeamWin/Team-Win-Recovery-Project/issues/1168 and https://twrp.me/leeco/leecolemax2.html
[ONE Android Development] SLAlmeida:
I installed the version TWRP 3.2.1-k2, and when I enter the recovery the smartphone does not turn on, if someone can help me, what changes do I have to make, to work?

Thank you
[ONE Android Development] Timmmmaaahh!:
Wipe cache from recov.. Oh wait, sike! >_<

No no no, seriously.. Did you check the md5 hash on the download? It should match eef99c8046ec3f59ee621b7974bda63d

And how did you install it, via System (with an app like Flashify) or via Fastboot? Do you see anything on boot when holding Power+VolDown? Be as specific as possible.
[ONE Android Development] Kutep0v:
Sorry for the delay, I was busy.

I can compile TWRP under Lineage 15.1 for test on the weekend.

This can happen if still used old firmware (4.4) or the file was just corrupted during download.
[ONE Android Development] SLAlmeida:
Thanks, Timmmmaaahh and Kutep0v

The file had corrupted.

New download and installation, all ok.
[ONE Android Development] Kutep0v:
[MENTION=6640136]tofu[/MENTION] try it. I didn't test, I don't use encryption.
[ONE Android Development] ctrlaltca:
I'm running TugaPowerO10 (Oreo 8.1.0) with data encrypted with a sequence pattern, tested decrypting using this 3.2.1-test TWRP, unfortunately it's not working.
[ONE Android Development] Kutep0v:
[MENTION=8118918]ctrlaltca[/MENTION] it's sad. Then I'll have to deal with this on the weekend. I'll encrypt and test it myself.
[ONE Android Development] tofu:
As someone already noticed, unfortunately still not working. I tested with Candy and Tuga roms.
[ONE Android Development] davbarr:
Hello [MENTION=7296006]Kutep0v[/MENTION]. I need to install old backup but with latest twrp 3.2.1-k2 i have an issue. So i need to download twrp 3.1.0-k1 When can i find this old version please?
[ONE Android Development] Ricardo reverb:
This version of twrp works on the oneplus X (onyx) devices??
[ONE Android Development] tejaskadaskar:
Nope.
[ONE Android Development] davbarr:
perhaps someone can share this old version ;) best regards
[ONE Android Development] :
A simple google search for twrp 3.1.0-k1 could have helped you in seconds.

https://androidfilehost.com/?fid=529152257862710466
[ONE Android Development] MurikBMX:
https://cloud.mail.ru/public/2ScZ/zxtiagQ5k
[ONE Android Development] davbarr:
Thanks but I found this file too but why "purple"?

and

Perfect [MENTION=5050366]MurikBMX[/MENTION]. Thanks
[ONE Android Development] Kutep0v:
About decryption.

Perhaps, many waiting for results... Unfortunately, there is nothing. I wanted to work on this on the weekend, but I didn't have time. In Russia there were holidays, so I left the city on 28th and returned only today. Bears, vodka, balalaika, ushanka, as usual.

I apologize if someone really waited. I'll try to start work in the near future.

P.S. [MENTION=4554809]davbarr[/MENTION], if you still need old version > https://kutep0v.bitbucket.io/TWRP/legacy/twrp-3.1.0-K1-bacon.img
[ONE Android Development] railcarchampion:
Is there is a way to use the md5sum file available to verify the ROM flash, how do I do that? This recovery skips the md5sum file every moment it gets. I try to verify a ROM or Gapps and it skips the md5sum file.
[ONE Android Development] 1plus:
Kutep0v, apparently currently there is no single TWRP build which works with Oreo encryption (February security patch). Is there a change to release a K3 version with such patch? Using TWRP is very convenient but no such option on any Lineage15 or other Oreo-based rom on oneplus.
[ONE Android Development] patp:
It would certainly be a big service to the community if you make a TWRP that can handle Oreo encryption!
[ONE Android Development] tofu:
[MENTION=7296006]Kutep0v[/MENTION] Any news regarding decryption issue? Did it turn out to be more difficult than excepted? cheers
[ONE Android Development] Kutep0v:
[MENTION=6640136]tofu[/MENTION] Yes. I have no idea what's wrong, there are no apparent reasons. I tolked with the maintainers of some other devices, they have similar problems.

I'm trying to solve problems, nothing more to say.
[ONE Android Development] thedamager:
Maybe something here can help: https://xdaforums.com/oneplus-5t/development/recovery-twrp-3-2-1-0-oreo-8-0-8-1-t3729673

Fingers crossed and thanks for all the hard work.
[ONE Android Development] xange:
Some how i have trouble believe this: https://xdaforums.com/showpost.php?p=76883580&postcount=14

OP mentioned he discussed with maintainers from other rom, but someone told me that decryption works well in RR6.0, and i'm pretty sure it is TWRP error since decrypt error only happen in TWRP and not when booting into Android. Well, i would rather wait for any development here or officially from TWRP team...
[ONE Android Development] tofu:
[MENTION=7296006]Kutep0v[/MENTION] Does this fix the decryption issues ? https://twrp.me/site/update/2018/07/01/twrp-3.2.2-0-released.html
[ONE Android Development] Kutep0v:
[MENTION=6640136]tofu[/MENTION] no.

Should I make a new version based on 3.2.2 if the decryption is still not fixed? :confused:
[ONE Android Development] tofu:
Ok, thanks for the info. I guess the latter question is up to you, sir :)
[ONE Android Development] Mustkeg:
If I want to encrypt now and be fine with the consequences (no working decryption, no TWRP backups, updates only via ADB, if I'm correct), can I just do so from the Android security settings? I'm on the latest official LOS and have /data F2FS currently.

Can I restore the latest TWRP backup if everything goes south?
[ONE Android Development] Diniodoc:
What do you mean with updates ? The SYSTEM partition is not encrypted and you can update it with TWRP if you store the *.img oder *.zip on USB-OTG. Or am I wrong ?
[ONE Android Development] Mustkeg:
Ah, I never got the OTG Part - now it's clear. You are right, that should work.
[ONE Android Development] kewinhw:
Encryption isn't working in 3.2.1 either right? If it isn't then I guess there is no need unless there are other fixes :cowboy:
[ONE Android Development] JoroBo:
I haven't looked at the changelog but I don't think you need to until decryption is resolved. Your 3.2.1 works great. Up to you of course :)
[ONE Android Development] Diniodoc:
Which changelogs? I can't find them :confused:
[ONE Android Development] JoroBo:
I meant for the official TWRP changes.

Edit: Actually all I see on TWRP.me is this:

What's new in 3.2.2-0:

* adb backup fixes

* OTA style update zips will now install automatically without prompting for decrypt

* minor tweaks to handling date/time on Qualcomm devices

* updates to some language translations
[ONE Android Development] bmark240:
I'm getting E: error creating fstab on your T1 & K2 builds.

It happened when I tried to uninstall Magisk to clean flash the latest build.

It also happened when I was updating my firmware too.
[ONE Android Development] Kutep0v:
[MENTION=7672643]bmark240[/MENTION] I can't reproduce. Any logs?
[ONE Android Development] bmark240:
Recovery log from TWRP folder the one I post?
[ONE Android Development] Kutep0v:
I don't see this error in the log.
[ONE Android Development] bmark240:
Would it be a problem with TWRP not mounting the file system tables?

I will try a roll back to pre error state.
[ONE Android Development] kewinhw:
[MENTION=7296006]Kutep0v[/MENTION] I'm wondering why the persist backup option is gone in 3.2.1-k2 it was available in the version before it if i remember correctly? :)

EDIT: maybe it isn't so important ^^
[ONE Android Development] Kutep0v:
[MENTION=8172975]kewinhw[/MENTION] I don't remember, but I can add persist backup, if necessary.

About decryption in 8.1: I need a little more time, almost ready.
[ONE Android Development] kewinhw:
you could add it back if it is one of the partitions that would be needed to be able to restore if suddenly imei and stuff like that is broken

I have a backup of firmware and efs in case that happens. If there are important stuff it would be nice to have persist back ^^
[ONE Android Development] Kutep0v:
Time for one more test: info (decryption and etc.)

Let me know if there are any problems. This is really important.

[MENTION=8172975]kewinhw[/MENTION] added persist backup.
[ONE Android Development] kewinhw:
heey thank you very much :D i cant test decryption tho because my phone isn't encrypted :/
[ONE Android Development] jmax91:
Decryption is working with latest LineageOS 15.1. Thank you so much.
[ONE Android Development] :
Decryption is also working with CarbonROM 6.1. This is amazing.Thank you.
[ONE Android Development] Diniodoc:
Also working for TugaPower O22 based on 8.1.!

Does anybody know if it is possible to set different PINs/PWDs for encryption and display-lock ?

Great work! THX
[ONE Android Development] Diniodoc:
encyption works fine, but there is another strange behaviour. When updating the system with your twrp-3.2.3-T1-bacon version via *.zip, the updated system is not able to mount the internal storage anymore (it does not care if encryption is enabled or not). Using the same *.zip update with twrp-3.2.2-0-bacon don't lead into that issue.
[ONE Android Development] Kutep0v:
[MENTION=5214431]Diniodoc[/MENTION] write step by step how to reproduce it. I tried to update using zip, the partition is mounting as usual.
[ONE Android Development] kewinhw:
I updated omnirom today with the latest test twrp version and it worked with no issues :cowboy:
[ONE Android Development] kewinhw:
[MENTION=7296006]Kutep0v[/MENTION] i have a backup of a clean fastboot flashed CyanogenOS 13.1.2, and i put the 3 persist files in it to make the full rom backup complete :cyclops: is it safe if i was to restore that backup now (persist files in it) now or will it break something? the persist partition did show up in that cos13.1.2 backup folder with the other partitions. but I'm just asking since the persist backup wasn't made the same time as the stock backup. :)
[ONE Android Development] ctrlaltca:
Decryption confirmed working on 8.1 (Tugapower O22), thank you!
[ONE Android Development] xakep666:
Decryption not working on official LineageOS 14.1 (says "decryption failed", using pin)
[ONE Android Development] kewinhw:
if i charge my phone to 100% and go into twrp with the test version the battery in twrp only shows 80%
[ONE Android Development] Kutep0v:
I'll check it on the weekend and try to figure out.

I can't confirm this (see screenshot below). Perhaps the problem is on your side.
[ONE Android Development] kewinhw:
hmm yeah might be my phone, i have noticed that sometimes when i install a new rom with 100% when rom is booted and i can check battery % it is sometimes at 80% oh well :) its getting old haha
[ONE Android Development] tofu:
Decryption working beautifully on TugaOreo.
[ONE Android Development] Diniodoc:
Sorry for the late answer, but I was very busy. I've tried many things today to narrow the problem down, but I don’t understand it. To anticipate, I have found a way to face the problem.

But let us start at the very beginning. In the past to make a full backup (incl. /data/media) I just renamed the /data/media folder into /data/media.backup and started the backup process. After renaming it again everything worked fine in the past.

With your new twrp-version there is a problem within the second renaming process, because after backing up twrp notices that there is no /data/media folder and creates a new one and mounts that folder additionally to /sdcard. So far no problem but then after deleting the new /data/media and renaming the media.backup into /data/media we run into trouble because the old folder is still mounted to /sdcard without throwing any error while renaming. That is really strange, but I think it causes the trouble that the system is not mounting the internal storage anymore. To avoid the issue, you just have to umount /sdcard before renaming and everything is fine.

The part I really don’t understand is, why I run into the same problem when updating the system after doing the backup process a long time before and in between the system ran without problems.

And the second question why there is no problem when using twrp-3.2.2-0-bacon?

I don’t know why but with the additional umount step everything is working fine for me.
[ONE Android Development] tofu:
There seems to be some issue with this new version after all. I've been using debloater zip and couple other flashable zips that for eg. replace gps.conf file in /system/etc with modified version, but with this new version nothing happens when I flash them.
[ONE Android Development] ImmortAlex:
Latest version working good for casual update official LAOS + OpenGapps. Decrypting is working too, at least for /data encrypted in LAOS 15.1.
[ONE Android Development] Kutep0v:
Quite strange actions. I think this is due to the fact that I compile using 15.1 branch, where there are many different changes for 8.1 in init files. Official TWRP for bacon compiles under 13.0 and our device tree has not been updated since 2015. :confused:

Perhaps the reason is updater-script in your zips. In this version I use toybox instead of outdated busybox 1.22.1 which was released 5 years ago. Some zip contain absolutely idiotic code like

run_program("/sbin/busybox", "mount", "/system");

which can lead to such problems. More precisely I can say pnly when you provide zips and recovery log.
[ONE Android Development] LPW00:
That may explain why all the addon.d scripts on my device stopped working.
[ONE Android Development] tofu:
I was suspecting that this problem probably has something to do with the busybox stuff.

This idiotic code has thus far served me well. Would be nice to learn the right way to do the same thing?

Zip attached.
[ONE Android Development] Kutep0v:
I've never seen this before. I think the problem is in something else. Can you upload scripts or give links to using zips?

I didn't mean to offend you, but there is nothing good in hard binding to busybox or toybox binary. In the future, this will lead to some new problems.

You can use such a "neutral" code.

mount("ext4", "EMMC", "/dev/block/platform/msm_sdcc.1/by-name/system", "/system", "");
...
unmount("/system");
[ONE Android Development] LPW00:
This for example. (It didn't want to upload the .sh file so I zipped it.) It should theoretically enable aptx and I modified it to also set the volume steps to 100 (because why not) and to include the nexus.xml file in /system/etc/sysconfig. While all the files get restored, the build.prop edits rely on busybox. Also Magisk didn't restore on my last update, but I use the beta and it has some new addon.d stuff, maybe that doesn't work, Idk. Maybe u wanna look at that too, it doesn't have busybox mentioned in the script tho. "All" was an exaggeration...

Edit: I used the 3.2.2 T1. Gonna update right now.
[ONE Android Development] hikumo:
OTA update (omnirom) works fine.

only one problem. install this flashable zip with latest twrp (TWRP 3.2.3-T1) and have error 7

[COLOR="Red"]Updater process ended with ERROR: 7[/COLOR]
[ONE Android Development] hikumo:
encryption/decryption working fine on omnirom oreo (TWRP 3.2.3-T1)
[ONE Android Development] Kutep0v:
[MENTION=7471495]LPW00[/MENTION] [MENTION=8410177]hikumo[/MENTION] in both cases the scripts refers to busybox, which was replaced in this version by toybox. I can only advise to change the code or inform the developer.

[MENTION=7471495]LPW00[/MENTION] about magisk. After the update, it didn't disappear, working as usual. I'll try again tomorrow with the latest beta.

About toybox.

I saw that almost all the problems of the test version are associated with the toybox, because some developers use links to busybox. Toybox contains fewer tools, but there are enough of them for almost all cases, syntax of almost all tools is identical. Honestly, I haven't seen a single script where binding to the busybox is really necessary.

Busybox hasn't been updated for a long time and hardly anyone will work on it. Now the busybox compiles only with a bunch of different hacks. Who knows, maybe at 9.0 it will not work at all. I think the sooner we leave, the better, but I can return busybox if it is really necessary.

What do you think of it?

Edit: I did a poll.
[ONE Android Development] Loriot:
For me toybox is OK. It is working perfect. No need to go back.
[ONE Android Development] LPW00:
So I just take the script and replace 'busybox' with 'toybox', right? I'd vote for toybox, but I use xda labs
[ONE Android Development] Kutep0v:
No. Just remove 'busybox'.

Edit: example https://gist.github.com/kutep0v/91f39d68b07cd68f491bbd0e7b0a554d/revisions

The solution may differ for other scripts.
[ONE Android Development] kewinhw:
In what situation is busybox and toybox used i know the names but i dont understand what they are used for like what does the code do ?
[ONE Android Development] hikumo:
sorry i didn't see twrp 3.2.3-T1 switched from busybox to toybox. im editing the script and no error 7. toybox or busybox is ok.
[ONE Android Development] 1plus:
[MENTION=7296006]Kutep0v[/MENTION]

super good work with TWRP encryption problem. Dirty flash of TugaPower 23 from twrp T1 -- no problem with /data decryption like before.