yashsingh1/xdaforums huggingface dataset:
[OnePlus 3 & 3T Cross Device Development] BillGoss:
So I've been using the latest 3.3.1-1 for a while and have noticed one major difference from *-0: what gets written to /cache/recovery at the end is now a zipped file last_log.gz rather than a plain text file.

That, in itself, is no big deal. However the contents of the .gz file is not just the last log. It looks like a number of partial copies of the log followed by the full copy at the end.

[MENTION=5091734]dianlujitao[/MENTION], I'm sure this isn't right and is a bug.

I've attached copies of the log copied to the sdcard using the TWRP>Advanced>Copy log function (recovery.log) and the unzipped (gunzip) last_log for three separate, sequential sessions of TWRP.

Just to show the problem, I did a search for "Starting TWRP" and "I:operation_start" on both files. This shows TWRP starting and the operations invoked. Here's the result from the second session:

u0_a208:/sdcard $ egrep "Starting TWRP|I:operation_start" 2-*
2-last_log:Starting TWRP 3.3.1-1-72d75b0b on Thu Nov 2 04:13:04 1972
2-last_log:I:operation_start: 'Decrypt'
2-last_log:Starting TWRP 3.3.1-1-72d75b0b on Thu Nov 2 04:13:04 1972
2-last_log:I:operation_start: 'Decrypt'
2-last_log:I:operation_start: 'Flashing'
2-last_log:Starting TWRP 3.3.1-1-72d75b0b on Sat Dec 14 01:08:17 2019
2-last_log:Starting TWRP 3.3.1-1-72d75b0b on Thu Nov 2 04:13:04 1972
2-last_log:I:operation_start: 'Decrypt'
2-last_log:I:operation_start: 'Flashing'
2-last_log:Starting TWRP 3.3.1-1-72d75b0b on Sat Dec 14 01:08:17 2019
2-last_log:I:operation_start: 'Copy Log'
<non-Blank line added>
2-recovery.log:Starting TWRP 3.3.1-1-72d75b0b on Thu Nov 2 04:13:04 1972
2-recovery.log:I:operation_start: 'Decrypt'
2-recovery.log:I:operation_start: 'Flashing'
2-recovery.log:Starting TWRP 3.3.1-1-72d75b0b on Sat Dec 14 01:08:17 2019
2-recovery.log:I:operation_start: 'Copy Log'

The last_log seems to have two partial logs (based on the 'Decrypt' lines) plus the full log at the end. Note that the "Starting TWRP" after the "Flashing" is a restart on the TWRP ui after installing a theme.
[OnePlus 3 & 3T Cross Device Development] jmadiaga:
This error message appears:

error cannot decrypt fbe
[OnePlus 3 & 3T Cross Device Development] BillGoss:
How did you get your phone using FBE (file based) encryption instead of FDE (full data)? FDE is the normal one used on our phone.
[OnePlus 3 & 3T Cross Device Development] Flounderist:
It also happened to me.

Device is f2fs file system for data and it is non-encrypted.

Probably 3.3.1-1 is set to work on Q builds.

As you may know, Q is using FBE encryption.
[OnePlus 3 & 3T Cross Device Development] BillGoss:
But an unencrypted data partition isn't the same as FBE. Maybe this is another bug in 3.3.1-1.

Unofficial Los 17 (Android Q), by the way, uses FDE.

I won't run my phone unencrypted. If our Q ROMs switch to FBE that'll be a real pain because one won't be able to revert back to P without formatting /data.
[OnePlus 3 & 3T Cross Device Development] Flounderist:
I assume the error notification is just cosmetic.

Maybe twrp is looking for any FBE encryption footer, it cannot find it and then it pop out.

By the way I've been able to backup and dirt flash latest LOS 16 build, so recovery isn't faulty.

Do you think reverting to 3.3.1-0 will be safe?
[OnePlus 3 & 3T Cross Device Development] jmadiaga:
i was doing a clean install of NOS (10/Q) when the error message appeared during install.

i did another clean install with LOS 16, error message still appears.

i'm guessing it's the TWRP latest ver.

however, despite the error message, installation was successful,

stable 9.06 firmware

LOS 16 (latest)

opengapps aroma dec. 12

magisk 20.1
[OnePlus 3 & 3T Cross Device Development] BillGoss:
Reverting is fine. I'm back on *-0 because of the last_log issue.
[OnePlus 3 & 3T Cross Device Development] jmadiaga:
is there a work around to the said error message: error unable to decrypt FBE?

thanks
[OnePlus 3 & 3T Cross Device Development] BillGoss:
Yes! Go back to *-0
[OnePlus 3 & 3T Cross Device Development] jmadiaga:
yup did it and also tried with one of the later versions of LOS 16 + opengapps aroma

the bootloop was fixed.

for several older LOS 16 versions, + opengapps aroma - there was boot loop

but not with opengapps pico or nano
[OnePlus 3 & 3T Cross Device Development] mmm4m5m:
recovery partition is restored after reboot

I have new/untouched/stock OnePlus 3T with android 9, OxygenOS version 9.0.6, model A3010, build ONEPLUS_A3010_28_191104.

I installed latest TWRP (twrp-3.3.1-1-oneplus3.img) BUT after reboot it is gone - recovery partition is reverted and restored to stock recovery.

Where is the problem? What is my mistake?

I checked few howto guides. Also from twrp site: twrp dot me/oneplus/oneplusthree.html

Flashing recovery is successful:

# fastboot flash recovery twrp-3.3.1-1-oneplus3_20191208.img
target reported max download size of 440401920 bytes
erasing 'recovery'...
OKAY [ 0.020s]
sending 'recovery' (27581 KB)...
OKAY [ 0.911s]
writing 'recovery'...
OKAY [ 0.201s]
finished. total time: 1.132s

After that, I tried different ways to reboot:

1) press Volume Down + Power and "fastboot reboot"

2) power off, then press and hold Volume Down + Power

3) in bootloader select "recovery mode"

In all cases TWRP is loaded and all looks good. Then I select Reboot/System.

Android OS is loaded and all looks good. Then I reboot to recovery and stock recovery is loaded.

What I am missing? Where is my mistake?

Thanks!
[OnePlus 3 & 3T Cross Device Development] BillGoss:
You're missing information. Stock OOS will automatically replace a custom recovery with the stock recovery when it boots up.

To prevent this happening you need a patched boot.img or use Magisk.

See https://xdaforums.com/oneplus-3t/how-to/dm-verity-disable-oxygen-os-t3922324/post79357241 for patched boot.

I'll let you find Magisk for yourself.
[OnePlus 3 & 3T Cross Device Development] mmm4m5m:
Thanks. TWRP is good now. And Magisk too.

(I was confused thinking that, before all, first I have to setup custom recovery. Need to sleep, will enjoy it tomorrow).
[OnePlus 3 & 3T Cross Device Development] milomak:
does twrp still limit the size of an update zip that will work from titanium backup to less than 2gb?

i ask because i want to do a clean update but i remember this being an issue last time i had to
[OnePlus 3 & 3T Cross Device Development] MirzaAwais:
No limit for me as OOS and all others roms flashes OK

Sent from my Phone
[OnePlus 3 & 3T Cross Device Development] Mohammed Gabr:
[MENTION=5091734]dianlujitao[/MENTION]

Can you please build a multirom twrp with your latest 3.3.1-1 twrp

https://tasssadar.github.io/

Please please please
[OnePlus 3 & 3T Cross Device Development] pcesar.cercal:
Hi there, this last update is ok for android 10?

no problems?
[OnePlus 3 & 3T Cross Device Development] Mohammed Gabr:
Yes
[OnePlus 3 & 3T Cross Device Development] knpk13:
Does TWRP still boot really slowly on f2fs? I've been on ext4 for a while now. And are any ROMs really using FBE?
[OnePlus 3 & 3T Cross Device Development] :
Never had any issues using f2fs, no boot delays or whatever.

Though would recommend ext4 because it is most common for custom roms and speed difference is questionable.

In real life (daily usage) you probably will not even notice the difference.

If you really want you can also speed up ext4 a little by using some exotic mount instructions, but would not recommend it.
[OnePlus 3 & 3T Cross Device Development] Flounderist:
It always took more than 10 seconds to pass from splash to home screen.
[OnePlus 3 & 3T Cross Device Development] :
How? There are no fbe supporting blobs from oneplus for op3t - but even if you make a rom with kanged blobs: first you need to make a fbe supporting recovery... I'm pretty sure, we would have heard/read of such development.

I also think this question would be better placed in https://xdaforums.com/oneplus-3t/how-to/info-list-oreo-roms-kernels-available-3-t3688139 or as seperate thread in https://xdaforums.com/oneplus-3t/help then here, since this twrp is made to support fde. You can consider all users of this twrp as users of fde roms.
[OnePlus 3 & 3T Cross Device Development] knpk13:
I asked because the person I quoted claimed Q roms used FBE, but they do not.
[OnePlus 3 & 3T Cross Device Development] :
Typical misunderstanding introduced by inaccuracy due to leaving out a single word. If you look at

https://source.android.com/security/encryption/file-based (red !caution box):

Just drop the word "new", and you get that claim... :(

All the g documentation refers to "devices originally shipped with version", since Oneplis has shipped the op3/t with android marshmallow (oos 3.5.x) Oneplus shipped it with fde. The updates up to android pie (oos 9.0.x) doesn't require the migration to fbe (from g's side as requirement for certification) and Oneplus has decided to stay with fde.
[OnePlus 3 & 3T Cross Device Development] milomak:
so i am running havoc-3.2

trying to run a titanium backup update.zip from a havoc-3.1 in twrp (latest- i think 3.1.1-1)

the zip runs without error. but no apps when the phone boots.

what have i missed?
[OnePlus 3 & 3T Cross Device Development] BillGoss:
I think that the way TB installs the apps (just copies the apk to /data/app) may no longer work in Android.

Why don't you take a copy of the TB apk and manually install that after the rom had booted and then install the apps using TB?

I find that this works, though it'll prompt you for each app (because the pro features don't work until you install the pro key from the Play Store).
[OnePlus 3 & 3T Cross Device Development] :
The recovery log. Though I guess [MENTION=2911853]BillGoss[/MENTION] is correct, without log it's just educated guessing...

[MENTION=2911853]BillGoss[/MENTION]: we both guessed wrong:

tb utilizes /sbin/busybox in update.zip's updater-script. This is no longer included in newer twrps. This has been discussed allready in this thread.

[MENTION=1830779]milomak[/MENTION]:

Replace all 4 occurences of "/sbin/busybox" with "/sbin/toybox" in /META-INF/com/google/android/updater-script in update.zip. BTW: flashing update.zip gives an error, but since the reboot hits immidiatly (because "/sbin/busybox sleep 5" isn't working, you can't see the error. You can remove the last line (reboot) to see the error message (before changing busybox to toybox, of course).
[OnePlus 3 & 3T Cross Device Development] milomak:
ah i haven't used titanium in ages so am not too surprised. thanks [MENTION=4405529]nvertigo67[/MENTION] and [MENTION=2911853]BillGoss[/MENTION]

---------- Post added at 03:59 PM ---------- Previous post was at 03:58 PM ----------

ah i haven't used titanium in ages so am not too surprised. thanks [MENTION=4405529]nvertigo67[/MENTION] and [MENTION=2911853]BillGoss[/MENTION]

---------- Post added at 04:40 PM ---------- Previous post was at 03:59 PM ----------

[MENTION=1830779]milomak[/MENTION]:

Replace all 4 occurences of "/sbin/busybox" with "/sbin/toybox" in /META-INF/com/google/android/updater-script in update.zip. BTW: flashing update.zip gives an error, but since the reboot hits immidiatly (because "/sbin/busybox sleep 5" isn't working, you can't see the error. You can remove the last line (reboot) to see the error message (before changing busybox to toybox, of course).[/QUOTE]

there were 3 instances

replacing didn't work

thanks for the effort
[OnePlus 3 & 3T Cross Device Development] :
There are 4. Remove the reboot line (just after the fourth occurence of busybox) and get a recovery log. "replacing didn't work" is not really helpfull.

EDIT:

If you use a rom with system-as-root, you need to adjust the paths in updater-scripts for system apps and previleged apps, of course. I strongly recommend restoring no system apps nor previleged apps!!!!!
[OnePlus 3 & 3T Cross Device Development] milomak:
no i didn't back up any system or privileged apps

the instances are

$ cat META-INF/com/google/android/updater-script| grep toybox
run_program("/sbin/toybox", "mount", "/data");
run_program("/sbin/toybox", "mount", "/system");
run_program("/sbin/toybox", "sleep", "5");

it seems, i do not have toybox

$ ls /sbin
ls: /sbin/.se: Permission denied
magisk magisk.bin magiskhide magiskinit magiskpolicy resetprop su supolicy
[OnePlus 3 & 3T Cross Device Development] :
How old is that zip? Made with 8.4.0.2 and PRO 1.3.2 it looks like this:

cat /mnt/export/data/Downloads/incoming/updater-script
ui_print("");
show_progress(2,0);
run_program("/sbin/busybox", "mount", "/data");
run_program("/sbin/busybox", "mount", "/system");
run_program("/sbin/busybox", "mkdir", "/data/data/.titanium_backup_data_restore");
set_perm(0, 0, 0555, "/data/data/.titanium_backup_data_restore");
set_progress(0);
ui_print("Extracting data: com.github.slashmax.aamirror");
package_extract_file("com.github.slashmax.aamirror-20200229-093929.properties", "/data/data/.titanium_backup_data_restore/com.github.slashmax.aamirror-20200229-093929.properties");
set_perm(0, 0, 0444, "/data/data/.titanium_backup_data_restore/com.github.slashmax.aamirror-20200229-093929.properties");
package_extract_file("com.github.slashmax.aamirror-20200229-093929.tar.gz", "/data/data/.titanium_backup_data_restore/com.github.slashmax.aamirror-20200229-093929.tar.gz");
set_perm(0, 0, 0444, "/data/data/.titanium_backup_data_restore/com.github.slashmax.aamirror-20200229-093929.tar.gz");
ui_print("Installing app: com.github.slashmax.aamirror");
delete("/data/app/com.github.slashmax.aamirror.apk");
delete("/data/app/com.github.slashmax.aamirror-2.apk");
package_extract_file("com.github.slashmax.aamirror-01c69939d107096e81629e716d9d6ab8.apk", "/data/app/com.github.slashmax.aamirror-1.apk");
set_perm(0, 0, 0644, "/data/app/com.github.slashmax.aamirror-1.apk");
set_progress(1);
ui_print("Installing app: com.keramidas.TitaniumBackup");
delete("/data/app/com.keramidas.TitaniumBackup.apk");
delete("/data/app/com.keramidas.TitaniumBackup-2.apk");
package_extract_file("com.keramidas.TitaniumBackup.apk", "/data/app/com.keramidas.TitaniumBackup-1.apk");
set_perm(0, 0, 0644, "/data/app/com.keramidas.TitaniumBackup-1.apk");
set_progress(2);
unmount("/data");
unmount("/system");
ui_print("Done !");
ui_print("Will reboot in 5 seconds...");
run_program("/sbin/busybox", "sleep", "5");
run_program("/sbin/reboot");

4 occurances of "busybox".

That's really strange: "/sbin/toybox" is part of twrp's ram disk (aka rootfs). Also strange: when I use twrp's terminal I get a root prompt ("#"), but you have a user prompt ("$"). Even stranger: no matter if magisk is installed or not, it shouldn't show up in twrp's ramdisk... What do you get, if you type "which toybox" in twrp's terminal?

I don't ask for logs a third time, but without logs this conversation is useless.
[OnePlus 3 & 3T Cross Device Development] milomak:
tibu v8.4.0.2

zip done friday

$ which toybox
/system/bin/toybox
[OnePlus 3 & 3T Cross Device Development] :
This is not the output of the twrp terminal. Please reread my previous posting carefully. The twrp terminal always has a root prompt.
[OnePlus 3 & 3T Cross Device Development] BillGoss:
You need to understand that:

- you are going you edit and change the zip file on your phone (or PC). But that is NOT where the installation happens.

- the commands in the zip file are executed in TWRP which is a totally different run time context to your phone or PC.

So, in TWRP the binary files in /sbin are different from this in /sbin on your normal phone system.

So, just change /sbin/busybox in the zip file to /sbin/toybox and then run it in TWRP.
[OnePlus 3 & 3T Cross Device Development] milomak:
ah sorry

OnePlus3:/ # which toybox
/sbin/toybox
[OnePlus 3 & 3T Cross Device Development] :
Let's continue here.
[OnePlus 3 & 3T Cross Device Development] milomak:
actually it seems there may be a bigger issue at play here

i notice that when i try to restore apps from tibu some of them don't restore after waiting for as long as 10 minutes

some restore fairly quickly

How do i get a twrp log? from the attached it would seem the apps have been restored even though that's not the case. View attachment 4962161
[OnePlus 3 & 3T Cross Device Development] :
TWRP-Main-Menu->Advanced->Copy Log

Perhaps you should start with a clean flash without restoring anything for now, and start learning the very basics of TWRP and edify before ruining a pretty fresh install by restoring a large amount of data without exactly knowing what you are doing.
[OnePlus 3 & 3T Cross Device Development] BillGoss:
That's because TB can't handle split apks (ones that have split_*.apk in /data/app). You'll have to restore these apps from the Play Store or use https://play.google.com/store/apps/details?id=com.aefyr.sai to backup/restore the apks).

PS: none of this has to do with TWRP, instead it's all about TB and should really be in a Q&A thread.
[OnePlus 3 & 3T Cross Device Development] milomak:
my apologies

i'll start one
[OnePlus 3 & 3T Cross Device Development] :
Don't blindly start one. Read before posting: https://xdaforums.com/showthread.php?t=2717159 , maybe the additional informatikn can be added.
[OnePlus 3 & 3T Cross Device Development] milomak:
this is happening on a clean install

the update.zip was done before i wiped

so for instance i can't understand what has changed in tibu that split apks would have been created now?

i have since about a year ago extensively used tibu and the update.zip specifically.

tibu and pro are the latest versions as well

though i should probably add that the backup options were greyed out

edit - recovery log added
[OnePlus 3 & 3T Cross Device Development] :
...and they seem to install:

Installing zip file '/sdcard/update.zip'
Checking for Digest file...
I:Update binary zip
Verifying package compatibility...
Package doesn't contain compatibility.zip entry
I:Extracting updater binary 'META-INF/com/google/android/update-binary'
I:Zip does not contain SELinux file_contexts file in its root.
I:has_legacy_properties: Found legacy property match!
I:Legacy property environment initialized.

about to run program [/sbin/toybox] with 3 args
mount: '/dev/block/sda15'->'/data': Device or resource busy
run_program: child exited with status 1
about to run program [/sbin/toybox] with 3 args
about to run program [/sbin/toybox] with 3 args
Installing app: com.barclays.absa.banking
Installing app: za.co.android.africanbank
Installing app: com.rdr.widgets.core
Installing app: com.google.android.apps.authenticator2
Installing app: me.bluemail.mail
Installing app: com.bottles
Installing app: com.app.cellc
Installing app: com.cetusplay.remotephone
Installing app: com.dvtonder.chronus
Installing app: za.co.discovery.consumer
Installing app: com.estrongs.android.pop
Installing app: com.ashwhale.sepush.eskom
Installing app: com.facebook.katana
Installing app: com.amazon.storm.lightning.client.aosp
Installing app: org.mozilla.firefox
Installing app: za.co.fnb.connect.itt
Installing app: com.mobilefootie.wc2010
Installing app: org.fedorahosted.freeotp
Installing app: com.ft.news
Installing app: org.dayup.gtask
Installing app: org.mmin.handycalc
Installing app: com.huawei.health
Installing app: com.huawei.hwid
Installing app: com.google.android.GoogleCamera
Installing app: com.identipet.IdentipetApp
Installing app: com.instagram.android
Installing app: org.xbmc.kore
Installing app: com.lastpass.lpandroid
Installing app: mega.privacy.android.app
Installing app: com.mtnsar3
Installing app: com.teslacoilsw.launcher
Installing app: com.teslacoilsw.launcher.prime
Installing app: com.microsoft.office.outlook
Installing app: com.diune.pictures
Installing app: com.d6.schools
Installing app: com.riteshsahu.SMSBackupRestore
Installing app: de.stocard.stocard
Installing app: com.touchtype.swiftkey
Installing app: com.jumobile.manager.systemapp
Installing app: com.quoord.tapatalkHD
Installing app: org.telegram.messenger
Installing app: com.waze
Installing app: com.keramidas.TitaniumBackup
about to run program [/sbin/toybox] with 3 args
Done !

Which apps, that are NO split apps arn't showing up?

This lack of understanding can be cured by reading:

https://www.google.com/search?q=split+apk+titanium+backup

"I've done it the same way I've used for years..." Famous last words - sometimes good enough to apply for a darwin award... ;)

Things change - especially with major version jumps. Reading helps to keep up.
[OnePlus 3 & 3T Cross Device Development] milomak:
none of those apps have installed.

but yes i will admit to have been lazy in keeping up with what is happening. i guess it is a case of having trusted the name.

thank you all for indulging me.
[OnePlus 3 & 3T Cross Device Development] :
Did you check the script if the missing mkdir is needed? Did you check if they are transfered at all (inside twrp)?

I doubt it's working witout that mkdir line I showed you.
[OnePlus 3 & 3T Cross Device Development] :
I assume you mean restore. If the boot freeze really occurs after backup ignore my posting.

This is not rom related. twrp has issues restoring system. Either use system image for backup/restore or dirty flash the rom again.
[OnePlus 3 & 3T Cross Device Development] tfae:
Hi all.

I know that this was already discussed somewhere on this thread but I can't find it:

My 3T screen broke and doesn't work. I can reboot into TWRP and list the device through ADB, but my phone is encrypted.

How can I enter my

pattern

through ADB to decrypt the device?

EDIT: found the answer here! https://android.stackexchange.com/a/133246
[OnePlus 3 & 3T Cross Device Development] BillGoss:
The TWRP documentation for this is at the bottom of https://twrp.me/faq/openrecoveryscript.html
[OnePlus 3 & 3T Cross Device Development] Johannes1098:
Hello,

can I backup /data in TWRP on a 64GB OP3T and restore it then on a 128GB OP3T?

Or can I use only 64 of 128GB then? Or is it simply failing.?
[OnePlus 3 & 3T Cross Device Development] Eric_Lev:
No sweat for a nandroid backup ... only the name of the backup directory/folder is different.

OP3T_64GB ... /TWRP/BACKUPS/XXXXX/your_backup_source

OP3T_128GB ... /TWRP/BACKUPS/YYYYY/copy_your_backup_source_here

For me, it worked.
[OnePlus 3 & 3T Cross Device Development] Johannes1098:
sounds good, will try it when I have the other op3t on hand..

Thank you :)
[OnePlus 3 & 3T Cross Device Development] :
Question which popped up today....

Why is /data and so /sdcard mounted after boot in recovery even when unencrypted and so without entering a password? Isn't this a security issue?

For me it sounds more logical when you would boot into recovery and had to mount /data and so /sdcard explicitly, so you do not have access when plugged in on USB.

May be even entering your password you have set unencrypted to boot into recovery?

When you want to erase everything it is still possible using fastboot, though it would make it more difficult to access your data unauthorized.
[OnePlus 3 & 3T Cross Device Development] ΦDroid:
I think the main argument against password protecting TWRP itself is that the security benefits of doing so would be limited. With no encryption, someone who steals your phone could easily install a modified version of TWRP with no password requirement to access /data if desired. Its like a gate where you can easily reach your arm around and open the latch from the outside.

As far as USB access goes, I think it would be wise to not plug your phone into computers you don't trust. I think this issue is not common within the community and thus the convenience of the community not having to take extra steps to mount an unencrypted data partition outweigh the desires of the few.

Regards
[OnePlus 3 & 3T Cross Device Development] :
Yes. That's why data should be encrypted and password protected. That's no twrp problem. If twrp would offer an own password, that wouldn't secure anything, because I can build my own recovery without... ;) The only way to protect your data if someone else has physical access is encrypting and passphrase protecting the data partition. Runnimg data unencrypted is the sec flaw!