yashsingh1/xdaforums huggingface dataset:
[OnePlus 3 & 3T Cross Device Development] :
I completely understand. Best setup is locked boot-loader, adb switched off and encryption enabled.

Unfortunately I am one of those guys who want to switch ROM's sometimes, just to see latest developments, compare some stuff or help people with their questions. Encryption makes this more difficult as I have to format and copy data every time.

But I understand this is absolutely not a reason to change TWRP's it's behavior with some 'fake' security using an extra password... ;)

So, question answered by ΦDroid and Nvertigo67! :good:
[OnePlus 3 & 3T Cross Device Development] knpk13:
Unless the ROM you're testing doesn't support encryption (I can only think of NitrogenOS), you don't need to 'format' data to switch ROMs, just wiping data is enough.

On newer devices, I've encountered some hardened ROMs (like GrapheneOS or GlassROM) that claim to use a different kind of encryption method compared to other "normal" custom ROMs, which require you to format data/metadata with fastboot -w, but I don't think any of that applies to the OP3 series.
[OnePlus 3 & 3T Cross Device Development] :
Never knew that encryption could be used cross-ROM and cross-install. Good to know, thanks; is making life much easier... ;)
[OnePlus 3 & 3T Cross Device Development] knpk13:
Don't really know how it works tbh and all technical details, but it works. The only time in the OP3's history I can recall where encryption was a problem (not compatible with all ROMs) was the transition from the 5.0.8 to the 9.0.6 OOS firmware. Details are somewhere in the thread. Since OOS is EOL for OP3, I doubt encryption will be a problem again.

I haven't figured out yet whether file based encryption (FBE) that's used on newer devices works across ROMs or not. But full disk encryption (FDE) that the OP3 uses does.
[OnePlus 3 & 3T Cross Device Development] Johannes1098:
Just wanted to mention that this worked well, No problems as ist only copies the data files from the backup to /data :)
[OnePlus 3 & 3T Cross Device Development] BrahmaGandhi:
Hi, i want to ask are the two latest official twrp 3.3.1.-0 and 3.3.1-1 still have issue with /system/ partition backup restore? or i should kept using the 3.3.1-SAR version instead? because i have installation issue with

(openGL+vulkan drivers l zip file) using the SAR version, so i want to try maybe using the official -0 or -1 would work. any suggestion to use which one? thanks
[OnePlus 3 & 3T Cross Device Development] :
I don't know, I use system image for backup and havn't tested file based system backup for quite some time.

Apart from that I'd suggest to use the sar twrp for sar roms and fix the zip in question for sar twrp.
[OnePlus 3 & 3T Cross Device Development] BrahmaGandhi:
it seems switched back to official -1 solved the issue, and re-flashed back to SAR version (not a comfortable nor efficient ways to do it though)

ok, did it on the related thread already
[OnePlus 3 & 3T Cross Device Development] Johannes1098:
damn it, wrong tab - i'm sorry
[OnePlus 3 & 3T Cross Device Development] BillGoss:
TWRP 3.4 official is available. See https://twrp.me/site/update/2020/06/24/twrp-3.4.0-0-released.html
[OnePlus 3 & 3T Cross Device Development] Johannes1098:
So System as Root is now fully compatible and we can use it for LOS 17.1, am I right?

Or should I continue to use twrp-3.3.1-20200107..
[OnePlus 3 & 3T Cross Device Development] knpk13:
Looks like it should fix the OTA updater issue with encryption as well. I'm testing it right now to make sure.

Edit: it works
[OnePlus 3 & 3T Cross Device Development] BillGoss:
Yes, 3.4 can be used to flash Android 10 (SAR) ROMs and also Android 9 or earlier ROMs.
[OnePlus 3 & 3T Cross Device Development] :
That should be great! When that is possible I can use addon.d and make it even easier to update and cleanup! :good:
[OnePlus 3 & 3T Cross Device Development] Eric_Lev:
FYI : https://xdaforums.com/showpost.php?p=82918439&postcount=1657
[OnePlus 3 & 3T Cross Device Development] MJPollard:
Too bad that z31s1g stopped developing his TWRP Materialized theme, and never updated it for TWRP 3.3 and beyond. That's really the only reason why I never upgraded past 3.2.3 (I don't use decryption so I never really needed 3.3), as the stock theme, which does look better than it used to, still looks clunky and old. The stock theme really, really needs a complete renovation, and not just a paint job and a few tweaks, but that's just MHO.
[OnePlus 3 & 3T Cross Device Development] :
backup/restore system

Finally backing up and restoring system with tar (not system image) works. Tested with nlps-16.0 on op3t.

Has anybody tried restoring data from a pre-3.4.0.0 backup?
[OnePlus 3 & 3T Cross Device Development] BillGoss:
I tested using 3.4 to restore 3.3-SAR backups of both Los 16 and Los 17.
[OnePlus 3 & 3T Cross Device Development] mk_1996:
Yes. It works just fine
[OnePlus 3 & 3T Cross Device Development] BillGoss:
I've been using the material black play theme for many years across all versions of TWRP and it's been flawless.

The only features available in the later versions of TWRP that aren't shown in the material theme are:

- reboot into EDL

- select language

Everything else is there though sometimes in different places (eg File Manager is in Advanced in stock, but a main menu item in material). Switching between stock and material theme is a simple rename of /TWRP/theme/ui.zip and restarting the theme.
[OnePlus 3 & 3T Cross Device Development] StressFull:
Does anyone know if f2fs is well managed in this new version ? I am using Arter's kernel and he was recommending his own recovery, I wonder if I can update with this new version. Thanks.
[OnePlus 3 & 3T Cross Device Development] BillGoss:
I'm on f2fs and I don't have any issues with TWRP 3.4. But I haven't tried to format the /data partition.
[OnePlus 3 & 3T Cross Device Development] :
Superb update

Let's update my post then
[OnePlus 3 & 3T Cross Device Development] MJPollard:
Might give TWRP 3.4 a whirl on my spare 3T, then (the one I use for development, versus my main 3T). I wasn't 100% sure that Materialized would still work on later TWRP versions; I know that additional features (like the ones you pointed out) aren't going to be supported. Glad to hear that the additional ones aren't all that important for our general purposes. :good:
[OnePlus 3 & 3T Cross Device Development] BillGoss:
There's a bug in 3.4. If the current booted system is NOT SAR (eg, Los 16) then:

1. If a backup has the system partition files named system_root.* (like those created with TWRP SAR) then TWRP doesn't show the backup as containing the system partition.

2. If you restore a SAR rom (eg Los 17) backup (system backup files named system.*), then you cannot boot the system - TWRP says there's no OS.

The only way back is to either restore a system image of the SAR rom or flash the rom zip.

Note 1: if the current booted system is a SAR rom (eg Los 17) then TWRP will show that the system partition is available in a backup whether the files are named system.* or system_root.*
[OnePlus 3 & 3T Cross Device Development] dianlujitao:
twrp 3.4 names both sar and non-sar system backups as system.* so you can backup/restore both types, system_root.* was produced by my unofficial sar twrp and is no longer supported, you have to rename manually to be able to restore on twrp 3.4
[OnePlus 3 & 3T Cross Device Development] BillGoss:
I like that it's now standardizing on system.*. But you don't actually need to rename the files if you're on an Android 10 rom. In that scenario TWRP 3.4 will properly handle both system.* and system_root.* files. But if you're on an Android 9 rom it will only recognise system.*.

But that still leave the problem of not being able to restore and boot an Android 10 rom (SAR) if you've be using an Android 9 rom (point 2), which is the bigger issue.
[OnePlus 3 & 3T Cross Device Development] :
Is the system backup shown, if you did the backup for system as image? If so: does it restore and boot?
[OnePlus 3 & 3T Cross Device Development] BillGoss:
A system image backup is always shown in 3.4, irrespective of what the current booted system is, because the file name is system_image.*
[OnePlus 3 & 3T Cross Device Development] :
From which twrp version the backup created? 3.3 unofficial/TWRP SAR?

Or restoring a 3.4 /system backup SAR rom also fail if you're on PIE?
[OnePlus 3 & 3T Cross Device Development] BillGoss:
I tried both. It makes no difference.
[OnePlus 3 & 3T Cross Device Development] :
Flashing SAR rom before restoring might do the trick then

Which one of these works? sorted by the fastest

Flash SAR rom > wipe > restore the backup

Flash SAR rom > reboot to recovery > wipe > restore the backup

Flash SAR rom > reboot to system > reboot to recovery > wipe > restore the backup

Thanks for the heads up :good:
[OnePlus 3 & 3T Cross Device Development] MJPollard:
Huh. Guess I'm gonna hold off on installing 3.4 onto my main 3T, then. I'll wait until I see if/when this bug is fixed. (I did put it on my test 3T, but since I almost exclusively clean flash ROMs on that one for testing and development purposes, I haven't been bitten by this bug.)
[OnePlus 3 & 3T Cross Device Development] Anghirrim:
Do you mean that TWRP 3.4 can replace the Treble compatible TWRP fork?

As long as /vendor partition is not changed, the TWRP 3.4 can be used to flash GSI ROM's?

I referring to this thread : https://xdaforums.com/oneplus-3/one...eble-lineageos-15-1-treble-oneplus-3-t3830455
[OnePlus 3 & 3T Cross Device Development] dianlujitao:
I'm not sure but the possible workaround might be:

wipe system

leave system empty and reboot twrp

restore your pie backup
[OnePlus 3 & 3T Cross Device Development] BillGoss:
Firstly a correction: the problem is restoring Android Q, not Pie.

This workaround doesn't work (just tested). If 3.4 last booted Android 9, then it has to successfully boot Android Q before it will successfully restore an Android Q system backup, unless you restore an Android Q system image or flash an Android Q rom.

PS: why does TWRP 3.4 even care or know what the last version of Android that was booted?
[OnePlus 3 & 3T Cross Device Development] dianlujitao:
because of system-as-root, it's indeed a corner case bug.
[OnePlus 3 & 3T Cross Device Development] pedrovb25:
This TWRP has F2FS with GC fixes? Thanks!
[OnePlus 3 & 3T Cross Device Development] Johannes1098:
what is a GC fix?
[OnePlus 3 & 3T Cross Device Development] :
Garbage collection. For f2fs this GC includes resetting the physical memory cells. The mentioned fixes should address the write speed degredation on a used (>60%) f2fs.
[OnePlus 3 & 3T Cross Device Development] Johannes1098:
Ah, didn't hear about this yet.

And with ext4, the garbage collection isn't neccessary? Or is this already included?
[OnePlus 3 & 3T Cross Device Development] :
f2fs and ext4 are fundamentaly different at this point. ext4 doesn't need gc - the freeing of physical memory cells is done by fstrim. On android fstrim is run by vold when the device is idle.
[OnePlus 3 & 3T Cross Device Development] :
Hey guys,

I recently booted my old OP3 running a really old LOS and TWRP build. I'd like to update it to the latest OOS, but rather then deal with sideloading twrp.img, I'd much rather have a flashable zip. Can a twrp zip be downloaded somewhere?

Thanks
[OnePlus 3 & 3T Cross Device Development] Viper The Ripper:
Try to flash it through your TWRP as a .img file and selecting the recovery partition
[OnePlus 3 & 3T Cross Device Development] :
I haven't even thought of that. That went flawlessly.

Thank you! :good:
[OnePlus 3 & 3T Cross Device Development] DarkSJ1998:
While you're on the "flashing spree", I'd suggest you go for a custom ROM ;)

I recently flashed a custom ROM on my mom's 3T, which has been running OOS since the start & trust me, it's day & night difference between >= OOS 9.0.5 & custom ROMs in performance.

OP really underclocked the 3/3T in final updates...
[OnePlus 3 & 3T Cross Device Development] tfae:
I always flash a bootanimation zip after flashing ROMs.

The script worked fine on the previous version but on TWRP 3.4.0.0 the bootanimation is not flashed even though there are no errors when flashing:

ui_print("- Mounting /system...");
ifelse(is_mounted("/system"), unmount("/system"));
package_extract_file("mount-system.sh", "/tmp/mount-system.sh");
package_extract_file("unmount-system.sh", "/tmp/unmount-system.sh");
set_metadata("/tmp/mount-system.sh", "uid", 0, "gid", 0, "mode", 0755);
set_metadata("/tmp/unmount-system.sh", "uid", 0, "gid", 0, "mode", 0755);
run_program("/tmp/mount-system.sh") == 0 || abort("Could not mount /system");

ui_print ("OnePlus Boot Animation");
ui_print ("*****************************************");
ui_print ("Installing...");
package_extract_dir("system", "/system");

ui_print("- Setting permissions...");
set_metadata_recursive("/system/media", "uid", 0, "gid", 0, "dmode", 0755, "fmode", 0644, "capabilities", 0x0, "selabel", "u:object_r:system_file:s0");

ui_print("- Unmounting /system...");
run_program("/tmp/unmount-system.sh") == 0 || ui_print("Could not unmount /system");
ui_print("Installation complete!");
ui_print(" ");
set_progress(1.000000);

When flashing another zip that I use to debloat I get the famous ERROR 7

set_metadata_recursive: Error on lstat of "/system/app": no such file or directory

With this example script:

ui_print("- Mounting /system...");
ifelse(is_mounted("/system"), unmount("/system"));
package_extract_file("mount-system.sh", "/tmp/mount-system.sh");
package_extract_file("unmount-system.sh", "/tmp/unmount-system.sh");
set_metadata("/tmp/mount-system.sh", "uid", 0, "gid", 0, "mode", 0755);
set_metadata("/tmp/unmount-system.sh", "uid", 0, "gid", 0, "mode", 0755);
run_program("/tmp/mount-system.sh") == 0 || abort("Could not mount /system");

delete_recursive("/system/app/Browser","/system/app/Calendar");
ui_print("- Browser");
ui_print("- Calendar");

ui_print("- Setting permissions...");
set_metadata_recursive("/system/app", "uid", 0, "gid", 0, "dmode", 0755, "fmode", 0644, "capabilities", 0x0, "selabel", "u:object_r:system_file:s0");
set_metadata_recursive("/system/priv-app", "uid", 0, "gid", 0, "dmode", 0755, "fmode", 0644, "capabilities", 0x0, "selabel", "u:object_r:system_file:s0");
set_metadata_recursive("/system/etc/permissions", "uid", 0, "gid", 0, "dmode", 0755, "fmode", 0644, "capabilities", 0x0, "selabel", "u:object_r:system_file:s0");

ui_print("- Unmounting /system...");
run_program("/tmp/unmount-system.sh") == 0 || ui_print("Could not unmount /system");
ui_print("Installation complete!");
ui_print(" ");
set_progress(1.000000);

What have changed?
[OnePlus 3 & 3T Cross Device Development] :
Boot twrp, run the mount script from terminal (check for errors; it's helpfull to use "set -x"), and check each path you are using in the flash script.
[OnePlus 3 & 3T Cross Device Development] Johannes1098:
I think it's because /system_root

I'm using this script and it's working fine with TWRP 3.4 & los 17

show_progress(0.020000, 10);
ui_print(" ");
ui_print("________________________________");
ui_print(" ");
ui_print("|------------------------------|");
ui_print("| Script by Johannes1098 |");
ui_print("|------------------------------|");
ui_print("| Mounting system |");
mount("ext4", "EMMC", "/dev/block/bootdevice/by-name/system", "/system_root");
ui_print("| |");
ui_print("| Enable Channel |");
ui_print("| Bonding on 2.4GHz |");
delete("/system_root/system/vendor/etc/wifi/WCNSS_qcom_cfg.ini");
package_extract_dir("system_root", "/system_root");
set_perm(0, 0, 0644, "/system_root/system/vendor/etc/wifi/WCNSS_qcom_cfg.ini");
ui_print("| |");
ui_print("| Installing YouTube Vanced |");
set_perm_recursive(0, 0, 0755, 0644, "/system_root/system/app/YouTube");
ui_print("| |");
ui_print("| Installing Bootanimation |");
set_perm(0, 0, 0644, "/system_root/system/media/bootanimation.zip");
ui_print("| |");
ui_print("| Unmounting system |");
unmount("/system_root");
set_progress(1.000000);
ui_print("|------------------------------|");
ui_print("| Success! |");
ui_print("|------------------------------|");
ui_print("________________________________");
ui_print(" ");
[OnePlus 3 & 3T Cross Device Development] tfae:
I'll have to test it when I find some time. :good:
[OnePlus 3 & 3T Cross Device Development] Joshmccullough:
Looks like a new version is out, but trying to download the .img from the website only downloads a .bin file.... Anyone knows where to find the 3.5 img?
[OnePlus 3 & 3T Cross Device Development] BillGoss:
I had no troubles downloading the .img file from https://dl.twrp.me/oneplus3/
[OnePlus 3 & 3T Cross Device Development] Joshmccullough:
Must've not liked mobile Firefox, it downloaded just fine on mobile Chrome. I should've thought about trying that first before posting....🙄
[OnePlus 3 & 3T Cross Device Development] BillGoss:
You might find that FF downloaded the file ok but didn't rename it properly. Verify that the md5 check sum of the .bin file is the same as the .img file.
[OnePlus 3 & 3T Cross Device Development] Joshmccullough:
I hadn't thought of that....I just flashed it fine with the Chrome-downloaded file, works like a charm.
[OnePlus 3 & 3T Cross Device Development] knpk13:
Haven't been using XDA since the new app came out, which is pretty cumbersome to navigate.

Saw TWRP 3.5 was out, but when I flashed it and rebooted to recovery I just got sent to fastboot. I flashed 3.4 again which worked, but noticed something strange. I am encrypted, but.. somehow TWRP did not ask me for my encryption pattern and took longer to boot and said that it had successfully decrypted using the default password. That is pretty weird. I haven't used my 3T much recently so I definitely have not made any major changes. Just using LOS 17 and updating it monthly. Is my filesystem dying/corrupted or did I screw something up? The system itself seems to be working fine and all my data is present. But I may have to clean flash to check and fix this if no one else is having this issue.

Edit: my issue probably isn't related to 3.5, because I remembered that TWRP stopped asking me for my encryption pattern while using the LOS updater at some point. I just don't understand if and how/why my encryption pattern got borked.
[OnePlus 3 & 3T Cross Device Development] moscic:
You can fix it by setting your password/pin again and requiring an on boot password.
[OnePlus 3 & 3T Cross Device Development] knpk13:
Ok that fixed the pattern requirement issue for me on 3.4, but 3.5 is still not working, it boots to fastboot.
[OnePlus 3 & 3T Cross Device Development] moscic:
I updated the recovery via the official twrp app and it works just fine. No issues.
[OnePlus 3 & 3T Cross Device Development] Nixeus2:
Hi, is there any changelog about this 3.5 twrp build ? Thanks
[OnePlus 3 & 3T Cross Device Development] moscic:
TWRP 3.5.0 Released

TWRP 3.5.0 is out now for most currently supported devices.

twrp.me
[OnePlus 3 & 3T Cross Device Development] BillGoss:
TWRP 3.5.1 is now available. See https://twrp.me/site/update/2021/03/17/twrp-3.5.1-released.html
[OnePlus 3 & 3T Cross Device Development] :
Can anybody confirm that zip sigmature verification with sha265 hash in a *.sha256sum file doesn't work anymore in 3.5.1_9-0?

If so: is this a regtession or intended?

Most roms come with md5 hash (wich is still working), but los only supploes a sha256 hash on their download page).

I suggest fixing (or readdimg) sha256 in the next release!
[OnePlus 3 & 3T Cross Device Development] BillGoss:
I was under the impression that the TWRP zip signature verification uses the digest information inside the zip file under META-INF (CERT.*, MANIFEST.MF), not the checksum for the zip file.

I've always used either the checksum feature in my file manager (Solid Explorer) or run xsum (where x is md5 or sha256) in a terminal.
[OnePlus 3 & 3T Cross Device Development] :
I'm talking of the twrp feature "Zip signature verification" (checkbox on the flash page). To use this feature you need a signed zip. Signing a zip will add - at least - META-INF/com/android/otacert. But flashing a properly signed zip with this checkbox checked will fail, as long as no checksum hash file is given (i.e. for hello_bill.zip it's hello_bill.zip.md5sum or hello_bill.zip.sha256sum in the same directory).

This behaviour makes sense to me, to avoid faked/spoofed/altered signatures.

Up to twrp-3.5.0_9-0 this works with md5sum as well as for sha25usum. Since twrp-3.5.1_9-0 only md5sum works.

The attached zip contains:

unzip -l hello_bill
Archive: hello_bill.zip
Length Date Time Name
--------- ---------- ----- ----
0 03-23-2021 09:28 hello_bill/
1026 03-23-2021 09:04 hello_bill/hello_bill-unsigned.zip
58 03-23-2021 09:07 hello_bill/hello_bill-unsigned.zip.md5sum
81 03-23-2021 09:08 hello_bill/hello_bill.zip.sha256sum
90 03-23-2021 09:08 hello_bill/hello_bill-unsigned.zip.sha256sum
49 03-23-2021 09:07 hello_bill/hello_bill.zip.md5sum
3175 03-23-2021 09:05 hello_bill/hello_bill.zip

The contained flashable zips (hello_bill-unsigend.zip and hello_bill.zip) don't alter anything, they just print "Hello Bill!" to the twrp console.

As you explained above, the unsigned zip will not flash at all with "Zip signature verification" checked.

The signed zip (hello_bill.zip) will fail to flash with no checksum file in the same directory.

The signed zip will flash in twrp-3.5.0_9-0 with sha256 xor (one at a time) md5.

The signed zip will flash in twrp-3.5.1_9-0 with md5, but fails with sha256.

Of course one can check the checksum outside twrp, or on the trp terminal. But using the twrp "Zip signature verification" checkbox is much more comfortable and saves each time about 10 seconds of my time compared to manually checking on the console.
[OnePlus 3 & 3T Cross Device Development] BillGoss:
@nvertigo67 Thanks for the info and test zips. I'd never noticed this feature before - cool!. And, yes, sha256 is broken in 3.5.1 but works in 3.5.0
[OnePlus 3 & 3T Cross Device Development] :
Thanx for testing and feedback.

@dianlujitao (though I doubt it's device specific):

Issue report created: https://github.com/TeamWin/Team-Win-Recovery-Project/issues/1602
[OnePlus 3 & 3T Cross Device Development] YechiamTK:
Hi,

I flashed the latest version (3.5) and usb otg doesn't work. Started going back, 3.4 - didn't work either.

Now I'm trying 3.0.0.2, but it nonetheless seem odd that on both these versions it didn't work.

The otg itself is fine (the phone detects it normally), so I'm not sure what's wrong.
[OnePlus 3 & 3T Cross Device Development] Not-Aus:
I had the same problem, on the fourth computer it worked.
[OnePlus 3 & 3T Cross Device Development] DarkSJ1998:
I assume you tried toggling the "Enable MTP" button as well as different USB cables?
[OnePlus 3 & 3T Cross Device Development] YechiamTK:
Enable MTP where? In recovery? Didn't see such option. I tried in mount screen and pressing the OTG did nothing (and it didn't detect the OTG). And the cable works just fine, the phone itself detects it (while not in recovery).

Edit: I see Enable MTP. It is enabled.

Also, I should note that my USB thumb drive lights up when it's activated. When I click on "mount storage" or am trying to mount it, it lights up. But nothing happens.
[OnePlus 3 & 3T Cross Device Development] DarkSJ1998:
Okay, yeah I meant the same button. It might have been renamed or I forgot what it was actually called, sorry for that :p

Actually I've had a weird experience with a cable before, it used to work totally fine in every scenario, except when I did an "adb sideload". It used to get disconnected in between in case of sideloads, but worked perfectly fine otherwise. Just try once with any other cable to double check.