yashsingh1/xdaforums huggingface dataset:
[OnePlus 3 & 3T Cross Device Development] Flounderist:
I downloaded the zip but I will test it later on. At the office at the moment.
[OnePlus 3 & 3T Cross Device Development] Dirk:
While that sounds great, i personally need to see some complete Mod zips using that method to understand how it all fits together in practice. (I couldn't make one from scratch but i can reverse engineer like a champ!) ;)

Thanks. Really appreciate it. If it works on 3.3.0-1, don't forget to attach your own Bootanimation and i'll edit it the same way for you.
[OnePlus 3 & 3T Cross Device Development] :
As mentioned in my longish posting, you can use addonsu-16.0-arm64-signed.zip as starting point.
[OnePlus 3 & 3T Cross Device Development] Johannes1098:
I'm using "mount("ext4", "EMMC", "/dev/block/bootdevice/by-name/system", "/system");" for mounting and it works fine in the 3.3 version :)

Edit: and as well on all TWRP versions since I own the OP3..
[OnePlus 3 & 3T Cross Device Development] :
That's what I refered to as "edify buitin mount command". As explained this is perfectly ok, if you know that all target devices for your zip have partition type EMMC and filesystem type ext4. I.e. if your zip is only for op3/t this will work (strictly systematically you should asset on the device type before trying to mount system), but if you want to make a generic zip with working mount on all devices, the mount/umount scripts are the way to go.

Best of all: the scripts give an error output to the twrp console, if mounting system fails. [MENTION=1763460]Dirk[/MENTION] would have seen on the console what [MENTION=2911853]BillGoss[/MENTION] found in [MENTION=7193743]Flounderist[/MENTION]'s logs (though with the mount/umount scripts the error wouldn't have happend in the first place.
[OnePlus 3 & 3T Cross Device Development] Dirk:
You wouldn't think there were so many ways to do the same thing! Alternative options may yet be useful though, so thanks! :good:

Attaching first effort. Seems easy enough to adapt for adding/overwriting files. Would of course need examples to get syntax and structure for additional things, but it's a very useful start! I assume the 'otacert' file is the same in all cases and just needs to be there?

Haven't signed the example.zip. Don't know if signing is necessary or if so, what method? Usually i use Zip Signer app on 'test keys'.
[OnePlus 3 & 3T Cross Device Development] :
You can throw away the otacert. The signing is generating the cert files. Signing isn't necessary. Signing with test keys is a bit like securing your homes door with a lock bar accessible from both sides... ;) You can do so, but you have not to do so... ;)
[OnePlus 3 & 3T Cross Device Development] Flounderist:
Sorry for being late at the party again but I had a crazy day.

The boot animation works, it's funny to point out I made the same for my Nexus9 a couple of months ago. :D

Using the apk to load some stuff is a nightmare. I will re-try later on. :silly:
[OnePlus 3 & 3T Cross Device Development] Dirk:
Great news, thanks!

Yeah, that Marshmallow/Nexus/Stock Animation is a keeper. Never found one better!
[OnePlus 3 & 3T Cross Device Development] Flounderist:
Hi Dirk,

here are the zips that I usually flash on my device, time by time. Feel free to modify the updater script as you want.

p.s. how I can get rid of the Google Boot logo? I noticed that dirt flashing a new ROM does not work.

edit: I sorted flashing the stock op3t boot-logo again.
[OnePlus 3 & 3T Cross Device Development] Dirk:
Here you go buddy. Done.

Strange about the Boot Logo. Every time i flash anything, the damn thing gets overwritten and i have to flash the mod again. Looks like you got it sorted anyway, but for future reference the file is 'logo.bin' and the directory it's written to is: /dev/block/bootdevice/by-name/LOGO.
[OnePlus 3 & 3T Cross Device Development] Flounderist:
I used the default logo.bin provided here: https://xdaforums.com/oneplus-3t/themes/mod-custom-boot-logo-pack-op3t-t3539019, by the way it doesn't look like the original one :(

Is there any way to restore the original?
[OnePlus 3 & 3T Cross Device Development] Dirk:
I've never seen that thread before!

(If you're wondering why that zip worked for you when the others failed, the Boot Logo mod doesn't need to Mount anything to do it's job).
[OnePlus 3 & 3T Cross Device Development] Flounderist:
I saw, but the logo doesn't look like the original one :( is there any way to get the original one back?
[OnePlus 3 & 3T Cross Device Development] Dirk:
I don't know what the original is but i'm sure it can be found on the forum. Try the Android Themes section. What ROM are you using? Delete the mod 'logo.bin' using TWRP File Manager and dirty flash ROM should work.

/dev/block/bootdevice/by-name/LOGO
[OnePlus 3 & 3T Cross Device Development] Flounderist:
I'm on Unofficial LOS16. The file LOGO is stored in:/dev/block/by-name/LOGO, while bootdevice is stored in /dev/block/bootdevice.

Which file must be deleted?
[OnePlus 3 & 3T Cross Device Development] Dirk:
I've been looking and i'm not sure you can access it directly. I don't know why dirty flashing ROM again wouldn't restore the original. Always works for me.
[OnePlus 3 & 3T Cross Device Development] Flounderist:
Reflashed ROM and custom kernel but nothing changed. The boot logo image looks smaller than the original one, btw it could be a placebo effect. I can survive with it.

We went really OT.
[OnePlus 3 & 3T Cross Device Development] :
/dev/block/bootdevice/by-name/LOGO is a very special file:

OnePlus3T:/ $ ls -la /dev/block/bootdevice/by-name/LOGO
lrwxrwxrwx 1 root root 16 1972-04-30 00:26 /dev/block/bootdevice/by-name/LOGO -> /dev/block/sde17
OnePlus3T:/ $ ls -la /dev/block/sde17
brw------- 1 root root 259, 1 1972-04-30 00:26 /dev/block/sde17
OnePlus3T:/ $

The symlink point to a special file representing a complete block device, which represents a partition. You can delete a partition, but you don't want that, belive me. (see fdisk if you inisist on wanting to delete sde17)

If you want the original bootlogo just reflash a the firmware package, you are using. If you are on op3t and doesn't use open beta firmware, you can use these: https://github.com/nvertigo/firmware-3t/tree/master/archive
[OnePlus 3 & 3T Cross Device Development] Dirk:
OMG, thanks you!

I've been tying to hunt down this sucker for hours! So how does a Firmware package restore an original logo.bin file? There's one in the package, but what is it? Where in a ROM zip would you find the ROM Boot Logo?

Yes, we did. :D

So nvertigo67 had the answer in case you missed it. Reflash Firmware!
[OnePlus 3 & 3T Cross Device Development] :
In OOS flashable zip it's in zip-root/firmware-update/logo.bin.

It's flashed by edify command

package_extract_file("firmware-update/logo.bin", "/dev/block/bootdevice/by-name/LOGO");

I'm pretty sure, you'll find that in all fw packages, too.

EDIT: if you need only the original logo.bin: https://github.com/nvertigo/firmware-3t/raw/master/firmware-update/logo.bin (if you look at the files history, it hasn't changed since ob22/ob31; it's the same file contained in the oos 5.0.8 zips from OnePlus)

---------- Post added at 15:12 ---------- Previous post was at 14:59 ----------

Not really: understanding devfs, block devices, partitioning, the content of some partitions and the difference of package_extract_file if targeting a file in a mounted fs, or an unmounted block device is necesarry to build flashable (working and compatible) zips for twrp... ;) sde17 and logo.bin are just examples.. ;)
[OnePlus 3 & 3T Cross Device Development] Flounderist:
As per good practice I always kept a flashable firmware in my download folder. Flashing latest stable 5.0.8 did the trick! :D Thank you!
[OnePlus 3 & 3T Cross Device Development] :
[MENTION=7193743]Flounderist[/MENTION], [MENTION=1763460]Dirk[/MENTION]:

Since the both of you are getting familar with zips flashable with twrp you may like to create your own boot logo and flash it with twrp: https://xdaforums.com/oneplus-3/themes/mod-splash-screen-image-injector-t3441999

Example: https://github.com/nvertigo/firmware-3t/raw/master/archive/nlos_bootlogo-v0.1.zip (unpacked and repacked with op3injector)
[OnePlus 3 & 3T Cross Device Development] Dirk:
Thanks! I've had a brainwave and will have a crack at making one. :good:

Edit: [MENTION=3456089]nvertigo[/MENTION]

Lol, it worked. I call it 'Dark Side Of The Android'. (The iconic Pink Floyd cover, but with 'Google coloured' rainbow).

(Flashable zip and preview attached).

I feel like i was given homework and am happy to hand it in completed! :D
[OnePlus 3 & 3T Cross Device Development] Flounderist:
Thank you :) I was aware of this program but, to be honest, I don't think I will dig more into it.

Basic 1+ bootologo is good enough for me :)

More likely, I will try to move the zip on a magisk module, as I already did for another device (Sony xPeria Z) :D
[OnePlus 3 & 3T Cross Device Development] Doc Sniper:
Getting some toybox related errors while executing a recovery script with TWRP 3.3.0:

Why is toybox trying to use /system/lib/liblog.so even if there is a /system/lib64/liblog.so ?

is this a know bug ?

Edit: toybox seems to be 64 bit:
[OnePlus 3 & 3T Cross Device Development] :
A dynamically linked binary isn't very usefull on a system without /system. / system is an empty mountpiont. Since twrp doesn't know if / system is mounted or not, it should be linked statically. The same goes for the (twrp) linker: it shouldn't search under /system, because the list of directories with shared libs would differ, if system is mounted.

The multicall binary (no matter if toybox or busybox) should be linked statically. The linker shouldn't search /system/ for shared objects.

---------- Post added at 08:59 ---------- Previous post was at 08:54 ----------

Awesome!

That's A+! :) :) :)
[OnePlus 3 & 3T Cross Device Development] Doc Sniper:
Thanks for your hint with dynamically linked.

Then I wait for a fix ... :fingers-crossed:
[OnePlus 3 & 3T Cross Device Development] :
But the hint is from your posting... :)
[OnePlus 3 & 3T Cross Device Development] Doc Sniper:
Yes I know, but it was late at night and all the gin and tonic ... ;) :silly:

---

Edit: Found the miss linked lib in: sbin

sbin/liblog.so: ELF 64-bit LSB shared object, ARM aarch64, version 1 (SYSV), dynamically linked, BuildID[md5/uuid]=6fb7d1a0f0392f871fb5a79e3154e368, stripped
sbin/toolbox: ELF 64-bit LSB shared object, ARM aarch64, version 1 (SYSV), dynamically linked, interpreter /sbin/linker64, BuildID[md5/uuid]=07d7b079c612733da07742538be21af5, stripped

Toybox, TWRP and Linker64:

sbin/toybox: ELF 64-bit LSB shared object, ARM aarch64, version 1 (SYSV), dynamically linked, interpreter /sbin/linker64, BuildID[md5/uuid]=eaa2ea579c6203fcc5315a9ab70b92b5, stripped
sbin/twrp: ELF 64-bit LSB shared object, ARM aarch64, version 1 (SYSV), dynamically linked, interpreter /sbin/linker64, BuildID[md5/uuid]=fd36bee6353bacbf613dd1867be9ea2c, stripped
sbin/linker64: ELF 64-bit LSB shared object, ARM aarch64, version 1 (SYSV), dynamically linked, BuildID[md5/uuid]=0ce1ada9dcd41254b91935801e32c139, not stripped
[OnePlus 3 & 3T Cross Device Development] :
Yep. And as long as /system/ isn't mounted the libs from /sbin/ are used.

BTW: historically the two dirs on unix like oses have the meaning:

bin - compiled binaries, dynamically linked

sbin - compiled binaries, statically linked (Satically linked Binaries; mostly system near programs needed to be executed at early boot time, when i.e. /usr (containing /usr/lib) isn't mounted, yet.)

If the multi call binary (here toybox, but also true for busybox) is dynamically linked (I would recommend statically linkage to make the multicall binary working, no matter what), it's important to put the dir with the needed libs (here: /sbin) in LD_LIBRARY_PATH before everything under /system/. Because the first name matching lib is usesd (here: /system/lib/liblog.so), which breaks toybox, as soon as system is mounted.
[OnePlus 3 & 3T Cross Device Development] Doc Sniper:
You are right, sbin contains binaries necessary for booting and for the superuser user (only).

But AFAIK the S in SBIN means SuperUser. Anyway, this isn't worth a discussion. ;)

Back to the facts, I booted recovery and had a look at LD_LIBRARY_PATH before and after mounting system, a screenshot is attached.

I both cases the LD_LIBRARY_PATH points to /sbin ... :confused:
[OnePlus 3 & 3T Cross Device Development] :
"superuser" wan't known when /sbin/ was invented. su was implemented AFTER dynamic linkage (and therfor after sbin was invented). Belive me, on pdp 7 (where one of the first unix like system was implemented by Ritchie et al.), it was called /sbin because of static binaries.

Cool finding. Then /system is hardcoded to the linker. Perhaps it's needed to be hard coded for the android system (twrp is build against the linker code of omni). In that case, building toybox statically or changing the order in the linker are the two options to fix this.

As temporary workaround I'd suggest to put a static busybox within the zip, package_extract_file it to /tmp, use metadata to make it executable, and use "/tmp/busybox" as first argument of run_program.

Considering the time [MENTION=5091734]dianlujitao[/MENTION]'s format fix needs to be found and merged, I doubt this will be fixed soon. The workaround won't stop to work, if this gets fixed (someday...).
[OnePlus 3 & 3T Cross Device Development] Doc Sniper:
Yes maybe, many things are rededicated. ;)

Made a comparison of busybox and toybox, unfortunately couldn't find a path, but toybox uses three more linked libs, And one of them is the "missing" liblog.so

TWRP 3.2.3-10 - busybox:
libcutils.so
libc++.so
libc.so
libm.so
libdl.so

TWRP 3.3.0-1 - toybox:
liblog.so
libcutils.so
libselinux.so
libz.so
libc++.so
libc.so
libm.so
libdl.so

Additions:
liblog.so
libselinux.so
libz.so

And I saw that LD_LIBRARY_PATH is set on boot in init.rc, so you're probably right that it's statically set.

on init
export PATH /sbin:/system/bin
export LD_LIBRARY_PATH /sbin
[OnePlus 3 & 3T Cross Device Development] BillGoss:
[MENTION=5091734]dianlujitao[/MENTION]

I've been playing around with the community OOS 9.0 beta and came across a problem with TWRP 3.3.0.1.

If I restore the system partition from a backup the phone will not boot. However, if I use TWRP 3.2.3.1 or your fix-3 version to restore the system partition then the phone boots.

Please see this post in the Los 16.0 thread https://xdaforums.com/oneplus-3/one...ageos-16-0-oneplus-3-3t-t3866517/post79379172

I've attached TWRP logs of the system restore process for both TWRP 3.3.0.1 and fix-3, in case it's of any help.

Edit: I've also attached the /sys/fs/pstore files taken after the failed boot. All the attachments are in the next post below.

I realise also that this happens on an OOS beta version so I wouldn't really expect you to look at the problem until OOS 9.0 goes stable. But I'm posting this so that others can know about the issue.
[OnePlus 3 & 3T Cross Device Development] BillGoss:
And here are the logs.

Edit: I've also attached the /sys/fs/pstore files taken after the failed reboot.
[OnePlus 3 & 3T Cross Device Development] Eric_Lev:
Unless I'm mistaken, the two log files are identical... Right?
[OnePlus 3 & 3T Cross Device Development] BillGoss:
I couldn't see any significant differences.
[OnePlus 3 & 3T Cross Device Development] Doc Sniper:
Seems like that, I couldn't see any relevant differences.

---------- Post added at 10:21 ---------- Previous post was at 10:00 ----------

@BillGoss: Did you try to restore a backup with 3.3.0, which was created with 3.2.3 and vice versa?
[OnePlus 3 & 3T Cross Device Development] Dirk:
I think we've seen both cases in LOS. The former fails and latter works. It's just 3.3.0 that can't boot after restoring. Not just that it seems, but also some Clean Installs failed to Boot on 3.3.0-1, but when 3.2.3-10 was used worked fine. (I assume the install procedure was done the same way on both builds).

[MENTION=3404276]kenboyles72[/MENTION] and [MENTION=7193743]Flounderist[/MENTION] both had this issue.
[OnePlus 3 & 3T Cross Device Development] :
Prior to the next fixing, may you check the reliability of backup/restore keys (pin/pass/pattern/fingerprint) in twrp backup functionality for the ease of use and also an ability to clear POB without the need to decrypt /data (use format /data function) if the user wipe internal storage or change FS.

[MENTION=5091734]dianlujitao[/MENTION]
[OnePlus 3 & 3T Cross Device Development] BillGoss:
I wouldn't know which version a backup was created with. I've been using fix-3 mainly but switched recently to 3.3.0.1. All I know is that no system backup restored with 3.3.0.1 boots up. I either have to flash the rom zip again or just use fix-3.
[OnePlus 3 & 3T Cross Device Development] :
Just a thought:

twrp offers two options to backup/restore system:

The first in the list is "System": this is basically a tar backup.

Additionally twrp offers "System (image)": this is a dd of the system partition.

[MENTION=227868]Chief[/MENTION]_tester_in_command, ähh, want to say [MENTION=2911853]BillGoss[/MENTION]: Can you test the second seperatly, please. If I havn't overseen it in the log, you've just tested the first - correct me, if I'm wrong.
[OnePlus 3 & 3T Cross Device Development] DarkSJ1998:
Correct me if I'm wrong. As per the experiences of a lot of people, people are unable to boot up on a TWRP backup restored by v3.3.0-1 right? Can it be possible that the new v3.3.0-1 isn't able to restore the /boot completely which causes the bootloop? Or it's the /system which isn't restored completely that causes the bootloop?

---------- Post added at 03:17 PM ---------- Previous post was at 03:14 PM ----------

I've never modified the partition types on my 3T. So I hope, this makes me a f2fs user right? As far as I remember, the 3/3T use f2fs partitions as stock.
[OnePlus 3 & 3T Cross Device Development] :
You can't format data with f2fs with 3.2.3-0. That's If you ever use the format data button on the main wipe page, you have changed your data partition to ext4, and you can't change back to f2fs with 3.2.3-0(*).

Why don't you check, which fs you are actually using, instead of making assumptions from what you think you've done or not done?

(*) as long as you don't know what you are doing
[OnePlus 3 & 3T Cross Device Development] DarkSJ1998:
And,

fastboot format userdata

formats it to f2fs right?

Ohh yeah, sorry my bad, totally forgot about it.

Thanks for your time though :)
[OnePlus 3 & 3T Cross Device Development] BillGoss:
I created a system image using 3.3.0.1 and then restored it using the same version.

...

And it worked!

This approach avoids two problems on :

1. The failure to restore a "normal" backup of the system partition of OOS 9.0 beta (all tested TWRP versions fail trying to restore)

2. Failure to reboot after restoring the system partition on 3.3.0.1

Edit: but this causes me problems because I do all my backups and restores using scripts (ORS) and the script commands don't allow for using system images in backup/restore

3.3.0.1 is fine with restoring /boot. I know because I use Magisk and the process I've been using is: restore /data, flash rom (/system, /boot), restore /boot (patched by Magisk)

Also, the test I just did using system image restored the backed up /data, /boot and system image and this worked.
[OnePlus 3 & 3T Cross Device Development] :
To my knowledge "fastboot format" honors the existing fs type. I can be overwritten by using the options described by "fastboot help":

format[:[<fs type>][:[<size>]] <partition>
Format a flash partition. Can
override the fs type and/or size
the bootloader reports.

You can also manually make a new f2fs within twrp-3.2.3-0, as long as you know, what you are doing...
[OnePlus 3 & 3T Cross Device Development] Eric_Lev:
I don't understand ... but I must have missed something... ?

I performed a Nandroid backup (boot+system+data) with 3.3.0-1 and restoring this backup with 3.3.0-1 fails but works fine with 3.2.3-10 on my side.

Same issue if the backup is performed with 3.2.3-10.It fails to be restored with 3.3.0-1.
[OnePlus 3 & 3T Cross Device Development] :
https://xdaforums.com/showpost.php?p=79380422&postcount=630
[OnePlus 3 & 3T Cross Device Development] Dirk:
As above. System Image was added as an option to preserve dm-verity, for people who might want to create and restore a clean backup so that they can OTA official ROMs.

Codeworkx has released his 3.3.0 build on the OP5. I've updated mine and taken a traditional Nandroid and a System Image for good measure. Nobody there has reported any issues yet, although it's early days yet. Nothing on the TWRP Issue Tracker either:

https://github.com/TeamWin/Team-Win-Recovery-Project/issues
[OnePlus 3 & 3T Cross Device Development] Doc Sniper:
Small partial success. After a restore with TWRP 3.3.0-1, I set the SELinux rights back to default and the system booted successfully. However, only up to the logo, here it unfortunately gets stuck in an endless loop. Will continue my research ...
[OnePlus 3 & 3T Cross Device Development] :
Bootlogo or bootanimation?
[OnePlus 3 & 3T Cross Device Development] Doc Sniper:
The boot animation, the cyan colored and animated LOS logo from /system/media/bootanimation.zip
[OnePlus 3 & 3T Cross Device Development] :
Then you can try to get a logcat (if adb is allowed and the key of your pc is regostered prior to the backup you try to restore). And/or get the pstore on the very next reboot.
[OnePlus 3 & 3T Cross Device Development] BillGoss:
[MENTION=5091734]dianlujitao[/MENTION], [MENTION=4405529]nvertigo67[/MENTION]

I've updated my post on restored backups using TWRP 3.3.0.1 on OOS 9.0 community beta firmware failing to boot https://xdaforums.com/oneplus-3/one...ial-twrp-touch-recovery-t3742894/post79379645

I've now included /sys/fs/pstore files taken after a failed reboot.
[OnePlus 3 & 3T Cross Device Development] BillGoss:
[MENTION=5091734]dianlujitao[/MENTION],

Just letting you know that I've raised an issue on TWRP for the bug that breaks the "wipe system" command in the latest official version.

https://github.com/omnirom/android_bootable_recovery/issues/254

I first raised the issue in this thread here https://xdaforums.com/oneplus-3/one...ial-twrp-touch-recovery-t3742894/post79046819

And, talking about TWRP issues, does the fact that 3.3.0-1 includes your fixes for formatting /data, does that mean that the fix is now officially part of TWRP and I can close the following issue? https://github.com/omnirom/android_bootable_recovery/issues/210
[OnePlus 3 & 3T Cross Device Development] :
Yes: https://gerrit.omnirom.org/#/c/android_bootable_recovery/+/33021/
[OnePlus 3 & 3T Cross Device Development] Doc Sniper:
Tried it, but wasn't able to get a log or pstore for this case :(

Got just only the pstore for a reboot directly after the restore, which ends with reboot to bootloader (attached as ZIP).

After resetting the selinux rights on system the phone boots up to the bootanimation and stucks there, but no adb available or pstore after a (forced) reboot.
[OnePlus 3 & 3T Cross Device Development] kamikaze666:
"Signed to work with locked bootloader" - do you mean that we can lock the bootloader after installing TWRP and a signed ROM image, like Lineage, for example?
[OnePlus 3 & 3T Cross Device Development] Eric_Lev:
Please read this post: https://xdaforums.com/showpost.php?p=79351881&postcount=1765
[OnePlus 3 & 3T Cross Device Development] kamikaze666:
Thank you, I appreciate it :)
[OnePlus 3 & 3T Cross Device Development] Dirk:
Can someone tell me what the 'System Image' option actually backs up? Is it just /system as the name suggests or is it an image of everything together?
[OnePlus 3 & 3T Cross Device Development] :
It's basically a disk dump of the system partition (dd man page, i.e.: http://man7.org/linux/man-pages/man1/dd.1.html).
[OnePlus 3 & 3T Cross Device Development] knpk13:
if i want to start over with a completely clean filesystem, how do i do it? theres a lot of conflicting information about what works and what doesnt. can i use the latest official twrp (3.3.0-1) to format data, or should i fastboot format userdata?