yashsingh1/xdaforums huggingface dataset:
[Android Software/Hacking General [Developers Only]] osm0sis:
No no, he's talking about custom ROMs, where OTAs are full, not incremental.

He did seem a bit confused while talking about my TWRP Retention zip though, which is specifically for A/B stock ROMs since they don't have addon.d-v2; I have an actual TWRP A/B addon.d-v2 script for custom ROMs.
[Android Software/Hacking General [Developers Only]] Nikhil:
No guys, may be I'm unable to explain properly (please bear with me)

I understand the concept of addon.d and how the files are backed up and restored. I have it perfectly implemented for my gapps and it works when a rom is dirty flashed. The scenario that I am talking about is the OTA update (specifically for my device I am starting to believe now).

What happens with stock oos for my oneplus 7 pro also happens for Resurrection Remix. The new update is installed to an inactive slot, after an ota update, we install magisk to inactive slot before reboot. There is no support for backuptools because it breaks safetynet.

Please check following posts as references.

https://xdaforums.com/oneplus-7-pro/development/rom-resurrection-remix-t4103115/post82651243

https://xdaforums.com/oneplus-7-pro/development/rom-resurrection-remix-t4103115/post82656097

I intend to work on a solution which is independent of backuptools. Which is why, i asked, once the ota update is installed through updater and we flash magisk to inactive slot to retain root, can we also copy the google apps (apks) to inactive slot so after reboot, when the slot is switched, the gapps are preserved.
[Android Software/Hacking General [Developers Only]] osm0sis:
This doesn't make sense, if this is stock ROM, there would be no GApps that need backing up, or if this is a custom ROM then it should have addon.d. I pass SafetyNet perfectly well in my OP7Pro on Lineage 17.1 with working addon.d-v2 for GApps, TWRP and Magisk.

To otherwise answer you though, yeah you can do whatever you want before you reboot after an A/B OTA. Why wouldn't you? For example on my 8 Pro I allow an OOS OTA to flash, then dump boot and recovery slots, then flash TWRP recovery.img to both slots, then Magisk to inactive, then reboot.
[Android Software/Hacking General [Developers Only]] Nikhil:
Yes, the expectation was that addon.d (backuptools) kicks in and does the needful backup and restoring.

It wasn't happening on RR and was breaking safetynet which led us to think in that direction.

Thanks for pointing out that it works for you on Los 17.1, I shall try the same with Los.

Yes, it makes sense. Thanks a lot.
[Android Software/Hacking General [Developers Only]] H3M@L:
I am noob in these things.i want to put a anykernel.zip inside a zip file(not a rom zip) with some stuffs of nethunter included and flash it via twrp like gapps zip inside a rom zip.

I tried both edify and shell method for kernel as they are used for gapps.

But i found out that anykernel update binary may have to be modified a little.can you help me?it is for A/B devices
[Android Software/Hacking General [Developers Only]] osm0sis:
It's pretty straightforward to use a zip to flash another zip, you just need to extract the update-binary and run it. See my FlashAfterUpdate addon.d-v2 script in my Odds & Ends thread for how to do so.

A/B does make things a bit trickier, but AK3 already currently detects an in-progress A/B OTA using the /postinstall/tmp directory as an indicator then will flash to the opposite slot.
[Android Software/Hacking General [Developers Only]] H3M@L:
Well i got your idea.but let me know you one thing.the thing i want to do is: a zip(named it nh.zip) file including a data folder including some apps, system/xbin and bin binaries, and the kernel.zip and META-INF.

now inside the meta-inf of the nh zip update-binary, i added the zip file to extract its all things in

tmp=/tmp/nethunter

extract "$ZIP" "$tmp"

Now it will put the kernel zip also in $tmp

Now i have to unzip the kernel.zip. according to your anykernel3 update-binary, first it makes a /postinstall/tmp/anykernel and your anykernel zip will be extracted in /postinstall/tmp/anykernel folder.

But if i use this zip, it does not work.i modified like this

unzip "$i" -d "$tmpzipdir" "$metadir/*";

chmod 755 "$tmpzipdir/$metadir/update-binary";

"$tmpzipdir/$metadir/update-binary" "$tmpzipdir/$metadir/updater-script" 1 "$i";

rm -rf "$tmpzipdir";

In my case:

i=/tmp/nethunter/kernel.zip

tmpzipdir=/postinstall/tmp/anykernel

metadir=META-INF/com/google/android;

Should i use like it?? Or this:

tmpzipdir=/tmp/nethunter/postinstall/tmp/anykernel

Or i am making things complicated??thanks
[Android Software/Hacking General [Developers Only]] osm0sis:
Well a couple issues, updater-script shouldn't be run, only update-binary, then the proper arguments for update-binary should include 3 as the API, again see my FlashAfterUpdate addon.d-v2 for an example.

You may want to look at my "Diffusion" installer setup too, since it'll do the data and system file work for you. I don't officially support it but it's pretty ready-to-roll. My latest Magisk modules are all working examples of configuring it.
[Android Software/Hacking General [Developers Only]] alecxs:
so you were able to run java in recovery mode? how is that working without android framework? stumbling across this while i am searching for the answer of a simple question:

How to make android install some .apk on boot which is installed from TWRP? I can remember it was possible to only place .apk in /system/app or /data/app and that's enough. But since android 8 the <pgkname> is not simple numbered but get's unique id (like /data/app/com.textra-th8jfhZal-Bk0JyCVIljhQ==/base.apk) so i don't know how to to it or if it is even possible
[Android Software/Hacking General [Developers Only]] osm0sis:
Yup. On latest Android in recovery one also needs to have APEX and vendor mounted so dalvikvm will run correctly, but it's enough for simple command-line Java utilities.
[Android Software/Hacking General [Developers Only]] alecxs:
do you think it's possible to create directory /data/local/tmp and install .apk as user app from TWRP recovery with /system/framework/pm.jar
[Android Software/Hacking General [Developers Only]] osm0sis:
Hmm, no I don't think that's possible on modern Android anymore. Magisk specifically waits for the device to be booted to install magisk.apk during a zip flash clean install.

You could always try, but I'm not sure how far you'll be able to get. :good:
[Android Software/Hacking General [Developers Only]] alecxs:
ah, thx for pointing me to magisk - yes that would be fine. unfortunately that's a dead end because seems all magiskinit binary stuff. do you know of any module that can install apk (basically just need idea how to batch-install apks from script, ideally flashable zip)
[Android Software/Hacking General [Developers Only]] osm0sis:
QuickSwitch does I think! Check it out here: https://github.com/Magisk-Modules-Repo/quickstepswitcher

Again not sure if that's possible in recovery.
[Android Software/Hacking General [Developers Only]] shadowstep:
[MENTION=4544860]osm0sis[/MENTION] I know you haven't been getting much Android time lately, but here's a request/suggestion for a function that mounts /system properly, with all the system-as-root and slot confusion, I think we'd all appreciate a function that can mount /system properly (and preferably bind mount /system_root to /system as well) so that we can straight away dive into doing whatever it is we want to do instead of worrying if /system has been mounted properly or not.

Also, probably trivial, but the mount function can be expanded so that if a normal "mount /cache" (for example) does not work, it does something like:

BLOCK=`cat /etc/recovery.fstab | cut -d '#' -f 1 | grep /cache | grep -o '/dev/[^ ]*/cache' | head -1`
mount $BLOCK /cache
[Android Software/Hacking General [Developers Only]] osm0sis:
I think robust mounting of /system_root and /system is outside the scope of a basic EDIFY replacement, but I can add a link to my work on a portable chunk of script for it which is in use with my AnyKernel3 project, Open GApps, all my Magisk modules and others. :highfive:

In the meantime check out my Diffusion Installer script and config in virtually all my Magisk modules: https://github.com/search?q=org:Magisk-Modules-Repo+osm0sis+xda-developers+in:README+topic:osm0sis
[Android Software/Hacking General [Developers Only]] shadowstep:
That would be more than handy. I'll just kang it straight up without credits obviously take inspiration from it and adapt it as per my use case. :p But yeah, having a link for reference would be great for everyone, I'm sure. Also, gonna look at the Diffusion Installer script in a few minutes. Thanks! :highfive:
[Android Software/Hacking General [Developers Only]] HemanthJabalpuri:
@osm0sis

Put the script in [code=bash][/code]. It would look great. Thanks to XenForo.
[Android Software/Hacking General [Developers Only]] osm0sis:
I just tried on the main OP and got "Oops we ran into some problems" so.. guess I'll try again later. 🤷‍♂️ 🤦‍♂️

Edit

: Ahah! Turning off the WYSIWYG editor so that it reverted to bbcode made it so I could edit the OP again! Switched it to code=bash now and it looks good. 🙂
[Android Software/Hacking General [Developers Only]] Git101-M:
i think its a bit off topic. But can system be mounted in dynamic partition devices using shell and install apps from twrp as system app?? dynamic partition seems to prevent app installation from twrp as system app.
[Android Software/Hacking General [Developers Only]] HemanthJabalpuri:
@osm0sis

I have made my own mod of signapk for recovery flashable zip file only.

See https://xdaforums.com/t/app-android-no-pc-sign-apk-zip-within-android.3835975/post-81324327 for more info.
[Android Software/Hacking General [Developers Only]] osm0sis:
Haven't seen the mapping part done yet via shell, but that should technically already be done via TWRP or LOSR. Then it's just a matter of mounting it rw; not every ROM has a super.img built to allow rw remount, stock ROMs almost certainly aren't. Here's how Open GApps has it, based on my AK3 mounting script: https://github.com/opengapps/opengapps/blob/master/scripts/templates/installer.sh#L935+L981
[Android Software/Hacking General [Developers Only]] HemanthJabalpuri:
Sorry for quoting very old posts.

I want to ask something that, why don't we just use minsigning with minsignapk.jar for signature verification?

Also as ale5000 asked here, is this error fixed with zipsigner by topjohnwu?

Sorry for disturbing you with these obsolete things.
[Android Software/Hacking General [Developers Only]] osm0sis:
Yeah zipsigner fixes everything, which is why I deprecated all the older instructions and packages.
[Android Software/Hacking General [Developers Only]] HemanthJabalpuri:
why don't we just use minsigning with minsignapk.jar for signature verification?

I think it will verify correctly in recovery.
[Android Software/Hacking General [Developers Only]] osm0sis:
It didn't work for shell script update-binary zips. I think all that's explained in the OP. Anyway, the zipsigner is all-in-one and covers all these cases, no point reinventing the wheel yet again.
[Android Software/Hacking General [Developers Only]] HemanthJabalpuri:
I saw a commit in signapk which removed jar signing from ota package signing https://android.googlesource.com/platform/build/+/0caa16a6d1b4349654956c895aab925c9522d2cf

Without jar signing we only need minsigning right.

May be I am wrong?
[Android Software/Hacking General [Developers Only]] osm0sis:
Not really relevant to us since that's about OTAs and we're talking about custom mod zips passing verification and unzipping correctly in custom recovery.

I think you have it backwards though. signapk -w (whole file) is the equivalent of minsignapk, IIRC.
[Android Software/Hacking General [Developers Only]] HemanthJabalpuri:
@osm0sis

The script in OP also needs to be changed according to this https://github.com/topjohnwu/Magisk/commit/ce84f1762c65e39b59b0677e7f50f1104afe78eb by you.
[Android Software/Hacking General [Developers Only]] osm0sis:
Yep. I'm aware.
[Android Software/Hacking General [Developers Only]] Nikhil:
Could you be anymore awesome? :P

I already use your anykernel3 mount script in my project and I've never had to worry about mounting partitions. All thanks to you!
[Android Software/Hacking General [Developers Only]] HemanthJabalpuri:
@osm0sis

Thanks for updating.

Also update signing source link to https://github.com/topjohnwu/Magisk/tree/v20.4/signing

Since flashable zipsigning is removed from Magisk.
[Android Software/Hacking General [Developers Only]] w1nst0n_fr:
@osm0sis

First, thanks again for all the things you do for the community.

I wonder if there is any reason to keep the updater-script. I managed to successfully flash a zip only containing an update-binary on a Samsung Galaxy S7 (Android 8.1).

I guess that if you explicitly wrote to create a dummy updater-script it's because there is a reason. But just to be sure, have you ever encounter an issue caused by a missing updater-script?
[Android Software/Hacking General [Developers Only]] osm0sis:
There were reports from I believe Chainfire back in the day of some recoveries and/or apps rejecting zips not containing both files. You're right it's not technically necessary, but to ensure broadest compatibility in all things, still likely best to do so. (y)
[Android Software/Hacking General [Developers Only]] zgfg:
Sorry, wrong thread
[Android Software/Hacking General [Developers Only]] lebigmac:
Hi @osm0sis thanks for the great work.

Can you please tell me what this error means when I flash my sample.zip ?

I also get the same error when signing the zip with the zipsigner.jar

Updater process ended with ERROR: 255
Error installing zip file '/data/local/tmp/sample.zip'

Inside this directory META-INF/com/google/android/

I have the updater-script with the dummy comment and another file called update-binary with your bash script from the first post.

I know in bash errorcode 255 translates to -1 in C language
[Android Software/Hacking General [Developers Only]] osm0sis:
Upload the zip? I can't guess what you've done.
[Android Software/Hacking General [Developers Only]] alecxs:
set_progress() not working anymore on TWRP 3.4.0 aarch64 but works fine on TWRP 3.3.1 armv7 do you know why?
[Android Software/Hacking General [Developers Only]] HemanthJabalpuri:
@osm0sis

Sorry for disturbing you for small things.

Why we need to mount /apex?

In which cases ANDROID_ROOT will /system_root and when /system?

I don't know these things because I have directly jumped to Android 10 from Android 4.4.2 Kitkat😂. I don't know what happened between these versions.

Thanks
[Android Software/Hacking General [Developers Only]] osm0sis:
News to me, worked fine in 3.5.0_9 and 3.5.0_10 last I tried.

ANDROID_ROOT will only be /system_root on old TWRP where they set it that way to try adding support for SAR (which they later realized was a mistake), so we must support it as well as a starting point.

As for /apex we support mounting it so that things may be run from /system/bin in mounted system partition.
[Android Software/Hacking General [Developers Only]] alecxs:
twrp-3.5.0_10-0-beryllium.img no progress bar to me. am i doing it wrong?

on my twrp-3-3-1-0-recovery-for-MT6580_cubot_king_kong.zip i can do `set_progress 0.5` and will get blue progress bar
[Android Software/Hacking General [Developers Only]] osm0sis:
Try show_progress 1.34 4; it should fill the bar in 4 seconds. And yeah set_progress 0.5; should fill the bar roughly half way. TWRP 3.5.1 is about to be released so I'll test there. Anyway, obviously not an issue with these scripts, more an issue with TWRP.
[Android Software/Hacking General [Developers Only]] lebigmac:
Do I have to bundle busybox with my flashable zip to make the unzip function work or is it preinstalled on every phone that has TWRP? Thanks
[Android Software/Hacking General [Developers Only]] osm0sis:
It's part of Android via toybox.
[Android Software/Hacking General [Developers Only]] alecxs:
not every toybox has unzip. furthermore not every TWRP has toybox. and usage differs. this works with busybox only (subdirectories part)

arch=arm64
unzip -o $zipfile \*-$arch/\* \*-$arch
Archive: test.zip
inflating: busybox-arm64
creating: tools-arm64/
creating: tools-arm64/bin/
creating: tools-arm64/lib/
creating: tools-arm64/lib64/
[Android Software/Hacking General [Developers Only]] osm0sis:
Seems to be working fine on TWRP 3.5.1 guacamole (OP7Pro); my AIK-mobile package installs and shows set_progress and show_progress correctly.
[Android Software/Hacking General [Developers Only]] MrCarb0n:
zip template available? 😅
[Android Software/Hacking General [Developers Only]] osm0sis:
Copy pasta!
[Android Software/Hacking General [Developers Only]] MrCarb0n:
how can I use keycheck? didn't find any such applet on BusyBox.

nvm, found getevent method from MMT-EX Add-on repo.
[Android Software/Hacking General [Developers Only]] lebigmac:
Hi @osm0sis do you know how to print an error message in red color similar to ui_print? Thanks.
[Android Software/Hacking General [Developers Only]] osm0sis:
No, I believe that's just something TWRP does for its own error messages, so isn't really exposed for others to use. I think the best you could do is send a message as the return value of a flashed zip, so it might display that in red.

e.g. `exit "Failed to run!"`
[Android Software/Hacking General [Developers Only]] wiki.alphajoza:
@osm0sis

Does this tool support execution of Dynamic Partitions operations? (such as resize, remove, add_group ...)
[Android Software/Hacking General [Developers Only]] osm0sis:
This isn't a tool, it's a collection of script functions. If you can find me command-line equivalents of those then I'll add them, but I'm not aware of any yet.
[Android Software/Hacking General [Developers Only]] aryankaran:
Any script for block_image_update

For those new dat + transfer list files -> raw image ..??
[Android Software/Hacking General [Developers Only]] osm0sis:
Ditto my last answer.
[Android Software/Hacking General [Developers Only]] ale5000:
Hi,

I'm trying to replicate keycheck without binaries but I'm stuck at one point:

hexdump -e '"%02x""\n"' < '/dev/input/event1' | while IFS='' read -r LINE; do echo "$LINE"; break; done

The first part never end and it never reach the "while" part.

Any idea?
[Android Software/Hacking General [Developers Only]] ale5000:
The solution was easier than what I thought:

# SPDX-FileCopyrightText: (c) 2016 ale5000
# SPDX-License-Identifier: GPL-3.0-or-later

_parse_input_event()
{
if ! test -e "/dev/input/${1:?}"; then return 1; fi
hexdump -n 14 -d 0< "/dev/input/${1:?}" | while IFS=' ' read -r _ _ _ _ _ _ cur_button key_down _; do
if test "${key_down:?}" -ne 1; then return 2; fi
echo "${cur_button:?}" | awk '{$0=int($0)}1' || return 1
break
done
}
[Android Software/Hacking General [Developers Only]] YZBruh:
Hello. I'm sorry for my bad english. Is it possible to use both updater-script and update-binary in the zip file? If possible, does update-binary or updater-script run first?
[Android Software/Hacking General [Developers Only]] osm0sis:
On a "normal" flashable zip update-binary is a compiled native binary that runs first and executes the commands in updater-script which is in EDIFY script format. This thread is about replacing all that with shell scripts, so that the update-binary is a customized shell script in itself, and updater-script is just a dummy empty file. Give the OP another read with that in mind and hopefully it'll all make more sense. :)(y)

By the way, I don't actively follow this thread anymore, so if anyone needs me just @ me. 🙌
[Android Software/Hacking General [Developers Only]] osm0sis:
Added ZipScriptSign packages to the OPs, with compiles of zipadjust for Win32 and Linux32 and quick little scripts to automate the documented signing process for shell script zips. Java still required in your path, of course.

The Windows script supports drag-and-drop (defaulting to test keys), but also supports a commandline for custom keys like:

sign[.bat] [keys] <filename>

e.g.: sign test UPDATE-Nexus.BootUnlocker.zip

The Linux compile is static so it should work on x64 as well with the usual compatibility layer most distros include. It is full commandline:

./sign.sh <keys> <filename>

e.g.: ./sign.sh test UPDATE-Nexus.BootUnlocker.zip

Keys must be in the included "bin" subdirectory and named like the included test keys. Both scripts should be able to be called from anywhere and will produce a "-signed.zip" suffixed file in the same directory as the supplied file.

Edit (1/27): Reuploaded the Linux32 version with a couple things fixed; should work now. :)

Edit (1/28): Reuploaded the Win32 version with some added fixes for long path names with spaces. :good:
[Android Software/Hacking General [Developers Only]] Lanchon:
FYI, i released this related tool...

Flashize: Turn Shell Scripts Into Flashable Recovery Zips
[Android Software/Hacking General [Developers Only]] _alexndr:
[MENTION=4544860]osm0sis[/MENTION]

Thanks for this very useful / helpful thread :good:

I'm moving further disscussion here as this thread seems to be better place instead of official SuperSU thread. I hope you do not mind...

Below is my original question in Chainfire's thread and your answer:

Unfortunately, CWM based recoveries (e.g. PhilZ) still displays this annoying message in case of my work ("using legacy property environment for update-binary...please upgrade to latest binaries") and I don't know why...

It is very interesting issue, because Chainfire's ZIP and your UPDATE-unSU does not tigger such message :confused:

How can I "update to latest binaries" if there is no real binaries at all, just shell script? :)
[Android Software/Hacking General [Developers Only]] osm0sis:
Might be that you're missing the dummy updater-script file.
[Android Software/Hacking General [Developers Only]] _alexndr:
Unfortunately, no... it is not a reason in my case, my "updater-script" dummy file:

#
# DevBase unified installer designed for Samsung devices
# by ALEXNDR (_alexndr @ XDA)
#
# (this is a dummy file, see update-binary)
#
[Android Software/Hacking General [Developers Only]] osm0sis:
Hmm I'm not sure then. SuperSU is signed, but unSU is not, so it can't be signing.
[Android Software/Hacking General [Developers Only]] Lanchon:
what happens with the "please upgrade to latest binaries" if you use flashize?
[Android Software/Hacking General [Developers Only]] _alexndr:
Just to let you know - problem just solved :)

http://xdaforums.com/showthread.php?p=65900543#post65900543
[Android Software/Hacking General [Developers Only]] matrixex:
Bro thanks, Truly i was searching for symlink() function in shell script but here i got all of it thanks

Chainfire's is a really dev AND YOU too
[Android Software/Hacking General [Developers Only]] Giark_RP:
good
[Android Software/Hacking General [Developers Only]] osm0sis:
Haha, and that's one of the easiest ones since all that changes is "symlink" for the actual command, "ln -s" ;)

Glad you found my thread helpful though! :)

Interesting! So basically it just depends on what functions are in the script. UnSU has neither so doesn't trigger it, however SuperSU doesn't even contain the word "meta" so I'm unsure why it doesn't trigger.

Edit

: Found it in the actual commit PhilZ linked, 'If "set_perm_" is found, it's probably a regular updater instead of a custom one. If "set_metadata_" isn't there, it's pre-4.4, which makes it incompatible.'

So since SuperSU only has set_perm, not set_perm_* (e.g. set_perm_recursive) that's why it doesn't trigger. Good to know! Honestly everyone should get out of the stone ages and use a recovery that's still being developed, like TWRP, but, good to know! :p

P.S. Luckily, my base script in the OP is fine since I wrote a set_metadata_recursive in the last big update. :good: