yashsingh1/xdaforums huggingface dataset:
[Android Software/Hacking General [Developers Only]] Zappo:
Sorry, should have re-read this thread before posting ;)

[MENTION=5787964]SuperR.[/MENTION]'s workaround version is not working for me at all:

Unzipping to system (as in all the examples previously posted)

Archive: /sdcard/installer.zip
creating: system/
creating: system/app/
creating: system/app/NewApp/
inflating: system/app/NewApp/NewApp.apk
mv: can't rename '/system/tmpunzip/system/app': Directory not empty

Unzipping to some other folder (package_extract_dir somedir /tmp/somedir)

mkdir: can't create directory '/tmp/somedir/tmpunzip': No such file or directory
unzip: can't change directory to '/tmp/somedir/tmpunzip': No such file or directory
mv: can't rename '/tmp/somedir/tmpunzip/*': No such file or directory

Unzipping to some other folder (package_extract_dir somedir /tmp)

Archive: /sdcard/installer.zip
creating: somedir/
inflating: somedir/keycheck

In this last example, the file "keycheck" ends up in /tmp/keycheck instead of /tmp/somedir/keycheck

Please note that I have no problems with this. I just thought I'd post my findings.

Actually, I find package_extract_dir "system" "/" clear and more logical.

--Edit--

However, looking at the unzip command, this doesn't seem logical at all....

unzip -o zipfile sourcedir/* -d destdir

Seems to suggest to copy everything inside sourcedir without copying the base directory...
[Android Software/Hacking General [Developers Only]] doppelhelix:
Don't know if someone cares. I made a github repo at https://github.com/doppelhelix/android_update-binary_replacement. Maybe easier than having new code scattered across the whole thread.
[Android Software/Hacking General [Developers Only]] Zappo:
I see you included the ZIP function by SuperR.

Did that work for you?

It totally fails on my device, mainly -but not only- because I can't move (-rename) to an existing directory...

So moving from /system/tempunzip/system/ to /system will throw an error.

As you can see from my post above, I tested different scenarios. They all failed.
[Android Software/Hacking General [Developers Only]] doppelhelix:
The way mv works is to rename the source name to the target name.

Though its man page doesn't document it, mv will refuse to rename a directory to another directory if the target directory contains files (which is actually a good thing).

I go around this behaviour by deleting conflicting files|folders first. Works across all my devices.

E.g.:

rm /system/media/bootanimation.zip
rm -r /system/media/audio

package_extract_file system/media/bootanimation.zip /system/media/bootanimation.zip
package_extract_dir system/media /system/media
[Android Software/Hacking General [Developers Only]] Zappo:
A couple of problems with that:

- Users of the script won't know this, and the script will fail unless the folder does not exist yet. (You don't know what other users have in their folders!)

- In the example above (rm -r /system/media/audio), you're removing all my custom ringtones and notifications!!!

- As described earlier in my post: package_extract_dir "customdir" "/tmp/customdir" will also fail.

- Unzipping a folder containing: /system/app/ AND /system/media AND /system/media/audio AND /system/whatever -- will fail.

- The purpose of the script is to make things easier. This is not easier, and your workaround is more complicated than simply:

package_extract_dir "system" "/"

which does exactly what is expected (Update/overwrite existing files and folders)

I'd say we need a better workaround, or just keep package_extract_dir as it is in the OP
[Android Software/Hacking General [Developers Only]] osm0sis:
cp -rf should work instead of mv, but I'll experiment to try and come up with a simpler replacement when I have some free time; working on some AIK stuff ATM. :good:
[Android Software/Hacking General [Developers Only]] Zappo:
Indeed. However cp takes a bit more time (as SuperR. pointed out earlier), and would need some extra free space to play with.

In the meantime, I'm gonna test the EDIFY version vs Script version and try some things myself.

-- Edit --

I think the main problem comes when <source> and <destination> directories match each other.

Instead of moving files around, I'm trying to correct the unzip command's destination.

-- Edit again --

For arbitrary source folders extracting to any location on device (existing or not), I added a copy routine.

If <source> matches (the end of) <destination>, the faster "direct" unzip is used:

# package_extract_dir <dir> <destination_dir>
package_extract_dir() {
leng=$((${#1}-1));
src=$1;
if [ ${1:$leng:1} = / ]; then src=${1:0:$leng}; fi; #remove trailing /
leng=$((${#2}-1));
dest=$2;
if [ ${2:$leng:1} = / ]; then dest=${2:0:$leng}; fi; #remove trailing /
ldiff=$((${#dest}-${#src})); # Calculate length difference
case $dest in
*$src) unzip -o "$ZIPFILE" "$src/*" -d "${dest:0:${ldiff}}";; #<destination_dir> ends with <dir>
*) if [ ! -d "$dest/tmpunzip" ]; then mkdir -p "$dest/tmpunzip"; fi; # Create all required dirs
unzip -o "$ZIPFILE" "$src/*" -d "$dest/tmpunzip";
cp -rf "$dest/tmpunzip/$src/." "$dest"; # Copy to final destination
rm -rf "$dest/tmpunzip";; # Remove temp dir
esac;
}

I believe this matches the original EDIFY version exactly.
[Android Software/Hacking General [Developers Only]] doppelhelix:
That would be awesome :good: It always bugged me to deviate from the original syntax. Will try it as soon as possible. (Just waiting for latest Lineage to finish downloading). Can you make a pull request?

One question though. Will the syntax be

package_extract_dir("/Source", "/Dest");

?
[Android Software/Hacking General [Developers Only]] Zappo:
Did pull request.

The syntax:

package_extract_dir "Source" "/Dest";

Both source and destination can have trailing slashes (will be removed): "Source/" and "/Dest/"

I didn't add a check for starting Source with slashes.... Could easily be added though... (added in my fork on GitHub)

P.S.

I also didn't add a check for how files are extracted (with or without Source folder), because as far as I can see it would require the unzip command to behave differently. After searching and reading quite a lot on "unzip", I haven't seen any mention of a version that does that...
[Android Software/Hacking General [Developers Only]] osm0sis:
[MENTION=4466791]doppelhelix[/MENTION] [MENTION=378915]Zappo[/MENTION]

Here we go! Added to the OP:

# package_extract_file <file> <destination_file>
package_extract_file() { mkdir -p "$(dirname "$2")"; unzip -o "$ZIPFILE" "$1" -p > "$2"; }
# package_extract_dir <dir> <destination_dir>
package_extract_dir() {
for entry in $(unzip -l $ZIPFILE "$1/*" 2>/dev/null | tail -n+4 | rev | cut -d" " -f1 | rev); do
if [ "$entry" != "-------" ]; then
outfile="$(echo "$entry" | cut -d/ -f2-)";
mkdir -p "$2/$(dirname "$outfile")";
unzip -o "$ZIPFILE" "$entry" -p > "$2/$outfile";
else
break;
fi;
done;
}

Zero compromises. :cool:
[Android Software/Hacking General [Developers Only]] Zappo:
I'll test when I have some time, but that looks pretty slick!

Awesomeness!
[Android Software/Hacking General [Developers Only]] Nikhil:
Hi [MENTION=4544860]osm0sis[/MENTION]

Please look at following code.

# file_getprop <file> <property>
file_getprop() {
grep "^$2" "$1" | cut -d= -f2;
}

As you know, this is helpful in fetching a value of keyword from prop files, this also has a tiny bug in it.

In case when the keyword is item.1.1 (we have it in aroma installer, the values are stored in this form), when user selects item number 1 and 10 (i.e. item.1.10), while using above function in following manner

"if [ $(file_getprop /tmp/aroma/dalvik_choices.prop item.1.1) == 1 ];

this returns false as the function returns multiple values. (grep matches item.1.1 in item.1.1 and item.1.10, item.1.11 and so on....)

Fix is to fetch the very first value which is matched, so following should work correctly.

# file_getprop <file> <property>
file_getprop() {
grep -m1 "^$2" "$1" | cut -d= -f2;
}

what do you think?
[Android Software/Hacking General [Developers Only]] osm0sis:
Sounds good! I'll do it with head -n1 though, just because there were some older recoveries that had head but not a full-featured grep (it was toolbox grep if I recall correctly), and I like to keep the functions as widely compatible as possible, whenever possible. :)

Edit: Added. I also noticed the grep fallback part of my getprop function was incomplete and would also have the same issue, so I fixed it up as well. :good:
[Android Software/Hacking General [Developers Only]] Nikhil:
Right. Great, thanks! :)
[Android Software/Hacking General [Developers Only]] Nikhil:
Hi [MENTION=4544860]osm0sis[/MENTION]

While executing following command

set_perm 1000 1000 0644 "/data/app/Google Drive_v2.7.063.14.35-1/base.apk"

I am getting following error.

Installing Google Drive_v2.7.063.14.35-1
chown: /data/app/Google: No such file or directory
chown: Drive_v2.7.063.14.35-1/base.apk: No such file or directory
chown: /data/app/Google: No such file or directory
chown: Drive_v2.7.063.14.35-1/base.apk: No such file or directory
chmod: /data/app/Google: No such file or directory
chmod: Drive_v2.7.063.14.35-1/base.apk: No such file or directory

It looks like if we have spaces in path then it is splitting the string and considering individual string as a separate directory.

is there anyway chown accepts the directory having spaces in it?
[Android Software/Hacking General [Developers Only]] osm0sis:
There really shouldn't be any paths with spaces in Android (outside of /sdcard, which is why this hasn't come up before), and that's not how /data/app handles its path structure.

I'll go through and make everything more respectable with quotes to contain these possible oddball paths, but you'd be better off fixing your zip to not push a directory with spaces, (use _ instead?)
[Android Software/Hacking General [Developers Only]] Nikhil:
Yes,

If we are not using shell script and use edify script along with device specific update-binary then it works. The path with spaces are accepted.

It would be wonderful if spaces are accepted while using shell script.

For now, I will use _ Instead and also try to identify a way to accept space.

I will update if I find a solution to this.

Thanks osm0sis. Please update if you manage to solve this. :)
[Android Software/Hacking General [Developers Only]] osm0sis:
You can, yes, but /data/app would never do that on its own. It's an easy fix for set_perm, just put quotes around the $1 $2 and $3 variables, so like "$1"

You're better off just not using spaces in your path, or quoting your path in your command.

Edit: Hmm, you did quote in your command but they got stripped... just noticed that command uses $*, so [MENTION=4867515]Nikhil[/MENTION] try changing all the $* in the set_perm function to $@ and let me know if that gets it working, please.
[Android Software/Hacking General [Developers Only]] osm0sis:
From PM:

Please try the current, new version from the OP:

set_perm() {
uid=$1; gid=$2; mod=$3;
shift 3;
chown $uid.$gid $@; chown $uid:$gid $@;
chmod $mod $@;
}

With your original (proper) syntax and quoting of the path with spaces:

set_perm 1000 1000 0644 "/data/app/Google Drive_v2.7.063.14.35-1/base.apk"

It should work, since $@ preserves the given strings, even with spaces, as originally entered. So your quotes around the path should then be sufficient.
[Android Software/Hacking General [Developers Only]] Nikhil:
Hi osm0sis,

you are right, $@ preserves the given string variables but it seems that it stores them as an array of strings and not as a concatenated string.

However, putting them in quotes worked for me. For some reason app didn't install but permissions were set correctly.

Please add quotes as then it works. :)

set_perm() {
uid=$1; gid=$2; mod=$3;
shift 3;
chown $uid.$gid "$@"; chown $uid:$gid "$@";
chmod $mod "$@";
}
[Android Software/Hacking General [Developers Only]] Nikhil:
Hi [MENTION=4544860]osm0sis[/MENTION],

Following are the edify commands which I used to install a zip within a flashable zip. As you can see below, the zip which I am flashing lies in Install folder.

Now, following command works and I am able to flash the zip in recovery.

run_program("/sbin/busybox", "unzip", "/tmp/Install/busybox.zip", "META-INF/com/google/android/*", "-d", "/tmp/Install");
run_program("/sbin/sh", "/tmp/Install/META-INF/com/google/android/update-binary", "dummy", "1", "/tmp/Install/busybox.zip");

However, when I am trying the same using shell script, the same is not getting executed.

The zip file is placed in /tmp/Install directory but following commands are not yielding any results.

Please look at below code and suggest if there is any mistake or if we have any alternate way of achieving this.

run_program "/sbin/busybox" "unzip" "/tmp/Install/busybox.zip" "META-INF/com/google/android/*" "-d" "/tmp/Install"
run_program "/sbin/sh" "/tmp/Install/META-INF/com/google/android/update-binary" "dummy" "1" "/tmp/Install/busybox.zip"

Any help would be appreciated! Thanks!

I am attaching the zip for reference.

Edit:

I found something.

unzip /path/to/supersu.zip META-INF/com/google/android/* -d /tmp	sh /tmp/META-INF/com/google/android/update-binary dummy 1 /path/to/supersu.zip

I'll try This. :)
[Android Software/Hacking General [Developers Only]] osm0sis:
run_program doesn't exist, because execute it directly.

E.g. second one would be /sbin/sh /tmp/Install/META-INF/com/google/android/update-binary dummy 1 /tmp/Install/busybox.zip

Edit: might need sh -c as well actually
[Android Software/Hacking General [Developers Only]] Nikhil:
Hi osm0sis,

it worked with following code

/sbin/sh /tmp/Install/META-INF/com/google/android/update-binary dummy 1 /tmp/Install/busybox.zip

Thanks! :)

using -c was not required for the busybox zip that I tried. I will try with -c if for any other zip it doesn't work! :)

Also, this command works when update-binary is a shell script and we are executing it using "sh". if it is not a shell script, is there any command which could execute edify script (updater-script)?

I tried executing "updater-script" with "sh" but it wasn't executed (may be it is obvious).
[Android Software/Hacking General [Developers Only]] osm0sis:
That's where -c comes in, then it'll work for any executable. You'd just need to set the update-binary 755 first. Maybe quote the entire command after -c; alternately you could just drop the /sbin/sh altogether since an executable should just run.
[Android Software/Hacking General [Developers Only]] Nikhil:
I tried with following commands,

unzip /tmp/Install/busybox.zip META-INF/com/google/android/* -d /tmp/Install
ui_print "setting 0755 permissions"
set_perm 0 0 0755 "/tmp/Install/META-INF/com/google/android/update-binary"

ui_print "executing normally"
sh /tmp/Install/META-INF/com/google/android/update-binary dummy 1 /tmp/Install/busybox.zip

ui_print "executing using -c command with quotes"
sh -c "/tmp/Install/META-INF/com/google/android/update-binary dummy 1 /tmp/Install/busybox.zip"

ui_print "executing using -c command without quotes"
sh -c /tmp/Install/META-INF/com/google/android/update-binary dummy 1 /tmp/Install/busybox.zip

it didn't execute, am I doing any mistake here?

also, please correct me if my understanding is wrong, in case when update-binary is not a shell script, update-binary reads commands from updater-script and executes it.

so we just have to execute update-binary. if it is a shell script, it will execute the commands written in it and if not then it will read commands from updater-script and get the job done.

also, updater-script should be placed in the same directory as update-binary after extracting!

Edit:

fetched from the logs, got following from command -c with quotes

wrong updater binary API; expected 1, 2, or 3; got dummy
[Android Software/Hacking General [Developers Only]] osm0sis:
My FlashAfterUpdate addon.d handles it like this:

unzip "$i" -d "$tmpzipdir" "$metadir/*";
chmod 755 "$tmpzipdir/$metadir/update-binary";
"$tmpzipdir/$metadir/update-binary" "$tmpzipdir/$metadir/updater-script" 1 "$i";
rm -rf "$tmpzipdir";

In your case:

i=/tmp/Install/busybox.zip

tmpzipdir=/tmp/Install;

metadir=META-INF/com/google/android;

So it looks like you're doing it pretty much right, except I guess I set 1 to that argument instead of dummy, and I execute the file directly without invoking sh.
[Android Software/Hacking General [Developers Only]] Nikhil:
Thanks a lot, you made it easier for me :)

I tried with following commands as it was not accepting "updater-script" as in input, I passed "1" instead of that (as you suggested). That worked for me. :)

tmpzipdir=/tmp/Install;
i=$tmpzipdir/busybox.zip
metadir=META-INF/com/google/android;

unzip "$i" -d "$tmpzipdir" "$metadir/*";
chmod 755 "$tmpzipdir/$metadir/update-binary";
"$tmpzipdir/$metadir/update-binary" 1 1 "$i" >> $OUTFD;
rm -rf "$tmpzipdir";

Please observe, I appended >> $OUTFD as my intention was to print whatever was getting printed in updater-script. It worked for me. however, attached is what I got on screen.

E:unknown command [minzip:]

this happened when the script was executing package_extract_dir function.

it seems normal coz the files were extracted, Is there anyway to handle this? to not get such error or not print such error maybe?
[Android Software/Hacking General [Developers Only]] osm0sis:
No idea, haven't seen that before. Type unzip --help?
[Android Software/Hacking General [Developers Only]] Nikhil:
Thanks osm0sis for helping me out. I read a lot about it but still I am getting the error mentioned in previous post. I will read more and try to solve it.

Thanks again! :)
[Android Software/Hacking General [Developers Only]] osm0sis:
unzip --help would tell you what binary your recovery is using.. please report the output here. I'm guessing you're using a TWRP that isn't using busybox anymore in favor of toybox and minzip.. hopefully they can bring busybox back soon since it's generally more complete.

Oh, and you piping everything to OUTFD is why recovery is trying to interpret all things as commands.
[Android Software/Hacking General [Developers Only]] ale5000:
Hi,

why double chown?

And what mean chown with dot?
[Android Software/Hacking General [Developers Only]] osm0sis:
Some older recoveries don't support proper POSIX with a colon for root:root, and instead require root.root, so we do both since it covers either case and there's no harm in doing it twice. :)
[Android Software/Hacking General [Developers Only]] ale5000:
Thanks for the reply.

In case I use a custom busybox to execute the commands will it support proper POSIX also on these recoveries or will still suffers the same problem?
[Android Software/Hacking General [Developers Only]] osm0sis:
Yeah that should work fine.
[Android Software/Hacking General [Developers Only]] X7SHaDe:
Your package_extract_dir will not work on zip files with spaces, and on extractions where the user would want to extract a dir that is NOT on the root of the zip (ex: package_extract_dir files/system /system) and I think this is a cleaner implementation (I just refactored your cool idea :cool:) tested to work on many different circumstances of zip structures, pathnames, filenames, function arguments etc. (There's not even a single regression ;))

package_extract_dir() {
for entry in $(unzip -l "$ZIPFILE" "$1/*" | grep "$1" | rev | cut -d" " -f 1 | rev | tail -n +2); do
pre1=$(echo "$1" | sed "s_/_\\\/_g")
pre2=$(echo "$2" | sed "s_/_\\\/_g")
out=$(echo "$entry" | sed "s/${pre1}\(..*\)/${pre2}\1/")
mkdir -p $(dirname "$out")
unzip -o -p "$ZIPFILE" $entry > "$out"
done
}

Btw, I used three backslashes because there are some shells that aren't doing what I have on mind with only two, which should be working for the purpose of adding backslash on forward slash to escape it on sed but isn't.

Fast, lean, and human readable... with real zero compromises :cool: Enjoy!

EDIT: Why not skip repeating commands once a command already succeeded? set_perm_recursive slowed down a lot due to that. You can just test cmd != 0 && test cmd != 0 && test cmd != 0 and so on, so that the function will stop iterating through all posible cmds aiming at achieving the same goal once a cmd returns successful.
[Android Software/Hacking General [Developers Only]] veez21:
I timed this one and it seems a little bit faster :good:.
[Android Software/Hacking General [Developers Only]] osm0sis:
Cool cool. I'll check this out and update it and the perms/metadata zips for proper fallbacks with || in my next revision. I'm working on some other things at the moment. :)
[Android Software/Hacking General [Developers Only]] X7SHaDe:
Ah right It is || not && hahaha my bad :p
[Android Software/Hacking General [Developers Only]] osm0sis:
Got the fallbacks in there now. The chcon ones got ... interesting, with the fallbacks having to operate on the for loop's "done" and then needing to fallback further within that with several commands for the recursive versions. Kinda ugly but it should work. ;) :)
[Android Software/Hacking General [Developers Only]] ale5000:
Hi,

apparently there is a recovery on an old phone without /tmp.

Is there an alternative to store temporary files?
[Android Software/Hacking General [Developers Only]] osm0sis:
Whoever maintains the recovery there need to fix that. It's the standard. Everything from SuperSU to GApps uses /tmp.
[Android Software/Hacking General [Developers Only]] ale5000:
I have asked to the user and he has DingDong Recovery.

I have never heard of it and certainly I cannot do a bug report in Russian (hopefully someone else will do it).

PS: Maybe I have found a way, creating /tmp and mount it from the shell script.
[Android Software/Hacking General [Developers Only]] osm0sis:
Finally got a chance to look at this. Sorry about the wait! :eek:

Where you've got the `tail -n+2` it drops the top-most entry when there isn't a match to `grep $1` in the name of $ZIPFILE; my placement of tail (before where the grep would be) works better. You were right about the failure with zip names having spaces but only because I forgot to quote the first instance of $ZIPFILE. ;)

Using grep is a good idea, but we can use it even more effectively to read the entries and that should also allow for files within the zip having spaces, which both of our previous attempts would not.

Using substitutions is also a good idea, though I'm a bit hesitant to use sed, and consciously switched away from it for most things since it was broken not that long ago in toybox.. so we can either use the shell itself to do the escaping ${1//\//\\/} and substitutions ${entry/$1/$2}, or we can go ahead and use sed (since it is far simpler when it works) but can at least cut down on escaping shenanigans by just using a different delimiter that shouldn't be in a filepath, like pipe: |

# package_extract_dir <dir> <destination_dir>
package_extract_dir() {
for entry in $(unzip -l "$ZIPFILE" "$1/*" 2>/dev/null | tail -n+4 | grep -o " $1.*$" | cut -c2-); do
outfile="$(echo "$entry" | sed "s|${1}\(..*\)|${2}\1|");
mkdir -p "$(dirname "$outfile")";
unzip -o "$ZIPFILE" "$entry" -p > "$outfile";
done;
}

Any particular reason you use \(..*) and \1 in the sed? That would just expand to the rest of the line I believe, and since the 's' replacement isn't run with 'g' it'll only do the first instance regardless, so it shouldn't be necessary. I've left it in for now since it appears to work either way, but I'd like to clear that up before I update the OP with it.

Thanks! :)

Edit

: No answer from [MENTION=8143568]X7SHaDe[/MENTION] yet, so I've added it to the OP without the \(..*) and \1. :good:
[Android Software/Hacking General [Developers Only]] mrrocketdog:
[MENTION=4544860]osm0sis[/MENTION] , do you just make this $h*t up as you go along¿

j/k ; very much appreciate your sharing , teaching & time.:highosm0sis:
[Android Software/Hacking General [Developers Only]] ale5000:
Hi,

on some old broken phones the unzip command fail with:

unzip: zip flags 1 and 8 are not supported

Is there ay workaround?

What is the meaning of this error?
[Android Software/Hacking General [Developers Only]] osm0sis:
Only when signed in the old way. That's what the special zip script signing documentation in the OP is all about. Check it out.