yashsingh1/xdaforums huggingface dataset:
[Android Software/Hacking General [Developers Only]] osm0sis:
In similar bughunting news, I just updated the OP to switch the $ZIP variable to $ZIPFILE so as not to conflict with the Info-ZIP (zip) binary if the variable ever gets exported (e.g. for subscript purposes) instead of just set like [MENTION=631273]Chainfire[/MENTION], myself and [MENTION=2777334]TKruzze[/MENTION] have always done. Might as well be future-proof. :good:

[MENTION=6404745]MastahF[/MENTION] and [MENTION=5423715]YashdSaraf[/MENTION] just spent many hours figuring out the issue over on Open GApps, so thanks to them for that and bringing it to my attention. :)
[Android Software/Hacking General [Developers Only]] _alexndr:
Exactly! IMO very poor check... Finally I have decided to leave my script as it is. In case of any similar reports I will link to this disscussion to let users UNDERSTAND the problem and consequently change the recovery...
[Android Software/Hacking General [Developers Only]] matrixex:
Hi

How did you figure this out all of it
[Android Software/Hacking General [Developers Only]] osm0sis:
Reading, then time and work. :p
[Android Software/Hacking General [Developers Only]] matrixex:
Got it :cyclops:

Any idea how to extract system.new.dat files in android?
[Android Software/Hacking General [Developers Only]] osm0sis:
Not sure. There are threads which offer tools to do it in Windows and Linux, so maybe they could be adapted for Android.
[Android Software/Hacking General [Developers Only]] bigsupersquid:
99.9% of anything you can do with desktop Linux you can do on an android device with a Linux chroot.
[Android Software/Hacking General [Developers Only]] LaserCrafter:
Can you show me the commands for this type of stuff because I am making a rom with a update-binary shell script and i don't know the commands to make it happen. :(
[Android Software/Hacking General [Developers Only]] osm0sis:
I'm not sure what you mean. All the commands are in the OP. Include the ones you need and the usage is in the comment above each one. :good:

Post your WIP script if you get stuck and want some help/clarification.
[Android Software/Hacking General [Developers Only]] SuperR.:
[MENTION=4544860]osm0sis[/MENTION]. Awesome job again :)

I would like to incorporate this into the kitchen, but want to make sure the syntax is correct first. Would you mind taking a look to see if there are any glaring issues with the script below? This is obviously not a complete update-binary. I have omitted all the functions to make it smaller for posting and I have only added one instance of the things I am currently working with. Thanks a lot :)

if [ $(getprop ro.product.device) != "bullhead" -a $(getprop ro.build.product) != "bullhead" ]; then
abort "This package is for bullhead devices, this is a $(getprop ro.product.device)"
fi
mount /system
show_progress 1 0
ui_print "Test rom Installer"
delete_recursive /system
package_extract_dir "system" "/system"
set_progress 0.75
symlink app_process32 system/bin/app_process
set_metadata_recursive /system uid 0 gid 0 dmode 0755 fmode 0644 capabilities 0x0 selabel u:object_r:system_file:s0
set_metadata /system/bin/ATFWD-daemon uid 0 gid 2000 mode 0755 capabilities 0x0 selabel u:object_r:atfwd_exec:s0
umount /system
package_extract_file boot.img /dev/block/platform/msm_sdcc.1/by-name/boot
package_extract_dir busybox /tmp/busybox
unzip /tmp/busybox/UPDATE-Busybox.Installer.v1.24.2-ALL-signed.zip META-INF/com/google/android/* -d /tmp/busybox
sh /tmp/busybox/META-INF/com/google/android/update-binary dummy 1 /tmp/busybox/UPDATE-Busybox.Installer.v1.24.2-ALL-signed.zip
[Android Software/Hacking General [Developers Only]] osm0sis:
Only thing I notice is that the way I had to write package_extract_dir it only requires the dir name (mentioned briefly as a caveat in the OP), so you'd put busybox and whatever in subfolders in a "tmp" folder in the zip root and then just use: package_extract_dir tmp

Similarly I believe only this should be required: package_extract_dir system

Extracting the boot.img directly into /dev/block may or may not work, let me know how that goes. If it doesn't, then extract to tmp first and write_raw_image should work. :good:

Let me know if the embedded detection script from my Busybox Installer still works when called this way too.

Cheers! :)
[Android Software/Hacking General [Developers Only]] osm0sis:
Refresh posted!

Changes:

- update set_perm, set_perm_recursive, ch_con, ch_con_recursive, and sha1_check to not use awk so current lacking recoveries (ie. CM Recovery) can be supported

- fix package_extract_dir to follow AOSP spec and use an output directory, since it may work after all (thanks [MENTION=5787964]SuperR.[/MENTION])

- notes/links to additional scripts for Embedded Zips and Systemless Root added the 2nd OP (thanks again [MENTION=1162986]Captain_Throwback[/MENTION] & [MENTION=631273]Chainfire[/MENTION])

Now I have to update all my zips to use the new set_perm functions.. CM Recovery is such a headache. :rolleyes:

Hopefully toybox adds awk soon, but either way, if something can be done with only built-in shell functions like shift it's more portable and better practice anyway, so here we are. :)
[Android Software/Hacking General [Developers Only]] osm0sis:
"Systemless Mod Expansion" link and info added the 2nd OP. :cowboy::good:
[Android Software/Hacking General [Developers Only]] doppelhelix:
Thanks a lot for sharing this script.

Two things I'm not sure about.

1. Line 12 reads

echo -e "ui_print $1nui_print" >> $OUTFD;

shouldn't there be a escape like this:

echo -e "ui_print $1\nui_print" >> $OUTFD;

Even so, what's the reason for it?

This seems to work too

echo -e "ui_print $1" >> $OUTFD;

2. package_extract_dir

In a old'n'busted edify script, to extract to e.g. /system/whatever I would write

package_extract_dir("system", "/system");

That gave me some headache. In the new script I wrote

package_extract_dir "system" "/system";

Took me some time to realise that the script is extracting to /system/system/whatever and I had to write

package_extract_dir "system" "/";

Is this correct? Will this work this way everywhere?

thanks advance
[Android Software/Hacking General [Developers Only]] osm0sis:
1. Thanks for noticing this. You're correct, it needs the escape but xda's Edit Project is a bit buggy and swallows it up unless I remember to go back and double escape it \\ every time I make an edit. :rolleyes:

As for the second ui_print, I remember testing removing it way back when I first started adapting [MENTION=631273]Chainfire[/MENTION]'s versions from his SuperSU zip to make them a bit more robust (and/or simplified), and if I recall correctly, the first prints the line in recovery but the second blank one is required to force recovery to go to the next line, or clear the ui_print buffer, or something.

Here is SuperSU's current version for reference:

ui_print_always() {
echo -n -e "ui_print $1\n" >> /proc/self/fd/$OUTFD
echo -n -e "ui_print\n" >> /proc/self/fd/$OUTFD
}

2. I was under the impression that this one worked like its EDIFY counterpart now. [MENTION=5787964]SuperR.[/MENTION] wrote it, and I know it's in use in his kitchen, so perhaps he can elaborate on what's going on. :good:
[Android Software/Hacking General [Developers Only]] SuperR.:
It should work as it is. I have not seen anyone mention it, but I have no idea how many people are using the update-binary feature of the kitchen. [MENTION=1162986]Captain_Throwback[/MENTION] kindly did the testing of the kitchen functions, and said everything is working. That is all I have heard unfortunately.

note: There is no need for quotes in your script for the package_extract_dir. However, I don't think it is causing your issues.

package_extract_dir system /system
[Android Software/Hacking General [Developers Only]] Captain_Throwback:
I actually had some issues with it on newer devices, but then I stopped building ROMs and didn't provide any helpful feedback. Sorry about that. If I get around to building again, I'll let you know what issues I'm having. Since I'm currently testing MultiROM on my current device, I might have more opportunity to play with this.
[Android Software/Hacking General [Developers Only]] SuperR.:
[MENTION=4466791]doppelhelix[/MENTION] too -

Try replacing the function with the following:

Change this:

package_extract_dir() { unzip -o "$ZIPFILE" "$1/*" -d "$2"; }

to this:

package_extract_dir() {
mkdir $2/tmpunzip
unzip -o "$ZIPFILE" "$1/*" -d "$2/tmpunzip"
if [ -d $2/tmpunzip/$1 ]; then
mv $2/tmpunzip/$1/* $2/
mv $2/tmpunzip/$1/.* $2/ 2>/dev/null
else
mv $2/tmpunzip/* $2/
mv $2/tmpunzip/.* $2/ 2>/dev/null
fi
rm -rf $2/tmpunzip
}

Basically, we create a tmpunzip directory in the target directory where we unzip the contents. Then we check if there is a directory in tmpunzip with the same name as the directory we just extracted. If so, we move the files from inside that directory to the target. Otherwise we move the contents of tmpunzip to the target. Once finished, we delete tmpunzip.

I have not tested it on a device but I am pretty sure this will sort out the issues. mv is pretty fast when operating within the same partition so hopefully it won't add much time to the process. Let me know how it goes :)
[Android Software/Hacking General [Developers Only]] doppelhelix:
Just another question

is there a reason for semicolons at the end of each line ? I was under the impression that newlines delimit commands, equivalent to a semicolon (except the find -exec double-semicolon of course ).
[Android Software/Hacking General [Developers Only]] osm0sis:
Ensures that even if it's copied and pasted from here and the newlines get gobbled up, it'll all execute correctly. It's also just personal preference. ;)
[Android Software/Hacking General [Developers Only]] aj20010319:
Can I combine APK Patcher with aroma? Like use the standard updater-binary and adding all these functions from updater-binary into a different shell script and executing it later?
[Android Software/Hacking General [Developers Only]] aj20010319:
Okay so I somehow managed to get aroma and apk patcher together, made separate zips and then made one zip flash the other.

I am trying to decompile framework-res.apk, but it's failing, logs show this error,

framework-res.apk
I: Using Apktool 2.2.0 on framework-res.apk
I: Loading resource table...
I: Decoding AndroidManifest.xml with resources...
I: Regular manifest package...
I: Decoding file-resources...
java.lang.NoClassDefFoundError: Failed resolution of: Ljavax/imageio/ImageIO;
at brut.androlib.res.decoder.Res9patchStreamDecoder.decode(Res9patchStreamDecoder.java:39)
at brut.androlib.res.decoder.ResStreamDecoderContainer.decode(ResStreamDecoderContainer.java:33)
at brut.androlib.res.decoder.ResFileDecoder.decode(ResFileDecoder.java:120)
at brut.androlib.res.decoder.ResFileDecoder.decode(ResFileDecoder.java:87)
at brut.androlib.res.AndrolibResources.decode(AndrolibResources.java:262)
at brut.androlib.Androlib.decodeResourcesFull(Androlib.java:131)
at brut.androlib.ApkDecoder.decode(ApkDecoder.java:108)
at brut.apktool.Main.cmdDecode(Main.java:163)
at brut.apktool.Main.main(Main.java:81)
Caused by: java.lang.ClassNotFoundException: Didn't find class "javax.imageio.ImageIO" on path: DexPathList[[zip file "/tmp/apkpatcher/tools/apktool_2.2.0-dexed.jar"],nativeLibraryDirectories=[/system/lib]]
at dalvik.system.BaseDexClassLoader.findClass(BaseDexClassLoader.java)
at java.lang.ClassLoader.loadClass(ClassLoader.java)
at java.lang.ClassLoader.loadClass(ClassLoader.java)
... 9 more
Suppressed: java.lang.ClassNotFoundException: javax.imageio.ImageIO
at java.lang.Class.classForName(Native Method)
at java.lang.BootClassLoader.findClass(ClassLoader.java)
at java.lang.BootClassLoader.loadClass(ClassLoader.java)
... 11 more
Caused by: java.lang.NoClassDefFoundError: Class not found using the boot class loader; no stack trace available

Decoding APK resources failed. Aborting...
[Android Software/Hacking General [Developers Only]] osm0sis:
[MENTION=7043986]aj20010319[/MENTION] - replied in the APK-Patcher thread.
[Android Software/Hacking General [Developers Only]] aj20010319:
Sorry I replied in the wrong thread:(
[Android Software/Hacking General [Developers Only]] doppelhelix:
Unfortunately this doesn't work as intended.

The way mv works is to rename the source name to the target name.

Though its man page doesn't document it, mv will refuse to rename a directory to another directory if the target directory contains files (which is actually a good thing). But we want to merge the content of the source (eg /system/app) into the target, which mv will not do.

I see only two possible ways. Either a function that will test not only $1 but for each directory inside of it. But I think that would be overly complicated (at least for my little scripting capabilities), or to ditch the whole unzip mess and use tar, as opengapps already does.
[Android Software/Hacking General [Developers Only]] SuperR.:
You are correct about mv not overwriting files, but the target directory should be empty before trying to move the files into it. Using /system as an example, The function extracts the system directory from the zip to a tmp directory, then checks to see if it extracted to a directory named system or extracted the files directly in the tmp directory. Then it uses mv to move the files from either /tmp or tmp/system/ into /system which should be empty to begin with. If /system is not empty to begin with, maybe we need to figure out why /system is not empty which may be the cause of your issue.
[Android Software/Hacking General [Developers Only]] doppelhelix:
Yeah well. I actually just want to move additional folder|files to /system/app|priv-app, /system/etc or system/media. Since i have some older devices or even no recovery at all (AMIDuOS) and I'm fed up juggling with different update-binary builds that have all different capabilities and different updater-scripts for each device, I just wanted a universal update-binary as a script.

It's not a big deal for me though that the functions here are not working as I am used to. I will just rewrite it to extract to DIR=$(dirname $ZIPFILE) and cp everything to destination. That way it won't be as fast as mv but that's not a problem for me.
[Android Software/Hacking General [Developers Only]] SuperR.:
Have you tried something like this in your script?

package_extract_dir "system/app" "/system/app"
package_extract_dir "system/priv-app" "/system/priv-app"
package_extract_dir "system/etc" "/system/etc"
package_extract_dir "system/media" "/system/media"

Assuming the files in these directories within the zip do not exist in the device, I think it should work for your needs.

edit: If you want/need full overwrite of existing files, then cp is the way to go. Or you could try "mv -f" in the function which forces overwrite, but I am not certain the built-in mv has this option.
[Android Software/Hacking General [Developers Only]] doppelhelix:
Of course it does. *Facepalm*

I was overly complicating things. Thanks a lot!
[Android Software/Hacking General [Developers Only]] CraschTD:
Input in shell script

Great work!!!!

I'm trying to build a simple ROM patcher executing some commands if user want to apply a particular patch.

Is there a way to capture user input from shell script?

#!/bin/bash
echo "Create swap file on cache partition? [S/n]: "
read choice
if [ "$choice" == "S" ] || [ "$choice" == "s" ]; then
echo "Swap created"
else
echo "Swap not created"
fi
[Android Software/Hacking General [Developers Only]] osm0sis:
Yes! I actually just noticed a clever way someone has implemented to do this in a fork of AnyKernel2 the other day.

First in your hashbang you'll need /sbin/sh for recovery, or /system/bin/sh since nothing is kept in bin on Android, and we typically don't have bash. busybox might contain bash, ash (most commonly) or hush, but none of that is a certainty on different setups.

Here's the choice code, timeout is part of busybox, but I'm not sure where the keychoice binary they reference comes from since I don't see it in the repo so maybe you can hunt it down:

https://github.com/TNAAOrganisation...META-INF/com/google/android/update-binary#L20
[Android Software/Hacking General [Developers Only]] CraschTD:
I tried to find the keycheck binary but I didn't succed!!!

The source is found here

They read from /dev/input and check key value.

If I succeed on compiling it (or finding already a binary file) I can put on my zip file and use it?

Maybe someone has already done that... (I'm lazy :D)
[Android Software/Hacking General [Developers Only]] osm0sis:
* Added setcap to set_metadata and set_metadata_recursive (may require setcap binary built from AOSP's libcap source)
[Android Software/Hacking General [Developers Only]] Captain_Throwback:
FYI - TWRP won't actually be including these binaries. Support is being added directly to libtar so that the capability information is included in the tar header. So you'd have to include those binaries yourself if that's how you'd be supporting setting capabilities with your script.
[Android Software/Hacking General [Developers Only]] osm0sis:
Ahh damnit. Everything else for file attributes is doable from command-line in TWRP though, so shouldn't/couldn't TWRP also include them?

Post edited above to reflect the binary requirement. :(
[Android Software/Hacking General [Developers Only]] Captain_Throwback:
Should, I'd say no. Could, maybe, but it would require source, and the repo would need to get added to Omni, so highly unlikely. Sorry. You'd have to plead your case on IRC with [MENTION=4878287]_that[/MENTION] & [MENTION=912474]Dees_Troy[/MENTION].
[Android Software/Hacking General [Developers Only]] Captain_Throwback:
[MENTION=4544860]osm0sis[/MENTION]

It does appear the TWRP will be updated to allow reading/writing capabilities as a numeric value, so once that's done you should still be able to calculate the proper capabilities for set_metadata. So all is not lost :)
[Android Software/Hacking General [Developers Only]] Ricky Divjakovski:
Honestly this is one of the best things I've came Accross on xda
[Android Software/Hacking General [Developers Only]] Captain_Throwback:
Looks like this minimal support will be all that's added: https://gerrit.omnirom.org/#/c/21382/

You can setcap using the same syntax that the capabilities in set_metadata uses, i.e. using the hex calculated capability for the binary in question. But since it's an ors command, it has to be run from shell preceded by "twrp". Hopefully that's enough for you to leverage for your script replacement.
[Android Software/Hacking General [Developers Only]] veez21:
[MENTION=4544860]osm0sis[/MENTION], can you have it in a txt file, I can't copy-paste cause i don't have a pc right now :)
[Android Software/Hacking General [Developers Only]] osm0sis:
Here's what it is currently: http://pastebin.com/AqzX8uDt

But I won't upload it as .txt since I want it accessible for people to grab functions easily and don't want to increase work for myself by keeping it in 2 places at once. :)
[Android Software/Hacking General [Developers Only]] veez21:
Thanks, It's hard to work on a phone. Thanks, really:)
[Android Software/Hacking General [Developers Only]] osm0sis:
Stumbled across what I presume is an arm compile of keycheck today, here: https://github.com/someone755/kerneller/tree/master/extract/tools

:good:
[Android Software/Hacking General [Developers Only]] Zappo:
This looks awesome, however it doesn't seem to work for me...

When I try to install any script, TWRP fails with ERROR: 255

Checked the log, which says:

E:Can't execute '/tmp/updater': No such file or directory

However, the file is actually there, and contains the script I'm trying to run...

Am I missing something?
[Android Software/Hacking General [Developers Only]] osm0sis:
They need to be named exactly as in the OP, update-binary with the commands and everything, and the dummy updater-script in the flashable zip at the usual location.
[Android Software/Hacking General [Developers Only]] Zappo:
Yeah, I know.

That's how they're named in the zip...

At first, I didn't know what "/tmp/updater" had to do with anything, but like I said; it contains the script from update-binary.

I thought it might be my own script, but even when I only use the script from the OP, I get the same error.

I also confirmed /sbin/sh is where it's supposed to be, and I believe busybox is installed properly.

Other "normal" EDIFY scripts I wrote are installed properly (and contain busybox commands as well)

---------- Post added at 03:44 PM ---------- Previous post was at 03:36 PM ----------

I'm an idiot; somehow the EOL settings for update-binary were set to Windows format.

The /tmp/updater file is still created, but runs properly now.

Thanks for your reply, and an awesome script!
[Android Software/Hacking General [Developers Only]] Zappo:
I wanted to share my customized Progress functions, in case someone is interested.

Instead of the weird 1.34 value for 100%, I adjusted the functions to take 0-100.

It makes it easier to set the correct values without thinking/calculating.

# show_progress <amount> <time>
show_progress() {
amount=$(echo "0.0134 $1 * p" | dc);
echo "progress $amount $2" >> $OUTFD;
}
# set_progress <amount>
set_progress() {
amount=$(echo "0.0134 $1 * p" | dc);
echo "set_progress $amount" >> $OUTFD;
}

P.S.

I found that "show_progress" isn't working for me: It always jumps to 100% immediately...

This also happens in "normal" EDIFY scripts, so I guess it's because of the TWRP version I have installed (some custom Xiaomi version)...

---------- Post added at 08:57 PM ---------- Previous post was at 08:29 PM ----------

That file works great!

The timeout file in the link didn't work for me (I think it's for 32bit systems), but timeout is included on my phone.

Here's the function I use for keycheck: (Make sure to adjust the location of the keycheck file if needed!!!)

# choose <timeout> <command1> <command2> [<default>]
# <command1> will be run if <timeout> runs out
# If no <default> is provided ("-" or "+") vol- is the default
choose () {
ui_print "You have $1 seconds to press vol+/-";
time=$1;
cmd1=$2;
cmd2=$3;
key=42;	# vol+
if [ ${4:-0} = "+" ]; then
key=21;	# vol-
shift;
fi;
timeout -t $time /tmp/keycheck;
if [[ $? -eq $key ]]; then
$cmd2;
else
$cmd1;
fi;
}

I'm not sure the return values from keycheck are the same for all devices.

To check the return values, start a terminal and run (from the directory containing keycheck):

./keycheck; echo $?

and press "vol+" or "vol-" to check the return value.

If "vol+" is not 42, or "vol-" is not 21, make sure to adjust the "key" variables accordingly!
[Android Software/Hacking General [Developers Only]] Zappo:
I think I found a bug (more like an inconsistency) with "package_extract_dir"

package_extract_dir("system", "/system"); # will unpack system to /system/
package_extract_dir "system" "/system"; # will unpack system to /system/system/

Obviously, an easy workaround is: package_extract_dir "system" "/";
[Android Software/Hacking General [Developers Only]] Captain_Throwback:
Yes, I noticed this too, but forgot to come here and post about it :). Thanks for doing that.
[Android Software/Hacking General [Developers Only]] osm0sis:
Hadn't heard anything more after this report a few pages back. Sounds like the version in the OP works on some devices but not others. [MENTION=378915]Zappo[/MENTION], does [MENTION=5787964]SuperR.[/MENTION]'s quoted newer workaround version work better for you?