yashsingh1/xdaforums huggingface dataset:
[Android Software/Hacking General [Developers Only]] osm0sis:
The great arm64-ing of my modules is upon us!

The Pixel 8 series has ushered in a new age of "arm64-only" devices, meaning arm (32-bit) binaries can't run on them anymore, which means it's time to update all the things!

I work on these projects in my limited time off, so if you like the progress I'm making, or enjoy anything else I've done on xda, please consider sponsoring my ongoing work using my GitHub Sponsors profile. For a one-time donation you can hit the donate link from my profile. Thanks for your support!

magisk-ci_installer.sh:

- reimplement needed find_magisk_apk function as latest Magisk removed it from util_functions.sh

TWRP_A_B_Retention_Script-Magisk(1).zip;

UPDATE-Adreno.Systemless.Installer-signed.zip;

UPDATE-Busybox.Installer.v1.36.1-ALL-signed.zip;

UPDATE-Nexus.Media.Installer-signed.zip:

- update to xdaforums links

Payload-Dumper-Go_for_Android-1.2.2(1223).zip;

UPDATE-adb.Installer.v1.0.41-signed.zip:

- add arm64 builds for arm64-only devices

- update to xdaforums links

UPDATE-Flashlt.Installer.v3.9-signed.zip:

- Diffusion Installer Backend updates

- update busybox binary (1.36.1) with the latest patches and fixes from my repo

- add direct support for init_boot and vendor_kernel_boot name parsing to avoid conflict with *boot* and *kernel* matching

- improve reading partition dump names by parsing out the _a/_b slot suffix

- add support for dumping and writing (including resize/replace) of logical partitions in super (active slot only) [thanks to @capntrips and his work on AnyKernel3!]

- improve MTD/NAND flashing by running flash_unlock on the target partition first

- update flash_erase, flash_unlock, nanddump and nandwrite binaries (mtd-utils 2.1.6)

- don't take first .img filename as partition name for dumping, always default to dumping boot if none specified

- add arm64 builds for arm64-only devices

- update to xdaforums links

UPDATE-nano.Terminal.Editor.v7.2-signed.zip:

- build my own libncursesw.a (v6.4) from upstream GNU ncurses official source

- add arm64 builds for arm64-only devices

- update to xdaforums links

UPDATE-mtd-utils.Installer.v2.1.6-signed.zip:

- add builds of flash_lock and flash_unlock

- add arm64 builds for arm64-only devices

- update to xdaforums links

UPDATE-ZipSigner.Installer.v3.0-13-signed.zip:

- build my own zip binary (v3.0-13) from upstream Ubuntu zip official source/patches

- add arm64 builds for arm64-only devices

- update to xdaforums links

Happy hacking!

Note: The mtd-utils build script is updated already, but the new zip build script (will be added to the zip installer info post) and the updated nano+libncursesw build script (will be within my existing nano build script post) will be up once I'm done polishing them - probably later this week!

Edit

: zip build script added! And nano build script updated!
[Android Software/Hacking General [Developers Only]] SageDroid:
I have removed the ads :) From version 1.1.0+ there will be no more advertising.
[Android Software/Hacking General [Developers Only]] osm0sis:
Oh wow! That's great news! Can wholeheartedly recommend this for everyone's go-to logcat reader now! 🎉😁
[Android Software/Hacking General [Developers Only]] SageDroid:
Thank you, I'm very glad! 😊 Of course I am still happy about everyone who buys the Ultra version to support my work. All proceeds so far have gone 100% into the app and ensure the continued development.
[Android Software/Hacking General [Developers Only]] gigavolthavoc07:
Does my system detects my phone as a Xiaomi 14 If i change the device and model as Xiaomi 14 in pif.prop? I'm using the v13 and it's still working fine but i want to try the v13.1's pif.prop feature
[Android Software/Hacking General [Developers Only]] osm0sis:
No, and ask stuff like that in the PIF thread.
[Android Software/Hacking General [Developers Only]] gigavolthavoc07:
i didn't notice, i'm in the wrong thread, my bad 😅
[Android Software/Hacking General [Developers Only]] HippoMan:
This is great!

I have a suggested enhancement. There was discussion here earlier about the possibility of randomly selected signatures. So perhaps we could do something like this:

If the script takes an optional `--random` command-line argument, or if perhaps an environment variable called RANDOM_PROP is set to a non-empty value, or something like this, the script would expect the build.prop file (or perhaps a different file) to contain a collection of property groups, and one of them would be randomly selected. Or perhaps there could be a group of build.prop files in a directory and one could be randomly selected.

I could implement this, but I won't have time to do the work for another several days or maybe a week or more.
[Android Software/Hacking General [Developers Only]] osm0sis:
Not really what it's for. Your idea would belong more in PIF itself.

My script is just for extracting a pif.prop from a ROM dump's build.prop (from /system, and possibly /product on newer devices), and there are lots of repos of those around if you do a bit of searching. 🙂
[Android Software/Hacking General [Developers Only]] HippoMan:
Oh, I see. I mistakenly thought that this script was meant to be run as part of PIF processing.
[Android Software/Hacking General [Developers Only]] knasiotis:
My previous device is a galaxy s9. Is there any way to extract the fingerprint without rooting it?
[Android Software/Hacking General [Developers Only]] osm0sis:
Factory Images, Full OTAs or someone else's dump. Might be able to get what you need manually from adb shell getprop as well, but I'm not sure.
[Android Software/Hacking General [Developers Only]] HippoMan:
Hmm ... maybe we should create some sort of clearinghouse for this info that we could build up over time.

If we do this, I suggest that it be secure: encrypted, requiring something like two-factor authentication to access, and perhaps even only accessible by invitation only. And it should not be constructed to use any facilities such as Gmail, Google Docs, nor anything else managed by the Googlefrinchians.

Thoughts?
[Android Software/Hacking General [Developers Only]] osm0sis:
Yeah I imagine someone will start a Telegram group or something.

I really just answered "how do I get my device's build.prop" which is highly searchable. 🤷‍♂️
[Android Software/Hacking General [Developers Only]] immortalwon:
I tried this on oneplus pro 9 and it didn't seem to work that effectively. What directory does the information get extracted to?
[Android Software/Hacking General [Developers Only]] osm0sis:
Working fine on my OP9Pro, in fact that's what I wrote it on.

Where did you get your build.prop? It should be in the same directory as the script, then pif.conf will be generated in that same directory as well.

I just updated it to use the build UTC to determine if it's past the March 16 2018 cutoff for not using SECURITY_PATCH, and to add API_LEVEL so that it'll work on latest module (v13.3 stable). Maybe try it again?
[Android Software/Hacking General [Developers Only]] immortalwon:
Why am I getting this error? Should I disable PIF first?
[Android Software/Hacking General [Developers Only]] osm0sis:
That means your supplied build.prop doesn't contain a fingerprint, as the error says. Make sure it's a /system/build.prop from an actual device dump.

I use FX File Explorer to run it from my Download directory, by the way, which is quite handy.
[Android Software/Hacking General [Developers Only]] HippoMan:
I have a request for a slight enhancement to this gen_pif_prop.sh script ...

Could it take an optional command-line argument which gives the value to be used for the `${shdir}` variable, instead of defaulting to the directory name of the script's pathname?

This way, we can optionally tell the script to run against build.prop files in other directories, without having to copy the script into any of those directories before we run it.

Here are unified diffs for this suggested change ...

--- gen_pif_prop.sh 2023-11-22 12:25:21.397197089 -0500
+++ new_gen_pif_prop.sh 2023-11-22 12:49:45.469272034 -0500
@@ -9,16 +9,21 @@
die() { echo "\n\n! $@"; exit 1; }
file_getprop() { grep "^$2=" "$1" 2>/dev/null | tail -n1 | cut -d= -f2-; }

-case $0 in
- *.sh) shdir="$0";;
- *) shdir="$(lsof -p $$ 2>/dev/null | grep -o '/.*gen_pif_prop.sh$')";;
-esac;
-shdir=$(dirname "$(readlink -f "$shdir")");
+if [ $# -gt 0 ]; then
+ shdir="${1}"
+ shift
+else
+ case $0 in
+ *.sh) shdir="$0";;
+ *) shdir="$(lsof -p $$ 2>/dev/null | grep -o '/.*gen_pif_prop.sh$')";;
+ esac;
+ shdir=$(dirname "$(readlink -f "$shdir")");
+fi

-cd $shdir;
+cd $shdir || exit 1

[ ! -f build.prop ] && [ ! -f system-build.prop -o ! -f product-build.prop ] \
- && die "No build.prop files found in script directory";
+ && die "No build.prop files found in script directory '${shdir}'";

item "Parsing build.prop(s) ...";
[Android Software/Hacking General [Developers Only]] osm0sis:
Not that simple. Sometimes you need both the system and product build.props to get all the needed info. I think it's fine the way it is.
[Android Software/Hacking General [Developers Only]] HippoMan:
But I don't see anything in your script other than "$shdir" being used for the pathname of any of the build.prop files (system and product). My change just lets us optionally set $shdir from the command line.

The name of the build.prop file itself is not expected to come in on the command line of my version of the script.
[Android Software/Hacking General [Developers Only]] osm0sis:
Yeah the script tells you how to include a product build.prop when needed by asking you to rename things in the current directory and run it again... It's all there in the script. 🤷‍♂️

People don't seem to understand the script is *not* for running on a device you want to extract a build.prop from, it's meant to be run on the device you want to *use* your custom.pif.prop *on*. That's so it can determine the correct API_LEVEL for your device. Therefore you don't need to point it multiple locations, you just need your dumped build.prop from some other device in the operating directory. Not sure what's so hard about all of this. 😅
[Android Software/Hacking General [Developers Only]] HippoMan:
Well, I used my own, slightly modified version to just generate build.prop data that I can examine, and it worked. That doesn't stop your way of using it from also working.
[Android Software/Hacking General [Developers Only]] osm0sis:
Purchased Ultra to say thanks and keep up the good work! Can you please make the Ultra unlock app hide itself from the app drawer once run or something though? 🙌
[Android Software/Hacking General [Developers Only]] SageDroid:
@osm0sis Thank you for your support, I'm very thankful! 😊 Hiding the app icon dynamically is unfortunately no longer possible since Android 10. I've been rolling out Ultra 1.0.3 for a week now, this version no longer has an app icon and can only be launched via the Play Store and the settings. The rollout is gradual because I don't want to take any risks and I'm a bit worried that Chinese OEMs might treat the app and the background service differently without the app icon.
[Android Software/Hacking General [Developers Only]] Vision:
With Total Commander without root, in the old devices go to root folder after system copy and paste the build.prop. This work for me in my old devices without root
[Android Software/Hacking General [Developers Only]] osm0sis:
Updated my custom pif script!

- output pif.json by default, but support pif.prop still using command line argument

- don't use running device first API level anymore, instead use the fingerprint device's level (requires its vendor build.prop)

- fallback to SDK version if first API level can't be found

- always include first API level to work around current module behavior triggering hardware backed eval when missing
[Android Software/Hacking General [Developers Only]] Usama-CW:
can you please help how can i do it?
[Android Software/Hacking General [Developers Only]] BillGoss:
Use . ./gen-pif-json.sh
[Android Software/Hacking General [Developers Only]] Usama-CW:
this didn't worked

i did

. ./gen_pif_custom.sh

it (cursor) goes to next line without any error and i can't see my custom .json file anywhere.
[Android Software/Hacking General [Developers Only]] BillGoss:
I think there's a bug in gen-pif-json.sh

Line 71 reads [ -z "$UTC" ] && UTC=$(file_getprop build.prop ro.build.date.utc);

And I think it should read [ -z "$UTC" ] && UTC=$(file_getprop vendor-build.prop ro.vendor.build.date.utc);
[Android Software/Hacking General [Developers Only]] osm0sis:
Nope. system build.prop contains that prop.

My suggestion would be to not use Termux.
[Android Software/Hacking General [Developers Only]] osm0sis:
From command line on latest Android you need root to run scripts it seems; you can run it directly with `sh gen_pif_custom.sh` from a root prompt and it'll behave correctly.

Honestly I'd suggest using a file explorer app that lets you tap to execute scripts like FX File Explorer. The script doesn't need root in FX either.
[Android Software/Hacking General [Developers Only]] BillGoss:
As usual, you are correct!

I must have copied the wrong build.prop file.

My apologies.
[Android Software/Hacking General [Developers Only]] Usama-CW:
showing inaccessible error
[Android Software/Hacking General [Developers Only]] osm0sis:
Probably a cut and paste error? I just pasted it freshly into a file to test and it works fine everywhere here.
[Android Software/Hacking General [Developers Only]] Usama-CW:
ok i am about to try again with this file
[Android Software/Hacking General [Developers Only]] osm0sis:
Yours is definitely broken. Fails here. Looks like Windows line endings? You need Linux line endings.

Also, wow! XDA finally allows .sh uploads?! 🤯

Opened it in FX, hit save and it fixed the line endings, so here you go. Attached it to the script post. 👍

Please remove yours so people don't get confused and try it.
[Android Software/Hacking General [Developers Only]] Usama-CW:
Worked, thanks for the explanation.

Edit : i deleted mine, yes i use windows. Also now i see that linux and windows line endings are different. never used linux before ':3
[Android Software/Hacking General [Developers Only]] TraderJack:
Hey @osm0sis ...I've been meaning to try your script for a while..finally got some dumps and trying to process. I'm a bit confused as to when/why you need all 3 props files. The /vendor/build.props I'm looking at for a device now appears to have all the appropriate data but I'm getting:

! No fingerprint found, use a /system/build.prop to start

Is there a reason you aren't using the following properties if found?

ro.vendor.build.fingerprint

ro.bootimage.build.fingerprint

?

thanks
[Android Software/Hacking General [Developers Only]] badabing2003:
because vendor print complements the system print, in most cases it by itself does not have all the necessary props.
[Android Software/Hacking General [Developers Only]] TraderJack:
the issue is that many dumps only contain a /vendor/build.prop

I have tons of dumps that don't have /system/build.prop or /product/build.prop

Additionally, I have found many working prints just from the data within vendor/build.prop.

It seems that by requiring all 3 to exist this tool is severely limited in how it can extract from other scenarios.

I might be missing something, but it would seem more flexible to have the script first parse the /vendor/build-prop file and pull out all values it can find. It can then parse the remaining prop-files to get anything else (or potentially overwrite earlier values if required).

@osm0sis is there a fundamental reason it doesn't/cant do this? I'd like to work with you getting all the appropriate values. I'd rather not just rewrite the code for my own use.

let me know.

some other examples..you are looking in /vendor/build.prop for:

ro.product.system.name

but in some it is:

ro.product.vendor.name

also you don't seem to be doing any error checking on things except FINGERPRINT and API, so it is possible that the other values just come out as null "".
[Android Software/Hacking General [Developers Only]] Homeboy76:
(y)(y)(y)
[Android Software/Hacking General [Developers Only]] osm0sis:
Plenty of vendor build.prop files don't contain much at all. Ones that do can technically have a different fingerprint from the rest of the system, hence system and/or product are the ones to use.
[Android Software/Hacking General [Developers Only]] HippoMan:
If I run adb shell getprop , does anyone know which of these build.prop files are queried? Could the output be the union of the data in all appropriate build.prop files that are found on my device? ... or does it only come from a subset of these files?
[Android Software/Hacking General [Developers Only]] TraderJack:
Can you clarify then for me if all builds should have a system and or build prop file?

Specifically will they have both?

I am seeing a lot of dumps that only have /vendor/build.prop

On some I see that have /product/build.prop, but I have not yet found one with a /system/build.prop.

Actually, I may have seen a system one on dumps of A8, but nothing on the 9/10/11 I've seen.

I have found dumps from A10+ devices that work fine just using the values from the /vendor/build.prop.

So, I guess what I'm looking for is:

- Which versions of Android should have which of those files.

- Assuming there is more than one of those prop files, which ones should take precedence over the others?
[Android Software/Hacking General [Developers Only]] osm0sis:
Pre-Oreo don't have a vendor necessarily and won't have a first API level, but they'll have an SDK level. My post and script explain this.

My script shows you the order of preference.
[Android Software/Hacking General [Developers Only]] TraderJack:
So maybe we won't agree on what the failbacks should be...so I might just adapt yours to suit my needs (thanks for it in any event, saving me work!). But I do have some questions that may lead into possible suggestions for enhancement...

Again, while your script probably works in all "ideal" situations, it doesn't seem flexible in situations where not all the prop files exist. Again, I've found dozens of workable prints from only vendor/build.prop files where those are the only ones that exist in a dump. However I had to modify your script because it won't pull the right values as they are named different there.

I was curious what you thought about the following structure change to the script (again cool if you don't want to incorporate it in your own, but in terms of actual functions I'm thinking about it for my needs). It seems to me since your grep command is only finding the last value wouldn't it make sense/work to just grep on all available files at the same time in order of least preferential to most preferential?

For instance something like:

PRODUCT=$(grep "ro.product.name=" build.prop vendor-build.prop product-build.prop | tail -n1 | cut -d= -f2-)

[ -z "$PRODUCT" ] && PRODUCT=$(grep "ro.product.system.name=" build.prop vendor-build.prop product-build.prop | tail -n1 | cut -d= -f2-);

[ -z "$PRODUCT" ] && PRODUCT=$(grep "ro.product.vendor.name=" build.prop vendor-build.prop product-build.prop | tail -n1 | cut -d= -f2-);

[ -z "$PRODUCT" ] && PRODUCT=$(grep "ro.product.product.name=" build.prop vendor-build.prop product-build.prop | tail -n1 | cut -d= -f2-);

[ -z "$PRODUCT" ] die;

I might not have the order above correct in preference... I think based on your script the priority should be:

/product/build.prop > /system/build.prop > /vendor/build.prop

In either event, I would have to modify your script to include valid entries in just the vendor/build.prop, so the above isn't really much more code. The idea is basically to search for each variable starting with the most desirable one to use and search across all the files, taking the last instance (being the most desirable file listed in the grep). If we can't find it, continue to the next most desirable variable and repeat until we either find one or die because we can't.

Again, I'm not asking you to necessarily adopt this technique, but I might to deal with my issues and wanted to know if you saw issue with it?
[Android Software/Hacking General [Developers Only]] osm0sis:
They're not all always needed though. Sometimes only build.prop is needed and vendor is basically empty/missing so it pulls SDK version props instead.

Some devices system and product fingerprints match but vendor fingerprint is completely different (seems like OEMs using GSIs basically), and the system ones I believe are still the ones which pass integrity in those scenarios from what I've seen.

Edit: I've adjusted the instructions to include "If the device doesn't have a vendor build.prop supply an empty file for it when prompted." 👍
[Android Software/Hacking General [Developers Only]] TraderJack:
I guess I'm not being clear in my statements, or else I'm missing your point.

There are situations when the only available file is "/vendor/build.prop"

So yes, if only /system/build.prop is needed to get the values and the file exists, great! We take those values and move on.

But if those values don't exist, or the file doesn't exist, we have to go with what we've got...and that might only be "/vendor/build.prop".

You are looking in only 2 places for the PRODUCT (e.g.):

/system/build.prop - ro.product.system.name

/product/build.prop - ro.product.product.name

But you never look to:

/vendor/build.prop - ro.product.vendor.name

I have dumps which only have ro.product.vendor.name in the vendor props file, so the script (as-is) can do nothing for me.

If I were to just leave it at that then I would basically forego all those dumps as worthless, but the truth is they have enough data in them to properly pass PI.

So if you get my issue now, the question is wouldn't you rather the script be able to handle these situations when the result is the ability to actually extract more usable fingerprint files from the available data?
[Android Software/Hacking General [Developers Only]] osm0sis:
No, there are no devices where only vendor build.prop exists, that's not how Android works. On ancient devices system is the only partition and there is still a system build.prop there...

And the vendor values often conflict with the correct ones from system build.prop on some devices (OnePlus 9 Pro Android 12+ for example), so no, I don't trust them and intentionally omitted their use.
[Android Software/Hacking General [Developers Only]] DiamondJohn:
And even if Vendor props do exist, that does not necessarily mean that the firmware will actually read/use the values. Personally, I get a copy of all three, which is a lie, as I dont bother much beyond system build prop ;)