yashsingh1/xdaforums huggingface dataset:
[Android Software/Hacking General [Developers Only]] peterlee928:
[MENTION=4544860]osm0sis[/MENTION] thank u for all ur fantastic work. using ur busybox - loving it! i left my former lover stericson busybox for this beauty. never looked back since.
[Android Software/Hacking General [Developers Only]] edzamber:
[MENTION=4544860]osm0sis[/MENTION]

Is there a way to correct this anoying red/orange color in TWRP for S9/S9+ when doing Recovery Screenhots ?

I tryed your GN_Recovery_Color_Correction_Injector_Addon-AK2 script but error 1 when flash.

Many thanks :)

Here my recovery.img if you need to look in ramdisk

https://mega.nz/#!a1MhhYrI!WY5mi2trZGxG43PCxaoI7kZ4I7l08SxzSgQBc2n-LSM
[Android Software/Hacking General [Developers Only]] osm0sis:
You just need to change the color space of the image. Usually RGBA vs BGRA. Imagemagick or ffmpeg can do this IIRC. Google for a build for your PC. Report the issue to your TWRP maintainer as "the incorrect color space is being used for screenshots"
[Android Software/Hacking General [Developers Only]] FriendlyNeighborhoodShane:
[MENTION=4544860]osm0sis[/MENTION]

I'm trying out installing apps to /data using zips just as a fun project. I wanted to ask you, when exactly did android shift from storing data apps in

/data/app/<package>/base.apk

to

/data/app/<package>-<user>/base.apk

? (I'm guessing KitKat but not sure)

Were there ever any places to store them?

And where did the pre-KK apps store their libs? Same as post-KK?
[Android Software/Hacking General [Developers Only]] osm0sis:
That's not user as far as I know, just a random string to prevent conflicts during app updates. That started in Oreo I think? Before that they were /dara/app/<package>-<#>/base.apk. And KK and below they were just named apks in /data/app, and libs were in /data/app-lib I think?
[Android Software/Hacking General [Developers Only]] FriendlyNeighborhoodShane:
Well, I'm on Nougat, and I see that -*. But, yeah, you're right, Now I notice some -2's too even though I have no 2nd user.

I'll download a couple ROMs now to confirm, looks like some flshing is in order.

Thanks
[Android Software/Hacking General [Developers Only]] m0han:
Wireless ADB issue

[MENTION=4544860]osm0sis[/MENTION], [MENTION=528201]NLBeev[/MENTION], tried to backup (another device) using Wi-Fi ADB. nothing gets backed up or logged.

i have renamed the earlier sdcard folder. how can i use the same setup for backing up other devices? thanks.

View attachment 4591793
[Android Software/Hacking General [Developers Only]] NLBeev:
In the screenshots I see in the command prompt the path. This path contains spaces. In bat-files path names with spaces needs to be within quotes.

Try first to get it working with a USB connection. When that works create a 2nd adbsync.bat with the wireless stuff added.

EDIT

When [MENTION=4544860]osm0sis[/MENTION] doesn't mind I can make a zip of all the files in my 'adbsync-working-folder' including the updated adb files.
[Android Software/Hacking General [Developers Only]] m0han:
if you meant the path c:\ADBsync sdcard Backup, it worked for me earlier. today, i replaced the line call bin\adb shell mount /data > nul 2>&1 with the line call bin\adb connect 192.168.1.4:5555. i hope you do make a zip of all the files in your 'adbsync-working-folder' including the updated adb files and share it here. meanwhile, is there anything else i can try? [MENTION=4544860]osm0sis[/MENTION]?
[Android Software/Hacking General [Developers Only]] jaager:
Currently I'm using Magisk 17.1 on LineageOS 15.1. I update my LineageOS build every week and I don't want to flash Magisk manually anymore after OTA update.

Can you please tell me where I have to put your script? I don't want to change anything on /system so is it possible to put the script in "/sbin/.core/img/.core/post-fs-data.d"? Will this work here without changes?
[Android Software/Hacking General [Developers Only]] guest4711:
Scripts in "/sbin/.core/img/.core/post-fs-data.d" will be executed if Magisk is installed only. So you cannot use it to reflash Magisk.
[Android Software/Hacking General [Developers Only]] marcdw:
The -# has nothing to do with users. Think of it more as incremental updates. For instance, Play Store update would be in com.android.vending-1. Way back in the day there were tips on keeping the Play Store app from updating itself (for whatever reason) by uninstalling the app and creating a read-only com.android.vending-1 directory. That didn't work as the Play Store still updated and was now in com.android.vending-2. If one tried it again it would create a -3.

Regardless, neither the -# nor the strings added in Oreo (more of a security measure) have anything to do with users. Nothing to worry about or research unless you want to know how things work.

Moto G5S Plus XT1806, balanceOS 2.0, MultiROM, XDA Legacy
[Android Software/Hacking General [Developers Only]] NLBeev:
ADBsync update

https://www.androidfilehost.com/?w=files&flid=282113
[Android Software/Hacking General [Developers Only]] osm0sis:
addon.d is addon.d, adding to system on a custom ROM is not a big deal. In fact Magisk has its own addon.d that it adds to system to keep itself flashed. On Lineage 15.1 that's addon.d-v2 for A/B devices, which my script doesn't support (since it'll require root for A/B), but Magisk's was updated to support. If it isn't working report it to the Magisk thread and/or on Magisk's GitHub.

I didn't realize people were still using it; I'll have to dust it off, give it a once over and update it for wireless and newer adb. :good:
[Android Software/Hacking General [Developers Only]] NLBeev:
Since Essential PH-1 and TWRP without touch it's a very useful.
[Android Software/Hacking General [Developers Only]] FriendlyNeighborhoodShane:
Okay... Thanks for that. I'll just make a random string (1 integer for kk-n, 24 alphanumerals + =/_/- for oreo and above). Right now, I'm more interested in how /data/app and /data/app-lib work on <kk, so if anybody can give me a few examples, I'd appreciate it.

BTW, I too hated that PS thing and just started using FakeStore (^_~)
[Android Software/Hacking General [Developers Only]] osm0sis:
Like I was saying lp-mm used the single numeral, o-p are the 22 alphanumerals+==.

kk was just /data/app/<package>-#.apk, no subdirs, and I believe used /data/app-lib/<package>/ just like /data/data/ does. Below kk was the same as that I think.
[Android Software/Hacking General [Developers Only]] FriendlyNeighborhoodShane:
Best I could come up with:

getrand() {

num="$1"

until [ "$(echo "$string" | wc -m)" == "$(($num + 1))" -a "$(echo "$string" | wc -l)" == "1" ]; do

string="$(busybox cat /dev/urandom | busybox sed "s|[^-A-Za-z0-9_]*||g" | busybox head -c "$(($num + 1))")" 2>/dev/null

done;

echo "$string=="

}

appdir="$package-$(getrand 22)"

It takes a second or so on average, but it does the job
[Android Software/Hacking General [Developers Only]] osm0sis:
A Google for "generate random alphanumeric string shell" led me here: https://gist.github.com/earthgecko/3089509

Which gives us this:

echo "$(cat /dev/urandom | tr -dc 'a-zA-Z0-9' | fold -w 22 | head -n 1)=="
[Android Software/Hacking General [Developers Only]] jcmm11:
I'll be curious to see if this works. I'm not sure when/how the system assigns a user-id for the app. I do know that the directories in /data/data are owned by the respective app. They shouldn't be an issue though.

I also know there's a bunch of files in /data/system/ that contain package information (including user-id). If these files are updated at first run you should be fine. If they're updated at install time you may run into problems.
[Android Software/Hacking General [Developers Only]] FriendlyNeighborhoodShane:
Thank you. That's comparatively much faster.

The loop was because sometimes the string came up with a newline as one of the charachters. I haven't seen this in this one yet, I hope it doesn't happen.

Also, catting /dev/

random will almost certainly lead to a pipe error and won't even work with android included cats. Your busybox works, my recovery busybox works, I hope other devices have no problems.

Yup, I'm not messing with /data/data (I maintain a MicroG packa, believe me, I know what conflicting data leads to).

I'll wipe all of the app's dirs, including

/data/data/$app

/data/user/*/$app

/data/user_de/*/$app

/data/app-asec/$app*

/data/app-ephemeral/$app*

/data/app-lib/$app*

/data/app-private/$app*

/data/app/$app*

and runtime-perms if the app is found in there. (Not sure I should do that with package-restrictions.xml, that seems important)

If I do that, I'm thinking a loop starting from the highest value from /data/system/packages.list +1 to get a UID, and then echoing the app with default perms onto there would suffice.
[Android Software/Hacking General [Developers Only]] FriendlyNeighborhoodShane:
[MENTION=4544860]osm0sis[/MENTION] [MENTION=6046143]marcdw[/MENTION] [MENTION=4487388]jcmm11[/MENTION] It's aliiiiiive! (Atleast on nougat, it is.) And I didn't even have to deal with packages.list (I'm guessing it's rebuilt on boot-time)

I'd like to Iron out some kinks, like e.g. the successful counter doesn't work (IDK how or why), test it on oreo, and then I'll upload it here.

Also, it hates spaces in filename (understandable, since I used a for loop). But then I changed it into a while read loop and it still behaves the same. It splits APKs with spaces into two entries. Any idea whats going on?
[Android Software/Hacking General [Developers Only]] jcmm11:
My general rule of thumb is - don't use spaces in filenames. That being said quoting the filename should work. Escaping the spaces with a \ definitely should work.

What's the exact command your using?
[Android Software/Hacking General [Developers Only]] FriendlyNeighborhoodShane:
It's looping through the results from find in the directory the zip is placed in. I always planned it as this. The user puts the apk's in an 'Apps' folder in the zip and zips it up, or puts the zip the APK's directory, or puts the APKs in a pre-defined directory. I could just warn them to remove spaces, but it'll be tedious, and I want to avoid that. I'll try sed 's| |\\ |g' on the list, but I'm kinda surprised a while loop doesn't work.
[Android Software/Hacking General [Developers Only]] FriendlyNeighborhoodShane:
Okay, update:

I've fixed the spaces issue. Some compromises, but done. I've figured out the counter issue. A while loop runs in a subshell, and even export could not get a variable through (IDK... Maybe catting and echoing to a temp file would work)

The primary issue right now is this:

Apps get extracted and permed, and everything looks perfect. Buuuut, only apps that weren't installed before show up in the drawer. Apps that were already installed don't. I'm guessing I need to remove them from some file (Nope, it's not packages.list, I can't find these apps there)
[Android Software/Hacking General [Developers Only]] jcmm11:
The way I usually run find, which will handle spaces in file names: (this example will simply get all files (not directories) in the current directory and do an 'ls -l' on each one, individually)

find -type f -maxdepth 1 -print0 | xargs -r0 ls -l'

-print0 causes each result to be null terminated

-r tells xargs not to process empty input

-0 tells xargs each input is null terminated and to not process spaces etc as separators

I prefer using xargs rather than the exec options of find, I just find it simpler in most cases.

(IIRC some older versions of toybox xargs didn't handle the -0 option so you had to use the busybox version. Within TWRP this isn't a problem in any case since that always uses busybox.)
[Android Software/Hacking General [Developers Only]] FriendlyNeighborhoodShane:
That can't exactly help, because it needs to run multiple actions on the file and it doesn't execute functions. For now, I'm stuck in a while loop (bad pun, IK). Maybe I'll just drop the counter thing, even though I wanted to have some summary to the user without having to open the logs.
[Android Software/Hacking General [Developers Only]] jcmm11:
Actually it can run functions, just not inline one's. Predefine the function then call it and it should work just fine. I don't have time at the moment to work up even a quick test but I'm better than 95% certain that would work. How the function handles the variable once it receives it may be another issue, but you should be able to just quote the variable.
[Android Software/Hacking General [Developers Only]] FriendlyNeighborhoodShane:
You'd be right, but we'd need to export the functions. And android exports don't have that option.

Anyway, I've tackled dirty flashing. I sed an app's codePath and nativeLibraryPath if it's found in packages.xml. But I can't seem to do that with system apps. If I sed a system app's path to a data dir, it returns to system on boot.
[Android Software/Hacking General [Developers Only]] FriendlyNeighborhoodShane:
Okay, so I've hashed out something rough, and I'm gonna be busy for a few days so I thought you guys would like something to play around with. It can now update already installed apps, but still can't update system apps (Won't cause any problems, RN it'll automatically skip them)

Also, pre-LP support is a stub and it will abort on these devices.

I've included an aapt for arm, which should also work on arm64, so for other arches you just need to put in a aapt-x86 or aapt-mips in there (The script already has support)

Thanks for all the help.

(And yeah, I do need to work on that name)

Edit: Dumb me, I wrote all that but forgot to give instructions. Y'all probably have figured that out by now, but the APKs are to be placed in this zip's directory or in internal-storage/MInstaller
[Android Software/Hacking General [Developers Only]] osm0sis:
Hmm so the exact error is as follows:

coreutils/date.c: In function 'date_main':
coreutils/date.c:267:23: error: 'struct stat' has no member named 'st_mtim'
ts.tv_nsec = statbuf.st_mtim.tv_nsec;
^
make[1]: *** [scripts/Makefile.build:198: coreutils/date.o] Error 1
make: *** [Makefile:744: coreutils] Error 2

Not sure what to do with that. Newer NDK headers than the ones I use (for compatibility) might handle it better. I'll look into updating my standalone compilers to the last NDK that shipped with gcc in all 6 arches and the unified headers and see where that gets us.
[Android Software/Hacking General [Developers Only]] Renate:
Actually it's only the colon and the backslash.

You want to use Cyrillic, Chinese, Arabic, graphic characters? No problem.

Actually, the Linux chats are full of all the headaches you can get putting colons in file names.

Even regular ADB on Windows can't handle it.

# cd sdcard
# echo hello > hi:there
# ls -l h*
-rw-rw---- 1 root sdcard_rw 6 2018-09-27 14:02 hi:there
# ^D
C:\> adb pull /sdcard/hi:there
0 KB/s (6 bytes in 0.031s)
E:\>dir h*
09/27/2018 14:02 0 hi

Asymmetrical character mapping is a hassle.

Should we map colons in Linux to dollar signs?

Then what do we do with dollar signs from Windows?

Maybe we should map colons from Linux as %3A.

Then should we decode all URI encoding sequences out of Windows?

Maybe we should map colons from Linux as U+02D0, Modifier Letter Triangular Colon.

That would be fun.

Maybe we should just patch offensive Android apps that gratuitously use colons?
[Android Software/Hacking General [Developers Only]] jcmm11:
It's a few more than just colon and backslash. Sticking with printable characters you have:

< (less than)

> (greater than)

: (colon - sometimes works, but is actually NTFS Alternate Data Streams)

" (double quote)

/ (forward slash)

\ (backslash)

| (vertical bar or pipe)

? (question mark)

* (asterisk)

Of those the only one forbidden in 'nix (and, by extension, Android) is / (forward slash).

The above was obtained from https://stackoverflow.com/questions...orbidden-in-windows-and-linux-directory-names

I knew there were more than just those 2 but had to look up the list. Interesting thread (I thought).

Also note that /sdcard is case insensitive (caused by the fuse file system it's mounted as)

Edit: apparently ^ is forbidden on FAT file systems.
[Android Software/Hacking General [Developers Only]] Renate:
Oops, I'm wrong, you're right.

In any case, strangely enough these limitations have never really put a crimp in my lifestyle.

Delving into this, I did discover that in Windows you can these characters in filenames:

U+00A0 No-Break Space

U+2002 En Space

U+2003 Em Space

U+2004 Three-Per-Em Space

U+2005 Four-Per-Em Space

U+2006 Six-Per-Em Space

U+2007 Figure Space

U+2008 Punctuation Space

U+2009 Thin Space

U+200A Hair Space

U+200B Zero Width Space

U+200F Narrow No-Break Space

The bottom line is, why would you want to use characters in filenames that would be annoying,

incompatible with major operating systems or problematic as a URL?

And why can't I put an EBCDIC "Numeric Backspace" in a Linux filename, huh?
[Android Software/Hacking General [Developers Only]] jcmm11:
As for the first question - I wouldn't ? Heck I don't even use spaces in my file names, and dislike it when others do. (They make things difficult at times).

As far as the numeric backspace goes I believe that you can. Just use the number 6

EBCDIC NBS = \x36 = ASCII 6 ?
[Android Software/Hacking General [Developers Only]] osm0sis:
My problem was always with :no_pm: or whatever that Google's old pre-Messages app used to add to pics downloaded from MMS. Shrug. Guess we'll mark it #wontfix ;)

Just thought about this some more. Anywhere that isn't encrypted you could use? You could try setting sdzipdir= at the top of 99-flashafterupdate.sh to somewhere else. That's why I included that up there to make it easy to change for user preference and cases like yours.
[Android Software/Hacking General [Developers Only]] jhedfors:
Hi [MENTION=4544860]osm0sis[/MENTION],

Your Magisk module "TWRP A/B Retention Script" links to this thread, but I cannot find it anywhere.

Do you have a post that you explain the module? Of so, perhaps you can fix the link in the README.md?

Thanks!
[Android Software/Hacking General [Developers Only]] osm0sis:
;)

Unreleased in this thread currently. The module readme and description in Magisk Manager describe exactly what it is and how to use it.

Also: https://github.com/Magisk-Modules-Repo/twrp-keep/blob/master/README.md
[Android Software/Hacking General [Developers Only]] jhedfors:
Cool thanks. Lookng at the Magisk install logs, it looks like you patch the new boot image with the same TWRP that is in the active one. Thanks for your all your hard work!
[Android Software/Hacking General [Developers Only]] osm0sis:
Bingo! :cowboy:

It would be pretty easy for Magisk Manager to do this itself, so hopefully [MENTION=4470081]topjohnwu[/MENTION] can add it as an option in the future.

We need an interoperable root ecosystem again. A/B and system-as-root were a big setback to the root community in this respect.

I've also submitted an updated generic TWRP A/B installer zip template to the TWRP team ([MENTION=912474]Dees_Troy[/MENTION] and [MENTION=2638973]bigbiff[/MENTION]) with fixes to allow it to be flashed from Magisk Manager and to keep it from putting the device into a loop booting to recovery when flashed over a rooted boot.img. [MENTION=1162986]Captain_Throwback[/MENTION] on the U12+ and [MENTION=3873953]eng.stk[/MENTION] on the OP6 are already using it in their TWRP builds. Hopefully it gets officially adopted! :fingers-crossed:
[Android Software/Hacking General [Developers Only]] NLBeev:
Tried the script in the Essential PH-1 with Magisk manager.

The recovery in opposite slot stays stock.
[Android Software/Hacking General [Developers Only]] osm0sis:
Oh weird.. Seems like the script's test for whether TWRP is present in the ramdisk is still passing, even though I had verified it working before.. maybe it's something that changed in Pie.. Thanks for the report.

Edit

: Ooh, looking at the log it's something that changed in the newer Magisk builds most likely.. flash_boot_image doesn't exist anymore apparently and the AVBv1 boot signer is broken. I'll figure it out and try to support both old and new..

Edit 2

: The boot signer thing looks like a Magisk bug that is fixed in the latest Canary (17202) since it works fine for me in a Magisk install there, and I have added a flash_boot_image wrapper function for flash_image when flash_boot_image isn't found. Should be good now! :good:

https://github.com/Magisk-Modules-Repo/twrp-keep/commit/5337a9d4e70a802f33094d425ccbcd1af19dfda8

Edit 3

: The boot signer wasn't actually fixed, it just only works when run from the Magisk installer. I've added another workaround that will at least work when Manager is unhidden. That said, signer more than likely won't be needed on A/B devices since they virtually all use AVBv2, not v1.

https://github.com/Magisk-Modules-Repo/twrp-keep/commit/5442b84e7f7ea52050d01dc3bf34bbea2825fc4c
[Android Software/Hacking General [Developers Only]] osm0sis:
TWRP A/B Retention Script Zip Module

We need an interoperable root ecosystem again. A/B and system-as-root were a big setback to the root community in this respect.

To help alleviate this issue, this atypical Magisk script zip "module" will keep TWRP installed to both slots when flashed from Magisk Manager after an A/B OTA has installed in the background, but must be used before you use Manager to install Magisk to inactive slot.

I've updated the README instructions to be more clear, and since the cat's well out of the bag, I guess we can consider this the release post now. :p

Changelog:

https://github.com/Magisk-Modules-Repo/twrp-keep/commits/master

Instructions:

https://github.com/Magisk-Modules-Repo/twrp-keep/blob/master/README.md

Enjoy!

Download or Install it again from the Magisk app Downloads section to get the latest version.
[Android Software/Hacking General [Developers Only]] NLBeev:
Tested again, after making sure there's no TWRP in the inactive slot.

Installed with Magisk manager. TWRP is installed on both slots now.

Works on the Essential PH-1.

Thx for this semi- (but great) module.
[Android Software/Hacking General [Developers Only]] notstevek:
Hmm, maybe if I get bored I'll try it out, thanks for the heads up on that, didn't even think about changing it tbh. it's not a big deal though, it's one of those things that aren't necessary, just cool/make your life just a little easier. thanks again!
[Android Software/Hacking General [Developers Only]] oF2pks:
[MENTION=4544860]osm0sis[/MENTION] , jic some needs it, I've upload an (android/system/bin/)pcre2grep version 10.22 2016-07-29 arm64 : (Attached Files/MagiskSystemless) https://xdaforums.com/android/software-hacking/dexdump-xodus-trackers-apk-static-t3833391

Google planned to move aosp to it, then switched back to regular : https://android-review.googlesource.com/q/grep+pcre

I'm also wondering about now missing (security ?), in customRoms, for aapt & curl: these can be very helpy for non-rooted devices to perform actions (like the ones used in linked εxodus thread).
[Android Software/Hacking General [Developers Only]] beez1717:
Does this mean we can do an ota update or do we need to sideload the ota?
[Android Software/Hacking General [Developers Only]] osm0sis:
It's not clear what you're talking about. If you mean the Nexus 5X/6P BLOD Workaround AK2 zip then read the OP of its thread for all the answers.

If you mean the TWRP A/B Retention Script Magisk Module then read the README.
[Android Software/Hacking General [Developers Only]] HemanthJabalpuri:
[MENTION=4544860]osm0sis[/MENTION]

Your init.d enabler is working well in my S4 mini running on stock kitkat 4.4.2 with custom kernel...

Can you please enable init.d with your init.d enabler manually to the below attached boot.img please..

Since copying boot from block adds some extra size to kernel.

Thanks
[Android Software/Hacking General [Developers Only]] osm0sis:
Nope. It doesn't make it bigger that's just the size of the partition... it's normal size when the zip repacks it. You can go to my Android Image Kitchen thread if you want to play around with doing it manually. It's the exact same tools and process.
[Android Software/Hacking General [Developers Only]] nxb:
basename is missing from busybox so I can't do an efs backup with efs pro. Kinda sucks.