yashsingh1/xdaforums huggingface dataset:
[Android Software/Hacking General [Developers Only]] jcmm11:
That's because a mac is based on BSD. BSD and 'nix are very similar but not quite identical. If this was a 'real' Linux distribution there's a good chance that ax (and possibly even -ax, with a warning) would have worked.

[MENTION=6037748]Zackptg5[/MENTION] I'll get you those logcats in a little while.
[Android Software/Hacking General [Developers Only]] jcmm11:
Logs attached. New coreutils works - no more bad system call. Want anything else just ask.
[Android Software/Hacking General [Developers Only]] Zackptg5:
Dang, I thought coreutils 8.31 fixed this issue, guess not for all roms. And just like before, the logs show nothing :/

Both sort and timeout binaries have this problem but the rest of the coreutils are fine. I'll look into it once my exams are over. Thanks!
[Android Software/Hacking General [Developers Only]] Zackptg5:
Think I have the paths worked out. There's a couple in here that you can change to whatever you want - I made it sdcard/coreutils

It's just a shell script so you can open it with a text editor and compare, I included the original one and the one I changed so you can see what I changed. Let me know if there's a better place to store locatedb. I chose sdcard because it didn't need root permissions to store it there
[Android Software/Hacking General [Developers Only]] micgeb:
I have added this 3 lines but now I get arg errors.

frcode="/system/xbin/frcode"

bigram="/system/xbin/bigram"

code="/system/xbin/code"

find does not support the parameters in the script. We need another version of find.
[Android Software/Hacking General [Developers Only]] jcmm11:
I haven't tried running this yet but just took a (very) quick look into the script. Saw this code:

# The directory containing find.
if test -n "$BINDIR" ; then
: BINDIR already set, do nothing
else
: ${BINDIR=/system/bin}
fi

# The names of the utilities to run to build the database.
: ${find:=${BINDIR}/find}
: ${frcode:=${LIBEXECDIR}/frcode}
: ${bigram:=${LIBEXECDIR}/bigram}
: ${code:=${LIBEXECDIR}/code}

It looks like GNU Utils installs to xbin for you. So you need to also add

find="/system/xbin/find"

the GNU version of find does support fstype (at least according to it's help). Toybox version doesn't (at least it's not referenced in help).
[Android Software/Hacking General [Developers Only]] micgeb:
I must copy find from the gnu folder in /system/xbin. Find was in this folder a symlink to busybox.
[Android Software/Hacking General [Developers Only]] micgeb:
locate find nothing but updatedb is created.
[Android Software/Hacking General [Developers Only]] Zackptg5:
[MENTION=7520659]micgeb[/MENTION] [MENTION=4487388]jcmm11[/MENTION]

Have a test build here for GNU tools mod for you all to try out.

https://zackptg5.com/testbuilds/GNU-Utils-Android_v2.2-test.zip

Current changelog:

* Attempt to fix paths in binaries and such

* Always install to /system/bin

* Have bash as default shell using mkshrc - true solution, doesn't break anything so far as I know

* Update awk to v5.0.1

* Add more files outputted from compile rather than just binaries

* Recompile all binaries - use NDK when possible

* Separate sort and timeout binaries from single coreutils binary again since some roms have issues with them

I'm looking more into locate and updatedb now and here's where I'm at so far:

You first must run updatedb as root (this is the same bahavior as in linux) - this will create /sdcard/coreutils/locatedb

Then you can run locate, specifying the above database:

su -c updatedb
locate -p /sdcard/coreutils/locatedb apk

I'm looking into how/if I can change the default database path for locate currently but the above works

Edit: Think I found something - compiling it and testing now. Also just noticed that I used sdcard/coreutils instead of findutils lol. That'll be fixed in next build too
[Android Software/Hacking General [Developers Only]] jcmm11:
clear gives me an error

"terminals database is inaccessible"

Still haven't played with updatedb or locate. I'll probably set an alias for locate

alias locate='locate -p /sdcard/coreutils/locatedb'

timeout/sort issues seem resolved. There was something else but I'll have to check what it was.
[Android Software/Hacking General [Developers Only]] Zackptg5:
Reuploaded test build. Recompiled findutils - think I got the paths worked out. I also changed coreutils symlink so it won't replace toybox applets (except cp and mv since they're special)
[Android Software/Hacking General [Developers Only]] jcmm11:
Only symlink you missed is awk (which points to gawk). So my scripts went from deleting 77 items to only 4.

awk

clear

[

test

Updatedb is still a problem. Not sure if it's fixable, at least not practically fixable. (There's ALWAYS ways to fix things. Whether it's worth it or not is another story. A rewrite from scratch will almost always do it ? )

See attached image for first problem. Or not - XDA is misbehaving again. Tried to run updatedb as root. Errored out with

# updatedb
/system/bin/find: File system loop detected; '/sbin/.magisk/mirror/system_root' is part of the same file system loop as '/'

So I tried to fix it with

updatedb --prunepaths='/sbin/.magisk'

That finished, apparently without error. So I try to run a locate. The response I got was

"locate: locate database '/sdcard/gnu/var/locatedb' is corrupt or invalid"

Question - what's the story with the awkscripts?
[Android Software/Hacking General [Developers Only]] Zackptg5:
Ugh, I don't know what's up with updatedb then, I can't even replicate that error :/

The awk symlink is meant to be there regardless of what's present - it's not a part of coreutils but is gawk (it's own separate binary) so that's by design.

For awkscript, I found those awk scripts after compiling and figured I might as well include them
[Android Software/Hacking General [Developers Only]] jcmm11:
Almost certain the error is caused by system-as-root (and the way Magisk handles it), which you won't have on older phones. Don't know if they'll get the locate error though.

awk (at least on this phone) is it's own compiled binary so I'd rather leave it. That way I have all 3 available (default, gawk, and busybox awk). But that's my choice.
[Android Software/Hacking General [Developers Only]] Zackptg5:
Makes sense, idk how to fix that issue though :/

Fair point on awk, I'll make it behave like coreutils symlink. On my device, there's no [ so a coreutils symlink is made for it. Test is tricky cause I guess it's a shell built-in - no binary/symlink present. Clear is a separate binary in this zip which is why it just replaces the toybox applet
[Android Software/Hacking General [Developers Only]] jcmm11:
test and [ are both shell built-ins (on both the default shell and bash)

[[ is a reserved word.

Pretty sure [ was causing some type of problem which is why I decided to remove it. Clear I removed because it's giving me an error (see earlier posts).
[Android Software/Hacking General [Developers Only]] fawazahmed0:
[MENTION=6037748]Zackptg5[/MENTION] kindly look into this:

https://github.com/Zackptg5/Disable_Dm-Verity_ForceEncrypt/issues/8
[Android Software/Hacking General [Developers Only]] Zackptg5:
Can you give this a shot and let me know how it works? Only problem binary/symlink left that you mentioned is clear. I'm still not certain if I want to remove it or not (I have no problems on my end)
[Android Software/Hacking General [Developers Only]] jcmm11:
Good news!

awk, test, and [ are gone

Bad news - 70+ others are back (base64, basename, cat, chcon, chgrp, chmod, chown, chroot, cksum - etc)

I'm also going to delete updatedb & locate since they absolutely don't work for me. If they work on older setups, I don't know.

Last point. I'm not sure which would take precedence, but I already have a custom mkshrc file so I deleted yours. I like having bash readily available, but I'm also happy with mksh (and the differences aren't that huge, although there are some).

If I delete mkshrc should I also delete /data/adb/modules/gnu/gnu-files?
[Android Software/Hacking General [Developers Only]] Zackptg5:
What's in the gnu-files folder? You can keep your mksh. All the install does is add "bash" to the end of it so whenever you open a terminal, it starts bash immediately
[Android Software/Hacking General [Developers Only]] jcmm11:
Not a folder. It's a file that contains 1 line

system/etc/mkshrc

I realized what it contains, I looked before I deleted it ?

Some additions you may want to add to bashrc

cu() {
coreutils --coreutils-prog=${@}
}

alias bb='busybox "$@"'

I tried making cu a simple alias as well but it didn't work for some reason. The reason for having them should be self-evident

Hmm. One thing I haven't tried, but should work.

#!/system/bin/bash

as opposed to

#!/system/bin/sh

Should force a script to use bash instead of mksh
[Android Software/Hacking General [Developers Only]] Zackptg5:
Good idea, I'll add that in.

Turns out I have way less free time than I thought I would so I'll be changing how I update this. The zip is impracticable since different people want different things and dropping a new zip for each update is annoying so I'll be reworking it. Not entirely sure how I'll do it yet but it'll likely be a term script mod or I'll just host the binaries and other files
[Android Software/Hacking General [Developers Only]] Zackptg5:
[MENTION=4487388]jcmm11[/MENTION] Got binaries and stuff up here: https://zackptg5.com/downloads/binaries

I'll be updating the website when I get the chance and maybe make a term script for them
[Android Software/Hacking General [Developers Only]] jcmm11:
Is any of that different from what I already have? I actually have no problem with the way you have it currently packaged. With the boot scripts I have in place it all ends up being transparent to me. I suspect most people don't modify the mkshrc file and prefer bash - although bash could probably be it's own package.

Other than that the only thing that I think isn't wise (and that you had been correcting) is to overwrite the toybox symlinks. Even if the GNU versions are better (and they usually are) something may be expecting the default behavior and if they don't get it things may break. But that's why I have the aliases so it's easy to grab a different version of I want/need it.
[Android Software/Hacking General [Developers Only]] Zackptg5:
Nope, same binaries. That's a good point. I think I'll drop a final release with a term script that can be used to update the binaries or something. I'll figure something good out :)
[Android Software/Hacking General [Developers Only]] Zackptg5:
Looked over the code again (it's been a long week) and those symlinks shouldn't have been created if a file or symlink exists unless it was the cp or mv applet so I'm not sure what the issue is there
[Android Software/Hacking General [Developers Only]] jcmm11:
I can try again, but I know they were there. The previous version they weren't. I'll run it again, with debug, and see if I can find anything.

Just a note that because of the interplay between them and /system/bin I always install both this and busybox in TWRP. My scripts prefer (for the most part) toybox, gnu, busybox (There's like 11 bb modules that I remove so they don't conflict with gnu).
[Android Software/Hacking General [Developers Only]] jcmm11:
Ok, I must be going senile or something. I may have installed the wrong ZIP before? I don't know what happened.

Anyway the only things I'm removing now are: clear, updatedb, and locate.

There's also no mkshrc? And no awk scripts? On those last 2 just wondering if I'm going nuts again. File I installed this time was GNU-Utils-Android_v2.2-test3.zip
[Android Software/Hacking General [Developers Only]] Zackptg5:
Finally pushed an official update - ended up calling it 3.0 since there were quite a few changes. Fixed the bugs with awk scripts and mkshrc and cu function so it'll work in mksh and bash.

cu cp --help

Thanks for all of the help troubleshooting!
[Android Software/Hacking General [Developers Only]] Zackptg5:
[MENTION=4487388]jcmm11[/MENTION]

Pushed another update to the gnu mod. Removed clear binary (not worth keeping for the trouble it causes) and tweaked scripts so it will cooperate and not conflict with terminalmods module (which I highly recommend - complements this great)
[Android Software/Hacking General [Developers Only]] Zackptg5:
Quick update: I switched back to OOS and found that theming was broken. Turns out that it requires on sort, timeout, or both and those 2 binaries have seccomp issues on some roms including oss. Deleting them solved the problem so if you're having weird issues with this mod, check that those binaries work and delete them if they don't (I made a note in the OP about it)
[Android Software/Hacking General [Developers Only]] jcmm11:
Not critical but

tar --help

Returns:

FORTIFY: %n not allowed on Android

Aborted

I get the same thing with

tar -?

Or just plain tar with no options, although in that case it does first tell me I need an option and lists what I need at a minimum.

Hope things are going well. Did you really think you'd have more free time after passing the boards? Silly rabbit ??
[Android Software/Hacking General [Developers Only]] Zackptg5:
Oh dang. Looks like some android security feature is blocking it

Ya, I was wrong about the free time thing, I have none of it haha :eek:

Edit: Seems it was a security thing of some kind related to ndk so I just recompiled with linaro and gcc. Updated zip and build script readme (zip had a couple other changes related to alias and bash calling stuff I was sitting on for a while)
[Android Software/Hacking General [Developers Only]] Harshiv989:
It breaks SCR Screen Recorder PRO app

OS: Android Lollipop 5.1.1(sdk22)

Arch : ARMv7a

before you say "go ask the SCR dev", disabling the module solves the issue, thus it is making some changes that cause issues.

When this Magisk module is disabled, SCR's "record internal audio" mode works fine, but when enabled that mode, which temporarily replaces audio libs and conf files, SCR errors out, saying that it can't put a text file in .bak backup in the same dir.

I know that as I went through the code(SCR is opensource, github) and the error occurs when the c++ function "open(filename,type)" fails to create/copy the original file into orig.bak style file.

Now this module doesn't replace any ".so" libs at all, so culprit could possibly be something from "/system/bin" as I tested the bash shell launching from "/system/etc/mkshrc", removal of exec'ing bash line doesn't solve it, thus isn't the source of problem. Thus anything in "/system/etc/*" likes of bashrc files should not be the culprit.

1. Is it possible that one of the binaries from "/system/bin/" can affect the working of a separate binary residing in "/data/data/" ? In other words, something from GNU coreutils affect the working of the said separate binary.

2. Is it possible that the changes of bash related files or alias files(I haven't added any) could affect running of the said separate binary ?

the binary or java code from SCR pro app doesn't exec anything from "/system/bin" other than calling to

1. "kill" a process called "mediaserver" from c++ code, 2. "mount,unmount" from java AND/OR c++ code,

3. Disable SELINUX temporarily with "setenforce"

Since none of the above 3 process results show an error in logcat, only backing up of .conf file does, I just can't seem to find what the module changes that breaks the SCR binary.

Update :

It was an issue with how Magisk mounts it's modules. What I did was to permanently put the GNU Utils binaries and corresponding symlinks in proper directories instead of allowing Magisk to

pseudo mount. This fixed the original SCR issue.

Drawback is that "/system/" is modified, I've modded tons of stuff in there anyway. Other thing is that when GNU Utils has an update,I'll have to manually installed stuff.
[Android Software/Hacking General [Developers Only]] Zackptg5:
I'm assuming you're talking about the GNU mod right? (This thread has several mods on it)

So the gnu mod primarily replaces and adds binaries to system/bin. Any binary in system bin is in PATH and so doesn't need the whole location spelled out when executed (so instead of saying '/system/bin/cp', you can just say 'cp' for example)

It's very possible that the app is expecting one of the binaries to behave in a certain way and it's not. The 2 problem binaries are sort and timeout. You can call those in terminal and see if they work of if they throw an error (don't use root when trying this). If they throw an error, then remove those from the magisk module folder and see if that fixes the problem. Otherwise, only thing I can suggest is to remove them all and try adding them back one at a time until you find the culprit

Assuming your on latest magisk, path is: /data/adb/modules/gnu/system/bin
[Android Software/Hacking General [Developers Only]] Harshiv989:
Thanks but the timeout and sort both seems to work fine w/o any errors.

I installed the Magisk module with GNU core-utils, and a symlink is made to a binary named the same in "/system/bin".

since the Java part of app depends on it's native part and native part has move/copy etc functions implemented in the code, it's a bit difficult to say app depends on those binaries, mount and unmount also works okay, restricted access would mean SELinux trouble I suppose.

The error specifically says, "device or resources busy" when copying "audio_policy.conf" to "audio_policy.conf.bak" and many system files are successfully overwritten before this file, the conf file is locked on by a program I suppose that isn't being killed by the app while the core-utils Magisk module is active, sadly nothing such is reported in the logs.

thanks anyway, guess I've to dig in android system code deeper...
[Android Software/Hacking General [Developers Only]] FuLygon:
will kernel sepolicy patcher cause any problem on stock miui boot image (after magisk) ?
[Android Software/Hacking General [Developers Only]] Zackptg5:
It shouldnt but always make a backup before flashing anything
[Android Software/Hacking General [Developers Only]] td32:
hi

is there any chance to have a grep binary with perl-regexp enabled

thank you
[Android Software/Hacking General [Developers Only]] jcmm11:
Version of grep in Gnu-utils

grep -P

or

grep --perl-regexp

PATTERNS are Perl regular expressions
[Android Software/Hacking General [Developers Only]] td32:
yes, -P is not enabled in the binary posted above
[Android Software/Hacking General [Developers Only]] jcmm11:
You're right. My bad.

grep: support for the -P option is not compiled into this --disable-perl-regexp binary

I know Zack is pretty busy nowadays but hopefully he can find some time to recompile.

[MENTION=6037748]Zackptg5[/MENTION] - while you're recompiling:

cpio

gzip

rmt

All are getting "FORTIFY: %n not allowed on Android" when using --help. That should be the last of those. I just checked each standalone compile.

grcat

pwcat

Are getting segmentation faults. Since we don't have regular group or password implementation these are probably useless in any case so can be removed. Their function is to dump the group or password database.

What is "toe" used for?
[Android Software/Hacking General [Developers Only]] Zackptg5:
Thanks for the details, I've been pretty swamped lately but I'll try to look into this. I have no idea what toe is but it's a part of ncurses so I just stuck it in there. The fortify errors are likely NDK related so I bet those will go away if I use a different toolchain. Segfaults are a headache so there's a good chance I'll just get rid of those binaries
[Android Software/Hacking General [Developers Only]] Surge1223:
First dev thread I've seen here in a long time. Keep up the good work, nice job on the cross compiles.
[Android Software/Hacking General [Developers Only]] Zackptg5:
Got around to revisiting one of my needs improvement projects: curl cross-compile and am happy to report that I finally got it where I want it. Old build repo is now defunct and new one is here if you want to check it out:

https://github.com/Zackptg5/curl-boringssl-android

Updated the curl unity addon too - it contains dynamic binaries with tons of features enabled https://zackptg5.com/android.php#curl.

If you need static ones for an installer zip or something (I can't see a real reason to use it over the dynamic ones unless you're running older than lollipop), you can grab them from here: https://github.com/Zackptg5/Unity-Addons/tree/master/External-Tools
[Android Software/Hacking General [Developers Only]] Zackptg5:
Finally got around to looking more into GNU and ended up reworking the build script and as a result, the mod here. Dropped v4.0 update. Changelog:

Update findutils to 4.7.0

Update bash with to 5.11

Fix Fortify issues with cpio, gzip, and tar

Add perl regex to grep

Switch everything to NDK

Fixed sort and timeout in coreutils (almost, sort can't read in files so just pass it through instead - ex: cat file.txt | sort)

Switched to dynamic link (except bash and gawk)

Removed ncurses - no reason for it

You can grab it here: https://zackptg5.com/android.php#gnu
[Android Software/Hacking General [Developers Only]] xdauser'17:
hi,

probably you can add a line like

[ "`which bash`" ] && exec bash -l

to mkshrc to make bash the default shell. i think this should be the most "correct" approach. even though sourcing aliases file is no longer meaningful here.
[Android Software/Hacking General [Developers Only]] Mickey03:
I can't use the latest version on CURL 2.0. I am still using Version 1.2. Because my device becomes bootlooped after the installation of the latest version and reboot.....

Can you help me please?
[Android Software/Hacking General [Developers Only]] Zackptg5:
Updated Curl and GNU mods to latest magisk module template.

Changed curl binaries to static - dynamic ones didn't work in Q for some reason - DNS error - and I don't have the time to look into it
[Android Software/Hacking General [Developers Only]] Zackptg5:
Pushed hotfix/redo to Curl - turns out static binary doesn't work in Q either if not run as su so I just disabled the mod completely for Q+ and restored the original dynamic binaries (have more features than static ones). Curl is now included in Q as part of AOSP anyways
[Android Software/Hacking General [Developers Only]] Zackptg5:
Pushed a flurry of minor bug fixes to curl and gnu :/
[Android Software/Hacking General [Developers Only]] Mickey03:
I am not using Q, I am on Nougat.

I can't use latest curl. It creates bootloop after installation and first reboot.

I am still using version 1.2
[Android Software/Hacking General [Developers Only]] Zackptg5:
I'll need logs or something cause it shouldn't. All it places is curl binary and a lib
[Android Software/Hacking General [Developers Only]] Mickey03:
How can I take log? It doesn't boot properly So.....

Is there a way to take the log?

Can u help me please?
[Android Software/Hacking General [Developers Only]] Zackptg5:
Unity is now deprecated and has been replaced with MMT-Ex which is magisk only. So my curl and gnu zips will be magisk only from now on. The old unity zips are still up on my site though
[Android Software/Hacking General [Developers Only]] Zackptg5:
Added new build script - Zsh shell. I'll likely be tweaking this down the road but seems to work fine
[Android Software/Hacking General [Developers Only]] Dexer125:
[MENTION=6037748]Zackptg5[/MENTION] Hi, I tried your Magisk overlay fix module with Quick switch and it doesn't work, GMS still crazy. My device is Xiaomeme Mi 8 (Dipper) Havoc OS, full Q blobs, latest Magisk canary. If you need tester, I'm here.
[Android Software/Hacking General [Developers Only]] Zackptg5:
Does your device have overlay mounts? I'd need you to post the output of /proc/mounts with and without the module installed?
[Android Software/Hacking General [Developers Only]] Dexer125:
It says can't execute: Permission denied. How to properly show it?
[Android Software/Hacking General [Developers Only]] Zackptg5:
Just updating the thread here, there's no overlay mounts on this device unfortunately