[Android Software Development] DualJoe:
Native ARM/static Linux binaries

(for all ARMv7+ compatible platforms)

Open-source Linux binaries that are either not available on Android (e.g. in Termux)

or make sense to be statically compiled (e.g. to run in TWRP/recovery for data recovery).

These are root tools and might damage your device severely. Use at your own risk. I take no responsibility whatsoever. If in doubt don't use them.​

Minimum CPU: ARMv7/vfpv3-d16. Compiled against musl-libc/Android Kernel 3.4. Binaries are static, bionic/libc independent and should run on Android, TWRP, emulator or any other compatible ARM device. Musl is patched (info)(info2)(patch file: patch -p0 -u -b -i musl-android-smp.patch) to iterate CPU cores by /proc/stat instead of _SC_NPROCESSORS_CONF/sched_getaffinity to prevent false detection due to ARM cpu core powersaving (permanently turning cores on/off). This should report CPU cores more reliably to multithreading apps.

Example instructions how to build EncFS can be found here.

Some Cryptsetup compile recipes are here.

Changelog:

20240622 - cryptsetup v2.7.3. Thx to @misterhsp

20240620 - 7za/7zsfx v24.07. Thx to @xenosaur

20240613 - f2fs tools v1.9. Thx to @xenosaur

20240602 - EroFS v1.7.1 and squashfs v4.6.1. Thx to @xenosaur

20240531 - Zstd v1.5.6. Thx to @xenosaur

20240519 - 7zip v24.05. Thx to @xenosaur

20240421 - mmc-utils updated. Thx to @xenosaur

20240409 - cryptsetup v2.7.2 updated. Thx to @misterhsp

20240322 - fscrypt v0.3.5 updated. Thx to @misterhsp

20240321 - gptfdisk v1.0.10 updated. Thx to @xenosaur

20240311 - veracrypt v1.26.7 updated. Thx to @misterhsp

20240308 - cryptsetup v2.7.1 updated. Thx to @misterhsp

20240126 - cryptsetup v2.7.0 updated. Thx to @misterhsp

20230812 - 7z-std v23.01 Thx to @xenosaur

20230613 - gocryptfs v2.4.0 Thx to @misterhsp

20230504 - fscrypt v0.3.4 Thx to @misterhsp

20230307 - gocryptfs v2.3.1. Thx to @misterhsp

20230215 - cryptsetup v2.6.1 updated. Thx to @misterhsp

20221213 - f2fs tools v1.15.0 fixed (uuid.h missing)

20221129 - cryptsetup v2.6.0 updated. Thx to @misterhsp

20220924 - fdisk v2.38.1 and file v5.43 added

20220922 - veracrypt v1.25.9 updated

20220913 - gocryptfs v2.3 updated. Thx to @misterhsp

20220913 - htop v3.2.1 added

20220910 - rsync v3.2.6 updated

Older:

Spoiler content hidden.

Data recovery tools:

- PhotoRec 7.2 - PhotoRec is file data recovery software designed to recover lost files including video, documents and archives from hard disks, CD-ROMs, and lost pictures (thus the Photo Recovery name) from digital camera memory. PhotoRec ignores the file system and goes after the underlying data, so it will still work even if your media's file system has been severely damaged or reformatted.

- Testdisk 7.2 - Recover lost partitions and partition tables. For external sdcards. Never use it on internal mmc unless you know what you're doing.

- ext4magic 0.3.2 (with supplementary gnu date binary that can handle relative time like 'date -d "-20minutes" +%s')

- fidentity - A little utility sharing PhotoRec signature database. It identifies the type of data contained in a file and reports the extension as seen by PhotoRec.

- debugfs - Might be helpful on ext2 systems or other stuff.

- strace 4.20 - For debugging. Mainly to catch syslog messages (as Android has no traditional /dev/log buffer).

- strace 5.6 - For aarch64.

- ddrescue v1.25 - Data recovery tool for block devices with errors.

Compression tools:

p7zip v17.01 (fork) - (Download) A new p7zip fork with additional codecs and improvements

pixz - Parallel, indexed xz compressor

xz - Multicore aware version of xz/lzma (use --thread=0)

tar v1.32 - Tar provides the ability to create tar archives, as well as various other kinds of manipulation. Download below. More builds from @mirfatif here.

fxz - (Download) FXZ Utils is a fork of XZ Utils. It adds a multi-threaded radix match finder and optimized encoder.

Misc:

- hexcurse v1.60.0 - Hexcurse is a curses-base hex editing utility that can open, edit, and save files, editing both the hexadecimal and decimal values. 'ncurses' ui layout depends on TERM env variable. Change temporary with eg. 'TERM=xterm-256color hexcurse <file>'. See /system/etc/terminfo for possible terminals (xterm-256color, linux..).

- nethogs v0.8.5 - ncurse/nettop-like per-app separated speedmeter and traffic counter supporting high refresh rate. Try 'nethogs -d0' (speedmeter) or 'nethogs -v1' (traffic counter).

- rsync v3.2.3 - rsync is an open source utility that provides fast incremental file transfer. (--with-rsyncd-conf=/data/etc/rsyncd.conf)

- smbnetfs v0.6.1 - SMBNetFS is a Linux/FreeBSD filesystem that allow you to use samba/microsoft network in the same manner as the network neighborhood in Microsoft Windows. More info see below.

- progress v0.14 - Linux tool to show progress for cp, mv, dd, ... (formerly known as cv). Download here.

- archivemount (20180801) - A fuse filesystem for mounting archives in formats supported by libarchive. Download here.

- squashfuse v0.1.103 - FUSE filesystem to mount squashfs archives Download here.

- FuseISO - FuseISO is a FUSE module to mount ISO filesystem images (.iso, .nrg, .bin, .mdf and .img files). It currently support plain ISO9660 Level 1 and 2, Rock Ridge, Joliet, and zisofs. Download here.

- HSTR v2.2.0 - HSTR (HiSToRy) is a command line utility that brings improved Bash/zsh command completion from the history. It aims to make completion easier and more efficient than Ctrl-r. (If history is empty try setting HISTFILE in /system/etc/bash/bashrc e.g. export HISTFILE=/data/.bash_history).

- GNU screen, tmux - Thanks to @mirfatif.

- dislocker, ntfs-3g, mount.exfat-fuse - Thanks to @mirfatif.

- f2fs-tools - Thanks to @mirfatif. Update: v1.14.0 here.

- parted v3.3 - GNU Parted (the name being the conjunction of the two words PARTition and EDitor) is a free partition editor, used for creating and deleting partitions. Note: It might be useful to partition external sdcards (e.g. to limit adoptable storage). I do not recommend to use it on internal memory. It might brick your phone.

- Several tools compiled by @Borovets

Spoiler content hidden.

Crypttools:

(These crypttools are mostly frontend tools for the main backend that resides in the kernel. If your kernel hasn't been configured accordingly at compile time you might not be able to use all features.)

Cryptsetup v2.3.5 - (Download) Cryptsetup is an utility used to conveniently setup disk encryption based on DMCrypt kernel module. These include plain dm-crypt volumes, LUKS volumes, loop-AES and TrueCrypt (including VeraCrypt extension) format.

eCryptfs-utils v111 - Frontend tools for the enterprise cryptographic filesystem for Linux. That's what Android/Google use for encryption. It's file-based (no container) and mounting can be automated by Termux widget. Needs shared libraries but is still portable. See notes below.

EncFS v1.9.5 - EncFS provides an encrypted filesystem in user-space. It runs in userspace, using the FUSE library for the filesystem interface.

gocryptfs - An encrypted overlay filesystem written in Go. Download here. Thanks to @mirfatif.

VeraCrypt - VeraCrypt is a free open source disk encryption software. Download here. Thanks to @mirfatif.

fscrypt 0.2.7 - (Download) fscrypt is a high-level tool for the management of Linux filesystem encryption. Needs at least kernel 4.1.

Crypttools info:

Cryptsetup:

General Notes:

- Features like TrueCrypt, VeraCrypt and LUKS2 need 'userspace crypto api' enabled in kernel. Most Android kernels are probably not configured for that and you have to recompile your kernel or contact your kernel maintainer. For kernel 3.4 you need this:

CONFIG_CRYPTO_USER=y
CONFIG_CRYPTO_USER_API=y
CONFIG_CRYPTO_USER_API_HASH=y
CONFIG_CRYPTO_USER_API_SKCIPHER=y

- If 'cryptsetup benchmark' is incomplete and says 'userspace crypto api not available' you might be affected. You can still use LUKS1 though. A full benchmark looks like this:

# cryptsetup benchmark

# Tests are approximate using memory only (no storage IO).
PBKDF2-sha1 249186 iterations per second for 256-bit key
PBKDF2-sha256 327680 iterations per second for 256-bit key
PBKDF2-sha512 58829 iterations per second for 256-bit key
PBKDF2-ripemd160 227555 iterations per second for 256-bit key
PBKDF2-whirlpool 33539 iterations per second for 256-bit key
argon2i 4 iterations, 208288 memory, 4 parallel threads (CPUs) for 256-bit key (requested 2000 ms time)
argon2id 4 iterations, 207817 memory, 4 parallel threads (CPUs) for 256-bit key (requested 2000 ms time)
# Algorithm | Key | Encryption | Decryption
aes-cbc 128b 77.8 MiB/s 88.4 MiB/s
serpent-cbc 128b N/A N/A
twofish-cbc 128b 58.5 MiB/s 61.9 MiB/s
aes-cbc 256b 61.5 MiB/s 68.4 MiB/s
serpent-cbc 256b N/A N/A
twofish-cbc 256b 58.5 MiB/s 61.8 MiB/s
aes-xts 256b 95.1 MiB/s 86.9 MiB/s
serpent-xts 256b N/A N/A
twofish-xts 256b 60.0 MiB/s 61.8 MiB/s
aes-xts 512b 74.1 MiB/s 67.2 MiB/s
serpent-xts 512b N/A N/A
twofish-xts 512b 60.3 MiB/s 62.0 MiB/s

LUKS:

** 10MB test image (luks.img) **
dd if=/dev/zero of=luks.img bs=1M count 10M
cryptsetup luksFormat luks.img
cryptsetup open luks.img myluks
mke2fs -t ext4 /dev/mapper/myluks
mkdir luks
mount /dev/mapper/myluks luks
** luks folder is ready here **
umount luks
cryptsetup close myluks

- If standard luksFormat cipher (aes-xts-plain64) doesn't work (not supported by your kernel) you can try one of the more compatible ciphers:

cryptsetup luksFormat -c cbc-essiv:sha256 luks.img myluks
cryptsetup luksFormat -c aes-plain luks.img myluks

- For LUKS2 (experimental) use:

cryptsetup luksFormat --type luks2 luks.img

- Use "cryptsetup -v --debug" for more verbose output (debugging). In case of errors.

Veracrypt:

cryptsetup open --type tcrypt --veracrypt veracrypt.tc myvera
cryptsetup status myvera
mkdir /data/myvera
mount /dev/mapper/myvera /data/myvera
umount /data/myvera
cryptsetup close myvera

- Use container from desktop system (created with real Veracrypt)

- "veracrypt.tc" is the veracrypt container name

- "myvera" is an arbitrary name (handle)

- Use "cryptsetup -v --debug" for more verbose output (debugging). In case of errors.

eCryptfs-utils:

General Notes:

These tools are not built statically as they explicitly rely on 'dlopen' (plugin system). Instead they are compiled with relative rpaths (./libs). That means dependencies (libraries in subfolders) must be present in the binaries folder and you have to be in the binaries folder itself (with 'cd') before invoking any binary. By this the binaries are still portable (system independent) as long as the subfolders are present. I've put the files into a tar.gz archive so permissions should be set +x already. Extract the archive into /data/local/bin for 'Example' below.

mkdir -p /data/local/bin
cd /data/local/bin
tar xf crypttools.armv7.20180204.tar.gz
cd ecryptfs
./ecryptfs-stat --help

More info: ArchLinux Wiki

Example:

Tested on /sdcard based on FUSE filesystem. sdcardfs untested. Might need selinux permissive.

We create a folder /sdcard/pics that can be enabled (files present) or disabled (no files present) by a click on a widget button (Termux script) and entering our password. The encryption is done on a per-file basis. The actual files are stored encrypted in /sdcard/efs/pics.

- You might need SuperSU or Magisk Superuser for 'su -mm'. That makes sure that all apps can see the mounted folder (mount namespace separation).

- Busybox needed

- Install Termux and Termux:Widget from F-Droid or Playstore

- Start it and enter:

pkg upgrade
pkg install tsu
exit

- Create script /data/data/com.termux/files/home/.shortcuts/efs-pics.sh and make sure permissions(700) and owner (take from parent folder) are correct.

#!/system/xbin/bash
su -mm -c "/system/xbin/bash -c /data/local/scripts/$(basename "$0")"

- Create script /data/local/scripts/efs-pics.sh (770/root):

#!/system/xbin/bash
set -e
PATH=$PATH:/data/data/com.termux/files/usr/bin

# Necessary because rpaths are relative
cd /data/local/bin/ecryptfs

# /data/myefskey contains the salted key.
# Don't forget to make a backup.
# Without it encrypted data is lost.
function enter_passphrase {
read -p "Enter passphrase: " passphrase
sig=$(printf "%s" "$passphrase" | ./ecryptfs-insert-wrapped-passphrase-into-keyring /data/myefskey -) || exit
sig=$(echo $sig | cut -d "[" -f2 | cut -d "]" -f1)
}

CPATH1="/data/media/0/efs/pics"
CPATH2="/data/media/0/pics"
if ! mountpoint -q ${CPATH2}; then
enter_passphrase
echo ""
mount -t ecryptfs -o ecryptfs_sig=$sig,ecryptfs_fnek_sig=$sig,ecryptfs_cipher=aes,ecryptfs_key_bytes=16 ${CPATH1} ${CPATH2} || (echo "$(basename "$0") mount failed!"; exit)
./keyctl clear @u
echo "$(basename "$0") mount successful! :)"
else
umount ${CPATH2} || (echo "$(basename "$0") umount error $? :("; exit)
echo "$(basename "$0") umount successful :)"
fi

# uncomment to force-close Termux window
# killall com.termux

- If your rom uses encryption already (/data/data) beware the './keyctl clear @u' command. It might flush *all* keys in the kernel including the Android encryption one (i'm not sure). This might lead to unpredicted behavior. Either comment it out (then your once injected key remains in the kernel keystore and someone could simply remount your folder without passphrase) or make yourself familiar with the keyctl command and handle it yourself. My phone is not encrypted so i cannot help here.

- Create random keyfile (/data/myefskey) and wrap it with passphrase. This might need 1-2 minutes depending on your devices entropy pool (/dev/random). Backup this key (/data/myefskey). Without it your encrypted data is lost. And don't forget the trailing '-' (minus) at the end of the line, it's important.

cd /data/local/bin/ecryptfs
read -p "Enter passphrase: " passphrase; printf "%s\n%s" $(busybox od -x -N 100 --width=30 /dev/random | head -n 1 | busybox sed "s/^0000000//" | busybox sed "s/[[:space:]]*//g") "${passphrase}" | ./ecryptfs-wrap-passphrase /data/myefskey -

- Create folders:

mkdir -p /sdcard/efs/pics /sdcard/pics

- Create Widget (Termux) and select 'efs-pics.sh'.

- Start it and enter your passphrase (you used above). If everything goes well (it will tell you) you can place files into /sdcard/pics and scrambled files should come up in /sdcard/efs/pics. Never write into /sdcard/efs/pics directly.

- Activate widget again. /sdcard/pics should get emptied.

- Optional: You can set /data/media/0/efs/pics to 700/root so no one can access/see the encrypted data.

SMBNetFS info:

Note: The library paths are relative. You need to be in the folder (with 'cd') to spawn the executable (./smbnetfs). You can extract the archive wherever you want though as far as the file/folder structure remains intact.

Example:

mount -o remount,rw /
mkdir -p /data/local/bin /mnt/cifs
mount -o remount,ro /
tar xf smbnetfs.tar.gz -C /data/local/bin
cd /data/local/bin/smbnetfs
export HOME=/data/local/bin/smbnetfs/home
* enter your smb credentials into smbnetfs/home/.smb/smbnetfs.auth (eg. auth "192.168.1.2" "${user}" "${pass}")
./smbnetfs /mnt/cifs
cd /mnt/cifs/192.168.1.2/${share}

I think it usually should list the samba environment in /mnt/cifs but i'm not sure which prerequisites are necessary for that (edit: maybe it needs real workgroup/hostname instead of IPs). If nothing comes up this should work:

The folder 192.168.1.2/${share} is unreachable by Androids folder picker (unless you can enter the path manually). So either pre-create the folder structure beforehand (mkdir -p /mnt/cifs/192.168.1.2/${share}) and add/register the folder to your app by folder picker (eg. MXPlayer) and then overmount that with the actual ${share}. Or bindmount the folder:

mount --bind /mnt/cifs/192.168.1.2/${share} /mnt/cifs2

Edit: Another option is to let smbnetfs create a static link (actually a symbolic link) to the share in the mountpoint root (/mnt/cifs). Its not as robust as the bindmount though. MXPlayer doesn't find any files (even though the folder picker shows the folders properly).

echo "link myfiles 192.168.1.2/${share}" > /data/local/bin/smbnetfs/home/.smb/smbnetfs.host
chmod 700 /data/local/bin/smbnetfs/home/.smb/smbnetfs.host

I've noticed that MXPlayer shows the samba folders just for a glimpse of a second. But if you enter one of the local folders and then go back all samba folders are there. Not sure why this is happening or maybe its just my system.

Edit2: Not yet tested but.. check the permissions. Its possible that SMBNetFS mounts with 755 or something. That's inaccessible for Android apps. Try this:

./smbnetfs -o umask=000,noatime,noexec,nodev,nosuid /mnt/cifs

Samba 4.8.3 configuration:

_idmap_modules=idmap_rid,idmap_hash,idmap_tdb2
_pdb_modules=pdb_tdbsam,pdb_smbpasswd,pdb_wbc_sam,pdb_samba4
_auth_modules=auth_unix,auth_wbc,auth_server,auth_netlogond,auth_script,auth_samba4

waf configure --prefix=/tmp/myout \
-C \
--sysconfdir=./conf/etc/samba \
--with-configdir=./conf/etc/samba \
--localstatedir=./conf/var \
--libexecdir=./conf/usr/lib \
--enable-fhs \
--with-lockdir=./conf/var/cache/samba \
--with-piddir=./conf/run/samba \
--with-logfilebase=./conf/var/log/samba \
--without-pam \
--without-systemd \
--without-ads \
--with-shared-modules=$_idmap_modules,$_pdb_modules,$_auth_modules \
--disable-cups \
--without-gettext \
--bundled-libraries=NONE,com_err,ldb,uid_wrapper,resolv_wrapper,socket_wrapper,nss_wrapper,ntdb,roken,wind,hx509,asn1,heimbase,hcrypto,krb5,gssapi,heimntlm,hdb,kdc,cmocka,talloc,tdb,pytdb,ldb,pyldb,tevent,pytevent \
--disable-rpath-install \
--disable-python --without-ad-dc --without-acl-support --without-ldap \
--hostcc=/usr/bin/gcc \
--cross-compile --cross-execute='qemu-arm -L /media/devpart/qemu/root'

waf build -j4
waf install