yashsingh1/xdaforums huggingface dataset:
[Android Software Development] afridi.shahriar:
yeah.

i have checked again, i have copied fsck.f2fs to system/bin as resize.f2fs; renamed ur binaries fsck.f2fs as resize.f2fs and executed from /cache/resize.f2fs ,, and version,, tried from recovery

but, still same thing is happening.

probably this is not working on arm.

anyways, thanks!

-----

BTW, this commit u linked , contain a fix for over-provisioning ,from v1.14-1.

but, our android's f2fs-tools (v1.13, v1.14, binaries build by you previously+ today) are stuck in worst condition.

where, u can't even resize a 100MiB f2fs partition to 1GiB. (i mean, simply cant increase the partition a lot).
[Android Software Development] DualJoe:
Maybe the dependency (util-linux) is the culprit. I didn't check for updates there. Also, i can omit all other commits from upstream and only cherry-pick the ovp fix exactly as Arch does. Will give it another shot tomorrow.
[Android Software Development] DualJoe:
Here are two new versions:

f2fs-tools_v1.14.0_g1603a3d1d_armv7_static_20210928:

- master (1603a3d1d)

- util-linux updated to 2.37.2 (libuuid)

f2fs-tools_v1.14.0_ovpfix_armv7_static_20210928:

- v1.14.0 stable + resize.f2fs: fix wrong ovp calculation (f056fbeff) (like Arch-3)

- util-linux updated to 2.37.2 (libuuid)
[Android Software Development] afridi.shahriar:
thank u so much. I'm gonna try.

u r so much helpful

BTW, why is it arm7? no arm64 bit?
[Android Software Development] DualJoe:
Supports more devices and smaller file size. ARM64 makes sense on compute intensive tools. For 'f2fs-tools' it shouldn't matter.
[Android Software Development] afridi.shahriar:
but, so sad!!!

both these binaries still has the exactly the same bug as before !

i have tested again and again!

You can also test it.

simply format ur /cust with 150000 sectors ( if, sector size is 4KiB)

mkfs.f2fs -f -w 4096 /dev/block/by-name/cust 15000

mount /dev/block/by-name/cust /data/anr

df -h /data/anr

umount /data/anr

----

now resize it to full 1GB size (my cust partition is 1GB)

resize.f2fs /dev/block/by-name/cust

----

now, try to mount , u can't even mount it!

mount /dev/block/by-name/cust /data/anr

-----------

BTW, thanks, bcz: u have tried a lot to build these binaries
[Android Software Development] DualJoe:
You initially were talking about the over provisioning bug. Now you say you cannot mount it at all. This rather looks like two independent bugs. Your kernel might have problems with resized partitions.
[Android Software Development] afridi.shahriar:
no.

overprovisioning bug was there, (at least) till f2fs-tools 1.14.0 , 1.14.0-1 for x86 linux distros.

which was fixed in 1.14.0-3 for arch/ manjaro.

-----

but, that particular bug, which corrupts the whole partition (while resizing to a large size) is there in android f2fs-tools.

so, i though the corruption bug & overprovision bug - BOTH should be fixed if u build from arch's 1.14.0-3 sources.

(as overprovisioning bug was there in every distros, so, also should be there for android)

but, even after your hardwork, still there are corruption bug.

(as it's becoming corrupted , so, can't test if there's any overprovision bug or not)

------

and, it's not kernel problem.

because:

if u just increase the partition a little bit (ex: 600MiB to 900MiB), you can't mount it, and, also overprovisioning bug is there -- at least in your latest f2fs-tools.

but, if u increase from 100/200MiB to 1GiB, partition will be corrupted!

so, it seems that: increasing to a 2X size = corruption

-----------------

as for corruption bug status of resize.f2fs -- a corruption bug still exists even in the latest f2fs-tools on arch/any other distros

which is, if u resize it using

resize.f2fs -s /dev/sdX

to a very large size (ex: 100MiB to 100GiB or, 100GiB to 200GiB) it will be corrupted.

(even fsck.f2fs can't fix it)

but,

if u use only "resize.f2fs" without "-s" , it won't be corrupted!

strangely, "-s" is for safe resizing, while it's actually unsafe.
[Android Software Development] afridi.shahriar:
can u plz update to v2.2.0? https://github.com/rfjakob/gocryptfs/releases/tag/v2.2.0
[Android Software Development] misterhsp:
I already have, aarch64.

/edit.

starnge, I can not attach a file

/edit2

Now it worked, why that did not work ... no idea
[Android Software Development] afridi.shahriar:
okay
[Android Software Development] misterhsp:
gocryptfs-2.2.1

fusermount3 (both aarch64)

...
[Android Software Development] zzzcoder:
wondering if you could do a pandoc build for armv7 ? Quite a useful piece of conversion software.
[Android Software Development] DualJoe:
I have no experience in Haskell cross-compiling and pandoc is a little monster (>1h compile time). Not sure, but there might be arm builds available already somewhere as there are some discussions on the github site claiming success.
[Android Software Development] zzzcoder:
Yeah I was interested in an armv7 build. Doesn't seem to be any reliable source. I too saw that it was a bit of a beast to build, might look at bash scripts instead for my needs (markdown to html etc). Thanks for the reply.
[Android Software Development] afridi.shahriar:
@DualJoe can u plz update rsync (arm64) to the latest v3.2.3? thanks
[Android Software Development] DualJoe:
rsync is in Termux for all architectures.
[Android Software Development] afridi.shahriar:
yeah. but, that's not static.... so i asked
[Android Software Development] DualJoe:
I can give you an armv7 binary with slightly reduced feature set.
[Android Software Development] afridi.shahriar:
ok . thank u so much!!
[Android Software Development] afridi.shahriar:
yeah. thanks.

btw, i wanna ask you a question:

is this cryptsetup -arm64 (build by you / @misterhsp as STABLE as the binaries available in debian-distors??

i mean, bcz of different compilation methods, can this crypcryptsetup-arm64 be less stable?

(not asking anything regarding security, just asking for stability)

thanks.

(asking it, bcz, i was thinking to replace the android-encryption with luks2 ! - just another replacement for FBE/FDE of google)
[Android Software Development] DualJoe:
Debian is quite conservative and has a huge user base for testing. The binaries here are tested by how many users? 10, 20? Most of the encryption stuff is done by the kernel/device manager itself though so not sure how much stability is affected by the frontend tools at all. I wouldn't say it's unstable, rather that the state is unknown or mostly untested.
[Android Software Development] afridi.shahriar:
yeah.

i didn't exactly mean debian, rather meant any compiled binaries from sources for amd64 .

but, the cryptsetup-arm64 binary is compiled with some "patches-musl" etcs (read in the OP),

so, my question is this:

will there be stability issues because of those patches for compilation-environment?

(regardless of kernels' stability/bugs/other arm64 possible bugs)

------

and, only asking regarding

cryptsetup luksFormat (luks2) and luksOpen (luks2)

not asking about other features.
[Android Software Development] DualJoe:
I don't know. Architecture specific differences in stability is way above my head. The distro patches should be rather harmless though.
[Android Software Development] misterhsp:
cryptsetup-2.4.2-static-aarch64
[Android Software Development] VR25:
Could somebody build the latest bindfs (static) for arm, arm64, x86 and x86_64?

Fbind needs those.

I would also appreciate some info regarding Android specific build requirements and potential pitfalls.

Meanwhile, I'll be researching this topic.
[Android Software Development] DualJoe:
You probably want bindfs with fuse3 that has meson/ninja build system. That's a completely different world than the gnu build system and i haven't done anything with that yet. So i'm probably not much of a help here at the moment.
[Android Software Development] DualJoe:
@VR25:

But looks like its no rocket science either. Here are the arm binaries.
[Android Software Development] VR25:
Superb!

Sadly, this does not seem to work with older Android kernels, probably because those lack fuse3
[Android Software Development] DualJoe:
Fuse2 is still maintained. I can post some binaries later.
[Android Software Development] DualJoe:
bindfs.v1.15.1.armv7.aarch64.static.fuse2.20211120.tar.gz
[Android Software Development] afridi.shahriar:
okay. thanks.
[Android Software Development] LNS:
I am getting this if I try to mount filesystem created with the same package

fs.Mount failed: fusermount exited with code 31

and this if I do it with sudo

Decrypting master key
18:33:47.791247 Unknown opcode 2016
Filesystem mounted and ready.
SwitchToSyslog: Unix syslog delivery error
SwitchToSyslog: Unix syslog delivery error
SwitchToSyslog: Unix syslog delivery error
SwitchToSyslog: Unix syslog delivery error
SwitchLoggerToSyslog: Unix syslog delivery error

Still doesn't seem to work.

Am I missing something? First time using gocryptfs and I don't have much knowledge with fuse/fusermount

What I did is copied gocryptfs to $PREFIX/bin and fusermount3 to same dir named fusermount cause it couldn't be found.
[Android Software Development] misterhsp:
Use -nosyslog option to mount gocryptfs. You can ignore the message 'Unknown opcode 2016', harmless. I have that here too. But everything works.

...
[Android Software Development] LNS:
If I sudo gocryptfs I can't access the mount point(as I can't without sudo)and I get question marks instead of info with 'ls -l'

d????????? ? ? ? ? ? mnt
[Android Software Development] bibikalka:
Great thread!

Would it be possible to request ARM compiled binaries for mmc-utils:

MMC tools introduction — The Linux Kernel documentation

The idea is to take a good look at eMMC, and understand its details.

Many thanks in advance!
[Android Software Development] DualJoe:
Here it is. Needs at least kernel 4.4 so aarch64 only.

Untested, feedback appreciated.
[Android Software Development] bibikalka:
Thanks!

The running requirements are steep (need a fairly recent device!), but I do have one that satisfied those! (Moto G7 plus). This compiled version seems to run, and some of the info it spits out looks legit, other - less so. Either the program is messing up, or EMMC mis-reporting! Things like size (it's a 64 GB device, not 1 GB), and date.

It looks like it needs root (?) to run. It spit some stuff for the EMMC, some of it checks out. So I noticed that "DiskInfo PRO" app also displayed some of the internal EMMC info, at least on older Androids. It worked on Android v5, and does not work on v10.

Originally I thought there would be a way to get some indication of the EMMC health, but I am not so sure that EMMC keeps track of this thing.

Anyway, here is some output:

lake_n:/data/local/tmp # cat /sys/block/mmcblk0/device/cid
15010044483644414202XXXXXXXXb600
lake_n:/data/local/tmp # ./mmc cid read /sys/block/mmcblk0/device
type: 'MMC'
manufacturer: 'Samsung/SanDisk/LG' ' '
product: 'DH6DAB' 0.2
serial: 0xXXXXXXXX
manufacturing date: 2008 jul
lake_n:/data/local/tmp # ./mmc csd read /sys/block/mmcblk0/device
type: 'MMC'
version: MMC v4.0-v4.3
card classes: 7, 6, 5, 4, 2, 0,
capacity: 1.00Gbyte (1073741824 bytes, 2097152 sectors, 512 bytes each)
lake_n:/data/local/tmp # ./mmc scr read /sys/block/mmcblk0/device
type: 'MMC'
lake_n:/data/local/tmp # ./mmc extcsd read /dev/block/mmcblk0 > emmc1.log

Edit: Using the product ID "DH6DAB", I found some random output from the same EMMC. The output there seems to match ! I've attached this random output in DH6DAB_emmc.txt
[Android Software Development] Borovets:
bash-5.1.16-[1]-[2022.01.05] -static-aach64 (with all available extensions)
[Android Software Development] Borovets:
.... and some upgrades
[Android Software Development] afridi.shahriar:
parted is outdated, can you plz update it? @DualJoe
[Android Software Development] afridi.shahriar:
hey, now your binaries are working properly. resize.f2fs does have too much over provision bug.

yes, now, i can mount these partitions too, after resizing.

probably it was one of my devices' kernel bug (mount problem).

regarding, overprovision : your f2fs-tools (latest) has fixed it.

tested it many times on many devices, your binary is working properly, as it should.

thanks.
[Android Software Development] DualJoe:
Parted-3.4 (changelog)

8x patches (3xTermux, 5xAlpine)
[Android Software Development] afridi.shahriar:
e4crypt isn't working.

e4crypt add_key shows "No salt values available"

(yes, i have done

mke2fs -t ext4 -O encrypt /dev/block/xxx

mount -o noatime /dev/block/xxx /oem)

still showing that msg.

any solution?

or, can k somehow use file based encryption on ext4/f2fs (e4crypt/fscrypt)??

(gocryptfs is too slow)

thanks
[Android Software Development] DualJoe:
Are you sure you have a supported kernel (4.1+ with CONFIG_FS_ENCRYPTION=y or CONFIG_EXT4_ENCRYPTION=y)?

Maybe check this:
[Android Software Development] afridi.shahriar:
yeah, It's supported!

(btw, cat /sys/fs/f2fs/features/encryption is also supported!

----------

can you plz build static gptfdisk binary?
[Android Software Development] misterhsp:
cryptsetup-2.4.3-static-aarch64

docs/v2.4.3-ReleaseNotes · master · cryptsetup / cryptsetup · GitLab

Cryptsetup and LUKS - open-source disk encryption

gitlab.com
[Android Software Development] sieger007:
bindfs for arm64-v8a. trying to compile one myself but hitting these errors

I prefer to compile myself - but if someone's got one already with fuse3 & fuse 2 compat. I could use it.

Can someone pl tell me HOW in the world compile it on Ubuntu ( I am using puppy based on Ubuntu ) . These guides bindfs-ndk fuse-ndk Cross-Compile-Build-Automator are not much clear
[Android Software Development] sieger007:
the source code for my stock ROM is available I would like to modify it so it can support ext4 natively without me writing a custom script to Mount it. How do I compile a custom kernel for my ROM with the kind of modifications I want .Can someone point me to a detailed guide without any ambiguity.
[Android Software Development] DualJoe:
There are some bindfs binaries here (and some posts before).

Regarding the standalone NDK errors i'm not sure. Usually it's a simple shell script that generates the standalone toolchain indeed. Maybe some env variables are wrong or it's the wrong script.
[Android Software Development] sieger007:
I managed to it arm-v7a working. I used this command

./b7 -o nosuid,nodev,noatime,context=u:object_r:sdcardfs:s0 -u 0 -g 9997 -p 0777 --create-with-perms=0777 --xattr-none --chown-ignore --chgrp-ignore --chmod-ignore -o,rw /data/sdext3/OGWhatsApp /mnt/runtime/full/emulated/0/OGWhatsApp
before that tried gid as media_rw , sdcard_rw & also /mnt/runtime/<default/write>/emulated/0/OGWhatsApp

before that I tried this of course

bindfs -o nosuid,nodev,noexec,noatime,context=u:object_r:sdcardfs:s0 -u 0 -g 9997 -p a-rwx,ug+rw,ugo+X --create-with-perms=a-rwx,ug+rw,ugo+X --xattr-none --chown-ignore --chgrp-ignore --chmod-ignore $DIR/WhatsApp /mnt/runtime/write/emulated/0/WhatsApp # DIR is /data/sdext3

with bindfs ( nickname b7 above ) the bind mount appears "transparently" as a folder compared to standard mount command where mount point appears as a "link" ..

A non root file explorer can rename existing file with bindfs but it cant create new files aka WhatsApp downloads fail

What seems to be the issue in the bindfs command . Can someone please help rewriting so file / folder creation is permitted.

Also on my Samsung Stock ROM SeLinux logging is disabled.

grep -i avc /proc/kmsg
dmesg | grep -i avc
logcat | grep -i avc

No results from above. So I cant figure out what's hurting the Create Perm

I ( hopeully ) recreated the scenario on another Android 9 phone with SeLinux state changeable ( the other 1 is perm. enforced). & I get this avc denial

9421 9421 W com.whatsapp: type=1400 audit(0.0:15540): avc: denied { create } for comm=4D65646961446F776E6C6F61642023 name="LW9n+rkip578Ox9pPGthBD5DGfO7rIMxJyDHRHjQ9ZY=.tmp" scontext=u:r:untrusted_app:s0:c122,c257,c512,c768 tcontext=u:object_r:sdcardfs:s0 tclass=file permissive=0

So the issue could be untrusted app or becos my target /data/sdext3 has unlabeled context ?

ls -Zd /data/sdext3 u:object_r:unlabeled:s0 /data/sdext3

BTW just FYI chron fails. Why so ?

chcon -Rh u:object_r:sdcardfs:s0 /data/sdext3` # before that I `chown -R`ed to `root` entire `/data/sdext3` o/p of ths is `chcon: '/data/sdext3/j37 WhatsApp/data/colornote/backup/1606263606732-AUTO.doc' to u:object_r:sdcardfs:s0: # Permission denied - throws this error for all files

.

Is there a seLinux Denial fix or bindfs option fix or Does it mean I have to recompile kernel with updated policy OR any work-around ?
[Android Software Development] Borovets:
Help, I need a static tar (latest version) with selinux support. I can't compile correctly because of selinux.
[Android Software Development] DualJoe:
Please test this.

- Internal temporary path is still set to /tmp. I'm not sure where to point it. Maybe it runs nevertheless (or can be configured at runtime).

- Not sure if latest selinux 3.3 will work. Maybe we need an older version.
[Android Software Development] Borovets:
Everything works great. Thank you so much for updating tar.

/tmp is the best option for different subsystem versions.
[Android Software Development] KaMyKaSii:
When I try to run testdisk and photorec I get the below error. Can someone help me?

~ $ su -c ./photorec
PhotoRec 7.2-WIP, Data Recovery Utility, December 2019
Christophe GRENIER <grenier@cgsecurity.org>
https://www.cgsecurity.org
Terminfo file is missing.
Extract all files and subdirectories before running the program.
Press Enter key to quit.
[Android Software Development] DualJoe:
Check if you have a /system/etc/terminfo folder. If not, try to search for and link it. Maybe it's placed somewhere else. This path is hardcoded (and valid on Lineage OS 18.1 indeed). On what environment/OS are you?
[Android Software Development] KaMyKaSii:
I searched with "find /system -iname *terminfo*" but there is no result. I'm using the stock ROM (ZenUI Android 9) on my ZenFone 4. Could you provide me with a copy of this folder? Thanks
[Android Software Development] DualJoe:
Try Termux/ncurses package (/data/data/com.termux/files/usr/share/terminfo).
[Android Software Development] KaMyKaSii:
Worked, thank you!

I'll leave below the commands I used in Termux in case anyone needs it. And as testdisk/photorec doesn't detect the userdata automatically, there is also the command to get the userdata path and pass it as a parameter.

apt install ncurses
su
mount -o rw,remount /system
cp -R /data/data/com.termux/files/usr/share/terminfo /system/etc
chown root:root -R /system/etc/terminfo
./testdisk $(mount | grep " /data " | cut -d' ' -f1)
[Android Software Development] LNS:
Guys can you give me a hand with gocryptfs being visible outside terminal? I tried 'su -mm', 'nsenter', 'unshare ..something' but can't get it done. I found an rclone script with fusermount that works in MiX however (so then I remembered gocryptfs it must work!) so I believe it's the same situation but could use some help.
[Android Software Development] DualJoe:
Maybe try 'global mountspace' in Magisk.
[Android Software Development] LNS:
Ohh it actually works if MiX is outside /sdcard so it gets root access, and no need for magisk global mountspace since it inherits anyway.

No idea why I can't get it to work on /sdcard like the rclone/fusermount script though(it's this one) but it really doesn't matter since you need root to mount all that anyway. Just for learning.

edit: ah apparently it's /sdcard permissions. It was the only part I kinda brushed aside thinking it's a personal rclone preference.

For anyone else(I don't know if it's posted in this thread already), to see files in other apps like explorers etc when mounting in /sdcard see detailed explanation here

tl;dr:

Set 0/9997 uid/gid permissions

and 0771/0660 directory/file permissions

edit2: Apparently however, after some use, there appears to be an error "The underlying filesystem does not support fallocate(2)' or something along the lines, which kinda messes the whole filesystem. Plus the 'unknown opcode 2016' which I'm also not sure what's the deal with that. So it's probably not very reliable.. but gocrypt/fuse themselves work nonetheless.
[Android Software Development] afridi.shahriar:
Can you plz build nano-static (standalone) aarch64 binary? (Without /etc/terminfo/* dependency)

@mirfatif @misterhsp
[Android Software Development] mirfatif:
terminfo (or termcap) database is a must for ncurses which is a dependency of (almost every CLI program that takes control of the terminal/console, shows colors, remembers command histroy etc. and so is the case with) nano. But luckily libncurses can be built with one or more terminfo entries inside it.

So what $TERM values you commonly have in your terminal emulator apps? xterm-256color seems to be the default for adb shell on recent devices at least. It used to be dumb on old devices IIRC.
[Android Software Development] afridi.shahriar:
Yeah.

echo $TERM

xterm-256color
[Android Software Development] mirfatif:
nano static aarch64 v6.3

Built-in terminfo:

ansi
dumb
linux
screen
screen-256color
tmux
tmux-256color
vt100
xterm
xterm-256color
[Android Software Development] afridi.shahriar:
Tanak u
[Android Software Development] NSLW:
Would it be possible to provide an armv7 package as well?
[Android Software Development] aryankaran:
what's the nanorc location. for colours?
[Android Software Development] DualJoe:
Yes, here it is.
[Android Software Development] DualJoe:
Use something like this to figure out. Start and close 'nano' (with 'strace') and then search for 'nanorc' in 'strace.txt'. My guess is $HOME/.nanorc might work that would be '/sdcard/.nanorc'.
[Android Software Development] mirfatif:
Correct. That's the default. Set whatever $HOME you want. And then $HOME/.nanorc may include any more syntax definitions you want like: include "/path/to/*.nanorc".
[Android Software Development] afridi.shahriar:
can you please update to v2.5.0?
[Android Software Development] DualJoe:
I think we should wait until final release. Its still rc1.
[Android Software Development] russanandres:
hello. Does anyone have the dialog binary?

Spoiler content hidden.
[Android Software Development] DualJoe:
It's available in Termux repo.
[Android Software Development] russanandres:
But I have android 2.3
[Android Software Development] DualJoe:
Do you have an armv7 cpu? Most Gingerbreads were armv6. Did you test some binaries from here? Are they working?
[Android Software Development] russanandres:
I have armv7 (Nokia N900 nitdroid 2.3.4). I've tested busybox 1.21.1 armv7 from busybox.net binaries.
[Android Software Development] DualJoe:
Ok, let's give it a try.

attached:

dialog13-20220526.armv7.static.20220724-2.tar.gz (with ncursestw and widec support)

dialog13-20220526.armv7.static.20220724-3-ANSI.tar.gz (with ncurses and no widec support)
[Android Software Development] russanandres:
It is isn't working..

Spoiler content hidden.

Photo is rotated, idk why. Sorry
[Android Software Development] DualJoe:
Try both new versions from above. I've recompiled ncurses and created another version without widec support (no UTF8). If none of these work there are probably problems with the old kernel or your CPU doesn't support vfpv3-d16 (that all dependencies and libc are compiled with). The binary with your cmdline from the screenshot works here.

Edit:

The N900 has a Cortex-A8 that should support VFPv3-D16.
[Android Software Development] russanandres:
these versions give the same error.

Is there a site where you get these versions from? Can you send it?
[Android Software Development] DualJoe:
The binaries are selfmade.

Not sure about the bug. Maybe the Cortex-A8 is not fully gcc/march=armv7-a compliant.
[Android Software Development] DualJoe:
This one has reduced cpu set and gcc7.

If this doesn't work i'm out of ideas right now.

attached:

dialog13-20220526.armv7.static.20220725-5-gcc7.tar.gz
[Android Software Development] afridi.shahriar:
can u plz update f2fs tools to v 1.15.0?
[Android Software Development] DualJoe:
attached:

f2fs-tools_v1.15.0_aarch64_static_20220728.tar.gz