yashsingh1/xdaforums huggingface dataset:
[HD2 Android NAND Development] HypoTurtle:
ROM-shell - just contains all the files for this stuff to work (for dev use mainly), just chuck in system and data folders - and zImage/initrd.gz and that's it a ROM that can be installed:

NAND

DataOnExt

NativeSD

DirectSD

with any filesystem identified by blkid and f2fs which isn't.

I'll test my script with update when I get the chance - I went for the basic folder option --- if there's something in /sdcard/NativeSD/$ROM_NAME_$TYPE/system then it's copied to /system after it's mounted then the folder is cleared and device rebooted (and same with a data folder), only thing is no permissions/commands are run; better than nothing I guess.

Edit. Can't find RobbieP's post but the f2fs patches are here if they're needed.

[MENTION=2906641]robbiep[/MENTION] Yes it'll extract the boot.img - was going to pull the 7za binary from the boot folder but it's still used for this purpose (initrd.gz and zImage don't need to be the /boot folder, it should work just as well with them anywhere in the zip). Correction It will if the .zip is named ${ROM_NAME}*.zip ; with the ROM_NAME taken from updater-script. I will change this; as if we are having the ROM_NAME in updater-script becoming $ROM_NAME_$type (i.e CM11_MACS_ NativeSD) then the zip would have to be CM11_MACS_NativeSD123ABC.zip, I'll change this to strip the _$type.

Edit. Have changed it in the ramdiskscript-update.sh so that the zip would only need to be CM11*.zip

Edit. OR you could put the boot.img into the boot folder.

I'll add the dropbox links to my sig. for easy access once I've checked the -update script
[HD2 Android NAND Development] Robbie P:
i noticed line 26 in rom-shell updater script needs another placeholder instead of cm11_mac

trying walter79 evervolv nativesd.....

Edit; Not sure what happened then, but it booted into mac4a.

the script didn't run.

I have attached aroma log

I put system and boot.img (and file.contexts file) in root of rom.zip, edited the updater script lines with "W79evervolv", zipped it, renamed zip to W79evervolv.zip and flashed it

will try again later, have to cook
[HD2 Android NAND Development] HypoTurtle:
My bad (Wrong EOL), glad I spotted that. shell fixed (in sig.), or open the sh in notepad++ =>edit==> EOL conversion==>UNIX/OSX.
[HD2 Android NAND Development] Robbie P:
tried again after changing eol to both .sh

it hangs on around 27% of install. tried with different recoveries. managed to get a dmesg of twrp 2.7.0.0 nand recovery trial. some lines about my ext4 partition being corrupted. might have to backup and reformat p2. will try again tomorrow evening.
[HD2 Android NAND Development] HypoTurtle:
Possibly caused by the f2fs format??

D/Loading evervolv now, and see if I can replicate.
[HD2 Android NAND Development] macs18max:
Wow. Thanks you guys a lot of effort put to the scripts.

Just a note, I just remembered whilst answering to "720/camcorder for HTC desire" that cm11 ROMs that I have built are in "user" mode and not "user-debug" mode. So you may not see a whole lot of lines in logcat. I made some changes In mac2 ROM to compile with user mode and haven't changed it back since. My bad. (Read in some thread that the build is lighter). I'll make sure that on the next release build ROM with user-debug flag enabled. So Robbie you may or may not see logs regarding f2fs as the ROM is build in user mode.

In regards to f2fs honestly don't know if the kernel supports it. Also I disabled ext2, ext3 and some few other filesystem support. You can have a look here. https://github.com/maniacx/android_kernel_htcleo-3.0/blob/leo/arch/arm/configs/htcleo_defconfig

I will use the patch that hypoturtle provided regarding f2fs when I come back. I was thinking android lollipop once I get back from offshore, but I think I'll wait until cm12 gets it first stable release and try and contribute to this development till then.
[HD2 Android NAND Development] HypoTurtle:
[MENTION=2473151]macs18max[/MENTION] I checked /proc/filesystems and f2fs isn't there - so patches needed

[MENTION=2906641]robbiep[/MENTION] - The shell works now for Evervolv (doesn't run as smoothly as CM11 though), and not sure why [MENTION=382340]walter79[/MENTION] stated the need for 2 ext4's.

Edit. p2 seems to be being used for some sort of dalvik on ext4 setup, haven't found p3's use yet. (Have found the script to autofind sdcard partitions though - thanks to Andrew Sutherland for that).

Update part of script doesn't translate properly across though - working on it.

What I want to add is a checkbox to aroma for installing system - so you can switch from nand to dataonext without reinstalling system. And throwing in a dpi choice shouldn't be an issue (180, 200, 240).

EDIT. That's done - Mark9 and ROM_Shell updated with the above.
[HD2 Android NAND Development] Robbie P:
Tried evervolv again with last shell script, got the attached aroma log (it booted cm11 BTW after install (which said failed)

I only changed the nativeSD placeholder in updater script, none of the others.

Shouldn't the last cm11mac rom be mark 10?;)

I had another quick look at the cm11, directSD rom (system-p2-ext4, data-p3-f2fs) and although wifi passwords are stored after reboot, I can't DL apps from playstore, it hangs. Looking forward to kernel f2fs support, it does seem a lot faster than dataonext (perhaps because it can't access /data ;):))

Not much time again tonight for testing:o
[HD2 Android NAND Development] HypoTurtle:
Technically yeah, mark 10 (I wasn't going to bother adding the link), My Evervolv is here, using an earlier version of ROM_shell. I had only checked clk dataonext. Just tried with native and it's looping - going to remove the init.d scripts and see if it boots.

Removed scripts 05, 10(apps2sd), 20, 97 and 99(libhtc) - causes a stalled boot, a dmesg may help diagnose. (might need to rewrite those or something else might be causing the reboots/stall.) DataonExt definitely works boots though...
[HD2 Android NAND Development] Robbie P:
tried your evervolv (:)) along with the init.d deletions, magldr NativeSD, and it hangs on htc screen. attached aroma log, dmesg and ramdisk

I found ramdisk in /sdcard/NativeSD/Evervolv_NativeSD_NativeSD folder, there were empty system and data folders in there too.

Edit; @HypoTurtle added attachments, sorry.

Edit2; I noticed you used evervolv 20141009, for my first try I used 20141015 from here http://d-h.st/users/walter79/?fld_id=36874#files ,it seems it is not "nand only". Other roms there use 2x ext4 partitions.
[HD2 Android NAND Development] HypoTurtle:
I'll wait for the attachments, only thing I've seen in my dataonext dmesg is some $sys...'s (which have been set in int.rc as * - these may need some actual values.), but still haven't seen p3's use.

Yeah I know file names are looking a tad odd now (I'm thinking it would be better to just have a unique name for dataonext and have the same one for Native/Direct - so that they share system (you could add in another name so that the data's are different run_program("/tmp/ramdiskscript.sh", "NativeSD", "PLACEHOLDERsystem", "PLACEHOLDERdata"); )); system and data folders were part of the update system (still now working afaik) - before switching to a update folder in the latest shell.
[HD2 Android NAND Development] macs18max:
[MENTION=3629322]Robbie P[/MENTION] [MENTION=4056517]HypoTurtle[/MENTION]

Hey I'm back!!!

So I have applied the patch provided by hypoturtle and compiled the kernel successfully. I have tested the kernel on cm11mac4a rom in this setup, clk, NativeSD, ext4 partition just to make sure it didnt break anything. So far everywork well same as without f2fs patches.

I have compiled 3 kernels with 3 different configurations.

1.

CONFIG_F2FS_FS=y
# CONFIG_F2FS_STAT_FS is not set
# CONFIG_F2FS_FS_XATTR is not set
# CONFIG_F2FS_FS_POSIX_ACL is not set
# CONFIG_F2FS_FS_SECURITY is not set

2.

CONFIG_F2FS_FS=y
CONFIG_F2FS_STAT_FS=y
CONFIG_F2FS_FS_XATTR=y
CONFIG_F2FS_FS_POSIX_ACL=y
# CONFIG_F2FS_FS_SECURITY is not set

3.

CONFIG_F2FS_FS=y
CONFIG_F2FS_STAT_FS=y
CONFIG_F2FS_FS_XATTR=y
CONFIG_F2FS_FS_POSIX_ACL=y
CONFIG_F2FS_FS_SECURITY=y

All 3 zImage are attached in the below file with description.

Also I have created a seperate branch on my git

https://github.com/maniacx/android_kernel_htcleo-3.0/tree/leo-f2fs

Another Note. I had some compilation errors (didnt read the intructions carefully :silly: ) and I did remember securecrt had patched f2fs on marcs 3.0.16 kernel earlier, so I was looking at his repo.

And I found this

https://github.com/securecrt/androi...mmit/a5a65fabdaeb232d5826c6100e6909ae6fd8db10

Description say that patch is needed on for magldr booting from sd. Does it mean that booting kernel from Nand in Magldr does not need to add 6MB to it initrd.gz.?

Also this commit seems interesting

https://github.com/securecrt/androi...mmit/2ad5761514ad9c67251a76dd189e2957dfe8feb5
[HD2 Android NAND Development] gilbert32:
[MENTION=2473151]macs18max[/MENTION]

NO :(

IIRC, securecrt was using windows phone on his hd2, so he was booting android via sd. but basically the same happens when you boot from nand.
[HD2 Android NAND Development] HypoTurtle:
I actually took it from his OneDrive, those are just the 4.4.3's I believe - I though I took the latest version.

From the dmesg it looks like system is being remounted from somewhere else as well as the init.d's.

Edit. I've looked through all the things that are called by the rc's and can't find the issue. I'll try the 4.4.4 later when I get the chance.

No that's the code that makes the 6MB thing work - as [MENTION=3907492]gilbert32[/MENTION] said.

For nand boot switching to rboot and flashing a clk boot.img should work and simple enough to add to the script once confirmed (it may need a different base while mkbootimging).
[HD2 Android NAND Development] Robbie P:
I reformatted sd-ext (ext4) and sdext2 (f2fs) and installed 4aDirectSD using the latest shell folder (with the update folder- how does this work? can I update the kernels using this?)

I have attached the aroma log and original kernel dmesg and logcat (p3 did not show up after mount from adb), along with kernel1 dmesg and logcat (p3 shows as f2fs, see screen, not sure about the log error at the bottom, it saved ok)

kernel 1 gave a reboot to bootanim after setting-up. I can't DL from playstore (error 920). The wifi didn't want to connect, but worked after a manual reboot. Logs are from the second (manual) boot.

will try other kernels tomorrow, is there any specific tests need doing?

Edit; sorry ppl, have had no time for testing today, real life stuff. Hopefully tomorrow evening:fingers-crossed:
[HD2 Android NAND Development] HypoTurtle:
You can see the update stuff at the end of the mountfs script, along the lines of:

systupd="/sdcard/NativeSD/${2}_${1}/update"
if [ ! -z "`ls ${systupd}/*`" ]; then
cp ${systupd}/* /
rm -rf ${systupd}/*
reboot
fi

I tested the test ( [ ] ) statement and seemed to be fine - it doesn't seem to run in practice though. Any ideas anyone?

Kernel wise, zImage is a copy into the NativeSD folder - so no need for it for that - modules could be changed this way (once it works), but permissions won't be set.

There is however the overrides in the script (added a couple lines to script in sig.) /sdcard/zImage and /sdcard/modules/*

with the permissions set to 644 (is that the right permissions here??)

I also have a sh with DalvikOnNAND as an install option - I'll test, package and upload tomorrow if I get the chance.
[HD2 Android NAND Development] macs18max:
[MENTION=4056517]HypoTurtle[/MENTION]

Is this script complete?

I have two ROM's waiting for a release

Updated cm11

And Carbon ROM.

I read your script. My lack of knowlegde makes it very difficult to understand what is been done with the script.

I will try it anyway now.

Thanks
[HD2 Android NAND Development] Robbie P:
Sorry again for not being around, you can see how much testing I have done on kernel2 from the attached screen (no sim in, standard battery)

Have attached 2 dmesgs and a log for kernel2, and a dmesg and log for kernel3

If I had to choose the best, it would be kernel3. But I have not really tested them:o
[HD2 Android NAND Development] HypoTurtle:
Yea the script from the original shell is complete - the update stuff is an issue as isn't working as it should but it doesn't have any effect. If you ignore the testing function the rest should be simple enough to understand, the testing part is probably a bit ott and took a fair bit of testing (and I was bored I guess).

I tried a DalvikOnNand install yesterday with CMTight - to maximise the NAND data size but it wouldn't boot... so it seems the ROM_Shell is only 4.4+, I couldn't pick out the problem (could be init, permissions, busybox used etc.) as I couldn't get a dmesg. Just tried with SlimKat and DalvikOnNand boots - but dalvik doesn't seem to be persistent.

ROM_shellv2

No problem man, haven't got round to checking f2fs out for myself, that screenshot looks weird to me though perhaps with Volta from 5.0 it wouldn't be as surprising.
[HD2 Android NAND Development] Robbie P:
tried the latest shellv2 directsd ext4&f2fs on an ics rom (blueorange mix) and it hangs on htc screen

tried it on JB miui 4.10.10 gtbluesky directsd ext4&f2fs and it boots

will post logs tomorrow

Edit; Have attached some logs, not sure why I thought a rom with tytung's kernel in it would support f2fs:o doh
[HD2 Android NAND Development] macs18max:
Hypoturtle robbie. I want to format ext4 partition to f2fs. And then install ROM in NativeSD

Questions

How do I format f2fs? (i dont want to use software on PC) I found a flashable zip but that didnt do the job

also

I believe your installer supports installing ROM in NativeSD on a f2fs partition.

Can we install ROM using non-f2fs supported recovery, such as extended TWRP?

extended twrp is using tytungs kernel which doesnt support f2fs, will it mount as f2fs partition by your script whilst installing?
[HD2 Android NAND Development] Robbie P:
Hi macs18max, I used the attached sd recovery from magldr ad sd. The kernel for the recovery is a 3.0.x kernel, compiled by HypoTurtle, with f2fs support. It has a few bugs, such as not shutting down fully and no extended battery support. The initrd.gz has 6mb of zeros added so as to boot from magldr. To boot from clk, you would need to remove the 6mb first, then fastboot the kernel and ramdisk (I think) or make a boot.img.

When using the recovery, it is best to format the partition first as ext4, then format it to f2fs.

Afaik, the recovery kernel needs to support f2fs. therefore needs to be a 3.0.x kernel. your X14f2fs kernel(s) might be used instead of the one in the attachment.

HypoTurtle has mentioned security issues with having system on f2fs, something to do with not having a later version of the kernel (eg 3.4.x or later) with better f2fs security fixes included iirc. So NativeSD might not be a good idea at the moment. Safer apparently is to have DirectSD (or NAND) with data on f2fs.
[HD2 Android NAND Development] HypoTurtle:
I concur. [MENTION=2473151]macs18max[/MENTION] all you need is the mkfs.f2fs binary like the one that's compiled into your roms. In terminal run:

/system/bin/mkfs.f2fs /dev/block/mmcblk0p2 (for DataOnExt - or p3 for DirectSD). If you bundle one of your f2fs kernels and add the mkfs.f2fs to a recovery you should be able to install (and attempt f2fs NativeSD - a dmesg would report pretty quickly the issue ( f2fs mount flags) - with /system), but you'd still need to format f2fs from terminal. The recovery RobbieP posted is simply extended twrp with nilfs2 changed to f2fs. There is one issue I recall - in the mount menu you can't unmount a f2fs partition, it unmounts fine in terminal though.

I was thinking about that and technically no as the filesystem needs kernel support - you could say run a ROM with one of your f2fs kernels, format a partition to f2fs, mount it and make the ROM_NAME/data folder on it then you could install DataOnExt or DirectSD with any recovery (you would need to remove/alter the SD-Ext_mounter script for DataOnExt).
[HD2 Android NAND Development] macs18max:
I will try to compile extended twrp recovery with f2fs flag enabled and try it out to install the ROM.

Could you modify an nativesd initrd.gz to mount f2fs instead of ext4 for system and data? (I dont know what need to be modified, I'm guessing we need some files in bin folder)
[HD2 Android NAND Development] Robbie P:
I used shellV2 to get new carbon rom to boot from DirectSD system-ext4, data-f2fs, using X14f2fs kernel3. It boots ok, but can't seem to get a dmesg, have attached aroma log.

have also attached a couple of androbenchmarks for NativeSD carbon (on ext4) and the DirectSD install, f2fs does seem a bit faster.

Will try to post a dmesg.

Edit; have attached a dmesg
[HD2 Android NAND Development] HypoTurtle:
If I can find the hard-drive i compiled mine on I'll post the changes I made to it... The script will make a NativeSD f2fs system/data; the only added thing to bin is the fsck.f2fs binary, other than that it's just the ext4's changed to f2fs in the mountfs-NativeSD.sh script. From RobbieP's logs it looks like the only flag thats unsupported is discard from the data mount flags (rw,noatime,nodiratime,nosuid,nodev,discard); system has (rw,noatime,nodiratime,barrier=0), so I might try an install now and report back.

Edit. Yeah seems to be a stall on f2fs NativeSD (Carbon) - for reference this is the mountfs-NativeSD.sh:

#!/bin/busybox sh
while [ ! -b /dev/block/mmcblk0p2 ]; do
/bin/busybox sleep 5
done
/bin/busybox sleep 2
# check the first f2fs fs
/bin/fsck.f2fs -p /dev/block/mmcblk0p2 > /fsckf2fs.log
/bin/busybox mkdir -p /NativeSD
[COLOR="Red"]#/bin/busybox mount -t f2fs -o rw,noatime,nodiratime,barrier=0 /dev/block/mmcblk0p2 /NativeSD[/COLOR]
[COLOR="Blue"]/bin/busybox mount -t f2fs -o rw,flush_merge,discard,noatime,nodiratime,background_gc=on,inline_xattr,active_logs=2 /dev/block/mmcblk0p2 /NativeSD[/COLOR]
# mount the data dir
/bin/busybox mount --bind /NativeSD/PLACEHOLDER/system /system
/bin/busybox mount --bind /NativeSD/PLACEHOLDER/data /data
systupd="/sdcard/NativeSD/PLACEHOLDER_NativeSD/update"
if [ ! -z "`ls ${systupd}/*`" ]; then
cp ${systupd}/* /
rm -rf ${systupd}/*
reboot
fi
# cleanup deleting /bin to free ram
/bin/busybox rm -rf /bin

will look and see what boot flags are used by other devices (if recall correct originally I had xattr stuff).

Edit: above changed with different flags; add the script into boot folder and try it - then check dmesg. I'd guess at least inline_xattr isn't supported.
[HD2 Android NAND Development] HypoTurtle:
ROM_SHELLv3 in sig

CHANGELOG:

PLACEHOLDER's removed from updater-script (uses ROM-NAME.prop, and installtype.prop). Also compatible with the current run_program("/tmp/ramdiskscript.sh", "NativeSD", "PLACEHOLDER"); so you can pick and choose if you want to have a different Rom-Name for different installations.

SWAP partition/file added.

Easily add another script - add it as a function to ramdiskscript; add function name to options="..." in optslist and execute with ("ramdiskscript.sh", "FUNCTION").
[HD2 Android NAND Development] Robbie P:
I've been trying v3 today, directsd dataonf2fs and been having problems with data persistency after reboot. decided to clear some space and reinstall. I didn't clear sd-ext though (thought I had, I used f2fs twrp to format sdext to ext4 but it apparently didn't work), and after re-install it hangs on htc screen. I have seen this happen with v2 when re-installing an existing rom. I wonder if /system is wiped before install, as it does with traditional nsd. I did see "out of memory" messages from dmesg when it hangs.

will post some logs later.
[HD2 Android NAND Development] HypoTurtle:
Yea logs will help.

With data persistence check "adb shell mount", when I've checked dataonf2fs; I have mistakedly installed without changing the rom kernel, the rom still boots but p2 isn't mounted as /data thus meaning /data is likely mounted to RAM.

Might need to switch the #s for the mount options for f2fs (atm there's none - i.e. letting it default) lines 350/367. I've also added a while loop to make sure that the initrd.gz added to the NativeSD folder is actually there (not 0MB); that I've noticed happening a few times - hopefully it doesn't infinite-loop.

ramdiskscript

I don't think /system is wiped for anything other than NAND/DataOnExt, would be simple enough to add: easiest way would be add:

##################
# System Install #
##################
if file_getprop("/tmp/aroma/gds.prop", "item.0.3") == "1" then
show_progress(0.9, 1400);
ui_print("@ Copying files...");
[COLOR="Blue"] delete_recursive("/system");[/COLOR]
package_extract_dir("system", "/system");
package_extract_dir("wallpapers", "/sdcard");
show_progress(0.1, 5);
run_program("/tmp/ramdiskscript.sh", "atp");
endif;
[HD2 Android NAND Development] Robbie P:
doh! didn't use f2fs kernel, I am dumb sometimes:o

Ok tried again, put x14-f2fs3 with module in rom.zip, put in new ramdiskscript.sh and added that line in updater script.

I now have persistent data, and it installed ok

Having a few probs with some app FCs. And with speed of rom.

Have attached aroma log, a couple of dmesgs, one from magldr and one from terminal of the same boot, and a log of a previous boot.
[HD2 Android NAND Development] HypoTurtle:
Aroma looks good (just a missing echo in script); not sure about this: [1.930419] 0x000005da0000-0x00001dea0000 : "syqtem" it's probably nothing. Not much I can see otherwise - is there anything in /e2fsck.log (if there is one) as;

[43.615112] EXT4-fs (mmcblk0p2): warning: mounting fs with errors, running e2fsck is recommended.

Speedwise it seems smooth for me (carbon dataonf2fs) although I didn't use it too much - the only thing missing is the read_ahead value - added now (you can manually add in adb/terminal:

write /sys/devices/virtual/bdi/179:0/read_ahead_kb 2048)
[HD2 Android NAND Development] Robbie P:
read ahead was 128kb, it does seem a bit faster with 2048kb.

Have attached screen of fsck

I might try and re-install with gapps in the appropriate folder, good idea.

I did jump a bit seeing darth vader on installing (thx -pixelfreak- ;)), but I am now prepared with Jedi training for my next encounter with the darkside.
[HD2 Android NAND Development] -pixelfreak-:
Hahaha lol you can change the picture in Aroma... I have insert my Themen and This picture....
[HD2 Android NAND Development] HypoTurtle:
Theme is set to ics btw. But yea picture (sample.png) and splash (HD24EVER) are there from yourself; might replace with placeholders if there's a need for a v4.
[HD2 Android NAND Development] macs18max:
[MENTION=4056517]HypoTurtle[/MENTION] Could you post a ramdisk using NativeSD on my CM11 ROM for f2fs file system.

I'm using a CWM recovery with f2fs. I was able to format the sd-ext and install the ROM in NativeSD method successfully.

However the ramdisk I use got stuck at boot. Probably I have created a wrong ramdisk. If you could make one I will test it.
[HD2 Android NAND Development] -pixelfreak-:
re

you can work with my picture ans splash.... no problem....

pixelfreak
[HD2 Android NAND Development] Robbie P:
Have you tried shellV3?

Merge /system and /data into v3,put clk-nand initrd.gz and f2fs kernel in /boot (delete the boot.img there), put a rom-name in rom_name.prop, zip it and install.:fingers-crossed:

no time for me tonight to try it:o
[HD2 Android NAND Development] HypoTurtle:
Don't have a working linux setup atm, so I'd probably just be using v3 as RobbieP said.

Can't believe I left a boot.img in there [removed]; yea he has system installed so just add zImage and initrd.gz (or a boot.img) to the boot folder of shellv3 along with a ROM_NAME.prop (CM11_MAC ??) and run without selecting to install anything else (system/data/gapps) and you'll have a ramdisk added to /sdcard/NativeSD[/CM11_MAC_NativeSD]

Yea I'll keep the splash and switch the picture to something more calming.

Edit: uploaded zip (renamed as .apk due to size) with the rom as above along with the NativeSD f2fs initrd and mountfs-NativeSD.sh that you can change and through in the boot folder if you want to change it in the initrd.gz