yashsingh1/xdaforums huggingface dataset:
[HD2 Android Q&A, Help & Troubleshooting and Genera] HypoTurtle:
Will be updating #1 with a slightly different way of running things - in theory it should work fine. It would run as follows:

Zip 1 = aroma rom - where you set up install preferences and files/folders are mounted accordingly.

Zip 2 = zImage, initrd.gz (nand) and system/data folder.

Zip 3 = ramdisk modifier.

This should mean that all the ROM specific stuff is contained in zip 2. Should also be able to reduce zip 3 down to one line added to zip 2.

Attached for testing will be a zip of kokotas' extended TWRP - with f2fs being used in place of nilfs2. It is using a 3.0.x kernel - so there mat be some issues from that - and doen't have full SELinux support (can't seem to get kernel to boot with 3.0.x kernel compiled with full support). Also attached is one of tytungs 2.6.X kernels which doesn't have f2fs support but does have full SELinux- using this kernel you can still format to f2fs but you can't mount it.

Edit. Robbie P - I see what you are saying. The reason for the Rname workaround was essentially for use when system wasn't mounted (therefore ROM_NAME need to be defined somewhere) and I presumed that if system is mounted then so would /boot and /boot_dir thus no need to remount them. Mounting /boot and /boot_dir agin just to be safe might be a good idea anyway. I would add back a conditional around ROM_NAME to take it from a file if system isn't mounted / a file is present (as before).

mkdir /NativeSD
mount -t ext4 /dev/block/mmcblk0p2 /NativeSD
if [ -f RName.prop ]; then
umount /system
umount /boot
umount /boot_dir
export ROM_NAME='cat RName.prop'
mount --bind /NativeSD/${ROM_NAME}/system /system
else
ROM_NAME=` cat /system/build.prop | grep ro.build.display.id | awk '{print $(NF-1)}' | sed 's/\(\[\|\]\)//g' `
fi
mkdir /boot
mkdir /boot_dir
mount --bind /sdcard/NativeSD /boot_dir
mount --bind /sdcard/NativeSD/${ROM_NAME} /boot

Edit. Just spotted it - RobbieP can you check your mount scripts (might be an issue with all of them)...

On the top line /sbin/busybox is used instead of /bin/busybox. Let me know if that fixes it.
[HD2 Android Q&A, Help & Troubleshooting and Genera] Robbie P:
Afraid not, still not booting.

Have attached my install folder to make sure we are on the same page.

Tried changing the first line of the other 4 mount files to /bin and, with nativesd.sh adding /busybox to first line. Similar dmesg to before, binder transaction failed.
[HD2 Android Q&A, Help & Troubleshooting and Genera] HypoTurtle:
nativesd.sh should still be #!/sbin/sh .

The scripts are using #!bin/busybox sh as it's using the busybox binary in the zip (added into the ramdisk) needed as the mount command / binary in ramdisks doesn't have the --bind option (at least I think that was the reason). You were able to run mount --bind in terminal so the busybox/toolbox in recovery can do it just fine.

The binder transaction stuff isn't related to mount --bind afaik ; it's something to do with images not able to be loaded (most likely the bootanimation), as /system isn't mounted

[ 2.794494] [USBH] msm_chg_detect_work: state:CHG_WAIT_FOR_DCD
[ 2.796966] init: cannot find '/system/bin/servicemanager', disabling 'servicemanager'
[ 2.797027] init: cannot find '/system/bin/vold', disabling 'vold'
[ 2.797088] init: cannot find '/system/bin/auditd', disabling 'auditd'
[ 2.797119] init: cannot find '/system/bin/netd', disabling 'netd'
[ 2.797149] init: cannot find '/system/bin/debuggerd', disabling 'debuggerd'
[ 2.797180] init: cannot find '/system/bin/rild', disabling 'ril-daemon'
[ 2.797210] init: cannot find '/system/bin/surfaceflinger', disabling 'surfaceflinger'
[ 2.797241] init: cannot find '/system/bin/app_process', disabling 'zygote'
[ 2.797302] init: cannot find '/system/bin/drmserver', disabling 'drm'
[ 2.797332] init: cannot find '/system/bin/mediaserver', disabling 'media'
[ 2.797363] init: cannot find '/system/bin/installd', disabling 'installd'
[ 2.797393] init: cannot find '/system/bin/keystore', disabling 'keystore'
[ 2.797454] init: cannot find '/system/bin/akmd', disabling 'akmd'
[ 2.804718] healthd: wakealarm_init: timerfd_create failed
[ 2.806152] init: cannot find '/system/bin/sh', disabling 'console'
[ 2.807373] binder: 1350:1350 transaction failed 29189, size 0-0

Is where we need to be looking
[HD2 Android Q&A, Help & Troubleshooting and Genera] Robbie P:
why not

IF bootloader=magldr THEN cat initrd >> 6MB

irrespective of what kernel it is. There would be enough space, and it doesn't matter if 2.6.x kernel has a big initrd.gz.
[HD2 Android Q&A, Help & Troubleshooting and Genera] Robbie P:
If nand, surely no need for patcher

If DataOnExt (or rather DataOnF2FS) there is enough room on nand for a bigger boot partition anyway, but can be booted from ad sd i think.

I tried the initrd.gz, it needs 6Mb of zeros to boot, have attached the zip of it below, extract the zip to get initrd.gz of 10.4Mb

I get a couple of errors attached after pushing 7 files and the build.prop :confused:
[HD2 Android Q&A, Help & Troubleshooting and Genera] HypoTurtle:
If you think of it as more then just a patcher - but also a ramdisk/zImage replacer. resizing boot on magldr just for this would be a bit overkill (as /system /data etc. need to be resized as well). I suppose boot ad sd would be a good solution for Magldr NAND roms running a 3.0.x kernel - should probably address that in the script.

Not sure about those errors... is /system mounted before the script is run?

the first two (line 2 and 5) are blank lines - line 175 is probably an issue try:

if [ ! -f /tmp/mount-${type}.sh ]; then
echo `$"${type}mount"` > /tmp/mount-${type}.sh
fi

instead (not sure why I had cat there).

For DataOnF2FS - now that you mention it - in this setup would require a mount-DataOnExt.sh file with the f2fs lines instead. (I could include that I suppose similar to the DirectSD stuff). It probably is best to keep /system on a filesystem known and trusted - at least until we get a more stable kernel, so no NativeSD f2fs.
[HD2 Android Q&A, Help & Troubleshooting and Genera] Robbie P:
got "unexpected else line 223 expected then"

/system isn't mounted before running script
[HD2 Android Q&A, Help & Troubleshooting and Genera] HypoTurtle:
then run 'adb shell mkdir /system' before sending the build.prop.

line 223 - there must be an open if command somewhere - can't see it myself.

perhaps -

## Checking zImage type
if [ `hd /tmp/zImage | grep '1f 8b 08' | cut -d ' ' -f '1'` == '00003f80:' ]; then
`dd if=/dev/zero of=/tmp/6MB bs=1M count=6`
fi

will perhaps fix it (in case it's that if that's interfering)
[HD2 Android Q&A, Help & Troubleshooting and Genera] Robbie P:
Still happening, i also tried;

# Checking bootloader
if [ "${bootloader}" != "cLK" ]; then
cat initrd-done.gz >> 6MB
cp 6MB /boot/initrd.gz
cp /tmp/zImage /boot/zImage
fi

#Checking if nand install

if [ "${bootloader}" != "cLK" ] && [ "${install_type}" -gt 2 ]; then
cp 6MB /boot_dir/initrd.gz
cp /tmp/zImage /boot_dir/zImage
else
cp initrd-done.gz /tmp/initrd.gz
chmod 0755 /tmp/mkbootimg
chmod 0755 /tmp/mkbootimg.sh
exec /tmp/mkbootimg.sh
if [ "${install_type}" -lt 3 ]; then
flash_image boot /tmp/boot.img else
flash_image sboot /tmp/boot.img
fi
fi

in case it was nested if-then that was causing it, but still the same.

mkdir /system returns can't create directory, file exists. It has bin in it.

deleting the lines 2 and 5 gets rid of the other errors.

could there be something in twrp ramdisk without a then?
[HD2 Android Q&A, Help & Troubleshooting and Genera] Robbie P:
weird, i get the missing "then" again, in line 3. Attached screen.

Edit; just tried reformatting p3 with new twrp and it seems to format ok, reinstalled rom and everything is mounted ok.

Also tried the previous ramdisk again and get same line 3 error, so it is not the ramdisk.
[HD2 Android Q&A, Help & Troubleshooting and Genera] HypoTurtle:
It's most likely a bracket issue - have updated my previous post #56 - have also changed it to use busybox, e2fsck (and fsck.f2fs) from /sbin so no need to push them anymore.

Edit. Yea I noticed the issues you had before from the ROM when it tried to mount the partitions - for me it seemed like an issue relating to things already being mounted or something wierd. I don't know if you noticed but in recovery p3 is automatically mounted if it's detected - as sdext2 but I can't seem to umount it.
[HD2 Android Q&A, Help & Troubleshooting and Genera] Robbie P:
I still get missing "then" from new nativesd.sh

i tried

#!/sbin/sh
then #
fi

at start, but got 'unexpected "then"':confused:

if i create a new .sh file with first 3 lines from nativesd.sh only and push it to /tmp i do not get an error.

if i save nativesd.sh as .bat, then rename it to .sh i still get the error.

can't think why this is happening.

Edit; BTW i can umount /dev/block/mmcblk0p3 from adb, but can't uncheck sdext2 from mount tab in twrp
[HD2 Android Q&A, Help & Troubleshooting and Genera] Robbie P:
syntax error: line 97: unexpected "fi" (expecting "then")
[HD2 Android Q&A, Help & Troubleshooting and Genera] HypoTurtle:
Updated #60
[HD2 Android Q&A, Help & Troubleshooting and Genera] Robbie P:
still get unexpected "fi" at line 97

also noticed "else" in line 159 should be on line 160, but it makes no difference to the fi error.

Edit; found a couple more elses at end of lines 67 and 71. Not sure if makes a difference or is missing a ;

also found > instead of >> and mount- instead of mountfs-

but still same error

I noticed that after running, no mountfs-* file is created in /tmp. The only new file there is nfo.prop (with magldr in it)

i also noticed that /NativeSD folder is missing in /, it should be created in line 8.
[HD2 Android Q&A, Help & Troubleshooting and Genera] HypoTurtle:
are you sure nfo.prop is new? (remove and see if it's remade) I believe that the script is run due to the fi error - thus no /NativeSD

#60

I believe all the > and >>'s are fine (I think I know what you are looking at though),

mount- is a mistake, those else's might actually be something as well - thought I had everything formatted fine.
[HD2 Android Q&A, Help & Troubleshooting and Genera] Robbie P:
confirm nfo.prop is new, and that /NativeSD is not made.
[HD2 Android Q&A, Help & Troubleshooting and Genera] Robbie P:
I get unexpected end of file at line 204 (expected "then")

nfo.prop created, no /NativeSD folder created. No other text echoed to screen.

This is bonkers.:)

Edit; ok if i just use the first 12 lines (11 lines + a blank line) i still get the unexpected eof.

I have tried first 12 lines in earlier nativesd.sh from the hide box in post#2 and i get the same. Edit3; But i know that the zipped file worked

I think it has to do with notepad++ and the way i am creating the .sh file, i am copying the text from the codebox, creating a new file in notepad++, pasting the text, adding a return at the end, saving-as nativesd.sh (type unix .sh, .bsh)

will try other ways of saving the .sh

Edit2; or perhaps it is due to the codebox, could you upload last nativesd.sh to ubuntuone, or zip it and attach?

Edit4; BTW there is an "elso" near the end, changing it has no effect on ueof.
[HD2 Android Q&A, Help & Troubleshooting and Genera] HypoTurtle:
Will run the sh again on my end and then upload. Notepad++ will work as long as formatting is set to ANSI as UTF-8 (Encoding > encode in UTF-8 without BOM).

Added that last if statement - I added after the fact, didn't notice the error as I'm running clk, so that if statement isn't entered for me.

FYI for testing I have

echo "ro.build.display.id [romn] [NativeSD]" > /system/build.prop
cp /sdcard/initrd.gz /tmp/initrd.gz
cp /sdcard/zImage /tmp/zImage

added to the top of script (after #!/sbin/sh)
[HD2 Android Q&A, Help & Troubleshooting and Genera] Robbie P:
right we are getting somewhere, if i copy and paste and run the script with those settings in notepad++, i don't get that ueof error, i do get other errors but will wait for you to upload latest so we are synched.

:D:D:D

Edit; Just saw you edited the above hide box, so i put those 3 lines (added rom name and changed to directsd) to nativesd.sh, pushed that and the mkbootimg files (why not:)) to /tmp, put nand initrd.gz and 3.0.* zImage to /sdcard and sh-ed nativesd.sh

i get the attached below

Edit2; after running, using twrp filemanager, /tmp/mt1.txt and /tmp/mt2.txt are empty

also /tmp/zI.txt has "00003F70 65 73 73 69 6f 6e 20 65 72 72 6f 72 00 1f 8b 08 ¦ession error....¦"

Which might explain why it didn't recognise p3 as f2fs and zImage as 3.0 respectively

md5sum of /tmp/zImage is 1cb7bcf80845f0178b99f7b9a5f80669
[HD2 Android Q&A, Help & Troubleshooting and Genera] HypoTurtle:
Can't get to a PC for a while to test a few things / upload. Those errors are more or less what I was expecting. Can you add rm /tmp/mt1.txt (and the same for mt2.txt and zI.txt) at the top of the script.

I also believe something's up with one of the f2fs lines ... Think it's the last one (remove the mt1.txt part if you can).
[HD2 Android Q&A, Help & Troubleshooting and Genera] Robbie P:
can't see which line to remove mt1.txt

i noticed around line 108

are we bind mounting data to system?

Edit; and should rOm be ${ROM_NAME}? it would save a couple of lines later.

Also found around line 181 "cp /tmp/zImage /boot_dir/ROM_NAME_new/zImage" should be ${ROM_NAME}
[HD2 Android Q&A, Help & Troubleshooting and Genera] Robbie P:
I don't think, for testing, we can do more than one run before rebooting, odd things happen. See attached.

It still isn't detecting f2fs on p3 and that it is a 3.0 kernel, changing 00003f80 to 00003f70 fixes the latter see 2nd attached

Edit; I think head is the problem with f2fs detection 3rd attached
[HD2 Android Q&A, Help & Troubleshooting and Genera] HypoTurtle:
I'll change hexdump check to that value then. Can you run 'adb shell' then run "hexdump ... | head ... | grep ..." I'm sure head works. | essentially starts another command , so other than just running adb shell first, running "adb shell hexdump ... | adb shell head ... | adb shell grep ..."

I've definitely had f2fs detected properly. Run 'hexdump -C /dev/block/mmcblk0p3 | head -30' and see if you can see f2fs, if it's not try rebooting and try again.

Edit. I think I see what you mean regarding reruns - that would be due to the mountfs. script not being made if one is already present in /tmp, so lets remove that after its used. (a moot point in the end as we should just clear /tmp)
[HD2 Android Q&A, Help & Troubleshooting and Genera] Robbie P:
I #ed the f2fs lines leaving just mt1="ext4" and mt2="f2fs" and i get an initrd.gz in the /sdcardNativeSD/*_new/ folder. It doesn't boot, attached dmesg, seems same as before.

If i do hexdump -C /dev/block/mmcblk0p3 on one line in adb shell it just fills screen with numbers etc, after ctrl C it can't accept it as input to "adb shell head -30"

head does work though, adb shell head gives me 10 input lines on screen.

I tried hexdump -C -n1000 /dev/block/mmcblk0p3 but can't find f2fs in there.

edit; will try again tomorrow
[HD2 Android Q&A, Help & Troubleshooting and Genera] Robbie P:
It was formatted using new recovery and the rom seems to be installed fine (looking from twrp's filemanager)

I changed the kernel to your quoted one in /sdcard/

I get the attached when running from /

it says "copying fsck for f2fs" but that is only checking for existence of mt2.txt, not whether anything is in it, it is empty (mt1.txt empty too). should the echo line be within the following if-then statement?
[HD2 Android Q&A, Help & Troubleshooting and Genera] HypoTurtle:
I put the echo there so that the first if statement (mt2 existance) has something to do, it should say something else I guess just need to add a 'mkdir /boot_dir/${ROM_NAME}-new' towards the top. mt1 and mt2 will both be empty if ext4, in hexdump are you seeing the other extensions (gif, png, mov etc.)

If you are running adb commands its better to run 'adb shell' first on its own to enter the shell itself.

adb shell

hexdump -C /dev/block/mmcblk0p3 | head -30

(or rotate the screen on device and run hexdump in recovery terminal)
[HD2 Android Q&A, Help & Troubleshooting and Genera] Robbie P:
Here you go, screens. Mt2.txt empty at each try, but no errors.

(BTW no need to thank all my posts, just happy to help;):D)

edit; would something like "mount > mtd.txt

if grep "sdext2 type ext4" then mt2="ext4"

if grep "sdext2 type f2fs" then mt2="f2fs"

or similar work better? might have to mkdir /sdext and do mount /dev/block/mmcblk0p2 /sdext first since p2 isn't showing up for me.

Also mount without -type seems to work for automount of p3

Edit; I just realised that the last recovery (ext twrp 2.6.3.5) didn't format sdext2. i didn't read this post carefully enough:o. Tried using tytung r2 but it still doesn't format it. Which tytung one did you use?
[HD2 Android Q&A, Help & Troubleshooting and Genera] HypoTurtle:
Mount would work yes but requires a kernel that can detect f2fs to work, using hexdump the partition doesn't have to be mounted. I believe the full selinux kernel was just the default extended twrp one. The f2fs formatting is done by the initrd.gz, kernel just makes the filesystem supported/detected, the f2fs just becomes an unsupported filesystem (like using ext4 in windows), only way to check it formatted properly would be to hexdump it. You'll notice the difference between ext4 and f2fs in hexdump. Regardless of the recovery (and kernel) used 'mkfs.f2fs -e f2fs /dev/block/mmcblk0p3' should format it so that it's detected by the script. (I could change the script I guess to look for one of the other extensions as they aren't there in the ext4 hexdump)

Edit. Didn't see the first screenshot before posting - yeah that's f2fs. Formatting with -e f2fs will add f2fs to the bottom of that list. Formatting in recovery - either format sd-ext or partition sdcard should already have that included.
[HD2 Android Q&A, Help & Troubleshooting and Genera] Robbie P:
Thought i would try magldr dataonf2fs using tytung's kk 4.5 on freshly formatted p2 using recovery (f2fs shows at bottom of list), it seemed to install ok (data files present on p2)

i pushed the zimage and the 3 files (mkboots and nativesd.sh) and sh-ed.

I had to edit a couple of lines around line 202

cat initrd-done.gz >> 6MB
mkdir /boot_dir/${ROM_NAME}-new
cp 6MB /boot_dir/${ROM_NAME}-new/initrd.gz
cp zImage /boot_dir/${ROM_NAME}-new/zImage

Since it couldn't find the ..directory.

i then copied the 2 files to sdcard/NativeSD and booted ad sd

It boots, but i get many error messages (process themes provider has stopped, google keyboard, settings, google play services). I think because the modules are tytung's ones. Can you post the modules for the md5sum=b4757fbfbc90c7fce419c9826b81770e kernel?

Edit; dmesg for the above boot attached
[HD2 Android Q&A, Help & Troubleshooting and Genera] HypoTurtle:
Kernel and modules attached. Good to see that DataOnF2FS is working.

I take it you are also still modifying / pushing build.prop though (so not quite automated), just seen a few other ..'s (will only come up as an error with clk).
[HD2 Android Q&A, Help & Troubleshooting and Genera] Robbie P:
my testing line is "echo "[NexusHD2-KitKat] [DataOnExt]" > /sdcard/installtype.prop"

rom is NexusHD2-KitKat-CM11.0_V4.5.zip

i get "sh:syntax error line 68: bad substitution"
[HD2 Android Q&A, Help & Troubleshooting and Genera] HypoTurtle:
not sure what that could be: could be the / I suppose try with the first / removed on the mkdir lines. Is there a /boot folder present? Could you try with Native/DirectSD?

Also change the bottom to read

testing
cleanup
echo "0 0 0 0 0 0 0 0 0x11800000" > /tmp/bootimg.txt
partnames
getpropfile
bootloadercheck
exportvar
mkdirs
if [ "${install_type}" -gt "1" ]; then
mountpart
fi
if	[ "${bootloader}" = "cLK" ]; then
bootorig
fi
if [ `find '/sdcard' -name '*.zip' | grep ${ROM_NAME}` != '' ]; then
systeminstall
fi
if [ ! -f /tmp/initrd.gz ]; then
bootingred
fi
if [ ! -f /tmp/mountfs-${type}.sh ]; then
mkmountfs
fi
mkinitrd
initrdcp
if [[ "${install_type}" -lt "3" && "${bootloader}" = "cLK" ]]; then
bootflash
fi
revertinitrd
showresult
exit 0
[HD2 Android Q&A, Help & Troubleshooting and Genera] Robbie P:
I tried removing /s but same error line 68

tried removing "-t auto" but same

added those lines too.

There is a /boot folder originally made from twrp, no /boot_dir, no /NativeSD folder and no /sdcard/install.prop made either.

tried "[NexusHD2-KitKat] [NativeSD]" > /sdcard/installtype.prop" and same
[HD2 Android Q&A, Help & Troubleshooting and Genera] HypoTurtle:
partnames() {
sdpart=/dev/block/mmcblk0p1
firstpart=/dev/block/mmcblk0p2
secondpart=/dev/block/mmcblk0p3
echo "setting partitions with ${firstpart} format"
}

Bad substitution at line 13

(also going to need to copy/replace 1stpart and 2nd part with firstpart and secondpart #83 )
[HD2 Android Q&A, Help & Troubleshooting and Genera] Robbie P:
replaced 10 occurences each of 1st and 2ndparts

added "mkdir /tmp/sdcard" at line 16

Line 69 (now 70) "mount --bind NativeSD/${ROM_NAME}/system /system" assumes that a rom is already installed, if not it errors. Is it meant to be installed already?

(Also the next line errors because /sdcard is already mounted)

Edit; also, i get

"unzip; can't open *Nexus-HD2-KitKat*.zip [.zip]"

if i change to "unzip /sdcard/*${ROM_NAME}*.zip system/*", i get "flags 1 and 8 aren't supported", see screenshot.

BTW i do like the modular format of the script, good job:)
[HD2 Android Q&A, Help & Troubleshooting and Genera] Robbie P:
have attached screens for the latest script
[HD2 Android Q&A, Help & Troubleshooting and Genera] HypoTurtle:
missing a !

getpropfile() {
if [ [COLOR="Red"]![/COLOR] -d /tmp/sdcard ]; then
mkdir /tmp/sdcard
fi

Those unzips seem fine (could there be an issue with the zip itself? can you check if recovery runs it), could try with -x I suppose
[HD2 Android Q&A, Help & Troubleshooting and Genera] Robbie P:
I tried with adding ! and got a different result (capture24)

I then tried a different zip (and different r0m and type) and got lots of inflating lines (capture25, first bit scrolled off screen area of command prompt)

So it seems my nexus zip is a bit duff:confused:
[HD2 Android Q&A, Help & Troubleshooting and Genera] HypoTurtle:
systeminstall(){
echo "installing system"
# rm -rf /system/*
unzip /sdcard/*${ROM_NAME}*.zip system/*
# /tmp/unzip /sdcard/*${ROM_NAME}*.zip data/*
unzip /sdcard/*${ROM_NAME}*.zip META-INF/*
unzip /sdcard/*${ROM_NAME}*.zip boot.img -d /tmp/
cat '/META-INF/com/google/android/updater-script' | sed -e '/assert(/,/);/!d' > /tmp/updater-script
cat '/META-INF/com/google/android/updater-script' | sed -e '/symlink(/,/);/!d' >> /tmp/updater-script
cat '/META-INF/com/google/android/updater-script' | sed -e '/setmetadata/,/);/!d' >> /tmp/updater-script
cp /tmp/updater-script /META-INF/com/google/android/updater-script
zip -r /META-INF
cp /META-INF.zip /sdcard/symlink-${ROM_NAME}.zip
}

should fix the META-INF errors, not sure about the nexus zip (perhaps changing to gunzip would work?)

updated #83

Is there a boot.img in the zip? (If yes what location) - Either way can you copy the zImage and nand initrd.gz to /sdcard and run the script on #83. Have attached the binarys (need to be pushed to /tmp and given execute permission 'chmod 777 ...' )
[HD2 Android Q&A, Help & Troubleshooting and Genera] Robbie P:
I tried #83 without bins (capture26)

it does not accept any option and is just waiting for input

There is no boot.img in either zip, they make it with mkbootimg.sh

Edit; i get exactly the same with bins added (27)