yashsingh1/xdaforums huggingface dataset:
[Nexus 5 Android Development] danarama:
I've not used TWRP.

It's all handled by one recovery image though.

So what you do is enable the migration and format the data partition to what you want (it prompts ext4 / f2fs). Once done and you leave that menu, migration is turned off. From then on, the format option formats each partition as it is already set.

So for example, system is ext4, cache is ext4 and data is f2fs....

Format system formats ext4, format cache formats ext4 and format data formats f2fs. You never need to do anything special again. Unless you want to migrate back.

Also FYI, Nandroid restore detects file system and restores appropriately.
[Nexus 5 Android Development] Sc4ryB3ar:
[MENTION=2872010]rootSU[/MENTION], I like testing things out and generally would be happy to do this (and have been eagerly awaiting PhilZ recovery to support F2FS), but honestly, I don't want to have go through the process again for the other partitions.

So with that said, I'm going to be waiting until it supports all partitions.
[Nexus 5 Android Development] danarama:
The recovery supports all partitions. The only limitation is the kernel YOU choose to use. Not all kernels support all partitions to be f2fs so if your kernel doesn't support system type f2fs, it won't boot.. The process for system is simple though. Nandroid backup system, format to f2fs, Nandroid restore system. Not really a hardship is it?

That said, what do you think the actual value of having the system partition formatted to f2fs is? The only partition that really matters is data, where all your read/writes happen. That's why some kernels won't support system as its pretty useless.

The process I outlined has more steps than other partitions because you're wiping your sdcard, which creates extra steps.

Also, you should know even if you do find a kernel that supports system type f2fs, everytime you flash a rom, if he updater-script format system, it's going to format it as per the script, usually ext4 so the steps for upgrading a rom would be pretty annoying....

flash Rom, flash kernel, flash moda, backup system, format system, restore system

The test build is the complete implementation. There will be no changes to it when it's released in the OP. I just want feedback from experienced root users to help ensure the process I gave is sufficient to understand.
[Nexus 5 Android Development] Unb0rn:
Migration works like a charm but I've had a hard time booting with kernel that is not used within recovery(the one from d-h.st). When I first flashed faux's kernel system just never booted, it boots well with denness544's kernel. Then I've tried to swap denness's kernel with faux's,-it booted(boot process took significantly longer), but everything was FC'ing. ROM used is Paranoid Android(it's AOSP, not CAF)

Any ideas? Different implementations?
[Nexus 5 Android Development] danarama:
I'm not sure. I use elementalx but it requires an f2fs kernel to be Flashed first. I booted fine with [MENTION=5499742]dennes544[/MENTION] kernel. In fact I'm still using it after my last test, but elementalx works great too.

I'll test faux myself.

I'm using slimkat

Edited > faux doesn't work for me at all. It asks for a decryption password. I don't know why this is. I can't see that using the [MENTION=5499742]dennes544[/MENTION] kernel to format to f2fs would be the problem.

What other f2fs kernels are there I can test flashing with?
[Nexus 5 Android Development] Sc4ryB3ar:
Oh, see, this changes thoughts for me....

NO ONE, and NO WHERE has this been explained so simply.....Thank you So much!

I had misunderstood the information I had read as far as being system partition being as important as data. I also was under the impression that if you had the data partition F2FS and System as eft4 that the mismatch would create an communication issue between the 2 that would not benefit making the change.

So I really appreciate the 'simply' put, a lot, Thank you :)

Now that said, I'll give this a whirl later this evening when I get home :)

Using Schism Rom/Uber Kernel(4.10 linario version) and I truly can't wait to get F2FS running with this already kickass combo ;)
[Nexus 5 Android Development] Unb0rn:
It looks like yo've just downloaded incorrect version of faux's kernel-he has a little folder named f2fs on his server)) (Test for attention or smth:laugh:) Standard faux's kernel asks for password, f2fs one does not boot at all for me.
[Nexus 5 Android Development] danarama:
Oh OK. Yeah maybe. Can you link me please?

Bricked supposedly f2fs was the same but again, maybe wrong version. Franco worked fine though.

No problem. Simply put, f2fs can be better for read / write speeds, but this isn't useful anywhere except /data really, where read/writes are frequent.
[Nexus 5 Android Development] Unb0rn:
Of course! Here are f2fs versions. And about franco kernel-I've tried it as well with no luck. Have you downloaded it from his thread or from f2fs one?(which, I think, may be modified)
[Nexus 5 Android Development] danarama:
Thanks I just found it.

Had same problem with faux as you.

There's a modified version of Franco which is an f2fs version, linked to from dennes544's thread. That worked fine. Elementalx just uses ramdisk of previous kernel.

Link to dennes544 thread

http://xdaforums.com/showthread.php?p=50777479

With Franco working I'm not sure I can say the problem is with recovery using dennes544 kernel. I've got more working kernels than not.
[Nexus 5 Android Development] Unb0rn:
No need, I was just wondering maybe I'm doing something wrong). Looks like stock franco does not support f2fs, and there are some problems with faux's kernel(which is strange, his works were very stable since my GNex' times. Anyway, great work! It looks like recovery is latest version again(with libtar updates) and f2fs supporting! Only one thing could be better... MultiROM(maybe via some ORS or something...) Anyway, don't think Phil is interested)
[Nexus 5 Android Development] danarama:
Recovery in test build is latest cm11 sources, latest philz sources.

Edit > I lie, there is a new commit to fix graphical bug since I built
[Nexus 5 Android Development] Phil3759:
You are doing an amazing work

Many thanks

I am happy this is the really first versatile f2fs/ext4 migration recovery :good:
[Nexus 5 Android Development] danarama:
No no, you're doing the work. I'm just testing and building :)

Yep, no recovery compares with your f2fs implementation. Now we just need kernels to catch up!!!

just updated change log and "recommendations" section. Happy to go live now!! Working great Phone feels brand new too!

One feature request, in the change log since the last N5 version, you added create /data/media after internal wipe. Would there be any way to detect whether /data/media/0 is required and create that too? I was thinking perhaps, see what /sdcard toggle is set to and create that dir? Because even after a wipe, the setting is still in place until you reboot recovery..

What do you think?
[Nexus 5 Android Development] Phil3759:
Sadly, this uses a "volatile" option

It is enabled by checking "/data/media/.cwm_force_data_media" file AND "/data/media/0" folder

After real wipe of /data, you end up with any of these, so recovery will default sdcard to /data/media

A workaround is to read into /system/build.prop for android os version. However, this is not a good way to go as we'll need to mount /system on start. It can also fail after a real wipe of whole phone, include /system

At the end, there is no reliable way to keep compatibility with some old phones that use /data/media and not /data/media/0

I could go with a flag to set default as compile time, probably the best alternative
[Nexus 5 Android Development] danarama:
That sounds like it would work. Some phones still need both media and media/0 so they'd have to stay with default.

So you think:

DATA_MEDIA_ONLY = TRUE

?
[Nexus 5 Android Development] Phil3759:
No, something like this:

https://github.com/xiaolu/android_bootable_recovery/blob/cm-11.0/roots.c#L615
[Nexus 5 Android Development] danarama:
USE_MIGRATED_STORAGE?
[Nexus 5 Android Development] Phil3759:
I just updated my repo

Please report if it now recreates /data/media/0 after full /data formatting
[Nexus 5 Android Development] danarama:
Updated source and ready to build. Do I need to set anything in .mk files?
[Nexus 5 Android Development] Phil3759:
no, by default it will use /data/media/0

note: you have to empty your PM box, you can no more receive messages!
[Nexus 5 Android Development] danarama:
Thanks. Emptied...
[Nexus 5 Android Development] danarama:
I had to create media and 0

dan@dan-Latitude-E6410:~/android/system$ adb shell
~ # ls
boot fstab.hammerhead sd-ext
cache init sdcard
charger init.rc seapp_contexts
data internal_sd sepolicy
datadata mnt storage
default.prop proc sys
dev property_contexts system
emmc recovery tmp
etc res ueventd.hammerhead.rc
external_sd root ueventd.rc
file_contexts sbin
~ # cd sdcard
/sbin/sh: cd: can't cd to sdcard
~ # cd data
/data # ls
/data # mkdir media
/data # cd media
/data/media # ls
/data/media # mkdir 0
/data/media # cd 0
/data/media/0 # exit
dan@dan-Latitude-E6410:~/android/system$ adb shell
~ # cd sdcard
/data/media/0 # exit
dan@dan-Latitude-E6410:~/android/system$
[Nexus 5 Android Development] Phil3759:
Are you sure you properly cleaned up build intermediates?
[Nexus 5 Android Development] danarama:
Generally between builds I just do this:

rm -Rf out/target/product/hammerhead/*

Edit > I'll do a full make clean, but I probably won't be able to fully test until tomorrow night now
[Nexus 5 Android Development] Koopa777:
Absolutely incredible you two. I'll take the plunge tonight. Quick question: If we are using a ROM with delta updates(or any ORS updates really), when we update we just dirty flash the new ROM, then flash this http://d-h.st/7V8 , followed by a kernel(if the F2FS kernels use the existing ramdisk, UBER for example) in that order?
[Nexus 5 Android Development] danarama:
Yes that will do the trick.

You can even use Philz to flash multiple zips at one, in alphabetical order, so you can just name the files:

01-ROM

02-F2FS

03-MyKernel

etc
[Nexus 5 Android Development] danarama:
edit > Never mind
[Nexus 5 Android Development] adrman:
RootSU, thanks for this and the detailed instructions. I converted my device this afternoon using your 6.43.4 build and everything is working perfectly. I did have to create the "0" directory in /data/media. The only thing I did differently is once my /sdcard had been adb pulled to my PC, I made a zip of the entire thing, then adb pushed it back to the phone after conversion and extracted it, instead of using mtp. I guess the Islay paid dividends. Oh, I had been running Trinity kernel and in that thread there was a post saying the "Any-Kernel" version of Trinity should work with F2FS. It does not.
[Nexus 5 Android Development] danarama:
Cool. The only issue I had with adb push was a permissions type error. For example, some directories, I couldn't create files in - so do check as many directories as you can

Yeah I'm not sure if it is something to do with using [MENTION=5499742]dennes544[/MENTION] kernel compiled into recovery or not. Certainly his AOSP kernel, ElementalX and "custom" franco build do work. I'm not sure if franco uses existing RAM disk or not. Hopefully dennes can shed some light. I'm new to all this F2FS stuff, so I don't know if it is possible that formatting whilst using this kernel as the stock kernel for recovery is an issue or not.
[Nexus 5 Android Development] adrman:
Just checked and all permissions look fine. I think having everything in one zip got around any permissions error. Interestingly enough, when I first booted back into android and hooked up to my PC (Ubuntu 14.04) I couldn't drag and drop files over via mtp. Normally, I see a "Internal Storage" directory on the device when connected via MTP. This didn't appear until I rebooted a second time and connected. Now I can drag and drop like normal.
[Nexus 5 Android Development] danarama:
The permissions can only be default as these are controlled by android when it mount the /sdcard or filesystem. The only test is trying to create files in your directories.

I too have had the issue where internal storage didn't show up until another reboot of the device. I may ad that to the recommendations section
[Nexus 5 Android Development] Phil3759:
6.43.6 on github fixes the bug you reported :good:

I am done for tonight
[Nexus 5 Android Development] danarama:
Yeah I'm already compiling it :)

Thanks very much...
[Nexus 5 Android Development] Koopa777:
So if I just adb pushed certain folders back to my sdcard, would I still run into the permissions error? Would it be better to mount /data in recovery and push that way?
[Nexus 5 Android Development] danarama:
If you have the time, try it. My personal experience was using adb push, I couldn't create files within those directories I pushed, where as the ones I moved in Android using MTP worked flawlessly. Test a small directory to not waste time and boot into Android. If you can create / edit / rename / delete anything in that dirctory, looks like it works ok for you.
[Nexus 5 Android Development] danarama:
6.43.6 uploaded. It is only one change, so I bundled it into the latest change log.

- create /data/media/0 after format of /data and /data/media
[Nexus 5 Android Development] Koopa777:
Windows 8.1 is an ASSHOLE for me when it comes to MTP. The Windows Copy process crashes occasionally on all of my devices running AOSP 4.4.X(Kindle, N5, N10, G2, etc.). I am pretty sure it's Windows being, well Windows...Anyway, I might install Linux yet tonight, because I'm positive MTP will fail. Out of curiosity, I would MTP in recovery, correct? As to not have the /data partition in use?
[Nexus 5 Android Development] danarama:
MTP is an Android only thing. In Android, you can MTP or adb push (although the latter can be quite annoying as you have to mount /data with read / write options unlike in recovery). In recovery, you cannot mtp, but adb push works easily.

Just try pushing 1 folder by adb after format and see which way it is going to go. I found my Linux MTP pretty damn slow tonight. It took hours to copy 8.6 GB back.

As suggested earlier, try using the zip method, maybe even with MTP

The permissions thing may only have been my issue, however - it is best to warn.
[Nexus 5 Android Development] adrman:
I tested six directories and all were good.
[Nexus 5 Android Development] danarama:
Excellent news!
[Nexus 5 Android Development] aheeejd:
Ok, I'm going to be the one to ask. I've been reading right along, but the f2fs is beyond me for now, which is fine, but my question is, for those of us who want to just stick with ext4, can & should we update to newest recovery. I mean, the f2fs feature, is available to those who choose to use it, but for those of us who don't, can we upgrade & philz will work like it always has, am I correct.

Sent from my Nexus 5 using Xparent Green Tapatalk 2
[Nexus 5 Android Development] danarama:
Yes. The recovery is one-use. It works for f2fs and ext4, so you can still continue to use it without usng the f2fs functions. It is worth updating s it still brings some new features and fixes with it.

That said, if you have any questions about f2fs, although it is technically off topic, I am happy to answer them here if I can
[Nexus 5 Android Development] FlyinGrub:
[MENTION=2872010]rootSU[/MENTION], it's me or the 2nd post's picture are dead ?
[Nexus 5 Android Development] danarama:
Maybe host delays....
[Nexus 5 Android Development] FlyinGrub:
Weird,

and it give me a "STATUS_403" when i try to open the picture in other tab
[Nexus 5 Android Development] rabbo3:
hi there is differences beetween kernel f2fs support and rom f2fs full ????
[Nexus 5 Android Development] danarama:
I don't understand the question.
[Nexus 5 Android Development] adrman:
[MENTION=2872010]rootSU[/MENTION] A bit OT, but you can add Uber Kernel to your "working with F2FS" list.
[Nexus 5 Android Development] Koopa777:
Carbon had phone crash repeatedly on F2FS. Purity and Uber work flawlessly.
[Nexus 5 Android Development] rabbo3:
there is difference from f2fs kernel.. and rom based f2fs
[Nexus 5 Android Development] danarama:
Yes. F2fs kernel exists. F2fs Rom does not exist.
[Nexus 5 Android Development] aheeejd:
Thank you very much, I appreciate the offer for help to, really I do. But I'm going to stick with what I know right now, I haven't even looked into multi-rom yet, but anyways, thanks again & thanks for the work you do. And lol at post above...
[Nexus 5 Android Development] rabbo3:
installing with adb not work with ext4 partion ... all rom doesn t boot.
[Nexus 5 Android Development] danarama:
1) you cannot install with adb.

2) if you want help, you should tell us exactly what you did.
[Nexus 5 Android Development] rabbo3:
i' can 't flash a rom by adb??? now i ve restor cmw recovery and all rom doesn t boot help?

---------- Post added at 11:35 AM ---------- Previous post was at 11:33 AM ----------

lol why?
[Nexus 5 Android Development] danarama:
"if you want help, you should tell us exactly what you did."

Edit > If you're using adb sideload (really not sure at this point your information is poor so I'm guessing at this moment).... then I tested it and its working fine
[Nexus 5 Android Development] rabbo3:
i solved i format \data \system \cache in f2fs... i must convert rom to f2fs compatible...and all works! very fast i use rastakat... which other rom work?????
[Nexus 5 Android Development] danarama:
All roms work. Its nothing to do with the rom. Its the kernel.

Also, I did advise to ONLY format /data NOT /system or /cache in instructions
[Nexus 5 Android Development] Unb0rn:
Hey, rootsu, isnt it what you've been trying ti fix?

6.43.8 (02.06.2014)

- fix recursive path creation bug for /data/media/0

- f2fs: do not try f2fs recovery.fstab mount options on an ext4 partition

- f2fs: do not redefine default recovery ext4 mount options when migrating from an f2fs recovery.fstab

- f2fs: error message when converting unsupported file systems

- fix Galaxy Mega variant background resolution
[Nexus 5 Android Development] danarama:
No
[Nexus 5 Android Development] Sc4ryB3ar:
Ok, not sure what to do, but since the other night i have been trying to get my phone to work through adb... I can not do an adb pull no matter what i have tried... Any suggestions?
[Nexus 5 Android Development] danarama:
You're going to have to give more information than that.
[Nexus 5 Android Development] Sc4ryB3ar:
Sorry, it was late and I was very tired and had gotten to the point of frustrated....

So, I have uninstalled and reinstalled (rebooting at the appropriate times) the nexus 5 drivers from the Google site a good 4 or 5 times. I have tried 3 different cables. I can get it to recognize as media, and transfer data to and from the PC to the phone, but for the life of me I can't get it to recognize the device using adb. I have turned debugging on and off in Dev options and rebooted the phone in between those times, but nothing seems to work. I'm on 64bit windows7 PC.

Any suggestions for getting adb to work?
[Nexus 5 Android Development] danarama:
If we're talking adb in recovery, usb debugging is of no use as that is an Android only setting and only works within Android itself.

Many people have adb driver issues. One thing you can do (which is what I had to do to get recovery to show up in windows 8.1 yesterday) was to use the adb naked drivers: http://xdaforums.com/google-nexus-5/development/adb-fb-apx-driver-universal-naked-t2513339

There is also a good thread about adb / usb driver troubleshooting in Q&A here: http://xdaforums.com/google-nexus-5/help/diagnosing-usb-driver-adb-issues-windows-t2514396

Hope these help

Edit > By the way, I always use adb push and pull from recovery as you can simply:

adb push c:\files\* /sdcard

or

adb pull /sdcard c:\files

Where as if you do it in Android, you must use adb root, you must mount as read / write etc etc. More trouble than it is worth. I simply cannot be bothered to use adb in Android
[Nexus 5 Android Development] bigdave79:
Thanks for all the info here and all your hard work.

I successfully converted my data and sdcard partitions to f2fs and got everything up and running again. But I'm having a weird issue when trying to read or delete files from the sdcard partition - I get a message saying the file doesn't exist even though it appears in the browser window. I'm using solid explorer (but I get the issue with other file managers as well) and I backed up and transferred all files using adb push/pull. Any ideas?

Thanks in advance.

Edit: The error message says ENOENT if that helps.
[Nexus 5 Android Development] danarama:
Yeah, this is the reason I said to use MTP to put your data back. Push does something that breaks the permissions.

Another suggestion made by [MENTION=4745442]adrman[/MENTION] was to zip your sdcard data up on PC, push the zip then extract that.
[Nexus 5 Android Development] adrman:
I've had no read or write problems since I did the conversion. One thing I didn't mention though, was when I unzipped, Root Explorer choked on the file and kept crashing my phone. I ended up using FX explorer to extract the pushed zip.
[Nexus 5 Android Development] Phil3759:
The best way to backup/restore /sdcard is to use Custom Backup menu and backup it to OTG USB

It will preserve all file permissions

MTP will only restore predefined permissions, not necessarily original ones if they were changed by a specific program that needs specific permissions for its data files
[Nexus 5 Android Development] Волк:
Did a full F2FS convert, went well. Have to head over to the F2FS-all conversion script thread as MTP isn't working. ADB works fine.
[Nexus 5 Android Development] bigdave79:
So I backed up and restored my /sdcard partition using adb pull/push but that screwed up all the permissions which meant I couldn't really use the /sdcard partition.

Fortunately I found a fix for it on this thread so I thought I'd post it here in case anyone else is having similar problems: http://xdaforums.com/showthread.php?p=40409356

Just flash the SD card fix permissions zip from recovery and your SD card will be fully functional again.

Sorry for the OT.
[Nexus 5 Android Development] danarama:
Great. adb push is much quicker than mtp and extracting a zip isn't too friendly either.
[Nexus 5 Android Development] onesolo:
What is all about f2fs thing??

Any benefits to the ones that are full stock or only to custom roms??
[Nexus 5 Android Development] danarama:
http://en.wikipedia.org/wiki/F2FS
[Nexus 5 Android Development] rabbo3:
With /system and /cache the device is more smooth...and ROM is installed in system..
[Nexus 5 Android Development] danarama:
I'm not sure how you can quantify that objectively. You can have discussions about that in the F2FS thread I posted earlier. I'd rather keep this as a recovery thread please.
[Nexus 5 Android Development] onesolo:
Still, I'm with 4.4.3 stock rom,

I'm seeing on Philz recovery, "toggle f2fs <-> ext4 migration" option. Is it this option? Any harm if I toggle it to f2fs?? Any app incompatibility or something like that??
[Nexus 5 Android Development] danarama:
f2fs migration off:

Press format data - no options, Just formats data

f2fs migration on:

Press format data - 3 options: Format default, format ext4, format f2fs.

If you're not about to do a migration, there is no need to toggle this setting.
[Nexus 5 Android Development] rabbo3:
About your recovery is good complementes [MENTION=2872010]rootSU[/MENTION]
[Nexus 5 Android Development] danarama:
Thank you. FYI, it is not my recovery. It belongs to [MENTION=4666971]Phil3759[/MENTION] - I just build it for him to save him time.
[Nexus 5 Android Development] petrescu:
I can't flash ANY ROM with this recovery and i do not know why. Gapps work fine, apps work fine, but any ROM not...
[Nexus 5 Android Development] danarama:
What's the error?

Please provide logs. /cache/recovery/
[Nexus 5 Android Development] petrescu:
Switched to twrp and managed to Flash MIUI, if i come back to this recovery and will have problems again i will tell you. Thanks for fast support