yashsingh1/xdaforums huggingface dataset:
[Galaxy S I9000 General] Chainfire:
Voodoo is definitely the most promising method IF they can get the kinks worked out. We'll see it when it gets there :)
[Galaxy S I9000 General] distortedloop:
@een625 makes a pretty good short comparison of all the methods, but leaves out Supercurio's method. This should be added to the second post:

-Supercurio's lagfix (aka Voodoo beta 1): Significant improvements in performance which can also be benchmarked. Requires a custom kernel.

This reformats the /data mount point as ext4. The conversion is automatic on first boot and no data is lost.

Pros: very fast, does not affect the amount of storage for apps, doesn't impact free space reporting, uses ext4 for data integrity which some argue is unnecessary, easy to install, easy to uninstall.

Cons: Semi-private beta release only, must be specifically uninstalled to upgrade some firmwares, uses ext4 which some argue is significantly slower than ext2, no longer easily available while beta2 is finalized

I personally use this one right now, it is the only one that I have never experienced lag with at some point, even after almost five days of continuous uptime and heavy use. All the others have issues from time to time, especially with the Market and installing apps. This one does not, in my experience.

Also, I think that there should be some mention that mimocan's method requires a custom kernel, and also that mimocan is pretty much the same as apps2sd on other Android phones (uses ext3/4 in a partition on the external card). Additionally, a con to mimocan is that the user must manually partition and format the external card, all of the other methods can be done without user intervention other than flashing a PDA or running a script/apk.

Finally, dkcldark had an update.zip method posted very briefly that reformats the /data to nilfs or ext4. Looked promising, but was pulled for some reason.

And last but not least, please, please, please, let's not let this thread devolve into the ext2 vs ext4 debate that so many of the individual lag-fixe threads turn into. Perhaps a thread dedicated to that should be started (and stickied)?
[Galaxy S I9000 General] Chainfire:
I'll keep the suggestion in mind :)
[Galaxy S I9000 General] Evans_Prophet:
RAID lagfix?

Speed is one concern, potential damage to the flash card is another. I would prefer to replace my external card than the internal one.

Btw, is it technically impossible to implement more advanced solutions such as using a combination of NAND (for heavy apps / data) and sd (for other apps / "bulk" storage)?

Moreover, I think that it would be possible to have some kind a RAID 1 arrangement that would increase read speed and reliability (esp. if combined with EXT2). Just think 2GB on internal and 2GB on external card. (Most importantly having - probably - the first mobile phone with a raid configuration. RAID is easy to implement under Linux so I don't see why this is not examined.
[Galaxy S I9000 General] dupel:
i think I might have even suggested something like this before, but it was either overlooked or its too hard to do. But the idea (i did not think about RAID) is awesome.
[Galaxy S I9000 General] lafaya:
I agree Voodoo lagfix is great but there is one problem/phenomena: CON: Big battery drains and we don't know where it's coming from :S
[Galaxy S I9000 General] r3loaded2010:
Wouldn't RAID put a big strain on the CPU?
[Galaxy S I9000 General] messatu:
I've tried MoDaCo, Mimocan and RyanZA lagfixes.. for a very brief review, I've to say that MoDaCo seems the fastest to me.. Followed by RyanZa 1-click lagfix and lastly the Mimocan lagfix. A pity that MoDaCo is limited to ~130mb of /dbdata space :(

Btw, just my personal opinion.. Some may disagree and feels that other lagfixes are faster.
[Galaxy S I9000 General] distortedloop:
That seems to be a YMMV kind of thing. I don't notice big battery drains with it, but there are those who say they do. Not sure if I am lucky, or others are unlucky.

Certainly couldn't hurt to do the whole wipe batterystats routine and see what's up after that.
[Galaxy S I9000 General] odyssey61:
Thanks for doing this. I had been pushing for the change for the last couple of weeks since RyanZA and Tayutama particularly bought out their fixes. We now have a great range with Voodoo being the best atm imo.
[Galaxy S I9000 General] Oletros:
Try Google search widget or an import of Spainsh speedcams with Radardroid lite
[Galaxy S I9000 General] newmail:
That's the same I experienced too, and frankly it's very much expected! It's impossible not to notice the difference between a fsync on close and not fsync on close.

Removing a fsync from every close, is super important for any hope for responsiveness. Even a laptop system would be hanging all the time if the filesystems were mounted with -o sync. I'm afraid rfs does implicit fsync in other places, not just on close, they really should make the sync_on_close a sysctl so we can disable it.

The quadrant benchmark isn't much affected because it helps only in 1 single test I think. This can't help the database write test that uses fsync explicitly. But not everything runs fsync in normal conditions.

Probably the galaxy is much faster than the other cellphones but the rfs is so bad at the fsync database write test, that quadrant gets screwed up. So especially with rfs the most important thing of all is to avoid synchronous writes, hence the removal of sync from close, to get better responsiveness. That's what my module does.

Thanks for testing. Also I had zero problems so far. But I'd like to find a way to move at least /data to ext4.... nevertheless. And I don't have more time to test, plus this is fast enough now for my needs.
[Galaxy S I9000 General] captive:
If you manage to compile a working ext4 module I guess we can apply supercurio's voodoo, which was working fine (superfine!) on my eclair: I might be wrong, but I remember someone here on xda already managed to unpack & repack froyo initramfs..
[Galaxy S I9000 General] newmail:
Ok, I'll try to build ext4 this weekend...
[Galaxy S I9000 General] newmail:
I got annoyed by k9 fsyncs, so I now disabled fsync as well. Removing fsync from close was theoretically safe. But removing fsync is unsafe and it breaks userland journaling, so if the battery is pulled out the fs state will be ok but the app state may not be ok.

k9 was unusable, now it works well... so this makes a definitive difference (just removing fsync from close wasn't enough to fix k9), but it's a broken setting, real solution is ext4. And hoping that ext4 fsync will be dramatically more efficient or we'll be back to the same issue if it's just a slow flash.
[Galaxy S I9000 General] RyanZA:
If you have the time, give the OCLF app in the market a try. It has a very nice effect on k9. :)
[Galaxy S I9000 General] newmail:
Yes, I'd like to upgrade to ext4 (ext2 not), but I run on froyo JPK (I'll try to build the ext4 kernel module over the weekend then maybe we'll find a way).
[Galaxy S I9000 General] RyanZA:
The OCLF market app does work on Froyo. It's a long way from being perfect, but it might tide you over until you can put together the EXT4 module and whatever else. It does have a very large effect on apps such as K9, and it's pretty easy to install and remove.
[Galaxy S I9000 General] newmail:
I'd need to verify if loop is buffered.... I'm pretty sure I had trouble with loop even on blockdevice (not file, but file on rfs is less safe if something) with journaling.

Anyway I attached ext4 for JPK. I rather not have loop.

My rfs_async tweak make quadrant I/O test an order of magnitude faster but I don't get anything better than 1027 as result even after removing fsync. The test 1/4 is already fully fixed by the "safe" removal of sync on close, the 3/4 database write is fixed by the further "unsafe" removal of fsync (but it may not be less safe than loop, I need to verify how things work inside loop and it depends if it's loop or dmloop). Unfortunately the "safe" tweak isn't nearly enough to make k9 usable, OTOH k9 is perfectly usable after making fsync a nooop... It'll be interesting to see how k9 does with ext4.
[Galaxy S I9000 General] newmail:
In other words we must change benchmark when discussing anything related to the write performance.
[Galaxy S I9000 General] sztupy:
Could you please post the kernel sources you're using for example on GitHub? It'd be useful for a lot of us.
[Galaxy S I9000 General] newmail:
You've to realize what a mess it is and how difficult it was to generate tun.ko with a tree in 90% broken state that has no chance to build, I've a total mess in a 1.7GB scratch directory, I build with SUBDIRS= tricks using only the headers that I fixed up...

This is just a temporary thing and I really wouldn't want to waste more time on it.... considering it's samsung's fault if they don't release the sources yet. They will release the sources with the official froyo release, so considering it's a matter of weeks (I hope) I doubt it's worth even trying to upload a 90% broken tree that very few people could ever find useful to github...

I uploaded ext4 so the voodoo project because it has taken 2 minutes to build and I was lucky. It's not actually guaranteed to work! :)

Nevertheless if you guarantee me you can build with SUBDIRS= and with tree in 90% broken state I can send you a diff against 2.6.32 android git tag. It shouldn't take too long, but just don't expect to run make and get something out of it :)
[Galaxy S I9000 General] woeds:
So, are you saying that with this file, a Voodoo-release for 2.2 would be possible?
[Galaxy S I9000 General] RyanZA:
Yep, if the posted EXT4 module works, then Voodoo for JPK should be easily doable! :)
[Galaxy S I9000 General] newmail:
Anything is possible given enough time :)

Well I've no way to generate a zImage, so if they can extract initramfs out of the stock zImage in JPK, modify it, and repack it in the same zImage kernel, it should be possible. I assume all mounts are done by initrd. Then please also remove also the annoying signature verification on update.zip that requires to swap kernels in order to root. this is open source stuff, signatures go away.

In the meantime my fsync removal makes k9 like 20 times faster and usable, no chance of placebo effect here. I decreased dirty_expired_centisecs to 5 sec and verified async written data is there after hard poweroff after 5 sec. So rfs plays at least right with pdflush and fs writeback. But this is risky and I want ext4!
[Galaxy S I9000 General] RyanZA:
This is already fully done by Chainfire. Check out this thread here for a JPK kernel you can flash with modified recovery, as well as modified initrd.

http://xdaforums.com/showthread.php?t=788108
[Galaxy S I9000 General] atinm:
Hey newmail - come join #project-voodoo on irc freenode and talk to us. Unhelpful and I have been pretty much working solely on building ext4.ko for the leaked 2.2 froyo kernel images the last few days and I have created/modified the scripts that were produced by dkcldark @ xda that can inject our own initramfs into the leaked kernel. We've got to the point where we can build the ext4.ko but insmod on it reboots the phone so there seems to still be some memory issue and so the next try will be to port over the s5pc110 stuff from .29 to .32 and see if we do better.

-Atin
[Galaxy S I9000 General] sztupy:
I'm still doing a multi merge between the éclair original-samsung kernel, the (who knows where it came from) git tree which supports s5pc110+discontigmem and the original froyo kernel, but still without much luck of compiling a working module. This is probably mostly the same mess as you have...
[Galaxy S I9000 General] newmail:
Thanks a lot Atinm, I'll try to join you this weekend. Porting all arch code isn't needed to build common code (I didn't). You've to get Kconfig right in arch however. I already did what is necessary so I hope I can save you the time and that my module works just fine (let me know if it doesn't and I'll figure it out). As written I can send a diff against 2.6.32 android but it's very broken state and it takes build tricks to make any use of it to circumvent the not ported arch code :). Let's talk the next days...
[Galaxy S I9000 General] sztupy:
extracting and modifying initramfs on linux is quite easy, you just need to use some magic numbers to separate the initramfis from the rest of the kernel. For repacking here is a complete toolchain. Signature verification was aded in the new recovery binary for froyo, the eclair ones don't have them, replacing it in initramfs solves the problems.
[Galaxy S I9000 General] vitalij:
Hi, I have tried to use your module. insmod command works fine. Cat /proc/filesystems shows ext4 as available filesystem. However if I try to mount it, mount command do not recognize -t ext4 option. Am I doing something wrong?
[Galaxy S I9000 General] newmail:
One thing comes to mind. It'd be interesting to experiment with -J -o size=$[102400*4096] (400M wasted for a logging filesystem isn't too bad consdering I won't ever get close to fill all 2G of space) plus a mount with -o data=journal (instead of the default data=ordered). I wonder if that will help speeding up fsync after database udpate, even further than on ext2, by allowing all updates to go in new disk locations (and then asynchronously internally to the journal code writing later to the final block destination which may be very slow if the underlying logging filesystem inside the hardware has to relocate the block somewhere else to avoid the wear out)
[Galaxy S I9000 General] newmail:
Does dmesg show something?
[Galaxy S I9000 General] newmail:
btw I posted strace for JPK on the forum, you should look into strace mount to be sure it executes the mount syscall.
[Galaxy S I9000 General] vitalij:
No, it doesn't, if I insert modules in order jbd2.ko, ext4.ko

And thats strace output:

/data/local/strace mount -t ext4 /dev/loop1 /data/ext4data
execve("/system/xbin/mount", ["mount", "-t", "ext4", "/dev/loop1", "/data/ext4data"], [/* 12 vars */]) = 0
uname({sys="Linux", node="localhost", ...}) = 0
brk(0) = 0x1dd000
brk(0x1ddd02) = 0x1ddd02
set_tls(0x1dd4a0, 0x1d7018, 0, 0x1d7be8, 0x1dd4a0) = 0
brk(0x1fed02) = 0x1fed02
brk(0x1ff000) = 0x1ff000
getuid32() = 0
getuid32() = 0
geteuid32() = 0
stat64(0xbeca7deb, 0xbeca76e0) = 0
mount("/dev/loop1", "/data/ext4data", "ext4", MS_SILENT, "") = -1 EINVAL (Invalid argument)
write(2, "mount: mounting /dev/loop1 on /d"..., 70mount: mounting /dev/loop1 on /data/ext4data failed: Invalid argument
) = 70
exit_group(-1) = ?
[Galaxy S I9000 General] newmail:
you can try debugfs /dev/loop1, stats command.

To create use mkfs.ext4 or the proper config option, maybe ext4 isn't there? dunno....
[Galaxy S I9000 General] vitalij:
Well it looks that problem was with loop device. When I created real partition on sdcard, it was mounted correctly. Thank you for your help!
[Galaxy S I9000 General] atinm:
Can you build .ko for ext4 and jbd2 without any debugging information? Right now ext4.ko is clocking in at size 3019596 which is way too big to fit in the initramfs if we try to inject it. Thanks. Ideally you'd put up your changes onto github and maybe just a terminal log of how you went about building the two modules as a Building_modules.txt in there too.

Thanks a lot,

-Atin
[Galaxy S I9000 General] newmail:
I didn't check the size :), I disabled it in .config now. You could get rid of it with strip --strip-debug I think but I rebuilt it just in case... Included the tun.ko module too, with 10 times smaller size too :).
[Galaxy S I9000 General] RyanZA:
Does anybody know of a version of either UMSDOS or perhaps a similar project that combines linux attributes with FAT32? This way, we wouldn't have to change anything - we could just mount /data as UMSDOS or something and get full speed benefits that way. Would be nice to test, at least. :)

For anybody wondering, UMSDOS is an old linux module that adds linux permissions to FAT32, similar to what RFS does. UMSDOS was really fast though (if very flakey as well!) and it would be cool to test. :) I don't believe UMSDOS is really maintained anymore though, so it probably won't work with the 2.6 kernel. Just something I thought of earlier and thought I'd share it in case it helps someone!
[Galaxy S I9000 General] vitalij:
So I was trying to use these modules with ext4 formatted partition on external sd. Then I moved to it /data/data. Everything looks to be working ok, but performance have not increased so much as with mimocan kernel on eclair. I was getting scores about 1000 in quadrant. Performance of realbench was also not good.

I tried ext2 loopback on top of it,then I was getting about 1900 quadrant result, but realbench was laggy and stalling for 2-6 seconds and sometimes fc.

So I am back to ext2. If any of you have better luck with froyo and ext4 please tell.
[Galaxy S I9000 General] newmail:
Hmmm I wanted to make some test with data=journal and a large journal size. However when I mount the loop device I got an alias of /mnt/.lfs mounted. I created a file in it and I can't delete it :)

# rm /mnt/.lfs/x

rm: can't remove '/mnt/.lfs/x': Operation not permitted

What's up with that j4fs weird filesystem? Maybe I should unmount it at boot. Is it possible to delete a file in it at all?
[Galaxy S I9000 General] vitalij:
Not sure about this file :)

However, I have tried to use data=journal and data=ordered with the same result: quadrant was about 1000. That was very strange, I have tried formatting partition both with paragon and with e2fsprogs: mkfs.ext4, that didn't help either. I even tried tune2fs -o journal_data_writeback with the same result.
[Galaxy S I9000 General] newmail:
I created the file in /mnt/.lfs with "touch x". myself It seems the filesystem allows file creation but the unlink operation isn't implemented in the file operations, so there's no chance to delete it. I'll remain sticky forever I guess (maybe flashing gets rid of it, I don't know). I'll have to leave it there, just make sure not to ever write anything to /mnt/.lfs as it can't be removed!

something's wrong with quadrant. From an usability prospective I got tremendous speedup in k9 and almost anything by just nullifying vfs_fsync (in addition to the fsync on close). Yet quadrant only goes up from ~950 to ~1030.

But the time it takes for the I/O test decreases from like 1 minute to 10 sec or so. I don't trust quadrant at all, computing on server side sending froyo details and everything, it doesn't mean a thing unless we know what it computes.

I repeat: we need to change benchmark, at least to know what a number means, when discussing lag-io issues. Making k9 like order of magnitudes faster is a very significant thing and it isn't almost measured by quadrant (if you wall clock the time it takes to complete the full benchmark it'll take half the time, yet the score improves less than 10%!!!).
[Galaxy S I9000 General] newmail:
This reminds me on the scary and funny side, I've been tried some magic code, and I found an option to flash the the firmware of the camera, the magic code page said "careful what you do in this page". I run the JPK froyo ROM image so supposedly the firmware of the camera is included in the JPK but I'm not sure. After flashing it asked me to reboot. So far so good. While the phone was rebooting I followed some other link of other magic codes, and for the same code I read "if you use this magic code don't flash the camera firmware or you'll brick it and you've to bring it to service center".... Panik!! Cause I already flashed it... The previous page said "careful" (I thought it meant not to pull out the battery while it was flashing, not that the firmware was dead wrong).

Luckily everything works perfectly still in the camera so I got lucky... But don't ever flash the camera firmware!!!!! Nothing has improved anyway it seems, it's identical...

If I keep going like this I'm optimistic I'll find a way to brick my phone :)
[Galaxy S I9000 General] googlymoogly:
A few respectable review sites have been using the softweg benchmark suite. In fact, I can't remember ever seeing Quadrant scores in serious Android publications.
[Galaxy S I9000 General] newmail:
Reading fsd_unlink asm (nothing at all calls that function, probably the f_ops->unlink was not compiled in but it has been implemented at some point if fsd_unlink exists) I figured it takes just one param only and it's a string. So I did:

unsigned long (*_fsd_unlink)(const char *name) = 0xbf19b4fc;

_fsd_unlink("x");

This deleted the "x" file from /mnt/.lfs just fine :). Now I carefully stay away from that /mnt/.lfs thing...
[Galaxy S I9000 General] sztupy:
I got the same result when I tried to load factroyfs.rfs in rfs mode with a loop device on JPK. So either factoryfs.rfs contains two partitions (one j4me and one rfs), or something is ultimately flawed somewhere and the j4ms parition gets mounted whenever something strange happens while mounting.
[Galaxy S I9000 General] RyanZA:
softweg isn't very good in it's current state. No good benchmark exists yet (that I know of).
[Galaxy S I9000 General] penfold:
i like my quadrant score after OCLF JM2..... 2350 :)
[Galaxy S I9000 General] newmail:
I guess the mount of j4fs partition has no magic number so it succeeds unconditionally, so making the default -t "auto" option always picking that filesystem because it registered itself in the filesystem list before ext4. I guess if I used -t ext4 I would have gotten it...
[Galaxy S I9000 General] newmail:
maybe quadrant checks if OCLF is installed and return a x2 result? I don't trust that not open source stuff.

It's unfortunate because quadrant looks to be exercising the right stuff, just I can't trust that number that it produces to heave any meaning.

Can you use a stop-watch to measure the wall clock time it takes to run the whole benchmark with and without OCLF. I can do the same on mine..... with or without my rfs_async that removes both fsync and sync on close. There's no reason to depend on a website to give a result back, measuring wall clock time is surely more reliable. It makes me sick that people is using closed source benchmarks on open platform.
[Galaxy S I9000 General] RyanZA:
LOL :)

No, Quadrant doesn't check that, it came out before OCLF did. ;)

Anyway, OCLF (loopback EXT2) gives much better effects than simply removing fsync from RFS (I've tried both). Give OCLF a try if you want to compare it, too. Wall clock time is noticeably different on loopback EXT2 as well (it's almost instant).

That said, Quadrant is NOT a perfect benchmark - all it does is open a sqlite database, run a few inserts and selects, and measures the time (appears to do this, anyway). This isn't that bad, as it does give a good idea of the I/O speed, but it's a long way from a perfect benchmark. :) (data set too small, operations too limited, single threaded, others)
[Galaxy S I9000 General] newmail:
OCLF uses ext2 + loop? If I mess around more than my current rfs_async_unsafe.ko that fixes k9 and speedup things significantly, it'll be only for ext4 or yaffs2, not for ext2 on loop.

So my question is, where does OCLF hooks into? If we can get an hook early enough at boot time before anything starts, I think we can do ext4 on the fly conversion and mounting without flashing or initramfs post install. Not a loop+ext2 hack but a real optimal safe and fast enough setup.

Undoing I don't care about, undoing only makes sense if you break stuff, and if the conversion breaks the undoing is even less likely to help. Let's not waste time on the undo feature... Nothing can perform worse than rfs I'm afraid.

My current idea is to move /system/bin/sh to /system/bin/sh.orig and replace it with this script:

#!/system/bin/sh.orig

sleep 10

mount / -o remount,rw

mount >/mylog

ps >>/mylog

mount / -o remount,ro

exec /system/bin/sh.orig

To see if it's possible to hook in there well before anything has started.

I hope samsung recovers from the NIH syndrome and replaces rfs in final froyo with yaffs2 or ext4, anything but not rfs will work. And not just from /data but from /dbdata and /cache too!