yashsingh1/xdaforums huggingface dataset:
[Galaxy S I9000 General] RyanZA:
I don't believe the firmware on the MoviNAND is in any way easy to modify. It would likely require very very specialized tools. It's probably out of our reach. Might not even be firmware, might have the instructions in hardware, even.
[Galaxy S I9000 General] tylerham:
Newer firmwares seem to have fewer lag issues (JM8 & JPK) than older ones.

Is Samsung slowly fixing write issues, masking them, or fixing other problems which were also slowing down the phone?
[Galaxy S I9000 General] D0ubl3_X:
Hey RyanZA,

which FS do you personally prefer?

Why not just use EXT3 to kill EXT4's online defragmentation which loads the cpu?

Thanks ;)
[Galaxy S I9000 General] xan:
@D0ubl3_X shhh ;p
[Galaxy S I9000 General] ragin:
I am an architect. Ours isn't a logical world. I put forth my ideas, & if someone differs, let him speak.

This is probably missing a lot of facts that we haven't uncovered yet. When we learn more, we can update what we know here

Boldly said, BUT, its good.
[Galaxy S I9000 General] ragin:
I think LAGs happen over a period of days.
[Galaxy S I9000 General] RyanZA:
Hahah! :D

I personally use EXT2, and I've had zero issues with it so far. :)

I would recommend either RFS for absolute safety, EXT4 with journaling for a good middle ground of quick boot ups and decent speed, or EXT2 for absolute performance.

I would probably never use anything besides EXT2 myself (with the possible exception of reiserfs or one of the new experimental filesystems) on the SGS, as it works the best FOR ME.

Obviously, if I learn otherwise that other filesystems are better, I'll be the first one jumping ship! I am always very very happy to admit if I am wrong about something. :)
[Galaxy S I9000 General] freakonaleash:
yeah!!

this is a great forum!!!

thank you ryanZA for your work and your lesson :D
[Galaxy S I9000 General] [Ramad]:
Did any one try YAFFS2 on SGS?

I know it is designed for NAND, is it not more logical to use a file system that is designed to work on NAND than a file system that is designed to handle HDD like ext2...3 etc.?
[Galaxy S I9000 General] borchgrevink:
Very interesting thread, but also a bit "disappointing", because no it seems that true fix is more or less impossible. Am I correct?
[Galaxy S I9000 General] supercurio:
I can't believe my eyes about what I'm reading.

RyanZA, please, stop trying to teach people when you

To make short. There is nothing right in RyanZA's original post.

It's not even worth explaining point per point why/how...

I re-read it 3 times... still can't find any point right... really this is absurd.

RyanZA, you don't get the whole picture, you don't get the details either... and now you say that what you think is "Reality" ?

Please !

Sorry but it just makes me sad.... really.

Why somebody tries so hard to publish and influence people on subjects he don't understand because he refuses to read everybody's else advices and experience. I don't understand.

People who read this. Forget these infos, will lead to complete misunderstanding on how filesystem work on Linux.

Hopefully.. well.. every other SGS developer think and act differently than RyanZA, because if RyanZA was leading SGS development this would go in a direction so wrong that you would want to sell your device.
[Galaxy S I9000 General] xan:
Oo?

You made me laugh with this post.

Not that I'm keeping RyanZA's side, no.

Just the way of expressing your opinion, you being the "smartest" without proving ANYTHING here to be wrong.

In OP, most of info is correct and can be confirmed, except ext2 vs ext4 and journaling parts, which are debatable.

Seriously, I thought you are so much more.
[Galaxy S I9000 General] woeds:
Well, it would certainly add some info to the discussion. Your post is pretty useless right now. I'm not saying you're wrong (you're probably right), but it would be nice to know the correct info regarding filesystems.
[Galaxy S I9000 General] supercurio:
Sorry xan.

It's because we are already so many having pointing errors in RyanZA publications day by day that it don't have the courage to detail this one :/

It's like an endless fight leading to nothing.

Maybe it's arrogant but I prefer using the same time to develop something that will stay and be useful :)

I understand why you are disappointed about my answer. sorry for that.
[Galaxy S I9000 General] jesperh:
Just for fun I tried ran this small test on my nexus and galaxy phones.

The loopcount was increased to 10000 to suppress the noise, and each test was run 3 times.

Nexus One: FRF91 (CyanogenMod-6.0.0-N1)

Galaxy S: FROYO.XXJPK (Stock ROM, only root has been applied)

galaxy	/data	111.16s	111.54s	113.64s
galaxy	/dbdata	047.68s	048.41s	048.14s
galaxy	/sdcard	064.88s	064.15s	066.87s
galaxy	/sdcard/external_sd	101.01s	099.06s	099.57s	(noname, 2x card)
nexus	/data	080.34s	076.71s	070.53s
nexus	/sdcard	177.73s	155.94s	173.47s	(kingston, 10x card)

This just tells me that the flushing is just a small part of the picture, and that my 10x Kingston card is kinda sucky :)
[Galaxy S I9000 General] bilboa1:
Hey that's nice someone took the time to make the test.

If i'm reading that right, the sdcard of the SGS+RFS is much much faster than the N1 (YAFFS on /data and FAT on /sdcard or ? I don't have one), but the nand is slightly slower (since its 100 iterations)

It would be nice if you could repeat the test with the SGS /data over EXT4 (and EXT2 to make Ryan happy!)

Remember everyone, that usually with properly coded applications fsync is rarely actually called, unless it's important, or a benchmark that wants to test the fs and physical media speed, so take the results with a grain of salt, it doesnt mean the N1 /sdcard is going to be all that slow (still it's probably kinda slower :p)
[Galaxy S I9000 General] xan:
Could you please provide compiled binary? :D
[Galaxy S I9000 General] giokou:
Thank you for the info and keep up the excellent work, you are a hope of our's! ;)
[Galaxy S I9000 General] dmulligan:
Supercurio: I know I would be happy if you would provide us with the correct facts. If you can not or don't wish to then you are just being a troll and you are not adding any value to this discussion.
[Galaxy S I9000 General] andrewluecke:
I'm pretty sure you only use YAFFS mainly if you are writing to the flash directly, have devices with limited resources, and a flash setup with no layer below it to handle wear leveling and such
[Galaxy S I9000 General] dcarr622:
Forgive my ignorance but why hasn't anyone tried native ext2 it seems like the most logical thing to do?

Also does anyone know what the cyanogenmod devs will be using? I am very curious to see what they come up with.
[Galaxy S I9000 General] ed10000:
Wow. That was pretty low. If you find any information to be incorrect, why don't you correct it rather than crying out loud that everyone else is dumb? Even though lots of devs on this thread have very different views and opinions, I don't see anyone calling each other names - they're providing their insight on the issue, but ironically people who (almost) always fail (or rather DON'T want to) share valuable information are always the ones posing as the ultimate linux gurus. Weird. I'm sorry but I had to say this, I hate it how you make everything personal.
[Galaxy S I9000 General] andrewluecke:
So lets take it 1 step at a time then.. The essence of his argument is mainly that Fsync on this device is slow, and RFS uses it whenever closing any file handle. If write caching doesn't exist, this is plausible. Or can you prove it's wrong?

I'm sorry, but really, it's time we ask why EXT4 is getting better performance, because FAT (which is is what RFS is based on) doesn't have this issue on computers, and EXT4 shouldn't be less laggy when it's a layer on top of the RFS layer. That seems to suggest RFS is only slow when creating,deleting,opening or/and closing files.

If we understand why the problem happens, we might find a silver bullet to solve it.. But as per anyone else, if you do believe it's all wrong, prove it.. Similarly, it would be nice for RyanZA to post some actual evidence regarding how these conclusions were drawn (profiling would be useful, but not sure if there are proper profiling tools for Android).
[Galaxy S I9000 General] dalingrin:
The MMC controller on the Nexus is terribly slow. Put the Kingston in the Galaxy S and I'm sure it will be faster.
[Galaxy S I9000 General] DamianGto:
RyanZA: that was alot to read.

But was nice to read.

To bad there is always someone that try to be better, but they can do write any better them self.

Keep up the good work.

You have already proved you know what you are talking about.
[Galaxy S I9000 General] Prasad:
+1

Very informative post! Being in the computer engineering field, it was an interesting read too... Looking forward to your future work..
[Galaxy S I9000 General] q123123an:
Yes, very informative post RyanZA, thank you. And what I found most valuable, is that you are open to discussion and other insights.
[Galaxy S I9000 General] supercurio:
Ok, i'll write a detailed response on this detailed post, using colors to show you what's right/wrong/pure invention.

Prepare yourself, it will be super long ^^

(and this will be my last post of this type, at least as long as I'm working on Voodoo beta5 because of time constrain issues)

Message to people not understanding my reaction: How do you expect me to react on RyanZAs FUD about Voodoo?

Just let it go and let him misinform people?

Just a sample:

Owooo, my FUD detector is beeping hard!

The content of this quote is a sample of affirmations which are 50% "ok", and 50% are plain wrong. What's in bold is just lies.

Yes, i'm tired of RyanZA's FUD. I call it like this because its recurrent and insistent since months.
[Galaxy S I9000 General] Evans_Prophet:
I really admire the fact that there is an honest intention for fruitful discussion and this has not resulted to a flame war.

This shows that that the sgs community has some very mature developers and an inquiring audience. For me it doesn't matter whether supercurio or ryanza is right. Besides, both have provided valuable input to the community. Kudos to both of them for conducting a high level, civilized discussion on technical matters...
[Galaxy S I9000 General] RyanZA:
Nice info, thanks! :D

Matches with what I wrote in the first post, but the difference is not really that much. So while the syncing is slower (at least 50% slower), it isn't slow enough to cause the huge lags. It probably does have an effect, but there are probably more factors involved here too.

It's interesting to note the nexus/sdcard is so much slower than galaxy/sdcard/sd. This shows that the actual data controller on the galaxy is far far faster than the controller on the nexus. So maybe the sync really IS that much slower.

ie. Normal Speed for the SGS = 10

Normal Speed for the N1 = 5

But sync speed for the SGS = 2.5

This means that sync is 4x more expensive on the SGS than on the N1 in relation to other file performance. Which is a pretty big difference, after all! :)

Need more testing. And I look forward to reading the explanations from supercurio, I hope they'll be enlightening! :)
[Galaxy S I9000 General] RyanZA:
I really wish I could, but there are no good profiling tools that I have been able to find! Most of my tests are just making some java or native apps, and doing little writes to disk and that type of thing. It's really not very good testing. However, that's what these discussions are for, to get people interested and testing, and if people get interested then someone WILL find the solution! :D

If you check out the tests by jesperh, they're definitely in the right direction, and you can already see some useful information peaking though!
[Galaxy S I9000 General] DubZyy:
ok, tell me what way i should use to make an EXT2 disk check when i had the 3 button bug?

my english is too bad to tell you the complete story but a member of the german androidpit board asked me 1 week ago if he could translate my experience with the ext2 data loss to you.

and of course i started to try everything to get my phone to work. so i started my phone i think 50-100 times and of course i waited few time till 30minutes for booting the phone... but after 5 minutes the boot images closes so the screen got black and the touch keys glow.

i pulled the battery out while android was running on the desktop. so i can say that with ext2 you can brick your phone (it dont have to...).
[Galaxy S I9000 General] RyanZA:
You can brick your phone without ext2 as well. :) I'm sorry to say it, but your experience doesn't really add up with most others (50 000+ currently using ext2), and you dont really provide enough info for me to work out what could have gone wrong...
[Galaxy S I9000 General] arvindm:
Great post RyanZA. Much appreciated..
[Galaxy S I9000 General] Oletros:
Install Radardroid Lite and import speed cameras from Spain.

You can see the differences using lag fix or not, or using GSG or a Nexus One
[Galaxy S I9000 General] RyanZA:
I was thinking about this test, and I dont think its the perfect test for the sync issue. What this test does is create one file, and then sync it. This should be speedy on movinand, since it's only one item in the movinand queue. The real test is to write many files, and then compare sync speeds, rather than just one file.
[Galaxy S I9000 General] ro4sho:
Not very constructive...
[Galaxy S I9000 General] dakine:
Yeah let's keep it civil. Supercurio if there are things that you perceive to be incorrect then just make a concise post pointing out the error, no need to get worked up over some post on XDA. RyanZA obviously just shrugs off any criticism anyway so I don't think bashing him is going to achieve much :)

You are both making great progress on this device so I don't see a reason that you have to necessarily agree on things. Worst thing that can happen is that we end up with some bricked phones....no big deal.
[Galaxy S I9000 General] bilboa1:
1: YAFFS is not made to work on the SGS because it doesn't have direct flash access, the SGS flash has this virtual layer on top, taking care of everything already (which might or might not be slower/better, but that's not the point anyway)

2:

As curio mentioned on IRC, /dbdata is the critical area which holds the most SQLite data (hey it's called db), and it's extremely fast on fsync() compared to all others.

3: ext4 on the sgs is working (for whoever asked curio to go write it.. he did, look at Voodoo project)

4: Ryan, once again even thus you will probably ignore what I write, others will read:

OCLF (which I assume you refer to) writes it's EXT2 data on top of RFS which is journaled (even thus it gets corrupted, it only seems to affect performance, not integrity).

Meaning, the 3 button manipulation will work even if EXT2 is corrupted, as far as I know you're mounting it later in the boot process, thus it's irrelevant. Using EXT2 as native file format is another story.

Your persistence is in my eyes only due to the fact that it's easier for you to use EXT2 (its included in samsung's kernel) and that you have used that in your own software already.

It's all fine and well presented that way, and actually useful for everyone!

Unfortunately you do not present it that way, and through your small fame on SGS XDA forum, you easily spread wrong information to the other people.

People who are actually able to understand what stands behind what we discuss are not your casual programmer, thus it's very easy to mislead a lot of people, just for some ego story.

Please, stop doing that !

Disclaimer: I don't claim to know everything. Sometimes I might even be wrong, but I tend to avoid that if I can :p I'm also a paid kernel dev. Even so, I don't know the whole kernel myself, no one does anyway. But what we discuss is not all that advanced and quite common.

I hope we can work together for a better SGS instead ;)
[Galaxy S I9000 General] [Ramad]:
Thanks for the answers guys.:)
[Galaxy S I9000 General] RyanZA:
Hey, it's awesome to have a paid kernel developer giving us advice! Thanks, I'm sure the info will help out a lot! :) Which part of the kernel do you work on, btw? I'm curious! :)

That said, /dbdata is only for storing the stock app databases. It only stores the settings database, as well as the stock calendar and email databases, etc.

User installed apps store their databases inside their own data folders, inside /data/data :) - This is why increasing the speed of /data is the most important, since most people use non-stock apps!

In regards to loopback EXT2 vs native EXT2, I've pulled the battery a number of times while using native EXT2 without issue! Not to say that it can't cause issues, but I haven't had any. I heard from another kernel dev that EXT2 will be as safe as EXT4 as long as filesystem checks are done. He may have been mistaken though, as he did preface it with "I'm not 100% sure, but" :)

Could you explain how EXT2 with a full filesystem check is less "data-safe" than EXT4 with journaling? The journals (afaik) are only there to enable fast filesystem checks by giving records of what was done to the disk -- any lost data on EXT2 would be lost on EXT4 as well. Perhaps the metadata itself could become corrupt on EXT2 and not on EXT4?

You mention as well that EXT2 loopback on RFS would be more safe than EXT2 native. I don't really understand how that is possible? All the RFS journal would say is that 'data was written to /data/linux.ext2 file', the journal wouldn't be able to help with real data loss.

My persistence on this is because I want my phone to be fast, and I want to know more about how this is all working. I don't really care if the conclusion ends up that EXT4 is always better - converting my app to use EXT4 is not a big task (I already have an EXT4 version after all, for 2.1 at least).

I read (and enjoy reading) everything you write, and I definitely don't ignore it. But I'd prefer if you backed up what you say with some facts so we could all learn from it! Telling us it's 'not safe' doesn't help me, I'm afraid. Why is it not safe? What does it do at a low level that makes it not safe? Any information in this regard would be very very helpful! Thank you for your time! :)
[Galaxy S I9000 General] g00ndu:
the reality of the rfs lag is samsung had messed up an otherwise excellent phone. that's the reality.
[Galaxy S I9000 General] RyanZA:
Sure, that's obvious. The real question is - what makes RFS lag!

Is it 1) Bad base filesystem (FAT32)? This one isn't so likely, since FAT32 seems to work quite speedily on PCs, and definitely no 5 second pauses! So is it MoviNAND+FAT32?

2) Badly made Linux RFS driver - does the Linux RFS driver call fsync far too much? This should slow things down a bit, but shouldn't be giving us 5 second pauses! Is it the combination of fsyncs and MoviNAND, such that MoviNAND behaves badly when told to fsync?

3) Is it the RFS journaling on top of FAT32? Is the journaling a MoviNAND problem, or is it that the journaling used is just very badly made?

4) Other?

Depending on what exactly causes the lag, a simple modding of the RFS driver could give us all the 'lagfix' we need, without any chance of messing things up. Or, it might mean that removing RFS is required. The whole point here is to work this stuff out, and working it out could mean that YOU get a faster device. Or we can just all resort to name calling and pretending that we know everything already. Your choice?
[Galaxy S I9000 General] g00ndu:
@RyanZA, you have been very civil about the whole thing and for that you do have my respect. :) I'm no kernel developer, heck I don't even know what does a kernel do. :( But what my previous post, though does not serve any meaningful insights to the working of the rfs system, highlighted the fact that if samsung had done a better job, we wouldn't be arguing all this in the first place.

And of course, who doesn't want the phone to be fast? from the way i see it, with a proper file system, and improved jit, coupled with the existing excellent graphics, this phone will still be tough to beat even in 6 months time.

i am really looking towards the day when these niggling issues are resolved once and for all. :)
[Galaxy S I9000 General] vlefevre86:
+1 :)

10char
[Galaxy S I9000 General] Kino87:
Before even starting: sorry for my bad english.

I'm not really good at this sort of things, I know a little bit of linux, programming and how computer works but that's all.

For what I can undestand having just one big file in the filesystem helps the controller (well... bypasses some slow operations) but obviusly having one FS on top of the other slows things down.

Usually simpler filesystem means faster access and inferior security.

Given this and the fact that RFS seems to be pretty buggy at the moment wouldn't be better for performance/security to have a pretty secure FS at the bottom (ext4?) and the simplest thing possible on top of it (even a raw partition like the one used for swap in many linux distributions, if possible.. i don't know) or the opposite (simple/fast on the bottom - safe/slow on top)? It's horrible, I know; but in this case seems that the most elegant solution isn't the fastest one.

Probably all this is just really really dumb, and I'm sorry; but maybe it's worth asking
[Galaxy S I9000 General] RyanZA:
Some useful information from newmail regarding why EXT4 with journals could be better than EXT2: http://xdaforums.com/showthread.php?p=8486028#post8486028

Sounds like EXT4 could be a better choice if power loss is expected, since what he says seems to make sense! :) I haven't been able to reproduce what he's talking about yet though.
[Galaxy S I9000 General] huxflux2003:
But you really don't have power loss on a phone. Once battery voltage is below a threshold, the phone gracefully shuts down. Pulling the battery is no normal and every day use-case.
[Galaxy S I9000 General] RyanZA:
True enough. I also can't seem to replicate the data loss with battery pull thing on EXT2. Might be more to it, since MoviNAND does have it's own controller? I'll do more testing on battery pulls on EXT2 and EXT4 and RFS with different file operations! :) That said, I haven't personally every pulled the battery out of my phone while it was still running except for testing purposes... so even if I do manage to get the data loss to occur, I don't think I'll personally switch because of it. Still very important to know! :)
[Galaxy S I9000 General] Project1.exe:
Perhaps this is the reason why the iphone engineers didn't want a removable battery, cause they might have an agreement with the software dept to use a filesystem that is fast but a chance of failure if battery is removed.
[Galaxy S I9000 General] RyanZA:
Yeah, in an interview I read, an iphone engineer claimed that a primary reason for the lack of removable battery was data loss, since the iphone doesn't use journaling. Plus, it reduces the number of support calls for users who use bad 3rd party batteries that don't provide enough voltage. It also makes the device a bit slimmer, and makes some revenue in battery replacements. ;)