yashsingh1/xdaforums huggingface dataset:
[Galaxy S I9000 Android Development] RyanZA:
I was referring more to a combination of Extents ("since an extent encourages continuous layouts on the disk" but also has a higher overhead afaik) as well as Multiblock allocation, which tries to ensure continuous layouts as well. I had thought that the online defragmentation was finished... but I guess not.

If filesystem block size != MoviNAND block size, then I'd think that a big performance boost may be in matching filesystem block size to movinand block size. If they aren't equal, then it would also be very important to ensure that they are at least aligned properly, else one filesystem block write could cause 2 movinand block writes.

More speculation here, but I believe that MoviNAND will rewrite an entire block if told to write data to disk. More diagrams!

Here is some data blocks that need to be written to disk. They are continuous, and 2 filesystem blocks = 1 movinand block.

___1234___

Here is the movinand blocks, however since 2 filesystem blocks = 1 movinand block, I'm going to use [12] and [34] to denote the blocks on movinand, since they cover 2 filesystem blocks

___[12]___[34]_

Here comes the interesting bit: the filesystem wants to write block 1 and block 2 to disk, so first it writes block 1. This causes the [12] block to be erased and rewritten with data that would contain new 1 and old 2. Then the filesystem writes block 2. This causes block [12] to be written again! This time with new block 1 and new block 2. So just because the blocks are continuous, doesn't mean we'r saving anything at all! :D

Although, MoviNAND is a 'smart' chip with a 'smart' controller, so it might have picked up that we asked the same block to be written twice and consolidated it into 1 write. Is MoviNAND really that smart? I personally doubt it! :)

As always, so much we don't know!

I don't think the probability is big. It could just as likely be leading to decreased wear, surely? The block size may even be the same. (I've been using a 4KB block size, we need to try and work out MoviNAND block size somehow)
[Galaxy S I9000 Android Development] jbrugger:
As stated before, i don't know alot about filesystems, but just a question, does this help?

http: //www.samsung.com/global/system/business/semiconductor/product/2010/4/16/078359S3C2450_51_16_IROM_ApplicationNote.pdf

(not allowed to post hyperlinks)

Sorry. Pasted wrong link...

http: //www.mobiletor.com/2010/09/07/samsung-launches-8gb-and-16gb-movinand-embedded-memory-chips/

About standards
[Galaxy S I9000 Android Development] dupel:
No, no. I think you misunderstood me. I never had data loss on OCLF (ext2), but had data loss on voodoo (ext 4).

And dont get me wrong Im not saying voodoo is worse than oclf or better, or not safe, etc. Just saying that even on "safer" fs you can have data corruption (and it doesn't really matter why (yes, it was my fault for playing too much with undervoltaging, thus having not responding phone)).

Which brings up the topic of "is it really safer (ext4 vs ext2)". And I dont know, as im not really a linux person, so just report that its possible and leave you to actually decide if its relevant, important etc.
[Galaxy S I9000 Android Development] RyanZA:
Noap, not too useful, unless you're trying to create a custom bootloader for the SGS. Which would be awesome, but I have no idea how! :)
[Galaxy S I9000 Android Development] RyanZA:
Ah...

I doubt your data loss on Voodoo was at all filesystem related. It was likely related to problems in the GPLed kernel, or the data transfer script in Voodoo. (Voodoo zips up your data and saves it on the sdcard, if you don't have enough space on the sdcard, then bad stuff can happen. ;) )

So it probably isn't EXT4 related at all. EXT is a pretty good filesystem, really. :)
[Galaxy S I9000 Android Development] xan:
As far I know, extents are in no way related to moving files around, making space for other files. It just distributes files assuring that there are relatively large contiguous free spaces (groups of blocks - extents). Multiblock allocation's functionality shouldnt be causing additional writes, it just decreases the number of block allocator calls a lot, and 'lets file system know' thus much data needs to be written, and for how big space it should look. Its based on predicting.

Neither of these functions involve 'moving' files/blocks.
[Galaxy S I9000 Android Development] RyanZA:
Makes a lot of sense, definitely shouldn't be slowing things down! I did try EXT4 unjournaled though, and the performance on the SGS was worse than EXT2. It was a real pain to set it all up, and I didn't do very good benchmarks though, and I may not have turned off all journaling either. So while I'm fairly sure that EXT4 without journals should perform at least as well as EXT2, I think in practice EXT4 without journals is probably almost identical to EXT2. Guess we need more testing, though.

Given the performance difference being small though, and EXT2 being supported by stock kernels while EXT4 requires a module to be built (currently impossible for Froyo?), it's probably best to stick with EXT2 for maximum compatibility.

Still needs more testing, though! :) Specially block size testing, to try and match MoviNAND exactly, as well as making sure that partitions are aligned properly. (The stock RFS partitions are aligned along the 4KB boundary, btw)
[Galaxy S I9000 Android Development] xan:
Actually for MoviNAND moving blocks is another "write" operation. It decides itself where to put the data. There is no way of determining the place it puts it, so there is no reason that green layouts are correct, thus there is no way of concluding that file system operations increase MoviNAND fragmentation. We need to assume (since wear-leveling algorithm picks block with least write count, which could be virtually anywhere) MoviNAND is heavily fragmented, with minimal suspected fragment being MN block size. There should be bigger fragments allowed. This is intended and normal.

So I assume, regarding fragmentation we came to point where it all depends on MoviNAND, which we don't know.

Another issue, being wear-related:

Is there any way to determine a number of actual writes? (modded driver?) ext4 has unique features limiting number of them, I'm curious if they actually decrease write count. If not, ext2 way seems to be better. (Not considering journaling)
[Galaxy S I9000 Android Development] supercurio:
Yes, this one is a good idea.

Until now I used ext4 default 4k block size because mkfs yelled hard at me when using bigger ones.

But this is interesting (but more complex than we could think at first look) track.

Still, i don't know any special guideline inciting using bigger block size than default 4096 on desktop SSD, and MoviNAND a little SSD.
[Galaxy S I9000 Android Development] andrewluecke:
DELETED (Sorry, missed a few posts)
[Galaxy S I9000 Android Development] RyanZA:
No flames about trying to use EXT2?!?! Awesome! :D

I'm glad this discussion has helped! Hopefully the block size tweaks will get Voodoo even better! :)
[Galaxy S I9000 Android Development] exadeci:
So why ext2 is not already the prefered one ??

Who cares about boot time if after it runs faster and use less cpu ???

What is the reason that some of you prefer ext4 ??
[Galaxy S I9000 Android Development] xan:
From where did u get this?

There is no proof presented from nobody, backing the hypothesis ext2 has lower cpu/controller overhead.

All of RyanZA's claims are based on assumptions, not facts, documents.

So, ext2 is not preferred one.
[Galaxy S I9000 Android Development] RyanZA:
All of the claims for EXT4 appear to be assumptions too - the speed difference of EXT4 vs EXT2 is very easy to test though.

It sounds like you have a bit of an agenda in declaring ext2 not the preferred one? You could just as easily be claiming ext4 as not the preferred one for the identical reasons! :)

That said, some useful information here on why ext4 could be better for data safety!

http://xdaforums.com/showthread.php?t=777610&page=18
[Galaxy S I9000 Android Development] xan:
There are documented features of ext4 which are not present in ext2. This makes ext4 a better filesystem.
[Galaxy S I9000 Android Development] dirk1978:
Regarding filesystems. We should be aiming for the latest and greatest surely? I've been using ext4 on all my Linux installs for the past 2 years and from a general experience point of view it's great. It's all well and good comparing features but ext4 is the default fs for almost every new Linux install aend surely in massively wider use than ext2 or maybe even ext3.

Personally I'd feel much more comfortable running ext4 as my system partition than ext2 or 3.

I'm still not sure movinand does wear-levelling, can anyone show me where it says this? (other than the Samsung movinand product page that just mentions wear-levelling but not specifically that it does).
[Galaxy S I9000 Android Development] RyanZA:
Unfortunately no - the only evidence of wear leveling we have is from the Samsung marketing material. It's almost impossible that it doesn't have wear leveling though, I think, and I can't see any reason for Samsung to lie about it.
[Galaxy S I9000 Android Development] RyanZA:
This part here is quite interesting in regards to ext2 vs ext4, and may be why I'm not seeing data loss on battery pull when using ext2?

At any rate, the block sizes are enormous! We won't be able to efficiently match a filesystem block size to one of those monsters. :) This leaves us with matching block size to page size I guess, which is almost definitely 4KB (since 4KB is far faster than 1 or 2 KB). The default partitions align on 4KB boundaries too, which lends some accuracy to the 4KB block size being correct.
[Galaxy S I9000 Android Development] xan:
Hardware ECC doesnt make data residing in buffers to be magically flushed to physical media. So this is definitely not the reason.
[Galaxy S I9000 Android Development] bilboa1:
I didn't have the time to review the replies in the other thread, but this one seems to go in the right direction, from ryan and others too!

Hopefully my posts made it go toward that goal and didn't make anyone feel bad :p

So yeah a bit of noise, don't reply to this post :p Just wanted to say good job continue ^^

In my mind eventually we'll end up with a ext4 fixup and a rfs workaround for the ones who don't want to reformat their partitions.

Also make sure to test all this stuff the database partition of the sgs too as it seems very fast on RFS (but must suffer the lag issues too?)
[Galaxy S I9000 Android Development] jbrugger:
Well let's look at RFS then as a starting point.

Samsung states:

RFS supports FAT16/32 and 512B/1KB/2KB/4KB cluster size and more.

We recommend you to use the cluster size greater than OneNAND™ page size.

Would this not count for moviNAND as well?

A flash memory does not work like an ordinary

block device; it is not possible to write twice to the

same memory location without first performing a

very time-expensive erase. NAND flash memory is

programmed on a page basis and erased on a block

basis. NAND flash memory cannot be overwritten

without erasing a block. And the size of block is

bigger than the size of a page because a block is

composed of N pages. An erase must be done on

whole block at a time and a common block size is

128KB. That is the reason why Samsung came out

with a new file system specialized for NAND based

flash memories. An already existing file system

would not work. For 'robustness', it provides a journaling

based error recovery mechanism, which

guarantees that the file system runs at all times even

if there is a sudden power loss.

See also: http://lwn.net/Articles/362020/

Ps about the moviNAND chip self:

Closed match i could find is that we have the KLMAG8DEFD-A301 (16 Gb) chip.

K: Memory

L: MOVI NAND/MCP (CTL : Controller)

M: moviNAND (NAND Function)

AG: 16G Byte (embedded NAND Density)

8: NAND8 + Microcontroller

D: MLC 16G Bit

E: VCC = 3.3V(2.5V~3.6V) , VCCQ = 1.8V(1.65V~1.95V) 3.3V(2.5V ~ 3.6V) mode= dual

F: Contoller???

D : 5th Generation

-

A: FBGA (Halogen-Free, Lead-Free, OSP) (12*16)

3: EVT3

0: None (speed)

1: Normal Sample

Searching another article:

2.4.1.5 Drawbacks of NAND Flash Memory

The page size of each NAND flash memory page should be 512 bytes. All the data

can be accessed bit by bit. Before accessing any byte of data, the address of the data

being accessed should be set. In other words, we need several steps while accessing each

single byte of data. This strategy is lack of efficiency. In addition, this serial operation

induces that it can’t be accessed by DMA (Direct Memory Access).

2.4.2 moviNAND Flash

moviNAND allows NAND flash to be instantly accessed through a standardized

MMC interface, manufacturers who use the new memory solution can develop mobile

phones more quickly and at less cost than with raw MLC NAND components soldered

into a handset.

2.4.2.1 moviNAND Architecture

The moviNAND is an embedded MMC card solution designed in a form of BGA

package. moviNAND operation is identical to MMC card and therefore is a simple read

and write memory using an industry standard MMC protocol of 4.2. moviNAND consists

of NAND and MMC controller. 3V supply voltage is required for NAND (VDDF)

whereas 1.8V or 3V dual supply voltage (VDD) is supported for MMC controller(Figure

2.6). Maximum MMC interface frequency of 52MHz and maximum bus widths of 8 bit

are supported.

See the following pdf (scroll down for english).

http://www.google.nl/url?sa=t&sourc...4RF7nbn4GzfAPDaHQ&sig2=qkVI_tqenKwiveNQQ3r5YQ
[Galaxy S I9000 Android Development] -MaxMad-:
fyi some additional information about ext4:

h-online.com/open/features/The-Ext4-Linux-file-system-746579.html
[Galaxy S I9000 Android Development] deteego:
To be honest, I see little reason in using the ext4 for a mobile device for numerous reasons

1. We are dealing with a NAND storage, which has practically linear access time regardless of the fragmentation. This is an issue on magnetic disks, as the head has to search for blocks that have been split up, but the impact on a SSD/NAND storage is negigable. In fact on SSD's (as an example), its highly recommend you defragment the disk because you won't get any real performance boost however you will waste a massive amount of write cycles (which is limited on these devices)

2. Ext4's extra features create overhead, this pays off massively for desktop systems which use a completely different type of disk (magnetic) and don't have issues with power consumption/longetivity, however again with mobiles its another story. extent allocation is used to help against fragmentation disk, again with #1 its not something that we should be concerned with. It can also lead (again) to unnecessary writes, since you allocate a portion of the filesystem and then write to it

So if we do a simple PROS

Ext4 Pros

- Extens allow for faster phone boot

Ext2 Pros

- Faster read time

- Faster write time

- Less writes onto disk (equates to less wear)

- Possibly better battery life (although this can be negligible)

There is only really one advantage of Ext4 (if you use its features), thats the better boot time on the mobile. Apart from that it arguably makes things (worse) in other areas for the mobile

Note that I am not bagging out ext4 as a filesystem in general (in fact its fantastic for desktop systems), however we are dealing with completely different hardware here, where we have to be wary of both read/write times (as they are more of a bottleneck on a mobile devices) as well as wear/tear on the NAND device

In fact RFS wouldn't be bad at all, if they only implemented buffering...
[Galaxy S I9000 Android Development] xan:
You missed your point with defragmentation. Preventing fragmentation is not defragmenting. Yes, ext4 papers specify 'online defragmentation' procedure, but AFAIK it is not implemented yet (and I doubt it will, given the time its in the drawer - since 2007)

Anyway, it has been taken that fragmentation is not an issue here, also when it comes to its relation with increasing wear. (which was supposed by me, but taken as not existent)

About performance comparisons you made, can you back this pros and cons with any benchmark results?

You may be terribly wrong, especially when it comes to writing. If go to list of features of ext4, there are some nifty writing technique tweaks, such as delayed allocation.

And what about extents allowing faster boot, because I dont get the relation?
[Galaxy S I9000 Android Development] deteego:
Arguably on NAND based devices (and on SSD's), you want to prevent defragmentation as much as possible, because fragmentation is not an issue (at all), and any attempt to prevent fragmentation (such as extents) creates an overhead for a benefit that isn't even utilized, and any attempt of defragmentation just wastes precious write counts

I don't have benchmarks specifically with what we are dealing with (Ext2/Ext4 on mobile devices), but as mentioned previously, the extra features that Ext4 does add do create additional overhead and do have a performance impact. These negatives are (negligible) on magnetic disks and end up giving massive performance benefits (such as defragmentation, which is a massive issue on magnetic disks), but on a NAND based disk the (positives) that they provide aren't even utilized, and so you are just gimping your mobile for no reason

Even if you do benchmarks and the performance impact only happens to be trivial on Ext4, there simply isn't a reason to use it over Ext2
[Galaxy S I9000 Android Development] xan:
There is one significant point you didnt count - journaling. Do you find it pointless on SGS?
[Galaxy S I9000 Android Development] deteego:
Thats a debate for Ext3, not Ext4. Ext4 implements extents (among other things) over Ext3, which implemented journaling

If we are arguing over journaling, then we should be arguing over ext3, and since I have been using the ext2 for the filesystems on my SGS and haven't noticed any data corruption whatsoever even with multiple shutdowns (with battery being taken out while phone is on).

Of course this is anecdotal evidence, although I suspect (and the reason why filesystems such as RFS/YAFFS2/ext2 have been used for mobiles for so long) its much less of an issue.
[Galaxy S I9000 Android Development] RyanZA:
Gotta say after months of using EXT2, with battery pulls and other funny stuff (hey, I've been messing around with root exploits here) - journaling is simply overrated in this device. I (and many others) have also been running loopback EXT2 which is even less safe than native EXT2, also without issue.

It is not required. Feel free to use it if you're really a fan though. I won't. :)
[Galaxy S I9000 Android Development] deteego:
Yeah exactly, which means the reasons for using both ext3/ext4 are not really tangible

Arguably RFS with proper file buffering would be better then ext2, because RFS has a much better startup time then ext2 (like when your mobiles screen goes to black because it goes to 'sleep', and then you wake it up by pressing the power button)

Unfortunately its propriety, and not having that file buffering is whats killing the Samsung devices
[Galaxy S I9000 Android Development] xan:
Well, topic says ext2,3,4 so... I believe journaling is viable for SGS. I cant find any filesystems benchmark comparisons on SSDs, that would stop speculation and let the facts speak.

About ext2 loopdevices, for me they didnt last longer than one week.
[Galaxy S I9000 Android Development] deteego:
Well SSD's are just a block of NANDS so.......

If you don't believe in regards to SSD's, just do some googling on recommended file systems/options to be used on SSD's. For ext4 for example, its recommended that you disable extents (among other things) for reasons stated earlier. Journaling is another story, desktop systems appear to be much more impacted by data corruption then mobiles are

I mean if you wan't a comparison for SSD's, you can look here for example

https://wiki.archlinux.org/index.php/Solid_State_Drives

Typically quoted here

This depends on how exactly the NAND on the SGS is implemented if journaling is an actual issue
[Galaxy S I9000 Android Development] RyanZA:
Not true, waking up with either RFS or EXT2/4 are all 100% instant. Not any room for improvement here. (Android 'sleep' isn't a proper power off/S1/S3, it doesn't unmount the drives or anything like that, it only uses the hardware low power mode settings on the numerous devices. )

EXT2 really is the best choice (for me, anyway, you do what you want guys!), as it gives the best performance, lowest number of writes, lowest CPU usage, etc.

Some people see the higher version number of EXT4, and want to use benchmarks and such from PCs/servers using spinning media, while disregarding the enormous differences involved. EXT4 is mostly just a tweaked EXT2 anyway - and the tweaks are all aimed at 1) spinning media 2) high processing power.
[Galaxy S I9000 Android Development] xan:
That is exactly what I've been implying.
[Galaxy S I9000 Android Development] deteego:
Hmm I just read through here, http://xdaforums.com/showthread.php?t=750805&page=2

Something was mentioned in there about RFS being better on wakeup, need to have a better look (might have been something else)

Yeah agreed, you use whats best for what we are dealing with.

In all honesty we should probably convince all those lag fixers creators to probably make do with ext2 (or there should be an option for ext2 mounting instead of the ext4 all thats used in ULK project)

Also all these options that Ext4 implement require extra CPU power, something that actually is an issue on mobile devices (in regards to both battery life and bottlenecks)
[Galaxy S I9000 Android Development] RyanZA:
Unrelated to filesystem. That is a bug in the power management in the 2.1 Samsung GPL sources. It's fixed in the 2.2 GPL sources.

As always, the filesystem is blamed (incorrectly) for anything that people can think of. ;)

Literally impossible! I have been insulted and threatened for even suggesting the idea! (Check out my talks with supercurio in asking for a EXT2 option)

I would highly recommend that you don't try to get others to use EXT2 if you value your sanity. EXT4 is something of a cult following, I think. :)

Not that there's anything wrong with EXT4+journaling+100 other options - all of my desktops use it. ;)
[Galaxy S I9000 Android Development] xan:
More choice > less choice.

sztypy's ULK doent support native ext2(yet, and if it will I'll definitely give it a try!), because he stated that detection of ext2 vs ext4 is "not that easy" (may not be actual words, but I remember it like that)

There are some people here who are bond to their statements beyond any discussion. I think discussion is nothing bad, there is always room for it.

Its IT, there are always 99 ways to do something, and everyone can have their own. (And that's why I love it!)
[Galaxy S I9000 Android Development] RyanZA:
Not true. Detection of EXT2 vs EXT4 is very easy.

if [ "$((compat&=EXT3_FEATURE_COMPAT_HAS_JOURNAL))" == "0" ]; then
# replace rfs with ext2
echo "ext2"
else
# ext3 or ext4
ro_compat=0x"`dd if=$part bs=$((RO_COMPAT)) skip=1 count=1 2>/dev/null | dd bs=1 count=4 2>/dev/null | od -vx | awk '{print $3$2}'`"
incompat=0x"`dd if=$part bs=$((INCOMPAT)) skip=1 count=1 2>/dev/null | dd bs=1 count=4 2>/dev/null | od -vx | awk '{print $3$2}'`"
if [ "$((ro_compat&=EXT4_FEATURE_RO_COMPAT_HUGE_FILE))" != "0" -o \
"$((ro_compat&=EXT4_FEATURE_RO_COMPAT_GDT_CSUM))" != "0" -o \
"$((ro_compat&=EXT4_FEATURE_RO_COMPAT_DIR_NLINK))" != "0" -o \
"$((ro_compat&=EXT4_FEATURE_RO_COMPAT_EXTRA_ISIZE))" != "0" -o \
"$((incompat&=EXT4_FEATURE_INCOMPAT_64BIT))" != "0" -o \
"$((incompat&=EXT4_FEATURE_INCOMPAT_MMP))" != "0" ]; then
# replace rfs with ext4
echo "ext4"
else
# replace rfs with ext3
echo "ext3"
fi
fi

Of course a better method is to just use fstab - check out https://github.com/z4/z4mod
[Galaxy S I9000 Android Development] xan:
I was just referring to what he said. I believe when 0.4 hits the public it will feature ext2 scheme.