yashsingh1/xdaforums huggingface dataset:
[Desire Android Development] melethron:
quote snipped - rest is below

Or maybe we used different setups. On my test i had the ext4 partition with journal (ordered). Ext4 without journal was faster than reiserfs with journal. But i didnt test unjournaled reiserfs (but i will try this also).

Here are my results (done with TAP Benchmark - 5 runs in a row and mean value of all db writes)

EXT4

(rw,nosuid,nodev,noatime,nodiratime,barrier=1,data=ordered):

db writes: 34

EXT4 unjournaled

(rw,nosuid,nodev,noatime,nodiratime,commit=19,barrier=0,nobh)

db writes: 57

BTRFS

(rw,nosuid,nodev,noatime,nodiratime,ssd_spread,noacl)

db writes: 16,2

ReiserFS

(rw,nosuid,nodev,noatime,nodiratime)

db writes: 40,2

Nand

(nosuid,nodev,noatime,nodiratime, + some other default stuff):

db writes: 54

But as i said i didn measure unjournaled reiserfs yet.

Hmm i cloned coutts99 git source (yesterday). In the all config files from the source he has:

CONFIG_SWAP=y

But i didnt check the config file of the zipped kernels. I would compile myself a kernel with reiserfs support but i dont know how i add it (compiling works but im to much a noob to add stuff to a kernel :D - much to learn for me - in never coded and im using linux for just one year now). At the moment i use the test kernel (from the btrfs thread) of neophyte and i pulled the proc/config .gz and swap is also enabled there so i give it a try with that kernel.

Did you use a loop file for the swap or did you make a realy swap partition for it? And what size did you use for the swap.

Hehe, I checked your v4 of course. Mount to /system/data_internal wasn't the problem with s-off i had. But i had "mkdir /system/data_internal" in my script and that was the problem. Now i simply create the folder when the zip is flashed (and permission in update-script). This way it works. Also about the data/property... this is already this way in cyanogens app2sd.... i should have guessed that this causes issues .....

About loop. I find it alright if it gets unmounted on shutdowns (but i still make it optional as a addon). For the test i did restart with snq power menu and if i used this the system wasnt shutdown proberly (partition was unclean and every second e2fsck reported errors). On normal switching off it worked. A bit of risk on a powerloss is ok but if switching off is like a powerloss it is not safe enough (for me at least ;) ).

Also note that i tried to make loop as you do in v4 (busybox mount -o loop, ...) and it didnt work. Loop didnt show if checked with mount and the sqlite score was the same as without loop. So i did:

busybox losetup /dev/block/loop0 /dev/block/mmcblk0p2;
busybox mount -o sync,noatime,nodiratime,nosuid,nodev /dev/block/loop0 /data

And this way it worked. You should check if you have the same issue. Im not sure why it didnt work for me this way......
[Desire Android Development] melethron:
On my HD rom i use it for the system apps. So i can split up the 370 mb of HD system between system internal and data internal ( ~220 mb system - ~140 mb data). Or it can be used for dalvik cache.... (**** i wanted to do that in my data2whatever script .... lol .... i copied the script from my rom for that and forgot that i dont need the space in internal data for system apps ^^)
[Desire Android Development] sibere:
Guys I would like to remind you what I stated in OP about SWAP: It causes (at least for me) quite a lot of FCs.

Apparently when the systems tries to get a swapped app, something goes wrong and leads to a FC.

For the swap no loop, just a partition or a swap file (see OP for that). You can use the nand for that (I/O sharing) :)
[Desire Android Development] melethron:
I saw that in op and your issues with fc was the reason i didn't try it yet. But it seems to work for ownhere .... so ill give it a try (later). I also thought about using nand (userdata) for it. Atm i have the HD roms system apps there to share i/o but for swap it should be better (if a system app is already running and in swap its less i/o on sd and so less lags). It should also prevent issues that way with unmount if it is not the external partition (you know ... mounting stuff on sd-ext ;) ).
[Desire Android Development] danarama:
Would dalvik on nand speed up boot?
[Desire Android Development] melethron:
The first boot yes since the dalvik VM does much stuff then. And maybe on the second. But in general it shouldnt speed up boot much. It makes app start faster over time since the dalvik VM optimize stuff. Thats also the reason the phone gets smoother after some time.
[Desire Android Development] ownhere:
my test use default options of FS, all enable journal.

reiserfs read performance is low, So I tend to continue to use the ext4.

swap partition, because this is better than swap file.

[/quote]

v4 is out of date:) It is before I pointed loop-device need losetup:)

'losetup' is my reply at another post to you.:D

my script V6 do follow things:

1. put /system/app to mtdblk0p5

2. mount mmcblk0p2 to /data

3. symlink /data/system,/data/property,/data/misc, and /data/data/com.android.providers.* to mtdblk0p5, The 'com.android.providers.*' supply sms/mms/contacts/accounts, so symlink it back to mtd improve safety.

4. mount mmcblk0p3 to /data/data, ext4, loop-device, journal enabled, data-ordered.

5. mkswap mmcblk0p4, swapon it.

6. /data/app,/data/dalvik-cache stay in /data(mmcblk0p2)

With upper things, my DHD port runs smooth, and no battery drains.
[Desire Android Development] sibere:
Well...

I'd say now that v6 has become data2sd2nand ;)

Merry Xmas all !
[Desire Android Development] Droidzone:
Thanks, iinga. I was the one who was wondering.. but for me just replacing the one file seems to work fine on the a2sd version itself.
[Desire Android Development] melethron:
splitted the quote up.

In the test i made i found that read isn't by far as important as write for smoothness. Also this google dev says that write is what can cause lags.:

http://android-developers.blogspot.com/2010/12/new-gingerbread-api-strictmode.html

I tried swap on internal. Because internal is no block device i needed to create a loopfile there and on this "fake" device i made a swapfile. But this is not good. Also because the double I/O for the bus (as you found out). Flush to loopdevice -> flush to to the mtd.

I try i a normal swapfile now. The reason is that it is easier to setup and can be done with a script. On a HDD a partition would be better because a swapfile would lead to fragmentation. But this doesnt matter on a sd-card so i try this. The only issue that this could make is that a swapfile could cause that /data doesn't get unmounted. Because if the swap is still used on a a shutdown the "umount /data" will cause a "device or resource busy". But i test this.

I thought you told sibere something about froyo app2sd when you talked about losetup. But if it works now its fine.

About 1: Thats the same i do on my ROM. I compared dalvik-cache on mtdblock5 with system apps on mtdblock5 and i found that system-apps is better to spread i/o.

About 3: system, misc and property is the same for me. i take a look at provider stuff. Maybe i do the same. thanks for the tip :) .

About 4: Im still not happy with this. If /data is a loop device with sync it gets unmounted on shutdown but /data/data on another partition doesn't get unmounted (afaik). I dont like it if switching off the phone is like a powerloss....

about 6: same for me :)
[Desire Android Development] ownhere:
Good resource, I found that Google engineers are to do battle with the lag, we are not alone, huh, huh

I think swapfile just like loopfile, system need more cpu-time/disk-io to handle app requsts. So swapfile will lead to battery drain. Yes, swappartition no "device or resource bussy" issue.So you should try it.:)

system/app on mtdblock5 have better experience than dalvik-cache on it, yes, I found it,too. So swap is more important to reduce apk(resources) reload time.

about 4: My statement is wrong,hehe. In my real script, the /data/data is symlink to /mnt/asec/extdata(mmcblk0p3), not mount direct to /data/data/,

yes, it is for unmount safety. :)
[Desire Android Development] melethron:
Hey,

Well best way to fix lag seems to be better coding. Fewer disk access is better than a faster disk access. So google devs do a better work then we do :D

Hope htc will learn from it. I think its different for the programmers to code for android (for example how often they call commit () ).

I tried loopfile on internal with swap on it now and swapfile on sd-ext. Internal was better so far, but i need to try real partition first.

Im also testing some different values of vm.swapiness ("sysctl -w vm.swappiness=NUMBER"). Higher swappiness makes new app start faster (because system keeps more ram available) but slows down already running apps (because they are on swap instead of RAM). Higher values (60 or 80 instead of default 40) reduces smoothness because swap is to much used. I think a lower value would be better so swap is only used when it is really needed.

About mnt/asec/extdata: Did you check with "tune2fs -l" if it is "clean" after shutdown. Im not sure if android does "umount -a" or just "umount /data" when it shutdowns. When i tested your v4 script i made the test with "snq powermenu" and as i say in my data2whatever thread this has issues cause it doesnt do unmount at all. So im not sure if it gets unmounted.

I read your v6 later. I first wanna setup real swap partition and make some changes to my rom.
[Desire Android Development] Droidzone:
Finally succeeded in creating a Stock Rooted Rom, Droid Alpha :) based on the latest 2.29 RUU. However I'm still new to Cooking Roms and I dont know where (which script) to add the commands which we perform initially while installing DATA2SD. I've added the necessary files and this Rom works if the partitioning and initial setup has already been done for another Rom. If the initial config steps can be added, I could release this as a DATA2SD Rom out of the box. Dont think there are any out there..

Could use some help..

Can I add these to the update_script after all the initial commands are done?

A list of commands from those in the OP and the script(s) in which they should go, would be great.. :) For the time being I'll stick to what Sibere originally intended, with two FAT32 partitions.
[Desire Android Development] hajen78:
I have recived a great offer on a class10 sd-card, but i have a class4 in my phone now, so will i gain much performance by getting the class10 card and put it in my phone instead of my class4 ?

Ps, of cause with data2sd i mean ;)
[Desire Android Development] melethron:
Not necessary. The smoothness of data2sd depends on stuff like the controller in the card. In general a more expensive card should have a better controller also but this isn't always the case.
[Desire Android Development] Droidzone:
Uh oh...I must swallow my words..melethron would know better. :)

Yet, for the four different cards that I have, I've noticed a definite improvement in performance in direct proportion to increase in Card Class.
[Desire Android Development] sibere:
I haven't tried the 2nd partition formatted to ext2/4 mounted via a loop device yet.

Obviously for better performance, partition 1 and 2 alignment are essential.

I did that with fdisk instead of gparted. Fdisk is not really user friendly but at least you know exactly what is done, specially with expert mode!

When I have a few hours spare time, I'll try ext4 with stride=32 for 128K data alignment on top of the partition alignment, no journal and loop device. Yes you can use RAID options on a single device! I'll do the symlinks also for the famous "parameters not saved at boot" bug :)
[Desire Android Development] Droidzone:
@Sibere..

Could you tell me how to convert these commands:

into commands necessary for the META-INF\com\google\android\update-script?

To release a Rom with built in DATA2SD support (So that the only thing users need to do before flashing the Rom, is partitioning their sdcard into 2 Fat32 partns), I'm guessing that these commands should be converted into a format necessary for this script.

The various files that DATA2SD uses, have been integrated into the rom.
[Desire Android Development] hajen78:
But if they both are kingston ? The nomal price is x2 for the class10
[Desire Android Development] qbus00:
DO NOT use Class10 cards for Data2SD.

Basically those cards are tweaked for sequence writing. They are really fast then (mine is able to write 40 raws in my camera's [Nikon D60] burst mode without any hiccups).

However they are extremally slow with random access. Even slower then class2 cards(!!)) because of the hacks mentioned above. My device was totally lagged when I moved DATA to class10.

On the other side class10 cards are perfect for App2SD (with dalvik cache) - I went into this approach and it's really, really fast (in daily usage I can easily spot the difference in phone's responsivness comparing to DATA on internal memory).

If you want to have Data2SD then use good class 4 or class 6 card and stay away from class10.
[Desire Android Development] Droidzone:
Can anyone else confirm? I've heard from a couple or more of people with Class 10, who're getting good performance.
[Desire Android Development] qbus00:
I have this problem with Kingmax Class 10 16gb card. Maybe other vendors use different controller and have no such problems but buying class10 card is a risky move IMHO. I tested this on ext2/3/4 filesystems. Maybe other FS have better performance but I haven't tested this.
[Desire Android Development] danarama:
16GB class 10 Kingston, sync on, ext 2, 2221 quadrant, no spike lags, stable as hell. Much better than my class 4 sandisk with sync off. About 250 less in quadrant but who cares about that, when data integrity is increased 10 fold
[Desire Android Development] Droidzone:
Good..All I care about are lags..
[Desire Android Development] danarama:
Ditto. Never really had any myself. Yours were down to sync on and low class card
[Desire Android Development] Droidzone:
Indeed. But I was about to order a Class 10, and this post made me wonder
[Desire Android Development] danarama:
I guess if you could borrow one from someone to test om your device first, that would be best. All devices are not equal so what's hood for one may not be for another
[Desire Android Development] hajen78:
Thanks

Well i have deleted my order on a class10 sd-card from Kingston, when i am not sure to get better performance i dont wont to by it.
[Desire Android Development] danarama:
If you use sync on, you will get better performance. If you currently can't use sync on, you will be able to with your class 10.

You will not however, be able to increase your fake sync off quadrant scores.
[Desire Android Development] wollyka:
I CONFIRM this. I have the same Kingmax Class 10 16Gb. Data2SD and Data2whatever were extremely laggy on this card. My old Nokia Class 6 card WAS MUCH FASTER with Data2sd.

The Kingmax is fast in windows (around 15 Mb/s),in any digital camera or the normal app2sd but slow when used with data2sd. It lags to the extent that if someone called me it needs some time before the phone starts ringing with the caller ID! I tried to call myself and sometimes it takes as much as 3 rings before the phone starts ringing!! NOT good for a phone. So i am back now to app2sd or i will have to go back to my old Nokia class 6 which only lagged when installing apps (which is normal!)
[Desire Android Development] hajen78:
I have gotten very confused, I am runing data2sd fat32 with the ext4 script iinga has made, is this still some of the best or what, it seems very stable on my phone, but if there is something better i would like to know ;)

I have treid data2whatever both with ext3 and ext4, but both where hanging on I/O writes for 3-4 minuttes and my score got very much lower 400+ points

I would like to hear what others think is the best ? specialy sibree
[Desire Android Development] sibere:
ATM I'm trying an EXT2 partition mounted via a loop device and even though the IO score is not better this way, I have to say that the phone is smoother that way. No big spike lags when writing big amounts of data on /data.

I've updated my script and I will update it again later if everything is OK to correct the "settings unsaved bug"
[Desire Android Development] Droidzone:
that's great. doing away with the re-formatting would be very nice. The only thing left for data2sd is a complete flashable zip. Iinga's is good but only for re-installation. Initial steps are manual. If a simple flashable zip could be scripted, it could be integrated in any rom. Right now, I've found problems only with hd roms. Others seem to work very well.

About the lag- I've never had serious lags with this even with mt kingston class 2. Only while installation of apps and while playing vids. Maybe the read ahead buffer size could fix it.

Edit: @Sibere, there seems to be a repetition in the last section where you mention ext2..

Going to try Data2SD2ext2 :) today with the new Oxygen Gingerbread..

Via T3 phone of the year 2010 - HTC Desire
[Desire Android Development] Droidzone:
Data2ext2

Ok, tried Sibere's updated commands for directly moving the /data to ext2, on the latest Oxygen Gingerbread RC4..and worked like a dream. It's so straightforward.

@Iinga, would it be possible to make a flasheable script for this?

@Sibere, I got a bit confused while jogging between Sections 5 and 7 :) Going to section 7, and coming back, one of the commands was missed.

Anyway, these are the list of Sibere's commands to be executed in order (as he has posted in the OP. I've just seperated them from the Data2sd part) to get Data2ext2..

adb shell
mount /system
rm /system/etc/init.d/10apps2sd
exit
adb push c:\busybox-armv6l /system/xbin/busybox
adb push c:\e2fsck_recvy /system/bin/
adb push c:\01data /system/etc/init.d/01data
adb shell
chmod 755 /system/bin/e2fsck_recvy
chown 0.2000 /system/bin/e2fsck_recvy
mount /data
mount /sdcard
chmod 755 /system/xbin/busybox
chown 0.0 /system/xbin/busybox
chmod 755 /system/etc/init.d/01data
chown 0.0 /system/etc/init.d/01data
/system/xbin/busybox mke2fs -m 0 -F -L userdata /dev/block/mmcblk0p2
cd /
mkdir /sd
system/xbin/busybox mount -t ext2 /dev/block/mmcblk0p2 /sd
cd /data
cp -a * /sd
cd /
sync
ls -l /sd
/system/xbin/busybox umount /sd

EDIT: There is a problem. The Rom restarts every time something is installed.

:confused:

Not random either. Happens every time something is installed.
[Desire Android Development] iroironz:
Hey Sibere,

Am a big fan of your work. Have been using data2sd for a while with no issues. have also been experimenting with losetup and an ext2 partition directly. Got it running no problem but am getting some strange random reboots. Moved back to the original data2sd method and random reboots were resolved.

Have you had any random reboots using your updated script?

I originally used melethron's method and noticed that he doesn't have the loop option in his mount (but you do in your 01data-2 script). Thought that might have been the prob but got some random reboots using the loop option as well.

What kernel, radio and busybox versions are you using?

Thanks again for your contributions.
[Desire Android Development] melethron:
There is a flashable script for that: data2whatever + life-on-the-edge does exactly that ;) . With the mod it removes the journal (that makes ext3 to ext2) it does "losetup" and then mounts the loop device.
[Desire Android Development] melethron:
loop mount option is redundant if used on loop device. This is set by default if /dev/loop is mounted.
[Desire Android Development] Droidzone:
Tried your script (without life on the edge) with oxygen and a Ext2 of 1gb. It showed up correctly on 1st boot. But on the very next reboot, that dropped down to 100 mb.

And there does seem to be differences in the way both the scripts are handled. With sibere's second method, oxygen rebooted every time I installed something. But space showed up correctly. With yours, the changes seem to disappear on second reboot. Weird.

Via T3 phone of the year 2010 - HTC Desire

Update: It seems to be a problem with Oxygen, rather than either of your methods. I tried Sibere's new ext2 script w/o any problems on my new Rom. Perfect. Now I can do Nandroid backups like previously! :) And for the record, melethron's latest script too worked flawlessly for me, though eventually I did get a reboot (I think caused by Titanium) followed by a loop which caused me to uninstall it.
[Desire Android Development] sibere:
There is no really "best" methods. They actually all use the same idea from the SGS: ext2/3/4 partition on a SD, mounted most of the time via a loop device to smooth things.

The only difference is the "add-ons" or options you have with the other scripts.

Mine is very simple and basic. It's a strength and a weakness at the same time :)
[Desire Android Development] sibere:
Hum I'm confused too ;) Yes with all the updates I made, it's getting a bit messy now ...

I should take some time to re-arrange the all OP to avoid jumping between sections.

Thanks for rewriting the command list. This will surely be of a great help for other users.

Hum , never happened to me. I just did a tar backup before moving to the new script. Everything was fine. I've installed a few apps since and no reboots at all. :/
[Desire Android Development] sibere:
Strangely you're not the 2nd to report that issue.

I have no problem here. I use stock rooted sense htc rom 2.09.405.8 with busybox 1.17.2 and my own kernel based on richardtrip's kernel

The strange thing is that I use loop on the original script, so the issue is not here.
[Desire Android Development] melethron:
Thats not overall true. There were lagfixes with loop on SGS, but most devs changed to normal mount because of issues with loop. Most people use the voodoo lagfix and they use normal ext4.

But there is a difference between SGS and Desire. The SGS has a BUILT-IN sd for data (and a pretty good one btw). So all user have the same SD ... makes it easier for the devs to make a good solution that work for all.
[Desire Android Development] melethron:
@sibere:

hey i want to compile a kernel with cgroup scheduler. I only need those in conf:

CONFIG_GROUP_SCHED=y
CONFIG_EXPERIMENTAL=y[/CODE

or do need to watch something else?
[Desire Android Development] danarama:
Sibere,

Droidzone said you managed to get the sd-ext nandroid size down with your new bit of script.

Which bit is that please? I can't find any reference.

If I already have Data2SD implemented, on the first revision of the instructions, how easy would it be to add?
[Desire Android Development] Droidzone:
Since it uses ext2, rather than Fat32, I did it from scratch by formatting my Fat32 to ext2.

How I did it:

Formatted fat32 as ext2.

Flashed a NonA2SD Rom

Booted into Android once.

Rebooted to Recovery.

Ran the following code.

adb shell
mount /system
rm /system/etc/init.d/10apps2sd
exit
adb push c:\busybox-armv6l /system/xbin/busybox
adb push c:\e2fsck_recvy /system/bin/
adb push c:\01data /system/etc/init.d/01data
adb shell
chmod 755 /system/bin/e2fsck_recvy
chown 0.2000 /system/bin/e2fsck_recvy
mount /data
mount /sdcard
chmod 755 /system/xbin/busybox
chown 0.0 /system/xbin/busybox
chmod 755 /system/etc/init.d/01data
chown 0.0 /system/etc/init.d/01data
/system/xbin/busybox mke2fs -m 0 -F -L userdata /dev/block/mmcblk0p2
cd /
mkdir /sd
system/xbin/busybox mount -t ext2 /dev/block/mmcblk0p2 /sd
cd /data
cp -a * /sd
cd /
sync
ls -l /sd
/system/xbin/busybox umount /sd

Note that the 01data we use now is the new file 01data-2.txt.

That's it. It's as easy as the old method.

I ported my apps with Titanium. The latest version seems very adept at transferring the data smoothly.

Sibere has described it in his Section 5 through 7. It's just a bit hidden in the post. :)
[Desire Android Development] danarama:
Oh right. Quite happy with the EXT2 file loop device. What are the other benefits of using an actual EXT partition?
[Desire Android Development] Droidzone:
Guess Sibere should answer the techie details. :)

Believe it or not, I have noticed a definite improvement in lags while installation.
[Desire Android Development] melethron:
Quoting myself: "fat sucks! loopfile sucks!" :D:D:D