yashsingh1/xdaforums huggingface dataset:
[Desire Android Development] Droidzone:
And I would politely disagree :D
[Desire Android Development] danarama:
All works well for me though ;)
[Desire Android Development] melethron:
Lol, still you move from fat+ext loopfile to a direct loop mount .....
[Desire Android Development] Droidzone:
Nope.. I just moved back, as of 5 mins ago. Got a boot loop with no apparent reason! :D
[Desire Android Development] melethron:
Bootloop has always a reason if you didn't check with adb what the reason is you shouldn't be so sure that its because of the "new" script.
[Desire Android Development] Droidzone:
Nah..I didnt check the logcat. But I didnt do a hot restart, or anything which usually loops the device. Just shut it down normally. And the next restart looped. Anyway, the log is now overwritten. :)

And I've never had a loop on the Fat method.

Edit:

You have a point. I tried a plain install of the Leedroid NonA2SD version and it hung at boot. Even before setting up data2sd.. I'm thinking that my card got corrupted. Recreating the whole filesystem and trying again.

Edit:

The lag on my Class 2 was too much for me. Switched temporarily to MTD hack till my new card arrives. :)
[Desire Android Development] nodiaque:
Hello everyone,

I just received my new Lexar 16gb class 6 card and tried all 3 way of mounting (2 fat32 with 1 ext2 file, 1 fat 32 -1 ext2 mounted directly, 1 fat32 - 1 ext2 mounted from loop; all with sync, 14gb/2gb partition scheme).

I'm with a desire, running leedroid 2.2f.

I've made a nandroid backup so everytime I boot, the ext2 file/partition have the exact same file that I restore, and the sdcard is nearly empty (only basic system file needed).

I've done partition alignement for 224 head, 56 sectors as per alignment guide, but the ext2 partition aren't aligned so it doesn't change anything.

First Fat32 partition using 32K cluster.

Second Fat32 using 64k cluster, since it only one big file. Alignment at sector 384.

My result are as follow:

2 fat32 with ext2 file mounted from a loop:

- Good as usual, better performance then my previous card

- Quadrant score: 1495

1 fat32 + 1 ext2 mounted directly:

- Very bad performance, I've waited more then 5 minutes for the phone to boot and froyo wasn't even ready to be used. Worst I had.

- Quadrant score: not even tested

1 fat32 + 1 ext2 mounted via a loop:

- very good performance

- Quadrant score: 1556

Now, I haven't used them for a lot of time yet so I don't have anything to say 'bout lag spike or other, just that for me, mounted directly was the worst of all.

I formated everything either into window or virtualbox in a linux environnement. I don't know why but fdisk from busybox and mkfs.vfat doesn't seems to do anything when I changed either the sector/header or cluster size.

Overall: it seems that mounted directly is a really bad idea (in my case) and that mounted from a loop to a ext2 partition is the way to go, giving 'bout 50 more points in quadrant score.

Since I had my phone in fat32 scheme before, I'll try with ext2 partition and see if I get a lot of lag

edit - update

In the last 2 hours, my phone reboot itself 3 times and the last time, it failed to boot and stay shut. When I boot it, it didn't mount /data (didn't run the script or the script didn't worked). I'm reverting to good old fat32 method with mirror mount; unless it's a rom issue (I would know it cause it shoul do the samething).
[Desire Android Development] sibere:
I had the same issue. Random reboots with latest sense rom and ext2+loop. I also had reboots when receiving a text when the phone was on standby for a while. I have reverted to Fat32+loopfile to see if I still have this problem.
[Desire Android Development] melethron:
What scheduler? Cfq?

Do you get cpu subsystem errors in logcat? Some cgroup settings cause much errors on 1.72 base. Check coutts thread about that.
[Desire Android Development] iroironz:
Hey sibere,

+1 for another person who moved back to ext2 file due to random reboots. Tried a whole bunch of stuff including different kernels, radios and busybox versions - always got reboots. Noticed could reliably cause the reboots by going somewhere with bad reception which would cause it to switch from 3g to 2g - bam would cause a reboot.

No problems at all since switching back to the good old ext2 file method.
[Desire Android Development] melethron:
Same question to you. Did you change the scheduler (or use my data2whatever script)? I simply cant reproduce this no matter how hard i try.... seems like kingston make real good microSD.
[Desire Android Development] iroironz:
Hey melethron,

Nah, didn't change the scheduler at all. although was using ext2 partition (looks like others were as well). Well retest with ext4 as you recommend over the weekend.

Jay.

PS: I have a class 4 kingston card.
[Desire Android Development] sibere:
Haven't tried different kernels and busybox versions. The problem might not be there then. I left the scheduler to CFQ and the phone was rebooting lately only when I was receiving a text when the phone was on standby for a while. That is really weird. Since I reverted back to loopfile, everything is fine.

Reboots are bad mainly because the ext2 partition isn't unmounted nicely and I ended up after a few reboots with some files in lost+founds, which is really really bad!

Fortunately I had backups :)

I'm only backing up /data using a gziped tar file. actually as long as you don't make any changes to you system, there is no need to do another full nandroid backup.

I keep one on my SD and I just redo a full /data tar file backup every now and then.

ATM I'm professionally really busy and I don't have time for trouble shooting this issue. I'm back and happy with my old settings. The phone is as smooth (or as laggy when installing apps) as it was with ext2+loop (both CFQ).

PS: I have a class 6 Transcend card.
[Desire Android Development] sibere:
Yep CFQ indeed.

I'm away from home with only my netbook and didn't have time to look through logcat.

I just restored everything to my old settings and since then, everything is back to normal.

Then only change I have now is the latest HTC sense rom rooted and init.d enabled (not with my little hack script).

I stick with cgroups tho because my phone is way much smoother and interactive with cgroups than BFS or anything else.

Before Cgroups My phone was laggy when it was coming from standby with cpu@128Mhz. Now It's just perfectly smooth.
[Desire Android Development] melethron:
Im pretty sure it is really direct loop mount. Alex-V had this issue and did some tests. He got reboots in almost all possible setups of a direct loop mount. So this should be the issue indeed. So i think i change life-on-the-edge to a loop file mount.

I still dont like loop btw and will stick with reiserfs. Alex-V also tested it because normal ext was to slow compared to loop and he really likes it too. I also see if i get IEF to make a crw recovery with support for reiserfs.

About cgroup. Almost every kernel has enabled it and this should be default. I didnt check the config.gz of the default kernel but i think it should be also enabled there. There is only an issue with cgroups on coutts kernel, but im sure hell fix it.
[Desire Android Development] sibere:
I enabled cgroups in my build of RichardTrip kernel b/c apparently it wasn't by default. Maybe I did something wrong when I pulled config.gz out of his kernel tho ;)

I'm back to loopfile ATM. As you don't like FAT32, you can always mount an ext4 file on an ext2 FS :D

I've just added a link to dalvik-cache on mtdblock5, pretty much like you're doing in your script and I have a slightly better responsiveness so I leave it that way. Next step will be to add links to settings on mtdblock5 too. Maybe I'll use a bit of you code if you don't mind ;)

What could help to is to transfer all apps in /system/app (96Mb) to sd/nand and use that freed space for dalvik-cache and be able to get almost unlimited space! But S-OFF would be mandatory.

ATM I have about 160mb of apks in app and 46Mb of dex files. even with standard partitions, I still have a lot of space for more app before filling the nand with dex files :)
[Desire Android Development] melethron:
Feel free to use whatever you like... open source 4tw ;) . But note that for some users dalvik on internal may be too small. Some users with about 300-400mb of apps will get full dalvik cache (happened on alex-V rom which uses/used my whatever script). Even if you add 96 mb to this this may still be not enough. Since i use a hd-rom i need internal for other stuff anyway so i have dalvik on ext (i have /system/app on mtd5... this way i get the complete 400 mb of desireHD system on nand without compression/stripping down/system stuff on sd)

About cgroups. I thought richards kernel use bfs?! Did you change it to cfs then? I also wonder what rom you use... you have "/dev/cpuctl type cgroup" in "mount". Either i was blind or it wasnt there before i used the desire hd/z with 1.72 base.
[Desire Android Development] sibere:
I have the latest teppic rooted htc sense rom with richard kernel but build @home with cfs and cgroups and few other mods :)
[Desire Android Development] melethron:
Did you check "mount". Has it "/dev/cpuctl". Cgroup is useless if you dont set it up yourself or it is set up by the rom.

Apart from that. Check this: http://xdaforums.com/showthread.php?p=10427606

This is a far better idea to skip fsync directly in sqlite than to use loop for that. Same effect and no double flush like on loopfile and no random reboots like on direct loop mount.
[Desire Android Development] sibere:
This is the bit of code I have in init.rc

# Create cgroup mount point for cpu accounting
mkdir /acct
mount cgroup none /acct cpuacct
mkdir /acct/uid

My rom is fully standard HTC sense 2.29.405.5. Nothing added, nothing removed. It's only rooted and init.d is enabled.

As you can see, HTC uses cpuacct. I do have /dev/cpuctl though.

For groups CPU sched they also use User ID and not Control Groups but I have added control groups support in my kernel.. you know.. just in case ;)

Back to topic: I've started to read those very interesting articles: http://ldn.linuxfoundation.org/blog-entry/aligning-filesystems-ssd’s-erase-block-size and http://www.nuclex.org/blog/personal/80-aligning-an-ssd-on-linux

Tell me what you think about it. If I've got spare time, I'll try it on my 1024K aligned ext partition to see if I can get a bit of improvement from it.
[Desire Android Development] melethron:
I didnt mean that part of ramdisk. That part in the ramdisk should be useless (not sure) without the following (this is from desire z 1.72):

# Create cgroup mount points for process groups
mkdir /dev/cpuctl
mount cgroup none /dev/cpuctl cpu
chown system system /dev/cpuctl
chown system system /dev/cpuctl/tasks
chmod 0777 /dev/cpuctl/tasks
write /dev/cpuctl/cpu.shares 1024

mkdir /dev/cpuctl/fg_boost
chown system system /dev/cpuctl/fg_boost/tasks
chmod 0777 /dev/cpuctl/fg_boost/tasks
write /dev/cpuctl/fg_boost/cpu.shares 1024

mkdir /dev/cpuctl/bg_non_interactive
chown system system /dev/cpuctl/bg_non_interactive/tasks
chmod 0777 /dev/cpuctl/bg_non_interactive/tasks
# 5.0 %
write /dev/cpuctl/bg_non_interactive/cpu.shares 52

Dividing interactive tasks and non interactive tasks in groups is rather a nice idea. If i get this right the apps in background get a max of 5% cpu as long as they are in the bg_non_interactive group. I also think it wasnt there in froyo before 2.2.1. Having cgroups without setting them up is useless....

Also this is only for the groups. I have not even a slight idea how threads are added to groups (fg / bg) in the code (this isnt in ramdisk ;) ).

About the article. I took a short look at it and i think this is different case there. Every optimization for raid is only needed for a journal (so that journal is splitted right by the controller and that metadata doesnt land on one drive only) and you dont even need a journal ;) .Also i dont think that partition geometry is changed on microSD. The specification of microSD are rather old (its almost the same spec as for sd-card) and i think the focus of this article is some modern ssd and not a microSD or a usbstick. But i'll read the full article later. On my card i noticed the difference between alignment and unalignment (in bench and reallife) so i think the "old" partition geometry should match.
[Desire Android Development] sibere:
I'm not talking about ramdisk either but my actual /dev/cpuctl/ directory with android running.

I have this so it should be fine:

# pwd
pwd
/dev/cpuctl
# ls -l
ls -l
drwxr-xr-x root root 2011-01-08 18:20 bg_non_interactive
drwxr-xr-x root root 2011-01-08 18:20 fg_boost
-rw--w--w- root root 4 2011-01-08 18:20 cpu.shares

and this in /acct:

# cd /acct
cd /acct
# ls -l
ls -l
drwxr-xr-x root root 2011-01-08 18:20 uid
-r--r--r-- root root 0 2011-01-08 18:20 cpuacct.stat
-r--r--r-- root root 0 2011-01-08 18:20 cpuacct.usage_percpu
-rw-r--r-- root root 0 2011-01-08 18:20 cpuacct.usage
-rw-r--r-- root root 0 2011-01-08 18:20 release_agent
-rw-r--r-- root root 0 2011-01-08 18:20 notify_on_release
-r--r--r-- root root 0 2011-01-08 18:20 [b]cgroup.procs[/b]
-rw-r--r-- root root 0 2011-01-08 18:20 tasks

All is coming from the init.rc where the all system is booted/started/mounted. I had the same with my old rom so I'm not sure that it's coming with froyo 2.2.1.

Setting up cgroups without having it in the kernel is also useless ;) That's why I don't know why some devs have build a kernel without this option.
[Desire Android Development] melethron:
Well BFS has no proper cgroup support. It can't be enabled proberly (some stuff is in though). Cgroup is in for quite some time (was wrong on that). The reason i didnt see it yet is maybe that i only checked mount on a BFS kernel.

Well but i think BFS on a sense ROM is not good idea. Cyanogen didnt only change the kernel when he added BFS (some changes to dalvik also). All those errors in logcat cant be good.
[Desire Android Development] Orka82:
Wrong thread. ;)
[Desire Android Development] sibere:
We've been a bot off topic too here.

Here is my last tries:

I've formatted my ext partition (aligned to 1024k) to and ext4 file system with a few special options but no journal and mounted it straight without any loop but with a bunch of options too.

This way Quadrant cannot be fooled (score is about the same as the NAND here) and I've run tap bench to see the score for DB reads/writes.

I have also done some "speed tests" with dd and different block sizes.

Finally , I have installed a big app like google earth which must write a fair amount of data when it is installed on the ext partition.

The result is pretty good.

Tap Bench with 6 iterations of 1000ms each

DB writes: 44

DB read: 270

DB deletes: 290

Synced write speed (tested with dd) is up to 10mb/s on my class 6 card.

# sync ; time dd if=/dev/zero of=test bs=1048576 count=50 ; time sync ; time sync
sync ; time dd if=/dev/zero of=test bs=1048576 count=50 ; time sync ; time sync
50+0 records in
50+0 records out
52428800 bytes (50.0MB) copied, 4.910217 seconds, 10.2MB/s
real 0m 4.91s
user 0m 0.00s
sys 0m 0.92s
real 0m 0.27s
user 0m 0.00s
sys 0m 0.00s
real 0m 0.07s
user 0m 0.00s
sys 0m 0.00s

That's for 1Mb buffer size. after the Flush I have here 4.91s+0.27s for 50Mb. Close to 10Mb/s.

But what about a stress test of 1K buffer size?! Usually this is very bad for performance.

# sync ; time dd if=/dev/zero of=test bs=1024 count=40960 ; time sync ; time sync
sync ; time dd if=/dev/zero of=test bs=1024 count=40960 ; time sync ; time sync
40960+0 records in
40960+0 records out
41943040 bytes (40.0MB) copied, 4.073609 seconds, 9.8MB/s
real 0m 4.08s
user 0m 0.03s
sys 0m 0.72s
real 0m 4.86s
user 0m 0.00s
sys 0m 0.00s
real 0m 0.08s
user 0m 0.01s
sys 0m 0.00s

Hum well, after the flush we have about 4.08s + 4.86s for 40Mb of 1K block writes. That's still more than 4.5Mb/s !

for 4Kb writes I have a bit more than 6Mb/s.

Consequences?

A fast phone without loop.

But you're gonna ask me "and what about the biggest issue: LAGS"

Hum well with my previous settings (ext2 file on fat32 partition mounted over a loop device) when I was running the 1Kb block stress test, the phone was becoming laggy at some stage.

Now? NOTHING!!! I'm really amazed!!

I'm keeping the setting I have at the moment and if I'm really happy about it (no lags, no random reboots, etc) I'll update the OP.
[Desire Android Development] melethron:
I get almost the same sqlite inserts on unjournaled ext4 (slightly higher ....~56 in tap ... but i benched on a fresh system without anything setup). You should try reiserfs. It is almost as fast as unjournaled ext4 but it has a journal (+ journal checksums).
[Desire Android Development] iroironz:
Hi Sibere,

Sounds great, looking forward to seeing the updated OP and giving this a go!

Cheers,

Jay.
[Desire Android Development] sibere:
The thing is there is much more kernels with ext4 support than reiserfs. I could build one for myself but the other users may not have the right kernel support.

Did you try raw speed with dd also on reiserfs?

If I had the sqlite3 tweak on top of this and it brings what it promises, I think that it'll be close to perfect for my needs. :)
[Desire Android Development] ownhere:
I use reiserfs for 3days. But back to EXT4 finally.

some issue about reiserfs:

1.mount with unclean flag is too slow.

2.force fsck is very very very slow.

3.journal is not stable, after unexpect reboot, file content be filled with random data.

Another tip:

with loop(file or device), JFS is the fastest FS. I can get IO score:5000-7000 with JFS(enable journaling) in a loop device. But it is very very unstable. any unexpect reboot crashed whole system.So if Apple iOS use JFS and care about power loss, it is the best choice.
[Desire Android Development] sibere:
On top of that, recovery doesn't know anything about reiserFS/JFS. Thus you can't backup the ext partition. With the backward compatibility of ext2/3/4, the ext4 partition can be saved by recovery and restored if anything goes wrong.

I don't want anything else ATM than ext2/4 FS. I got rid of loop mounting now and it's very stable. Since I moved to direct ext4 mount, I never experienced reboots. I just had to upgrade my e2fsck to an ext4 compatible one. ;)
[Desire Android Development] Droidzone:
This post is only indirectly connected with DATA2SD..

Wanted to see your opinions on good Class 10 16 GB cards..I'm about to order one of these for DATA2SD..

Kingston is over my budget range..At least two people have commented that Kingmax doesnt work well...The two I'm considering getting are A-DATA and Memorette..Does anyone have any experience with those? Any other good and cheap alternatives?
[Desire Android Development] sibere:
I'm not sure that you will get much better performance with a class 10. You'll probably get the same with a class 6. Some incompatibilities have been reported with class10 cards on the Desire.

FYI mine is a 8GB class 6 Transcend card.
[Desire Android Development] Droidzone:
Even if I get the same performance with DATA2SD on Class 10 as with Class 6, I was attracted by the (almost) double data transfer speed while transferring huge files/backups from/to the PC and the Desire. Would the speed of a Class 10 with DATA2SD be lower than a Class 6 by any chance (b/c of said incompatibilities)?
[Desire Android Development] sibere:
UPDATE

FYI guys, I've completely updated and review the OP

No loop anymore but an optimized ext4 formatting/mounting. I do recommend to aling your partitions too! (see the very good comments and scripts of our fellow dev melethron for this).

On top of that for a better performance, I've added a patched version of libsqlite.so for sense 2.29.405.5 (FRF92) thnaks to the great work of our other fellow dev ownhere. :D
[Desire Android Development] melethron:
About 2: How did you force check? With "reiserfsck -S"?

About 3: same applies to ext4 with journal=writeback ;)
[Desire Android Development] sibere:
You should hopefully get better transfer rate on big files with a class10.

If you can afford the price difference, go for it! :)
[Desire Android Development] melethron:
Extracted recovery-clockwork-3.0.0.5-bravo.img ... replaced the kernel image with the zimage of coutts kernel .... repacked the image ... flashed with fastboot .... problem with recovery solved:

/dev/block/mmcblk0p2 on /sd-ext type reiserfs (rw,relatime)

Wipe is working. I test nandroid in a min .....
[Desire Android Development] sibere:
This is a bit beyhond standard users skills isn't it ? :)
[Desire Android Development] melethron:
flashing a recovery is beyond user skills??? I test it a bit more and make a zip for that ...
[Desire Android Development] sibere:
Come on dude! Don't make me say what I didn't.

You obviously know that I was talking about the entire process of hacking the recovery, don't you?

Anyway, ATM I remain with my settings and unless reiserFS brings a lot more, I'll stick with EXT4.

But this is a bit off topic ;)
[Desire Android Development] Fightspit:
I think I can do it the same with the ClorckWorkMod 2.5.0.7 which is the most stable recovery for me for an Amoled Desire.
[Desire Android Development] sibere:
Most stable for SLCD desires too :)

I never had any problems with this recovery.
[Desire Android Development] Fightspit:
Yes, my Desire *bricked* too many times with the most recent recoveries after I did a nandroid backup :/
[Desire Android Development] melethron:
If i "hacked" it then user don't need to do that anymore.

Result of ReiserFS may vary but there are quite a lot people that like it.
[Desire Android Development] alexcavaco:
Hello sibere,

I've got two questions, please note that I have the latest oficial stock Sense ROM (2.29.405.5), rooted with unrevoked. Kernel is the original.

I've seen your OP and you have removed your script for enabling init.d scripts. Since the stock ROM doesn't have, can you put back that script? Or was it causing trouble?

Do you know if the stock ROM supports ext4? Is there a simple I can check it?

Thanks
[Desire Android Development] melethron:
Kernel support can be checked with "adb shell cat /proc/filesystems" but im pretty sure that stock kernel only supports ext2/3.
[Desire Android Development] alexcavaco:
Yep, I've check, it does not support ext4...

Thanks
[Desire Android Development] peterme58:
Hi sibere thanhk you for the work, i follow all the steps in op and i want to say that i failed with amonra recovery 3 times, and went everything ok with clockword 2.5.07 , i want to ask you how i buck up my libsqlite.so in recovery i have never done it before, becouse i really want to try the patched one, and another question can i take nandroid backup with this script?

thank you all in advance
[Desire Android Development] Droidzone:
@Sibere,

What's the 2nd partition with the newest method? I think my eyes are playing tricks with me. I dont see the part about the filesystem to choose for the 2nd partition..You do mention that I need a kernel with ext4 support, and I see the mk2fs making an ext4 system. So is the 2nd partition Fat32/ext4?

Edit: Never mind. Just saw post #1114. So 1st is Fat32, Second is ext4. Trying it now on Leedroid..
[Desire Android Development] Droidzone:
That version is buggy (bugs while restoring and backing up sd-ext). I've reported that in udk's CWM thread. But maybe you compiled one yourself and it's good?

By the way, loved the method of extracting and repacking recovery that you taught Alex. He taught us.. :D It's just brilliant..

And the best version for me is the Alpharev 2.5.1.8.. As stable as 2.5.0.7 IMHO, but with a lot of goodies
[Desire Android Development] sibere:
Hi. There is a little bunch of ext4 sense compatible kernels around the place. You should find one easily.
[Desire Android Development] sibere:
Simply copy the original one on your sd card or do a nandroid backup of the entire phone.

And yes you can do a full backup with the script with clockworkmod :)
[Desire Android Development] sibere:
I ve used teppic rom. I posted the link on the OP. This one is init.d ready. But you have to flash an ext4 compatible kernel. I'll post mine on the OP when I'm home :)
[Desire Android Development] sibere:
Yep. 2nd is ext4. I thought it was clear enough on the op ;)