yashsingh1/xdaforums huggingface dataset:
[Asus  Eee Pad Transformer Prime] Tempie007:
Desire HD:

/dev/... /system ext4 ro,noatime,barrier=0,data=ordered 0 0

/dev/... /data ext4 rw,nosuid,nodev,noatime,barrier=1,data=ordered,noauto_da_alloc 0 0

/dev/... /cache ext4 rw,nosuid,nodev,noatime,barrier=0,data=ordered 0 0

SGS3:

/dev/... /system ext4 ro,relatime,barrier=1,data=ordered 0 0

/dev/... /data ext4 rw,nosuid,nodev,noatime,barrier=1,journal_async_commit,data=ordered,noauto_da_alloc,discard 0 0

/dev/... /cache ext4 rw,nosuid,nodev,noatime,barrier=1,journal_async_commit,data=ordered 0 0
[Asus  Eee Pad Transformer Prime] Hobbesian:
Intriguingly it seems most other devices use the ordered flag, the prime is the only one to use writeback? Curiouser and curiouser. I'm pondering now if we might have stumbled upon our I/o culprit in some way shape or form.
[Asus  Eee Pad Transformer Prime] Peter Galbavy:
From what I have been reading I feel it's the nodelalloc option that would have more impact but there is no harm trying ordered writes too.

There was a link I posted on using tune2fs to alter these...
[Asus  Eee Pad Transformer Prime] dagrim1:
Hmmm, what if I fail to remount/mount properly... is that restorable through CWM?
[Asus  Eee Pad Transformer Prime] leppie:
You should be able to remount at runtime, as long as you have root. Wont be permanent either ;)
[Asus  Eee Pad Transformer Prime] jazzy__b:
/init.cardhu.rc contains the following lines:

on fs
mount ext4 /dev/block/mmcblk0p1 /system wait ro
setprop ro.crypto.tmpfs_options size=128m,mode=0771,uid=1000,gid=1000
setprop ro.crypto.umount_sd false
mount ext4 /dev/block/mmcblk0p8 /data wait noatime nodiratime nosuid nodev nodelalloc,errors=panic
mount ext4 /dev/block/mmcblk0p2 /cache wait noatime nodiratime nosuid nodev nodelalloc,errors=panic

I tried remounting /cache and /data with delalloc instead nodelalloc with no notable impact on the SQL benchmark (mine vary between 80 and 120 seconds and this one came in at just under 100). So my current mounts are:

/dev/block/mmcblk0p1 /system ext4 rw,relatime,user_xattr,acl,barrier=0,data=ordered 0 0
/dev/block/mmcblk0p8 /data ext4 rw,nosuid,nodev,noatime,nodiratime,errors=panic,user_xattr,acl,barrier=1,data=ordered 0 0
/dev/block/mmcblk0p2 /cache ext4 rw,nosuid,nodev,noatime,nodiratime,errors=panic,user_xattr,acl,barrier=0,data=ordered 0 0

As you can see, on AOKP B38 I already have data=ordered.
[Asus  Eee Pad Transformer Prime] banderos101:
Some interesting stuff, thanks:o
[Asus  Eee Pad Transformer Prime] dagrim1:
Can you explain how you did that remounting? Which commands, and where (terminal emulator)?
[Asus  Eee Pad Transformer Prime] jazzy__b:
from adb shell:

mount -o remount,delalloc /data

Then, to check what I did:

mount | grep /data
/dev/block/mmcblk0p8 /data ext4 rw,relatime,errors=panic,user_xattr,acl,barrier=1,data=ordered 0 0

---------- Post added at 08:14 PM ---------- Previous post was at 07:56 PM ----------

Does anyone know why data=journal and data=writeback don't seem to be supported on AOKP?

mount -o remount,data=writeback /data
mount: Invalid argument
[Asus  Eee Pad Transformer Prime] djmcnz:
Did you try 'busybox mount' just in case?
[Asus  Eee Pad Transformer Prime] jazzy__b:
I get a more detailed error message but the same result:

busybox mount -o remount,data=writeback /data
mount: mounting /dev/block/mmcblk0p8 on /data failed: Invalid argument

(wish I could get approved to get past this 5-minute posting limit)
[Asus  Eee Pad Transformer Prime] Tempie007:
getting the same errors, can remount system with data=ordered with no problems, but on the data mount i get this same error..

- jazzy's system allready had the data=ordered , no improvement for sqllite bench

- i was able to remount without nodelalloc, no improvement either

i think the way the volumes are mounted does not give us the effects we want.

going to look into the fsync checkpoint behaviour....
[Asus  Eee Pad Transformer Prime] leppie:
If you change the ordering (especially of the cache), it probably needs to be cleared/formatted to be efficient.

It might also help, if you reinstall RL benchmark after you remount the data partition in case the database is already 'fragmented'.

Random farts :p
[Asus  Eee Pad Transformer Prime] dagrim1:
Remounting doesn't seem to help with anything here... remounted data, system and cache with the options I got from sgs3 (noacl, nouser_xattr, barrier=0, etc) but results stay the same.

And of course a reboot resets everything to default settings...
[Asus  Eee Pad Transformer Prime] jazzy__b:
I tried many different options and I couldn't get the 1000 inserts to move away from 40 seconds. I didn't try reformatting; I guess that would need a recovery script or custom ROM. I may try changing the boot mount options to make everything clean. The earlier RL fragmentation suggestion is interesting, perhaps I should clear data between tests. Hopefully I will get some time on this over the weekend.

Sent from my Samsung Galaxy S II
[Asus  Eee Pad Transformer Prime] dagrim1:
http://www.sqlite.org/speed.html

Seems this was used as a base for the RL Benchmark... Quite interesting.

Might want to give something else a try. Changing the sync value to 2 instead of 3 (full sync) or 1 (no sync).

*Edit*

Nope, nothing either... And appearantly on AOKP the ext4 fs are already mounted with optimal settings (and performance is the same as the sqlite without patch) so the mounting doesn't seem the place to be either.
[Asus  Eee Pad Transformer Prime] Peter Galbavy:
Ah well. Thanks to those who tested. Looks like it's not the mount options then :)

Reformatting will make zero difference BTW, the filesystem structure will be unchanged between remounts.

Back to the "thinking board" ...
[Asus  Eee Pad Transformer Prime] dagrim1:
And thank you for the info, at least I learnt from all this... So came out wiser (and amd kind of enjoying this process of figuring stuff out as well ;) )
[Asus  Eee Pad Transformer Prime] banderos101:
Just so you guys know, you got one spectator right here:D

Im luvin it when you guys throw ideas at one another, primes very own thinktank:D

Im glued to this thread:D Goodluck fellas

Edit : Sorry, i keep telling myself not to disturb the thought process going on here, but i just see potential here, and get as giddy as a school girl:D ill try and keep it to a minimum
[Asus  Eee Pad Transformer Prime] jazzy__b:
For the record the options I played with were:

- delalloc on/off

- no_auto_da_alloc

- barrier 0 vs 1

- some atime/diratime combinations

- couldn't try data=writeback or data=journalled: see earlier post

Next things I would like to try are running an app such as browser or play store under strace to get more info and adding some debug logging to sqlite code but I'm going to need some help.

Does anyone know of a precompiled strace I can get from somewhere? Also how do I launch an app from adb shell?

I've started working through a post I found to compile CM9 on MacOS Lion. My understanding is that CM9 and AOKP are binary compatible. Is this correct?

Sent from my Samsung Galaxy S II
[Asus  Eee Pad Transformer Prime] dagrim1:
You mean if you can use the binaries from CM9 with AOKP and vice versa? Yeah, sure... as long as theyŕe compiled from the same-ish source it's not an issue. It's all linux basically :P

About executing a binary... say you have a binary called tune2fs, you can executing it by typing:

./tune2fs

in the folder where the binary is (and if it's in a folder in the general path you can just type that command anywhere).

Make sure you set the properties to executable though!

And if you need superuser access (often needed with this kinda stuf), type: su

The sign will change from a $ to a #
[Asus  Eee Pad Transformer Prime] jazzy__b:
I'm fine with the Linux stuff. Where do I find the executable for an app, e.g. Play Store? If I can't launch it I guess will just need to strace the running process by id but I find launching from shell easier.
[Asus  Eee Pad Transformer Prime] dagrim1:
Hmmm, well... the android apps themselves are java. So there are no binaries for those afaik...

Btw, the TF101 is also suffering from the same-ish issues (even though the RL benchmark finishes in 83 secs there with 54 ssecs for the 1000 inserts, versus 120 secs for the TF201 and 80+ sec for 1k insert). Are they consistently using bad memory? Or is it truely some asus softwareside thing...
[Asus  Eee Pad Transformer Prime] jazzy__b:
Doh! Of course. I'm going to need to do this the hard way.

Sent from my Samsung Galaxy S II
[Asus  Eee Pad Transformer Prime] dagrim1:
Hehe, but too easy ain't no fun either right...

But thinking about it, given the effect that the prime (and tf101) are the only ones really having issues here with the various custom AOKP/CM9/AOSP roms it is not in that source department but in the remaining code...

The Asus kernel?

But I'd think the real wizards here (the ones releasing their own custom kernels) would have detected the issue already in that case.

Because if it was in the nvidia modules, I'd expect for example the tegra3 HTC ONe X to have issues as well...
[Asus  Eee Pad Transformer Prime] Tempie007:
Yesterday i did a binary comparision of the kernel sources of one x vs. tf201 (beyondcompare)

There are some pretty big differences in the MMC and NAND drivers/stack folders but that might be because of the prime having more connectivity options.

What i did see in the MMC files is that HTC has support for high speed MMC >45MB where asus doesn't, but this is MMC, not NAND but it might have some overlapping?

fsync.c is identical. Will post details on it later on tonight.

Im not a linux guru so might need some help...
[Asus  Eee Pad Transformer Prime] jazzy__b:
I don't know but I'm half expecting to find that some lazy developer has put an exclusive lock around an unnecessarily large section of code in a driver somewhere. This would mean that the sqlite call to fsync will block while other I/O is taking place. I've seen this type of thing dozens of times before. If this is the case I'm not sure if we will be able to do much about it. strace will hopefully help.

---------- Post added at 02:51 PM ---------- Previous post was at 02:41 PM ----------

Now I feel stupid. Hunted round on the web for a precompiled strace for Android, downloaded and it seems that AOKP has it built-in.
[Asus  Eee Pad Transformer Prime] leppie:
From what I could gather in the kernel, the MMC should be running at 104MHz SDR mode on the Prime.
[Asus  Eee Pad Transformer Prime] jazzy__b:
I can strace the sqlite profiler and I can see from 'top' that it appears to be handing off some of the CPU load to mmcqd and then to jdb2/mmcblk0p8 processes but I cannot strace those processes. sqlite profiler only hits 3% to 5% CPU, confirming what we already know - we are not CPU bound. It looks similiar to the link below but the resolution there does not work (I am already logged in as root in any case). Any ideas?

http://stackoverflow.com/questions/5154634/android-adb-linux-problem

root@android:/data # ps | grep mmc
root 90 2 0 0 c042c7e8 00000000 S mmcqd/0
root 91 2 0 0 c042c7e8 00000000 S mmcqd/2
root 99 2 0 0 c01dc888 00000000 S jbd2/mmcblk0p1-
root 101 2 0 0 c01dc888 00000000 S jbd2/mmcblk0p8-
root 103 2 0 0 c01dc888 00000000 S jbd2/mmcblk0p2-
root 479 2 0 0 c042ac90 00000000 S ksdioirqd/mmc1
root@android:/data # strace -p 90
attach: ptrace(PTRACE_ATTACH, ...): Operation not permitted
root@android:/data # strace -p 91
attach: ptrace(PTRACE_ATTACH, ...): Operation not permitted
root@android:/data # strace -p 99
attach: ptrace(PTRACE_ATTACH, ...): Operation not permitted
root@android:/data # ls -l /system/xbin/strace
-rwsr-xr-x root shell 121840 2008-08-01 13:00 strace

The setuid on the strace binary above was set by me, it is not standard AOKP, but it makes no difference.
[Asus  Eee Pad Transformer Prime] st33med:
eMMC is a type of NAND flash storage. You might have found something there. So we can hope. :D
[Asus  Eee Pad Transformer Prime] leppie:
eMMC is actually SDIO (which is like NAND + a MMC/SDIO controller).
[Asus  Eee Pad Transformer Prime] Tempie007:
I've attached some of the different files\folders of the One X kernel vs Stock 9.4.2.21 kernel. Not all files in the zip are different, but with a compare tool like beyond compare you'll see which files are actually different.

Same differences are in Motley's kernel.

Not a clue if this is relevant or not...

Might be worth it if someone with decent linux knowledge compares the whole kernel sources?

\drivers\mfd --> Multifunction Devices

\drivers\mmc --> Multimedia Card

\drivers\mtd\nand\ --> Memory Technology Device

\include\linux\mmc ''
[Asus  Eee Pad Transformer Prime] whycali:
Did this i/o issue exist when the prime was on Honeycomb?
[Asus  Eee Pad Transformer Prime] yumms:
So the patch works, but has this caused any issues with anyone yet?
[Asus  Eee Pad Transformer Prime] Tempie007:
no issues here so far.. the chances that something goes wrong due to disabled fsync are relatively small, however the impact can be big ( corrupted data )
[Asus  Eee Pad Transformer Prime] djmcnz:
My wife died... :mad::(
[Asus  Eee Pad Transformer Prime] whycali:
more time for you
[Asus  Eee Pad Transformer Prime] Hobbesian:
Played around with fstab mounts, yeah, I was kinda hoping nodelalloc was the problem but that only seems to be an element thereof, so this comes down to either the controller OR the software that deals with it. Taking out fsync allows the CPU to skip the whole ordering of writes and doesn't require it to commit to storage, that's why it's going faster, it's not having to concern itself with the I/O flush. Brrr. This is going to be a controller issue, but then it's a case of where do you look? My knowhow gets fuzzy at this point...
[Asus  Eee Pad Transformer Prime] Peter Galbavy:
OK, I've been playing source code detective a bit more but I don't have the facilities to test anything here. What I have discovered is that the kernel sources that Asus make available (which may or may not be the ones that the real kernel is built from, so treat this lightly) include MMC drivers from a long time ago.

mVidia very kindly make their tegra source tree available via public GIT web interface and so I have used that to track various changes - these merge the Linux 2.6 tree with nVidia's own changes.

The file in the TF201 sources for drivers/mmc/card/block.c appears to be a version of Oct 2010: http://nv-tegra.nvidia.com/gitweb/?...it;h=12578f66b9058e63f193fbfe1552e1d1bfa361bb

Later versions include support for CMD23 multi-block writes (with count) which I have learnt about from this page: http://elm-chan.org/docs/mmc/mmc_e.html - it appears that you should use CMD23 with a block count and not open-ended for better performance of "some MMC cards". This change was introduced with this commit: http://nv-tegra.nvidia.com/gitweb/?...it;h=d0c97cfb81ebc5b416c0f92fa2fc18d2773e3023

Looking at the sources that Samsung publish (for my GNote) they are using a much later block.c (and related) files. This is all still 2.6 kernel and not 3.x specific.

Now - whether any of this makes the slightest bit of difference I cannot say as, I repeat, I ddon't have the tools handy here to do much. I would love to verify if this is the MMC driver that the TF201 kernel is using for internal storage but I can't and I would love to build a new standalone kernel with more up-to-date MMC drivers, but I can't... if anyone can, please!
[Asus  Eee Pad Transformer Prime] jazzy__b:
This is a steep learning curve! I've just built CM9 from scratch with kernel debugging turned on for ext4 and mmc. I'll let you know later if it runs without crashing :-)

Sent from my Samsung Galaxy S II
[Asus  Eee Pad Transformer Prime] dagrim1:
Absolutely great to see all the research going on in here... I wish I had more (read, some) experience with all this (c/c++ programming, linux, etc) but this is all a bit too much for me...

I keep on checking back and fighting the urge to delve into this myself more as well (not too much spare time atm) but am very curious to see where all this is going...

I wonder if a 3.x kernel would help, the sgs3 is 3.x, but afaik the sgs2 wasn't... Anyway. Great reads and thank you all for all the effort :)
[Asus  Eee Pad Transformer Prime] Peter Galbavy:
Browsing the source trees at http://nv-tegra.nvidia.com/gitweb/ hints to me that nVidia have not yet done a full 3.x build hence the 2.6 kernel. But then it's only a nu,ber at the end of the day - as you can see from my note above it's quite possible to built a newish looking kernel and yet carry around 2 year old driver code - that's Asus, not nVidia.
[Asus  Eee Pad Transformer Prime] Peter Galbavy:
I have managed to build a stock kernel in a VM at work so assuming I can one day unlock this thing I might be brave enough to play. For now I am going to wait for .28 to see what may have been done...
[Asus  Eee Pad Transformer Prime] dagrim1:
That's cool... Since I have unlocked mine I wouldn't mind giving it a go (afaik there is no risk, just maybe being unable to boot but then I can flash back again in CWM... right?)
[Asus  Eee Pad Transformer Prime] demandarin:
Great find. ill make sure to pass all this info to asus. this whole thread is great info.
[Asus  Eee Pad Transformer Prime] Peter Galbavy:
As of now I've not made any changes and tomorrow is a public holiday in the Philippines so I am not doing anything for at least 36 hours... I then need to make and apply the minimal mmc block.c related changes and build. Not sure what else is required apart from vmlinux ... but I'll learn.
[Asus  Eee Pad Transformer Prime] Tempie007:
willing to test it once you built it :)
[Asus  Eee Pad Transformer Prime] leppie:
If you feel adventurous, try merge the latest changes from the L4T tree.

Specifically:

http://nv-tegra.nvidia.com/gitweb/?...2ab86fc1f8dfffc8687bc450b8619b;hb=l4t/l4t-r15

But the updating the entire mmc section would probably be a good idea.

Edit:

I am tempted to unlock too now. Just to fix this sh1t. I hardly use it due to sluggishness.
[Asus  Eee Pad Transformer Prime] tf201:
I think I had my first issue with it. Nothing major browser just crashed then wouldn't open, hard reset later and good to go.

Are there any later patches to try?
[Asus  Eee Pad Transformer Prime] dagrim1:
Dunno if that's cause of the patch though, had those kinda crashes more often... Or that pages suddenly wouldn't load.

Usually for me terminating the process is enough (haven't had any so far though).
[Asus  Eee Pad Transformer Prime] Emandroid99:
Dagrim, then you work for asus? These will be made official when your corrections in an update from asus?
[Asus  Eee Pad Transformer Prime] Tempie007:
same here. this happens on all roms.
[Asus  Eee Pad Transformer Prime] jazzy__b:
I have some kernel debugging output for pastebin shortly. I may need to add some more debug options to my kernel however.

Sent from my Samsung Galaxy S II
[Asus  Eee Pad Transformer Prime] st33med:
Doit. DOITNOW!

http://www.youtube.com/watch?v=u6ALySsPXt0
[Asus  Eee Pad Transformer Prime] valentinrascu:
Demandarin user is woking with asus

Inviato dal mio Transformer Prime TF201 con Tapatalk 2
[Asus  Eee Pad Transformer Prime] jazzy__b:
What's a good web page for me to load up and get kernel debug logs?

Sent from my Samsung Galaxy S II
[Asus  Eee Pad Transformer Prime] jazzy__b:
Here's the first kernel debug output. I have not reviewed this in detail yet. Happy to take suggestions on what to patch or tweak to see what changes.

http://pastebin.com/CrenimFS

My 1000 insert test increases from 40 sec to 220 sec as a result of the increased load of debugging.

---------- Post added at 09:59 PM ---------- Previous post was at 09:43 PM ----------

Looking at it now...
[Asus  Eee Pad Transformer Prime] demandarin:
I just sent Asus/Gary all of this info and link to this thread. hopefully this gives them new insight or possible solution to this issue, if they don't know about it already.
[Asus  Eee Pad Transformer Prime] Peter Galbavy:
I did the same a couple of days ago but I suspect being a tester your name will merit more reaction amongst all the crap he gets...
[Asus  Eee Pad Transformer Prime] leppie:
Thanks :)

Notes:

1. Speed is only 52MHz, according to the CSD of my memory (64GB model), the max clock is 200MHz. I highly suspect the updated driver (as pointed out in my previous post) will help with this. It has some 'auto-tuning' method to get the max speed from the device.

2. Another issue I spot is this constant 'fiddling' with the device and the clock.

[ 8830.482688] tegra_sdhci_set_clock mmc0 0 enabled=1

[ 8830.483036] tegra_sdhci_set_clock mmc0 52000000 enabled=0

This seems to happen on every write, effectively causing a delay of 0.35 milliseconds (which is astronomical in this day an age).

(Actually this does not really make sense! Perhaps the enabled flag is is really disabled)

3. tsac 1600 ms nsac 4000 these 2 numbers represent the advertised speed as found also in the CSD. They should be the same on the device. From investigation MLC (as in flash sticks) has a high tsac and SLC (as in SSD) is much lower. I still think ASUS conned us here by using the cheapest form of memory they could find. Paying a $100 for an extra 32GB should be equating to high quality SLC memory. And not memory from a $20 USB flash stick.
[Asus  Eee Pad Transformer Prime] st33med:
So we know a probable fix, any idea for what might be causing the second one?As for the 3rd one, less latency would be enjoyable, but if you think about how the OG Transformer performed memory wise, it isn't too bad. Of course it is a bit of a rip off, but that's ok right now. Let's just make the tablet operate as best as it can right now.
[Asus  Eee Pad Transformer Prime] leppie:
I scanned through the updated driver code yesterday, and it seems they 'fixed' this for builtin memory as the power management is triggered by WIFI.

I could be wrong, but this is what I gathered. There is no reason to enable/disable the clocks when the device is in use.
[Asus  Eee Pad Transformer Prime] st33med:
Excuse my ignorance, but do you mean updated drivers from Asus with .28 or nVidia?

Never knew wifi could affect so much.
[Asus  Eee Pad Transformer Prime] leppie:
I am talking about the kernel drivers supplied by nVidia for it's L4T (Linux 4 Tegra) tree.

http://nv-tegra.nvidia.com/gitweb/?p=linux-2.6.git;a=shortlog;h=refs/heads/l4t/l4t-r15
[Asus  Eee Pad Transformer Prime] Namidairo:
Gee, I wonder if:

"mmc: host: Disable SDIO card clock when idle for Tegra 3 only"

Does anything :P

That said, some of the devs might find it a good idea to start backporting patches from the L4T tree if they don't want to wait for another Asus release.
[Asus  Eee Pad Transformer Prime] jazzy__b:
Last night I started looking at the nvidia gitweb tree but didn't find a way to download the sources and I couldn't deal with trying to compare code through a web page. I downloaded the SGS2 update 4 source from Samsung and it is clear that they have much newer mmc code than we do so I began a merge. There are many changes in there, including a number that relate to things we have discussed. The merge is not entirely straightforward as we have a number of tegra changes that are not in the Samsung code and we want to keep those. A number of common header files have been update so I have some knitting to do to get this all consistent - probably another couple of hours editing work there. There may also be some newer nvidia changes we want to pull as well. Once I have done all this I have to check and hope that I have merged it all consistently.
[Asus  Eee Pad Transformer Prime] jazzy__b:
Wondering if this thread should be moved to a dev forum...
[Asus  Eee Pad Transformer Prime] dagrim1:
My guess is that it would maybe get more attention from others able to help, it is now starting to be more development targeted anyway.