yashsingh1/xdaforums huggingface dataset:
[General Omni Discussion] Twiq:
maybe this is pretty outdated but not that old, for those who didn't know there has been several changes in AOSP to sdcard binary for kitkats.

Project name: android_system_core

*Elliott Hughes - Make sdcard log to the log rather than stderr. https://github.com/android/platform_system_core/commit/f6d6737529ea164fd2ea79f02cc78a46bb9260f5

*Elliott Hughes - Fix sdcard's FUSE_FSYNCDIR handling https://github.com/android/platform...b9260f5#diff-d8c7e9d093a2787934c6e58aecf36d05

For a file the FUSE fh is a struct handle containing an int fd; for a directory it's a struct dirhandle containing a DIR*. Fix handle_fsync to extract the file descriptor appropriately in both cases.

*Daisuke Okitsu - Handle FUSE_FSYNCDIR as FUSE_FSYNC https://android.googlesource.com/platform/system/core/+/b2831a2db9e0370c5f797c037c135108025f2522

Handle FUSE_FSYNCDIR as FUSE_FSYNC

There have been issues with sdcard data corruption even after successfully calling fsync for /sdcard. This is caused by the sdcard daemon doing nothing in this case.
[General Omni Discussion] kicsitian:
It happens like every 5 minutes on my phone (music played from the external SD card stops playing, player crashes or becomes unresponsive until I kill it). I am more than willing to try capturing anything useful, I only need detailed instructions on how to do so.

(Samsung Galaxy S2 i9100 / CM version: 11/20140708-SNAPSHOT-M8-i9100)
[General Omni Discussion] Entropy512:
Won't help since those aren't the problem we're having.

The problem we're having has been clearly traced to a REALLY strange kernel bug, in which pread64() can corrupt variables in the function that called it if certain interrupts come in during the pread64() call. (Normally, a function can only alter its own data, global variables in the same process space, or data that has been passed to it by reference. In this case, variables not even passed to pread64() are getting corrupted, which says "FUNKY KERNEL BUG". Now there are some workarounds, such as changing some of the structures to force the data to be in RAM instead of being optimized into a register, but those don't fix the raw underlying problem which is likely manifesting itself as all sorts of other strange issues throughout the system, which is that registers are getting clobbered by pread64() that shouldn't be getting clobbered.
[General Omni Discussion] Twiq:
After reading back to http://xdaforums.com/showpost.php?p=51241452&postcount=258

Now i get what you mean by funky :silly:

I gotta do more reading ?
[General Omni Discussion] zeitferne:
So the next step would be to find some of these interrupts and look what they have in common? It looks like you're already quite close to finding the bug!

And just a speculation: There have already been found several buffer overflows in the kernel. It could be that the registers are corrupted by one in the place where they are saved in memory.

EDIT:

Found a fishy difference between the original samsung JB i9100 kernel (directly from opensource.samsung.com) and the CM11 sources in the mshci_irq handler (which is somehow related to the problem, see http://xdaforums.com/showthread.php?p=52820068&highlight=+irq#post52820068) (): The assignment "timeout = 0x10000;" is missing before the while loop in line 1803. So when the above if-branches are taken and the timeout reaches zero there, timeout will go below zero in this loop, causing this to potentially wait for a very long time.

The other change (apart from added #ifdefs) is a timeout related debug output in line 1666.

EDIT2: FAT32 file copy still get stuck, though. So this was not the culprit. Pity! It looked just right, with the interrupt and the timeout and all that. There is a slight chance that I did something wrong while building but I don't think so.
[General Omni Discussion] Sivabalan:
I am running pa_i9100-4.43-20140716 paranoid ROM . even here also the issue exists. the music player goes unresponsive after 5 min and after that there is a huge battery drain.
[General Omni Discussion] lerkin:
"EVEN"
[General Omni Discussion] Sivabalan:
As mentioned in this post I have reformatted both the internalSD and the externalSD to exFAT format and everything working fine now.

1. Tested file transfers between the device and the computer.

2. Tested the data transfer between the SDcards using the ES file explorer

3. Ran music player morethan 10 minutes and after that changed tracks multiple times.

4. Watched the battery drain by the mediaserver.

Everything seems working normally without [MENTION=3995045]cgx[/MENTION] fix. Thanks everyone for the valuable wrokaround for this issue. otherwise I would have gone back to stock JB ROM.

Please anybody tell is there any drawback because of this exFAT formatting of the storage spaces?

Thanks and Regards,

Sivabalan K
[General Omni Discussion] lerkin:
Some specific apps can not access exfat formated storages (Navitel).
[General Omni Discussion] Entropy512:
There's also the fact that it doesn't solve the root cause - which is likely causing all sorts of other weird reliability/stability issues.

Interrupts corrupting the call stack is a VERY nasty bug that can cause a wide variety of issues.
[General Omni Discussion] HippyTed:
Its strange that we aren't seeing (m)any other issues caused by this, ... I would have expected many more complaints in the forums if this were the case. Wouldn't you?
[General Omni Discussion] Entropy512:
I suspect many of the issues are probably more subtle... I'm actually shocked that we aren't seeing more issues. But some of the random issues people are having that not everyone can reproduce (some people saying they sometimes can't get baseband to work, some people complaining about random excessive battery drain, etc.) could be stemming from the same kernel flaw.
[General Omni Discussion] HippyTed:
Yes, I've had problems with the modem not connecting, occasional excessive battery use, etc., but nothing that a factory reset and clean install hasn't cured.

There are still bugs in cm-11-20140724-NIGHTLY, for example (MINOR: the quick settings panel offers two ranges of display sleep delays, the Settings menu/Display/Sleep times have a wider range...), (NOT SO MINOR: adb logcat still reveals an occasional NullPointerException), and so on. - I've always felt that these problems must have been caused by software bugs or out-of-date data hidden away that wasn't being reset properly when the rom or an app is upgraded.
[General Omni Discussion] damiend:
Spotify was stopping and hanging several times per day on my Galaxy S2, latest SNAPTSHOT. After formatting the SD card as exFAT from Clockworkmod Recovery (no other method worked) and making sure that Spotify stored its cache on that partition, I am happy to report that I haven't had any further hangs.
[General Omni Discussion] Gigahawk:
Works just fine on my i9100, haven't noticed any side effects.

I wrote an init.d script to do it automatically here if anyone wants it - https://www.dropbox.com/s/ql5tg2x7eufgjkj/99MusicFix

First time writing scripts and I have no idea what I'm doing but it seems to do what it's supposed to.
[General Omni Discussion] lerkin:
del
[General Omni Discussion] Sivabalan:
[MENTION=3440805]lerkin[/MENTION], Sorry for the asking this...

"del" means...? The previously posted info is being deleted?

Thanks,

Sivabalan K
[General Omni Discussion] Sivabalan:
Will this Fix work here?

Hi,

[MENTION=3995045]cgx[/MENTION] , Will the below mentioned commit in CM 11 will fix the Music playback and file IO issues from the SDcard?

http://xdaforums.com/showpost.php?p=54438265&postcount=1988

Regards,

Sivabalan K
[General Omni Discussion] lerkin:
no
[General Omni Discussion] snow:
[MENTION=591147]Entropy512[/MENTION] FUSE from official 4.4 N7100 KitKat kernel could be useful? I mean, we have an unofficial FUSE which works for i9300 but maybe that official FUSE daemon from SAMSUNG is better. :) I know N7100 has Exynos 4412, but maybe SAMSUNG applied some tweaks or little changes in their official kernel that make the difference (maybe which are good for Exynos4 chipset). :)

At least we can try this. :D

EDIT: I've tried to use "sdcard" file from N7100 Kitkat and it seems phone was more stable. No stutter at all. :) And the very interesting thing is that "sdcard" from N7100 has only 9.484 bytes. Maybe SAMSUNG removed a lot of stuff and kept only important code for FUSE to work. :)

EDIT 2: It seems I'm not able to acces sdcard (so apps installed on sdcard don't work). But I can enable USB Mass Storage and I can see files on my PC. WTF? Weird...
[General Omni Discussion] Twiq:
Just commenting..

It should be related to emulated storage. it points at the right mmcblk, but it's looking at different path.

Can you link the file here? Maybe it needs some "patches" to work properly with our device

on Sabered SlimKat 0726 | powered by dorimanx test kernel v116
[General Omni Discussion] snow:
I've attached the file. Here you go. ;)

P.S.: Change "sdcard.zip" to "sdcard"
[General Omni Discussion] Entropy512:
No. The bug is not in the FUSE daemon itself, this is well established and has been for over a month.

I'm going to repeat again - THIS IS A KERNEL BUG. pread64() should NEVER corrupt the call stack but it sometimes does if an interrupt comes in during the call.
[General Omni Discussion] AndDiSa:
...probably related to GCC and not directly a bug in the kernel itself? Linus was recently complaining much about the quality of the GCC results.
[General Omni Discussion] grzwolf:
pread64() call from sdcard.c with temporary disabled interrupts

How about this?

system/core/sdcard/sdcard.c

...

/*mod-beg*/

spinlock_t mLock = SPIN_LOCK_UNLOCK;

unsigned long flags;

spin_lock_irqsave(&mLock, flags);

/*mod-end*/

res = pread64(h->fd, handler->read_buffer, size, offset); /* line 1231*/

/*mod-beg*/

spin_unlock_irqsave(&mLock, flags);

/*mod-end*/

...

At least the mod didn't harm my home build cm11 during the last week.

I didn't notice any side effects.

Although, I never got hit by the music bug - even w/o the mod.

So no positive proof from my side, whether the mod might be effective or not.

Thanks to cgx: zip uses his updater-script.
[General Omni Discussion] Entropy512:
That's what I was thinking at first, but this is affecting multiple versions of GCC and is ONLY occurring on Exynos 4210 devices. That said some of the behavior seems very similar to the sort of thing Linus was ranting about - but despite some general similarities in behavior I think it's a different actual issue since that recent rant was specific to GCC 4.9.0 with an x86-64 target. Same compiler/kernel source tree is fine on 4412 devices, so it's probably:

1) Exynos 4210-based hardware-specific errata that doesn't have a workaround in the 4412 source tree or

2) Some common errata workaround is only enabled in the 4412 defconfigs but not the 4210 defconfigs

No I haven't had the time to even look at the defconfigs, but that's a place someone else could possibly start. Another possibility would be to look at the old smdk4210 kernel defconfigs to see if they are enabling any ARM errata workarounds that don't exist in the 4412 tree.

I didn't think that sort of thing was legal to do in userspace... It's really questionable, and there's the issue of this bug potentially still existing in less obvious cases elsewhere in the system. Like those people who randomly have their baseband not show in rare cases.
[General Omni Discussion] grzwolf:
I don't feel save too, therefore I didn't dare to manipulate pread64() directly on lower level - on the first shot.
[General Omni Discussion] HippyTed:
How far back do we need to look?

Looking at the latest checkouts of CyanogemMod jellybean (linux-3.0.15) and kitkat (linux-3.0.64).

The cyanogenmod_i9100_defconfig have the same ARM_ERRATA and PL310_ERRATA enabled, although there are other things that do differ -- see the attachment at the end.

ARM_ERRATA and PL310_ERRATA appeared in just these five files (in the smdk4210 tree for linux-3.0.15 and smdk4412 tree for linux-3.0.64):

[FONT=Courier New]
arch/arm/mm/proc-v7.S Changes to [B]linux-3.0.64[/B] based on [B]CONFIG_HARVARD_CACHE[/B] and [B]CONFIG_ARM_THUMBEE[/B]
arch/arm/mm/cache-l2x0.c Identical in both versions.
arch/arm/mm/cache-v7.S Use of [B]CONFIG_PREEMPT[/B] adds small change in [B]linux-3.0.64[/B]
arch/arm/kernel/smp_scu.c Identical on both versions.
arch/arm/mach-exynos/cpu-exynos4.c Identical in both versions.
[/FONT]

I dont understand why the change in arch/arm/mm/proc-v7.S is masked by #ifndef CONFIG_ARM_THUMBEE in linux-3.0.64, rather than #ifdef, but I've not properly worked with assembly code since 8-bit micro days!

Is there a definitive document on the ARM ERRATAs? Google has been unhelpfull and all I have so far are the "help" comments in the arch/arm/Kconfig files. Linux devs must have obtained this information from somewhere.
[General Omni Discussion] grzwolf:
RE: pread64() call from sdcard.c with temporary disabled interrupts

http://xdaforums.com/showpost.php?p=54592044&postcount=467

I'm running the workaround for meanwhile 3 weeks with no side effects.

Any other feedback available?

Despite of concerns, it would be great to learn whether the workaround is effective.

Reason:

I cannot reproduce the failure scenario even w/o the workaround - therefore your feedback is very welcome.
[General Omni Discussion] pasche:
just some more information from me

Hi all,

I just wanted to give some other issues I have. But it's also related to music play. But most of the time, the problem is, that the music does not stop directly within a song. It stops just between the songs (most of the time).

The first time i recognized this issue, i went to Terminal and searched my "music" path with "lsof" and - in normal operation - there seem to be 4 processes...2 media processes and 2 sdcard processes holding the "current" song and the song it should play next.

I also have this issue with several music players (rocket player, apollo, etc) BUT I think I have more issues with apollo. When the music stops "switching" to the next song, I used to press the "skip to next song" within the lockscreen widget of apollo. That seem to work several times but at some point, pressing the "skip to next song" did nothing. I again searched with "lsof" and found a "couple" of "media" and "scard" jobs! Sometimes about 10 "media" songs and 10 "sdcard" processes. (as if there were Threads started for every "skip to next song" press)

And then, I just had this a couple of times, the music just stops within a song and switched to another song, so as if there were 2 players running (one in background - mute) and when the one playing "died", the one in the background could be "heard".

Now, then I have this issue, I go into apollo into the current "playlist" and press the "play" button (to pause the song which actually didn't start to play) and then press the play again. Then music continues to play "another" song, but continues.

Using this method works "better" for me, but does not really solve the problem.

So I also hope that this very strange bug will be found and fixed one day. I really like the OmniRom :) But this music play but is really annoying.

Kind regards,

Robert
[General Omni Discussion] lerkin:
Tested your fix for a night, unfortunately still not working for me (gdrive sync hangs).

After all that time kitkat on i9100 still unusable for me. Still on cm 10.2.1 with memory leak workaround.
[General Omni Discussion] bleggy:
I went on a road trip today and listened to every beatles album (so, 13 total) beginning to end. no freezing, skipping, pausing, reboots. exfat + cgx's patch still awesome.
[General Omni Discussion] grzwolf:
Too sad.

As Entropy512 already said, there's surely an issue in a so far well hidden place.

'gdrive sync' might not take advantage from the workaround at all.

Perhaps the 'spinlock pread64' thing could be applied at a lower level - which would make this botch even more ugly.

Not promising anything, but I could to try it.
[General Omni Discussion] philippe734:
I formated in ext4 my internal sdcard and music bug is gone. Also rare lags while video recording are gone. Here is a way to format your internal sdcard to ext4 and keep your data : link
[General Omni Discussion] Hank87:
Ext4? Or you mean exfat?
[General Omni Discussion] philippe734:
Ext4, not exfat. Ext4 because I read it on wiki on cm about this bug. And because :
[General Omni Discussion] lerkin:
I tried ext4 once for both storages, bug were still there.
[General Omni Discussion] philippe734:
I’m surprised of what you said. Because this bug believed to be due to the bug being highly timing-sensitive and different file systems (FAT32, exFAT…) have different timing behavior (source : here). And also because EXT4 is native filesystem on Android, while FAT/exFAT is not. I’m not the author of this workaround, I read this on xda and cm forums ;) Since I follow this workaround, music bug and lags while video recording are gone. Here proof of my internal sdcard in ext4 :
[General Omni Discussion] lerkin:
Gdrive sync and dropbox syns hang fuse then I tried to format my storages to ext4.

Don't remember about copy files in file managers.
[General Omni Discussion] pasche:
best solution for me right now (as far as i can see)

Hi,

the best solution for me - right now - is to use "poweramp", because it does not need the "mediaserv".

Before yesterday, I used "apollo" on OmniRom 4.4.4, which used "mediaserv" to access the files. As I posted before, I had problems that after some time, the number of "mediaserv" processes accessing my music files increased. Had sometimes 3 or 4 "mediaserv" processes accessing the same music file - which had been played abount 30 minutes ago. Looked like the process could not "release" the file.

Right now, I created a ".nomedia" file in most of my top level directories on my external sdcard. In "poweramp", I only added my "music" folder to the library and also deactivated every automatic "scanning" for new files within settings.

I store all of my music on the external sdcard only, which is still formated with "FAT32"!

Now I don't see *any* "mediaserv" process accessing my "music" folder. Everything is playing just fine...

I'll report, how this continues....

pasche
[General Omni Discussion] Hank87:
Have you tried exfat?
[General Omni Discussion] lerkin:
Yes, and sdcard workaround.
[General Omni Discussion] pilgrim011:
What's wrong with sd card workaround? Works for me.
[General Omni Discussion] lerkin:
Not for me
[General Omni Discussion] Gexiaojiang:
This is useless, and bug was reproduced on Poweramp also.

The problem was not caused by media server itself, and root cause is within kernel side.

If you use the top command to show CPU usage after problem occurred, you will found that the player process is in zombie state, and eating all cpu cycles on that core(50%), so battery drains quickly.

A userland program cannot made such result, also try to use spinlock to disable interrupt during file access this idea is very unpractical.
[General Omni Discussion] Tann San:
Just formatting the external SD card helped reduce the frequency but it still occurred. In the end I bit the bullet and repartitioned+reformatted my internal to exFat as well and I haven't had a single problem in the 1-2 months since. I also disabled Fast Dormancy using a free app from the store. I'm not sure if they both contributed since I disabled FD as soon as I had the latest OmniRom nightly installed after the reformat. I was having insane battery drain on an almost daily basis until then.
[General Omni Discussion] lerkin:
FD app from playstore doesn't work since android 4.1.

Try this app instead:

http://xdaforums.com/galaxy-s2/themes-apps/fast-dormancy-toggle-app-version-1-5-t2647015
[General Omni Discussion] Tann San:
Ah that explains why I kept seeing secril_fd-interface in BetterBatteryStats. I stopped thinking about it once the battery drains stopped. I guess that means FD didn't play a factor in the battery drain issue I was having which is cool since my preference was to have it enabled so yay! But yeah, just formatting the external SD to exFat wasn't enough to solve my problem on it's own, I had to format the internal as well and since then boom magic. I'm running almost the latest OmniRom nightly on my Galaxy Note N7000.
[General Omni Discussion] philippe734:
The music bug will be gone if format in ext4 not exfat.
[General Omni Discussion] Entropy512:
IIRC that wakelock name is a little bit misleading... It says fd-interface but IIRC it is only minimally connected with Fast Dormancy and you'll see it even on a device with FD disabled.

No it won't. It'll still be there, just far less likely to trigger.
[General Omni Discussion] philippe734:
I don’t agree with you. When my internal sdcard was in original file system, I has often the music bug because I listen often music, sometimes with headphones, sometimes with external speaker. I has got music bug several times a day. I formatted my internal scard to ext4 the august 18th. Since, I’m listen more often music to check the music bug. Since, I listened exactly 21 albums of about 76 min (total about 26 hours) and zero music bug ! I don’t lie. In my opinion, it’s not a coincidence, it’s the fact that the music bug is gone after ext4. I have the same effect about the small barely perceptible lags while video recording. I film often my kids so I know well these lags. These are gone too. No coincidence, it is a fact too. Remember, ext4 is the native file system of android, and chosen by Google to release android device since 2011. May be, it’s why since Galaxy S3 all Galaxy S series have their internal storage ext4. I’m a dev too, and I can understand your position. But for me, the facts are more important that source code ;)
[General Omni Discussion] AndDiSa:
If the bug is somehow related to fuse ext4 is not affected, because it's supported natively ... In that case you would notice the bug only in fat / vfat / exfat / ...
[General Omni Discussion] Entropy512:
The problem is that in KitKat, even native filesystems use the fuse sdcard daemon for permissions derivation.

In fact the "workaround" some people have used (exfat) changed timing by adding, of all things, a SECOND fuse layer.

The bug is in the kernel. That is unambiguous, we've got plenty of pread64() sometimes corrupting the call stack which is something that should NEVER, EVER happen under any circumstances. Changing the filesystem won't get rid of it, it will only make it far less likely to trigger to the point where many users may consider it fixed.
[General Omni Discussion] tusko5:
If the problem is always pread64 could be a workaround to just use lseek + read system calls instead?

I found a code snippet about how to do that here:

ftp://ftp.netbsd.org/pub/NetBSD/NetBSD-release-6/src/tools/compat/pread.c

However, it is not thread-safe. Not sure if this is needed for fuse to work correctly. In this is the case, a lock should be placed around the call. Though it does not need to be a spinlock.
[General Omni Discussion] j1gga84:
Does the work of [MENTION=2376110]dorimanx[/MENTION] help here in any way?

https://github.com/dorimanx/exfat-nofuse

Regards
[General Omni Discussion] Entropy512:
There's a good chance that will cause performance issues, and it may still suffer from issues with interrupts corrupting the call stack.

No. In fact it might make things worse than the current exfat workaround.
[General Omni Discussion] tusko5:
Alternatively mmap could be used.

I think that mmaping the part of the file to read, then copying the mapped data to the buffer in userspace would be the best method to avoid the kernel bug caused by the interrupt.

The reason is that mmap is heavily used by dynamic linking, and can produce page faults, which are common situations that if were wrongly managed by the kernel, our devices probably would not even work.

Also, locks are not needed in this case.

Of course, this is only another (possible) workaround.