[Epic 4G General] rsage:
I've been experiencing some maddening issues with corruption of sdcard data. After much googling and finding some other reported occurrences of this, but not finding much info that's been helpful in solving the problem, I thought I'd compile and post some data in the hopes that others could add to it - hopefully leading to some kind of conclusion and maybe even a solution. I apologize because this is going to be a long post - hopefully it will be worth it.

Issue description:

There's a directory called LOST.DIR on sdcards in android phones. From what I've gathered, it contains lost/corrupted/orphaned files from the sdcard. When these files are moved to LOST.DIR, they're renamed with seemingly random strings of numbers and any file suffixes are stripped. Other OS's (Mac, Windows, etc.) have similar directories that serve similar purposes. The problem that I and others are experiencing is an alarming rate of sdcard data corruption, causing the LOST.DIR directory to fill up rather quickly, sometimes reaching multiple GB's in size. I’ve seen up to a dozen new files per day appearing in this directory. It's gotten to the point where I no longer trust that any data I put on my sdcard will remain intact.

I've read a few theories on how/why this happens. I've read that this is a Mac-related issue and only happens when mounting the sdcard on a Mac. This is not the case, as I've had this issue occur regardless if I'm mounting the card on a Mac, on a Windows PC (XP or 7), or even when mounting the card in a card reader and not from the phone directly (though I don't think the data corruption is occurring during mounting the card, but rather when the card is unmounted from the phone either when shutting the phone down or when unmounting the card from the phone during a request to remount the card via USB). I've even had a few (rare) occurrences of new files appearing in LOST.DIR without mounting the card on any device except the phone itself. I've also read that it's ok to just delete the files in this directory to reclaim space on the sdcard, but I've found that the files in LOST.DIR are often files I need, or files that various applications need to function. For the most part, when files are found in LOST.DIR, they've actually been removed from their original location. Deleting the files in LOST.DIR means you'll never be able to recover those files.

Once files end up in LOST.DIR, it's painstaking to recover them. Since they no longer have file types/suffixes, you may have to try opening the files in a utility to be able to tell what they are. Or by trial and error try appending different suffixes until you can open the files in an appropriate application.

Examples of files from my sdcard that have ended up in LOST.DIR:

whole mp3's

partial mp3's (more on this below)

photos & videos taken with the Epic's camera

xml config files

cached browser files

sql lite db's stored by various apps on the sdcard (causing one or two apps to FC since the sql lite files are no longer where the app expects them to be)

flashable zip files

etc.

Basically, any file I have on my sdcard can become corrupted or orphaned and end up in LOST.DIR in a near-unidentifiable state. To add to the pain, sometimes the files in LOST.DIR are not complete files, and sometimes they're not moved there intact. I was playing an album stored on the sdcard, and one track cut off in the middle. I found the other half of the mp3 file in LOST.DIR. I've lost entire photos and videos to LOST.DIR, but for the most part the files that are moved there are munged together with other files. If you view some files in LOST.DIR with a utility that can open any file as text, you'll see that a single file in LOST.DIR can really be multiple files or parts of files munged together into one. I've seen parts of text files mixed in with jpg's from browser cache mixed in with html mixed in with xml mixed in with other unidentifiable data. Obviously if this happens the data is not recoverable.

Some theories which I've ruled out or partially ruled out:

Mounting/unmounting method: I always shut down, mount, unmount, etc. the phone and the sdcard properly. If I have the card mounted on a Mac or PC, I always use the appropriate OS-specific method of unmounting the card before turning off USB mounting on the Epic. I ran tests where I kept track of every step I took in booting, unmounting, remounting, etc. the sdcard to confirm this. Other users have also ruled this out as a cause (though I'm sure improperly unmounting a card can cause data problems)

Journaling: I confirmed this happens regardless of whether journaling is turned off or on

Ext4 file system: don't think this has anything to do with it, as I've seen this happen on Ext4 and RFS

ROM's: I confirmed that this seems to be happening on multiple ROMs: midNIGHT/Bonsai, Baked Snack, stock deodexed, etc.

Kernels: Seems to be happening on multiple kernels: Bonsai kernel, Baked Snack kernel, maybe one or two others I've tried

Card file system corruption: I've reformatted my sdcard more times that I can count in trying to get rid of this problem, doesn't seem to have any effect

Epic4g itself: this seems to be happening on many different android phones, so I'm ruling out a design flaw with the Epic4g

Theories I haven't ruled out:

android (froyo?) bug related to data integrity when mounting/unmounting the sdcard (let's hope not)

damaged or defective sdcard

I've read that some people have had success in banishing this problem by moving to a higher-class sdcard (doesn't really make sense to me, but whatever). I'm currently using the OEM 16gb (I'm assuming class 2 or 4) card that came with the Epic. In researching class 6 cards, it seems that the 8gb cards are generally more reliable than the 16gb cards, so I ordered a highly-rated 8gb class 6 card (still waiting for its arrival). I'll gladly give up storage space if I can count on data integrity. If it's a low-level android bug, then who knows if/when it will ever get dealt with.

Here are two links that have more info and user experiences with this issue:

- Google Code issue:

http://code.google.com/p/android/issues/detail?id=2500

- ZDNet article:

http://www.zdnet.com/blog/sybase/could-your-sd-card-cause-your-android-smartphone-to-crash-or-lose-data/1128

To be honest, I don't know how long I've been experiencing this problem. I don't know if it happened on Eclair, but it definitely happens on Froyo.

I apologize that this is so long, but hopefully it will help someone else and possibly even lead to some helpful information in dealing with this problem. I'll post my findings once I receive the new class 6 sdcard.

Peace.