yashsingh1/xdaforums huggingface dataset:
[Epic 4G General] zman519:
I have also had this same problem. Its really disgusting. This is about all I can take from this phone. I'm about to smash it and go back to a dumb phone. What's the point of a smart phone is the data is not relable.

OK sorry about the ranting.

I started a thread about this in Q&A. I refomatted the card yesterday again I think that's the 5th time. And so far its good AFAIK 21 hours in. O all the things/Devices I had moded hacked I have never had data ****ing up like this.

Sent from the Drivers Seat of my Suby txting and Driving doing 100MPH+ in a school zone! Ha.
[Epic 4G General] rsage:
I feel your pain. Found your thread in Q&A, I missed it when I was searching the forum before I started this thread, probably because there's no reference to LOST.DIR. A lot of similarities in our setups and the issues we're seeing. Keep us posted.

By the way, you mentioned that you've had files go down to zero bytes but then at some point go back to normal. I've never had a file come back from this kind of corruption. Do you have any idea how this happened - did you run any kind of utility on the sdcard from windows or android OS for example?
[Epic 4G General] mkasick:
How repeatable is the corruption?

If, for example, you reformat the SD card, restore its contents from backups, is it the same files that are corrupted after some time? Roughly how long after a reformat is corruption observable?
[Epic 4G General] rsage:
Only answering for myself - never been the same files for me. Seems that it could be any file at any time, I've never actually seen the same file get corrupted twice before or after a reformat (assuming I was able to restore it after the first corruption). I can see new corruption after a reformat as soon as I start unmounting/mounting the sdcard. If I reformat then never unmount/mount the card via USB, occurrences are much rarer (though I have seen a few occurrences after just rebooting the phone, which of course unmounts & mounts the card as well).
[Epic 4G General] mkasick:
And these are confirmed missing files that are no longer in their original location, or are zero-length or something?

When you say "unmounting/mounting", do you mean just on the phone, or only after mounting/unmounting it on a PC? For example, if you were to reformat, copy a bunch of data over, verify (on the phone) that it's correct, then click "Unmount SD card" on the phone a bunch of times (remounting inbetween), does that pretty much guarantee corrupt/lost files?

Also, do you store any apps on the SD card? If so, do those appear to be OK, or can they get corrupted as well?

Edit: The more times you mount/unmount the SD card, does corruption appear to worsen? Is this independent of how much you otherwise actually use the card?
[Epic 4G General] rsage:
Well, I've seen both. I've had files go completely missing (photos taken with the camera, an mp3 or two, etc.), I've had files go to zero bytes (zip's, sql lite db's), and I've had files stay in place but get partially truncated (other mp3's, maybe some others).

I'd say 90% of the issues I've had have been noticed after mounting the card, while in the phone, on a Mac or Windows PC. I notice the issue after I then unmount the card from the Mac/PC, at which point the Epic mounts the card. However, I've also seen it happen when shutting down the Epic, removing the card, and mounting it on a Mac or PC with a card reader (I see the evidence after reinserting the card back in the Epic and booting back up). I'm not sure if it's happening when the card is unmounted from the Epic, or when it's mounted on some other device, I could try to track that down.

I have less often seen it happen when shutting down and restarting the phone. I haven't tried repeatedly hitting unmount/mount in the Epic settings.

Good question - until a few days ago, I had no apps on the SD card. I've been noticing the data corruption well before that - weeks, if not a couple months. Really don't know how long it's been going on. Within the last few days I have had to move some apps to the SD card. I haven't had an actual app/apk get corrupted (that I'm aware of), but I have had data get corrupted for apps that store data on the SD card.

Edit:

Yes, and yes
[Epic 4G General] zman519:
I did not run any thing on the sdcard with the data on it, win or android. At frist I saw this happening to videos I hid with "pandora" (its a media hider) so I thought that was messing up the data. But then my nandroid imgs where not the right hash & some times 0 bytes so at that point I ruled out pandora.

This is just an idea I have know testing to conferm it. BUT what if the problem is simpley a junk samsuck sdcard? I mean look at it, the usb cable is junk that came with the phone, froyo from them is junk, bluetooth, gps on 2.1 (if it where not for the devs hear the phone would be totaly junk to me and I would have got a new phone)

Sent from the Drivers Seat of my Suby txting and Driving doing 100MPH+ in a school zone! Ha.
[Epic 4G General] mkasick:
I think what we're establishing here, and correct me if I'm wrong, but data corruption appears to be correlated with mount events on the Epic. The more times you mount, the higher probability of encountering data corruption.

Furthermore, there's instances in which the corrupt data was observed to be OK on a PC prior to the remount. And it's independent of how much the card is actually used on the Epic between mounts.

If that's all the case, then there's definitely something suspicious about the mounting process. It's certainly not spurious corruption. It could still be "bad cards," in the sense that the mounting process is correct but tickles something in the controller. But I'm inclined to believe it may be a software-level issue afterall.

I'm still curious if the problem persists across multiple SD cards in the same device.
[Epic 4G General] rsage:
That's possible, new card is coming today so hopefully I'll have new info soon
[Epic 4G General] rsage:
mkasick, my perception of the problem lines up with your statements. Without very deliberate & detailed testing I'm hesitant to say anything beyond that because I'm not examining the hundreds/thousands of files on the card every time I mount it on a computer, and I'm not checking it 100% of the time when I remount it on the phone. Actually, lately, I have been checking the file count and size of LOST.DIR frequently, so I do have a fairly good idea of when files are being moved to that directory, but I wouldn't take my casual testing as definitive at this point.

Me too, installing a new highly-rated card tonight, should know more soon
[Epic 4G General] zman519:
The problem is back. Check out my thread for more info.

Sent from the Drivers Seat of my Suby txting and Driving doing 100MPH+ in a school zone! Ha.
[Epic 4G General] rsage:
That stinks - so you're seeing corruption from just rebooting the phone. I saw a little of that but not as much as mounting via USB. Couple questions - are you on custom ROM/kernel (which one), and do you have journaling on or off?

I'm currently up to 372 files in LOST.DIR, about 1.33gb. Largest files are around 150mb.

Edit: can we get these two threads merged?
[Epic 4G General] zman519:
Merged yep ;-) sounds good to me.

Rom = midnight rom 5.2 rfs journaling On kernel that comes with the rom bonsai 4.0.1

Sent from the Drivers Seat of my Suby txting and Driving doing 100MPH+ in a school zone! Ha.
[Epic 4G General] parasense:
To recover files on a damaged sdcard, use zar recovery. Short for zero action recovery, or something like that.

The files get corrupt because of volitility. The sdcard controler could be cheap which is why a better card solves, sandisk is rhe best and most expensive.

Thw android media scan is to blame in many case. Shutting phine down durring a process writing ro card, even one you not aware of, like background service.

Edit: also, format the partition and format the sdcard two or three times. I sugest you zero all the bits with a full format.

This type of issue is mostly fs corruption if the fat file system. If your using a journaling system, stop. It is way too much overhead for sdcard. These are volitil memory.

Deliver by Epic 4G xda app
[Epic 4G General] rsage:
Found it, will look into it, thanks

Have tried that, it didn't help

Hmmm, that goes against everything I've read and understand about file systems and journaling. Can you provide a link with more information?
[Epic 4G General] rsage:
Experiences with new sdcard

Ok, I've been rocking a new sdcard for about 24 hours. I've flashed a few ROMs, kernels, themes, shot photos and videos, copied tons of files to and from the card, & rebooted, unmounted, and mounted the card more times than I can count. I would normally have a couple dozen new files in LOST.DIR by now. Currently, LOST.DIR is clean, zero files, and I've had no data issues.

Something else interesting, I (along with many others) have never been able to consistently shoot HD video on my Epic (always got "recording failed" after a few seconds). Never was able to solve it, but that problem is gone now too. Probably due to the higher class of this card (6), or the quality of the card, or both.

I'm going to keep an eye on it, but so far so good.
[Epic 4G General] zman519:
Cool that's good to hear. What sd card did you buy?

I'm looking for a 32GB high class, but if its a lot of $ I may end up buying a 16GB.

Sent from the Drivers Seat of my Suby txting and Driving doing 100MPH+ in a school zone! Ha.
[Epic 4G General] musclehead84:
I also noticed that a lot of my sd card corruption stopped after I got a class 6 8 gb card.

Sent From My Evo Killer!
[Epic 4G General] zman519:
So you still have corruption?

Sent from the Drivers Seat of my Suby txting and Driving doing 100MPH+ in a school zone! Ha.
[Epic 4G General] rsage:
I got the same card I'm using in my nook color - 8gb class 6 transcend. I think it's going to be too small for my needs in the long run, but I wanted to get a card that I knew was high-quality to test the data corruption issue with.
[Epic 4G General] parasense:
I used to work on filesystem design years ago, but I know a thing or two about filesystems, volatile storage, and how things go wrong. But don't believe me, I'm just somebody on the internet. Here is what I have to say: The FAT32 filesystem is the most common problem people have, though not sure you were using FAT? Anyhow, most of the time the data blocks (metacdata) that points to (referrences) the actualy data (your pictures, etc) gets corrupted. That is to say the reference gets trashed up in the FAT tables where an index is kept. Other times the FAT table get corrupted in various interesting ways, and when new data is sent to save on the SDcard the regionof the FAT table is writen to again, but perhaps not completely. The basic idea is a block of storage used for the FAT table is made relevant so it's now scanned when the filesystem is traversed, but parts of that data block are still having old pointers, and things get weird.

I'm just trying to stimulate your mind here, and I'll stop going into all the scenarios.

So anyhow the same goes for journaling, SDCARDS should never have journaling enabled in my humble opinion. They might help recover files, at the cost of destroying your sdcard. All those meta data and data sync's to the sdcard are expensive, and each bit on an sdcard has a 1 billion bit-flip cycle, which means the areas of the sdcard that are written to a billion times are forever gone. A journaling filesystem will used a dedicated area of the disk for the journal, and that area is constantly being written to.

That said, my advise before about the full format of the sdcard would touch every bits at least once, and shorten the lifespan. Pretty much anything, so it's not that big a deal, but something to keep in mind.
[Epic 4G General] dito33:
In a search of the most stable settings for my phone experiencing no boot with ext4 when no journaling also even with journaling on I have no fate in ON or OFF went to stock RFS thinking this is it for me. But guess what it happened the worst ever I always was afraid of not recovering SD card with all the tools on my pc I have lost pics and videos forever. When sd card is inserted in pc calls for format but then can't be done, and phone wouldn't recognize it. All this happened when I was backing settings of AWD-EX when phone froze on a generic 16gb card that I got from my insurance replacement with the phone. I'm puzzled what's next I know of backing up but sick of it, wanting to go dumb phones maybe. So I think writing to sd as in earlier post was mentioned causes the most corruption vs mount unmount.
[Epic 4G General] zman519:
So if your sd card is FAT32 and you have jounaling ON in a rom. It dose not mean that your sd card has jjounaling turnd ON.... correct?

One more Q for you. What size do you recommend the file allocation size be for a FAT32 partiton on a micro sdcard?

I just orderd a 32GB lexar class 10 sdcard. And a few of the reviews on amazon talked about 64KB.

Sent from the Drivers Seat of my Suby txting and Driving doing 100MPH+ in a school zone! Ha.
[Epic 4G General] DroidApprentice:
Reading this has me a bit concerned. Even though I only run rooted stock with RFS and jounaling, I believe. Is the Samsung 16GB card OK or not? Sammy makes it themselves, and so far nothing bad has happened with mine. Just looking for opinions.
[Epic 4G General] rsage:
Unless this is happening to everyone with Sammy cards, probably just isolated bad batches, or even more likely just the usual failure rate with this kind of media. Judging from the sdcard media reviews on newegg and other retailer sites, even the best make/model cards have some that are DOA or fail shortly after purchase. Just my best guess...

Edit: though these cards don't seem to be experiencing outright failure, so I don't know... hopefully someone more knowledgeable can chime in...
[Epic 4G General] zman519:
How long have you ownd you epic/samsung sdcard. Mine it from late 10/2010

Sent from the Drivers Seat of my Suby txting and Driving doing 100MPH+ in a school zone! Ha.
[Epic 4G General] Anon9mouz:
My phone runs fine with no corruption at all I just looked no files

Regular 16gb SD that comes with it
[Epic 4G General] bobo9900:
Hi rsage,

I've been having this same issue on my defy. stock rom eclair with 2gb sandisk.

whenever I copy file to sdcard, after mount/unmount sdcard or restart phone, I see some of my files gone, and lost.dir folder filling up.

Tried to format several times, format with fat or fat32, just the same.

I notice when I run chkdsk from windows while phone connected to computer as mass storage, the drive sometimes forcely dismount and remount back.

Can you please confirm, does a higher quality sdcard like class 6 fix this corruption issue?
[Epic 4G General] rsage:
That's a tough one to answer definitively. A new, higher quality, higher class card seems to be alleviating these issues for me. But it could just be because my old OEM card was defective in some way. Any new high-quality card may have worked (regardless of class).
[Epic 4G General] bobo9900:
I see, glad it works for you.

Btw, after reading from htt p://code.google.com/p/android/issues/detail?id=2500

I believe the problem is on linux kernel as in comment 118.

eclair use kernel 2.6.29, froyo 2.6.32, and the problem with sdcard is not solved until kernel 2.6.34 ( see htt p://code.google.com/p/sim1/issues/detail?id=2 )

so, real solution is true gingerbread (kernel 2.6.35)

for the mean time, my only chance is new high quality sdcard.

thanks..
[Epic 4G General] mkasick:
First, SD cards typically (always?) implement wear leveling in their memory controllers, so even if the same logical space of the file system is overwritten, it's spread across many flash blocks.

Second, the fact that journaling file systems use log-structured journals is rather convenient, as incremental appends to the journal do not require block erase operations. It's true that there's more overall wear from having to encode metadata twice (once for the journal, second for metadata blocks), but the actual metadata block writes, since they're already encoded in the journal, may be delayed longer and coalesced, also minimizing erase ops. Thus for some workloads, I would expect a sufficiently-advanced journaling file system (e.g., ext4) to exhibit fewer erase ops than a non-journaled one.

I'm not sure an SD card's speed class is really related to its quality. And I wouldn't say that a faster card makes a difference for this purpose (aside from the camcorder).

Keep in mind that many folks are using the OEM Samsung cards and are not experiencing corruption. There's just probably a few bum cards out there, and simply using a different card solves the problem.

I'm not sure the problem addressed by that patch is being experienced here. For one, if the MMC controller were returning write errors, the cards should remount read-only, which doesn't appear to be the case.
[Epic 4G General] bobo9900:
Update:

I found a spare 2gb microsd samsung with adapter.

Here is my test :

A. Old 2gb sandisk

B. New 2gb samsung

Speed compare copying 400+MB 300+files

copy file from card to computer using adapter

A. 10MB/s

B. 10MB/s

No problem in both cards

copy files from computer to card using adapter

A. 0,9 MB/s

B. 5 MB/s

No problem in both cards, no corruption

copy files from computer to card attached to phone

A. 1,5 MB/s

B. 2,5 MB/s

Card "A" gives data corruption, card "B" no problems

Analysis:

When write process occur to card within the phone, android always try to use maximum speed and use RAM to cache recieved files althought the card isn't capable of that speed. At my test, card "A" real write speed only 0,9 MB/s, but when I use phone to copy files, it accept 1,5 MB/s as extra 0,6 MB/s is temporary cached to RAM while queue the writes to complete to the cards. With big files, when RAM is full, android couldn't handle it, thus gives data corruption.

Conclusion:

Fault at android kernel, temporary workaround is use card capable of write speed above 2,5 MB/s (class 4 or above)

Notes:

Those who use class 4+ cards and still got memory corruption, maybe should try to copy files to card using adapter and check the speed, maybe you got a bogus low quality card.

If corruption still occur, then my analysis was wrong.. :)
[Epic 4G General] bobo9900:
It indeed happened once, the card remount as read-only. Maybe its just glitch. I had to unmount the card from menu, turn off the phone and restart to use the card properly.

Btw, I see Epic 4G got leaked gingerbread rom with 2.6.35 kernel (EE03 based), maybe someone who still got the problem can try to update.

My defy just got leaked gingerbread rom too, strangely its kernel is still 2.6.32. Maybe some other time when it got official, as my card problem temporary fixed and I have zero experience in android rom update.

Cheers..
[Epic 4G General] mkasick:
I'm not sure this conclusion can be drawn from the above data.

For one, the filesystem itself is manipulated by the mounting computer, the card itself is presented as a block storage device.

But more to the point, the "old" card is class 2 right? Class 2 cards have a minimum sustained write throughput of 2.0 MB/s. Neither the card reader nor the phone appear to exceed this. I agree it's strange that the write throughput is different for both, yet both devices can write faster than 2.0 MB/s using a higher class card. However, as this is a single-instance of a file-system level copy operation, I'm not sure the numbers are representative of the general case.

To compare: I performed full-card, block-level zero-writes to a class 2 SD card with a USB card reader and my Epic. I had a write rates of 2.0 MB/s and 1.9 MB/s respectively, neither of which violates the card's rated sustained write throughput rate (which is a minimum anyways).

I also performed a few tests whereby I formatted the card (on the phone), mounted it on my computer (via the phone), copied nearly the card's worth of photos, mounted it back on the phone, checksummed (on the phone) the files, and verified that the checksums matched the originals. In none of the tests did I witness corruption.

Not that I'm suggesting the corruption issue is exclusively limited to "bad cards", or even that it's not correlated to lower-class cards. Rather, I'm just refuting the idea that corruption on low-speed cards is omnipresent.
[Epic 4G General] RATMdrumer:
I have a ton of files in the LOST.DIR folder on my SD Card. I always remember the folder being there, but I don't recall it containing any files when I was running 2.1. I've been on 2.2 for about 1.5 months. I noticed that all of the files in the folder were last modified Dec 31, 1979 11:00PM and seem to all be 32KB. I also noticed that the folder was last modified Jan 23, 2011 1:48PM, which seems like the exact time that I purchased/activated the phone (Sprint Samsung Transform). I can't make any sense of it.
[Epic 4G General] rsage:
The directory was probably created the first time the phone was booted. The files showing up with date/timestamp of 12/31/1979 seems normal - I think it's the OS display default since the files themselves seem to have NO actual timestamp associated with them. I also don't think the last modified date on the directory gets updated when new files are added to the directory.
[Epic 4G General] Falstafff:
This has now happened to me twice, and I've lost a lot of really important pictures and videos. Probably the single most annoying "tech fail" i've experienced recently.

Hate if you want, but the only thing I've done that I believe caused it was ... I had just previously flashed the ACS ReRFS StockEC05 rom.

Seriously. It has happened twice now. Does that even make sense? Hmmmm
[Epic 4G General] sneezy316:
This has happened 4 or 5 times. Really annoying. It is usually when an app starts to act funny, and I reboot the phone. The sdcard will then show as missing/not formatted.

I am on EI22 stock rooted with CRT and 1% bat odex mod. It has also happened on my oem card and Patriot 16 GB card.
[Epic 4G General] dmanp84:
Any updates? My phone freezes on official gingerbread only after installing apps due to the sdcard (unfreezes if i remove the sdcard or format it)...same issues as you guys...and my sdcards are not faulty, they work perfectly fine in android 2.2
[Epic 4G General] TeamERA:
Quick reboot could be the cause of your problems, I never use it.
[Epic 4G General] dmanp84:
Don't know if that's for me, but I don't use quick reboot
[Epic 4G General] dmanp84:
Copying ".nomedia" file to a folder that contained 100s of pdfs fixed my problem
[Epic 4G General] redvexx:
Its happend to me twice recently. I lost all my photos and music.

Not sure if it is related or not but it started after I flashed ACS ICS rom.
[Epic 4G General] erikserif:
Hi! I am new here but i have the same problem...

I have a sony ericsson live with walkman phone,

and a 32gb Sandisk CL6 sd card wich do the same issue what is in the thread.

Any time i copy something to my sd card from my pc it goes to LOST.DIR when its unmounted.

I have had changed the sd card at the shop where i bought it but with the new card the same happens again and again and again. :S

Do you have any ideas?
[Epic 4G General] squshy 7:
Possibly a bad card port in the phone? Even if not, it sounds like your phone (or some part of it) is the culprit.
[Epic 4G General] erikserif:
Thank you for reply!

I dont Think its possible because i bought my phone about two weeks ago but maybe. After holyday i will go to my phone supplier and discuss it.

---------- Post added at 12:07 PM ---------- Previous post was at 12:02 PM ----------

I forgot something.

I have a 2gb sd card wich given by the manufacturer and its working...

My brother have an xperia mini pro with the same 32gb Sandisk sd card and he dont get any kind of these problems.

This is strange...