yashsingh1/xdaforums huggingface dataset:
[Samsung Epic 4G Touch] garwynn:
Possible if #2 or #3 was CWM based on an ICS kernel. Then again you may have gotten lucky.

If you can try a stock ODIN One-Click (GB version preferred) and see if goes through. If you get stuck at /system every time you're probably affected. (According to what I've read on the Note's issues this seems to be the partition affected.)

What we've found is that the kernel that CWM or recovery is based on appears to be how you can tell. ICS builds have a new erase feature that doesn't agree with the eMMC and causes it to lock up (freeze) - GB didn't have this as it wasn't introduced until HC.

a) At the moment replacement is the way to go. You can temporarily repartition to restore use but even the guy who has made the custom PIT file (hg42) is not suggesting this for long term use.

b) This seems to be the sticking point right now because Samsung has told others that it won't do so on custom builds. If you had done this on pure stock without CWM you'd probably have an easy case.

What most people have been doing with the E4GT is flashing what they can of a stock build and then taking it to Sprint and asking them to exchange/replace - and then using insurance as a fallback. Don't know what options you have but might help.

And one last thing - if you're on the AT&T note you can ignore this post. Every sample we saw said the I717 is on the new eMMC revision and should not be able to be affected by what we're discussing here.

Hope that helps!
[Samsung Epic 4G Touch] niko26:
Thanks a lot for your help, garwynn! :)

Is there any other way how to verify whether any blocks have been affected?
[Samsung Epic 4G Touch] garwynn:
If it is the bug poking around in the affected area will cause the device to freeze.
[Samsung Epic 4G Touch] monoko:
(don't do this if you are beginer)

in terminal or adb as root

dd if=/dev/block/mmcblk0 of=/dev/null bs=1024 & watch -n 1 kill -USR1 $!

It will read whole emmc till the end. Watch will update you about the progress every sec at the end you need to press CTRL-C to kill watch.

(hope mmcblk0 is on note the emmc).

As beginer

You boot into CWM and run backup. It will finish or lockup (if it finishes you likely ok, if it fails, I would sugest the dd if=.... .
[Samsung Epic 4G Touch] niko26:
Do you mean poke as on the Commodore/CBM C64's PEEK and POKE? ;)

In other words.. is it sufficient to read the blocks (PEEK(x)) or do I need to write the blocks (POKE(X))?

Here is the funny thing, I already tried dd'ing, it went through fine. However, CWM and backup does *not* finish.
[Samsung Epic 4G Touch] monoko:
I din't see yet the bug myself.. but All reports in this thread are saing if you Access invalid block it will freeze. So if dd is ok (till end of the device), thant I would expect you don't have the corrupted flash internals

about backup freeze I would guess some other corruption. Kernel should complain. You can try to triger the problem an look at log

tar --exclude=sys --exclude=proc -cvf /dev/null /

(backup of / to /dev/null (except /sys /proc )

and check log with

dmesg

tested on i777
[Samsung Epic 4G Touch] niko26:
Hm, I didn't get any error report. Still don't understand why backup in CWM aborts with a reboot of the device :(
[Samsung Epic 4G Touch] monoko:
Unless someone responds, I don't thing you affected and we should not continue in this thread, as well I have no idea why is cwm crashing on you. I can try to help you, but I would do some sort of "SAFE ERASE" and/or REFLASH. And continue somewhere else as in this thread. Be careful about the ERASE.
[Samsung Epic 4G Touch] garwynn:
Did you try using an ODIN one-click instead of CWM? I'll bet that will get you back to a safe point - then reload CWM and try again.
[Samsung Epic 4G Touch] sfhub:
Updated the first post with select posts from our discussion that detail the call path in the source code of how the problem occurs and also the discussion on where a workaround can be placed and the pros/cons of each. Easier than reading 40 pages.
[Samsung Epic 4G Touch] musashiro:
i'm reading posts on twitter and stumbled upon samsung support page on facebook.

i tried to explain the situation and please correct me if i stated wrong facts.. i tried my best and i will post later the whole conversation.

they reply quick!

edit: too bad, they only support US and ICS is not available there.. now they are asking my location so they can redirect me to proper support team.. lols... i tried to reason that they should use the information to patch the bug on US release...

im not sure if my "little" attempt contributed anything to the issue but here is the full conversation, please do correct me if i stated anything wrong and i will message it to them..

ME

16 minutes ago

Samsung Support

13 minutes ago

ME

10 minutes ago

Samsung Support

5 minutes ago

me

about a minute ago

Samsung Support

sorry if i failed to go technical regarding the bug itself, but i gave a link to here... my apologies..

by the way my name is Ken but i'm not in any way related to Ken Sumrall :D
[Samsung Epic 4G Touch] sniperbr0:
I have a question that I dont know if it has been answered, if we disable the MMC_CAP_ERASE flag, then wiping data as we do when we switch roms will never happen, right? how do we wipe then?
[Samsung Epic 4G Touch] Entropy512:
The full "wipe" offered by ERASE is only needed for privacy concerns, it's unnecessary for flashaholics - all a flashaholic needs is a format. (Which is all Gingerbread recoveries offered.)

(This is why Google added the MMC erase stuff to ICS recovery - better for privacy because it truly erases the data instead of just clobbering the data structures that point to it.)

In theory, if ERASE were working, you'd get better write performance to a filesystem after a full ERASE - but right now disabling MMC_CAP_ERASE makes you as good as Gingerbread, but not worse.

Mr. Sumrall's recommendation of replacing the wipe functions with zero-overwrite was only for those who want to ensure that wiping is as good for privacy as the ICS ERASE-based method - but if you're not prepping the phone for sale/return there is no need for this.
[Samsung Epic 4G Touch] sfhub:
So I wanted to answer the question of whether "nandroid backup/restores" using ICS-based Recovery are safe.

Nandroid backups are definitely safe.

Nandroid restores have the same issue as wipe data/factory reset. Namely they will eventually call make_ext4fs() (from libext4_utils.a) when restoring *ext4* partitions (that means /system /data /cache).

For "images" like zImage, they do not call make_ext4fs() and I verified they don't actually erase anything when restoring images on our phone.

The conclusion is that when you restore, using an ICS-based Recovery compiled and linked against existing unpatched ICS libext4_utils.a, the boot.img is restored safely, but /system /data and /cache are NOT restored safely.

So right now, if we were NOT to make any changes to address the issue, feel free to *create* your backups using ICS-based CWM Recovery. However if you want to *restore* your backups, flash a GB-based CWM Recovery, boot into the GB-based CWM Recovery, then restore your CWM backup. This should allow you to backup and restore safely w/o needing to wait for any devs to fix things.

This is the call trace I used to come up with this conclusion (this is all pseudo-code with lots of stuff eliminated to aid readability):

nandroid.c

nandroid_restore()
nandroid_restore_partition(/boot)
nandroid_restore_partition(/system)
nandroid_restore_partition(/data)
nandroid_restore_partition_extended(/cache)

nandroid_restore_partition()
if (vol->fs_type == "emmc") {
format_volume()
restore_raw_partition()
return 0
}
return nandroid_restore_partition_extended()

nandroid_restore_partition_extended()
if ("*.ext4.tar") {
backup_filesystem = filesystem;
restore_handler = tar_extract_wrapper;
}
format_device()

tar_extract_wrapper()
cd FILESYSTEM_PATH ; tar xvf FILE.ext4.tar

roots.c

format_volume() [for emmc case]
format_unknown_device()

extendedcommands.c

format_unknown_device() [for emmc case]
erase_raw_partition()

format_device() [for ext4 case]
if (strcmp(fs_type, "ext4") == 0) {
reset_ext4fs_info();
int result = make_ext4fs(device, NULL, NULL, 0, 0, 0);
if (result != 0) {
LOGE("format_volume: make_extf4fs failed on %s\n", device);
return -1;
}
return 0;
}

flashutils/flashutils.c

erase_raw_partition()
switch (type) {
case MTD:
return cmd_mtd_erase_raw_partition(partition);
case MMC:
return cmd_mmc_erase_raw_partition(partition);
case BML:
return cmd_bml_erase_raw_partition(partition);
}

restore_raw_partition()
switch (type) {
case MTD:
return cmd_mtd_restore_raw_partition(partition, filename);
case MMC:
return cmd_mmc_restore_raw_partition(partition, filename);
case BML:
return cmd_bml_restore_raw_partition(partition, filename);
}

mmcutils/mmcutils.c

cmd_mmc_erase_raw_partition()
return 0;

cmd_mmc_restore_raw_partition()
mmc_raw_copy()

mmc_raw_copy()
while ( ( ch = fgetc ( in ) ) != EOF )
fputc ( ch, out );
[Samsung Epic 4G Touch] mobilevil:
does it has anything do with TRIM command? which affect performance.

or the TRIM command is working fine ?
[Samsung Epic 4G Touch] sfhub:
I also looked into whether it made sense for CM9/AOKP to package their update.zip with the "safe" EL29/GB update-binary. It turns out their Edify install script uses some extended functionality introduced in the ICS source. Basically their updater-script is not comptiable with a GB update-binary.

So are they out of luck? I believe with a "simple" change to their install script, they can make their ROM update.zip files safe for flashing in ICS recoveries. Basically just replace the format("/system") with delete_recursive("/system"). This will ensure make_ext4fs() is not called and thus the ioctl() in wipe.c will never be called.

*** updater-script-cm9 Thu Feb 28 20:33:00 2008
--- updater-script-cm9-new Fri Jun 01 07:44:33 2012
***************
*** 9,17 ****
set_perm(0, 0, 0644, "/tmp/backuptool.functions");
run_program("/tmp/backuptool.sh", "backup");
show_progress(0.500000, 0);
! unmount("/system");
! format("ext4", "EMMC", "/dev/block/mmcblk0p9", "0");
! mount("ext4", "EMMC", "/dev/block/mmcblk0p9", "/system");
package_extract_dir("recovery", "/system");
package_extract_dir("system", "/system");
symlink("Roboto-Bold.ttf", "/system/fonts/DroidSans-Bold.ttf");
--- 9,15 ----
set_perm(0, 0, 0644, "/tmp/backuptool.functions");
run_program("/tmp/backuptool.sh", "backup");
show_progress(0.500000, 0);
! delete_recursive("/system");
package_extract_dir("recovery", "/system");
package_extract_dir("system", "/system");
symlink("Roboto-Bold.ttf", "/system/fonts/DroidSans-Bold.ttf");
[Samsung Epic 4G Touch] Esoteric68:
Samsung really needs to hire you, garwynn and Entropy.
[Samsung Epic 4G Touch] Entropy512:
Hmm. CONFIG_MMC_DISCARD is enabled on I9100/I777 Gingerbread kernels, which is why I said that MMC_CAP_ERASE was enabled (MMC_CAP_ERASE is #ifdefed based on CONFIG_MMC_DISCARD) - I'll look elsewhere tomorrow to see if maybe the "chain of doom" was broken elsewhere.

It's a lot easier to find stuff when you know what you're looking for. :)

I believe that still, removing MMC_CAP_ERASE from mshci.c is guaranteed to render you safe - but even if it's enabled, there are other places in the kernel where the chain could be broken.
[Samsung Epic 4G Touch] sfhub:
I am not surprised I9100/I777 source tree differs from E4GT in some areas. I saw a bunch of #ifdef CONFIG_MMC_DISCARD_MERGE code which indicates to me they were (for our tree) in the process of merging the discard code so it probably could have gone either way depending on when GB for a particular platform was released.

It is easier to search when you know what to search for :) In this case I wasn't starting with MMC_CAP_ERASE though.

I traced top-down from the wipe.c ioctl() from userspace to kernelspace and eventually down to mmc_erase() through the block interface indirection layers. Then I traced from mmc_erase() upwards to see who else might be making calls to it. It was all a bit messy because of the CONFIG_MMC_DISCARD_MERGE code and the resulting multiple definitions of functions.

I kept wondering why it seemed like the ioctl() function was seemingly left undefined by the ifdef's, then realized the function table wasn't populated thus the ioctl() functionality was essentially disabled for the mmc driver (on our platform) I tried to stress a few times above this was based on our platform and that I wasn't sure about others.

Most likely that is correct.

My goal in the previous post was not necessarily to determine where to do a workaround in the kernel, but to answer the question of why the "unsafe" update-binary included with our CM9/AOKP installs didn't trigger mmc_erase() (and expose themselves to the EMMC lockup/superbrick) when run on a GB kernel/recovery which I thought had mmc_erase() enabled. I found that for our platform, the most immediate reason was the ioctl() was disabled.

The difference in the CONFIG_MMC_DISCARD ifdef status in E4GT and I9100/I777 might end up resulting in slightly different bricking behavior when coupling an "unsafe" update-binary with a stock GB kernel/recovery.

I say "might" because I am not sure if the issue is with ERASE, TRIM, SECDISCARD, all three, or some combination of the three. It is possible only SECDISCARD is problematic but ERASE and TRIM are ok because the SECDISCARD code wasn't even in the GB source (for our platform) so it would be something new introduced in ICS (and also we can see it is being used in wipe.c). It is also possible they are all bad.
[Samsung Epic 4G Touch] garwynn:
Then I would have to leave my current job, which I can say is the only job in 17 years that I love. The biggest reason are the managers and coworkers. Without the people and mindset in that office I could easily see a polar opposite of working there.

It is honestly a miracle how I've had opportunities that I did... Much like getting a response from Samsung, Sprint and Google on inquiries with no previous working relationship. :D
[Samsung Epic 4G Touch] _yupa_:
A simple question: how do deleting a large file is suspected to trigger the bug? And how much large the file have to be?
[Samsung Epic 4G Touch] sfhub:
I don't think I ever stated that and I haven't found code which can support that statement with the bug as we understand it.

There may be a separate bug that supports the statement you are saying, but it isn't the bug we are dealing with here, though I really think we would have heard more complaints if such a bug existed.

Perhaps you are confusing with the 32KB zero bug. I don't know if we are afflicted with that bug. fwrev 0x25 definitely is and possibly we are too (which is basically what Mr. Sumrall assumed), but that one would likely end up corrupting data in a recoverable manner (as in you might lose data, but you can reformat/reflash and things should be ok), rather than a superbrick.
[Samsung Epic 4G Touch] RustedRoot:
Once again, SF, you do a remarkable job of clarifying a nettlesome and thorny issue. Thanks my friend.
[Samsung Epic 4G Touch] musashiro:
if i understand correctly, flashing CM9 or AOKP based roms over ICS kernel can trigger the bug..

while flashing directly from GB is safe...?

i tried CM9 on GB rooted and came back using nandroid backup just fine...
[Samsung Epic 4G Touch] sfhub:
It is not flashing them "over" ICS kernel per-se that triggers the bug.

It is running an ICS-based update-binary while in an ICS kernel and having that update-binary process a "format()" command in the Edify install script.

I think that is what you meant, but "flashing over" could have different meanings and I wanted to clarify that point.

Once the CM9 and AOKP get rid of the format() command in their Edify install scripts, then the above scenario should be safe.

Yes, that is correct. Even with the ICS-based update-binary and the format() currently in the Edify install script, if update-binary is running under a GB-based kernel, the problematic "erase" code initiated by make_ext4fs() will be blocked by the GB kernel, so no issue arises.

Thanks for trying that. The behavior you describe is as expected. When you flashed CM9 on GB rooted, the GB kernel blocked the CM9 update-binary from making the problematic call when the install script asked to format("/system") so that is why there are no issues.

Then when you "came back" using a nandroid backup, that is being handled by Recovery. Assuming the Recovery was compiled against GB source, which I believe the Recoveries you get from here have, then it will Recovery will not make the problematic "erase" call when it restores, even though the restore process is calling make_ext4fs(). The GB source for make_ext4fs() just doesn't try to erase at all.

BTW may I ask which Kernel/Recovery did you use to do your Nandroid restore?

The only "Recovery" I believe you need to worry about is CWM Touch fake-flashed onto an ICS kernel, as I believe (but have not confirmed because I can't find source code) that Recovery has been compiled with an ICS version of make_ext4fs().
[Samsung Epic 4G Touch] musashiro:
i tried CM9 long ago, and i never knew it would contribute anything..

before trying CM9, i messaged XplodWild if nandroid restore is safe and he boldly said "Yes". i trusted him and i tried CM9 for a day..

the recovery that i used is the one (if im not mistaken) that is included in the CM9 safe kernel. flashing their CM9.zip includes the kernel as stated by Entropy on cm9 thread
[Samsung Epic 4G Touch] sfhub:
I didn't even realize you were talking about Note until you mentioned some unfamiliar names and I looked at your sig.

My answer was based on E4GT.

To answer based on Note and your newly provided info, if it was a "safe" ICS kernel with a Recovery compiled against ICS sources, then your Recovery would have called the problematic make_ext4fs() when you did the Nandroid restore, but the erase command would have been blocked by your "safe" ICS kernel, thus your nandroid restore worked as expected.

Same result, but the in-between happenings were different.
[Samsung Epic 4G Touch] Entropy512:
That's bringing back something I suspected long long ago back when we knew far less about this bug, but I'm still not sure... Obviously a userspace-triggered erase operation will fire ERASE commands at the card - but what about TRIM (which is a variant of ERASE) - is it blocked by another mechanism? (I suspect so but I'm not entirely sure.)

As to musashiro's questions - the CM9 kernel for N7000 has MMC_CAP_ERASE removed, which prevents attempts to erase from ever reaching the eMMC chip.
[Samsung Epic 4G Touch] temporarium:
What about using a different file system?

Apologies if this was already tried/suggested, but since we're dealing with a Gnu/Linux kernel, and if setting up the ext4 file system triggers the bug in the eMMC firmware, what about using a different one? (See en.wikipedia.org/wiki/Flash_file_system)
[Samsung Epic 4G Touch] Entropy512:
It has nothing directly to do with ext4 itself - it's just that recovery was modified such that it would issue ERASE commands prior to formatting the partition for privacy purposes.
[Samsung Epic 4G Touch] sfhub:
The way I answered that question for E4GT is to observe that ERASE, TRIM, DISCARD basically essentially all do the same mmc operation but pass different arguments to the mmc command. I believe in the ICS source tree they all get filtered down to mmc_erase() so if you should do something to short-circuit mmc_erase() (like not setting MMC_CAP_ERASE in the host capabilities) TRIM and DISCARD will get blocked also. They won't be blocked explicitly, but since mmc_erase() checks the host's ability to do MMC_CAP_ERASE first, it will return with EOPNOTSUPP because presumably if your host doesn't support ERASE, it wouldn't support TRIM and DISCARD since ERASE is the older base functionality.

In the case of the E4GT GB source tree the only references I saw were having it called from userspace through the ioctl(). Since the ioctl() is disabled in our GB build, there is no way to eventually reach mmc_erase(), thus TRIM and DISCARD should never be reached.

Since our build has CONFIG_MMC_DISCARD disabled, the discard code isn't even compiled. I believe the TRIM and ERASE code is still there, but since the ioctl to switch to kernelspace to run mmc_erase() isn't present it won't ever get run.

I still am not sure if ERASE, TRIM, and DISCARD versions of erase are all borked or some smaller subset are the issues.

So to answer your question, if you have blocked mmc_erase() as you described earlier, by not enabling MMC_CAP_ERASE in the host capabilities, based on the generic ICS tree, that should block TRIM and DISCARD as well. I haven't looked in the Samsung specific source tree though.
[Samsung Epic 4G Touch] sfhub:
I'm pretty sure the wipe code that was added between GB and ICS only gets called when creating ext4 filesystems. However I didn't specifically check if the code paths for the other filesystems would reach some variation of the wipe code.

It is probable the other filesystems aren't doing any wipes so you could theoretically change the FS and the bug wouldn't be triggered. You'd have to change all the config on your ROMs/kernels/recovery yourself to use a different FS though and it would be more intrusive than just making sure your recovery, update-binary, updater-script, and in some cases recompiled kernel are safe.

To be clear, it isn't a problem with the ext4fs filesystem itself. It is that the userspace "utility" function make_ext4fs() (this is not part of the kernel filesystem code) eventually tries to erase the partition and this leads to a bad reaction with EMMC firmware that has the lockup/superbrick bug. make_ext4fs() could be renamed joes_erase_ext4_partition() for all we care. It is just some arbitrary utility function being used by Recovery and update-binary to create an ext4fs() filesystem. Don't let the ext4fs() lead you into thinking it represents some problem with generic ext4fs code in the kernel.
[Samsung Epic 4G Touch] sfhub:
Entropy, do you recall how it came about that deleting large files was a possible suspect?

Was it just a theory or was there specific code people were looking at that led to that belief?

For completeness purposes I would like to research if this possibility exists.
[Samsung Epic 4G Touch] blackhorses777:
im curious if its a possibility also. i regulary delete large files off my sd card and ive been using a ICS kernel for a couple of months now.
[Samsung Epic 4G Touch] sfhub:
Without looking at source code, my guess is you would hear of more bricks when people cleared out files if this was a likely scenario.

I don't like to completely rule something out until/unless I look over the source code to verify though.

Clearly the wipe data and format from ICS-based update-binary are the most likely scenarios where people see superbricks as the boards are littered with anecdotes for those two.
[Samsung Epic 4G Touch] Entropy512:
Well yeah - if MMC_CAP_ERASE is removed it's safe. I'm just trying to figure out whether there's a possibility the stock N7000 ICS kernels might fire an ERASE at the chip in scenarios other than recovery.

And as to where this theory came from - it's just a theory coming from the fact that TRIM will eventually hit the same code path (mmc_erase() ). However there might be some other aspect of the filesystem code that keeps the TRIM code from ever being executed in the first place.

AAAAH found it - Mr. Sumrall briefly mentioned this but I think we were all thinking about it having an effect on recovery, not something else:

http://sites.google.com/site/lightrush/random-1/howtoconfigureext4toenabletrimforssdsonubuntu

To enable TRIM, the filesystem must be mounted with the "discard" option. No initramfs I know of does this - so TRIM commands will never be issued. Deletion of large files should be safe.
[Samsung Epic 4G Touch] jgaviota:
It would also be extremely useful if someone could get Samsung to release the errata for the emmc firmware 0x19 there should be all the info needed to know exactly how the bug can be triggered.