yashsingh1/xdaforums huggingface dataset:
[Nook Color Android Development] webyrd:
We're on a 10meg/sec cable connection which was also being used to stream netflix, play Halo, etc, like I said, it took a while. Last nights re sync went much faster and reported "repo synced". Thanks, will keep at it as I find time, hoping to actually be able to help one day.
[Nook Color Android Development] skwalas:
Re: [DEV] CM9/CM10 Work-In-Progress

Wow, good reason for me to not even bother. We're on satellite internet, with top speed around 4mbs. Guess I'll wait to start building.
[Nook Color Android Development] bytte:
Re: [DEV] CM9/CM10 Work-In-Progress

4 mbps will complete an initial sync overnight. Subsequent syncs will take only a few minutes, depending on what has changed.
[Nook Color Android Development] bytte:
Re: [DEV] CM9/CM10 Work-In-Progress

I'm still running an old build from early Jan, booted off an sdcard.

While browsing the web it appeared to crash and rebooted into stock B&N NC.

Powering off and rebooting proceeds as normal except that around the point where the boot animation would stop it once again crashes and boots into stock.

Investigation revealed that the /data filesystem had been badly corrupted - too bad for fsck to sort out. I created a new ext4 filesystem and all is now well.

I have no mods in kernel, and can offer no clues as to what had gone wrong.

But I thought you would like to know.
[Nook Color Android Development] leapinlar:
Re: [DEV] CM9/CM10 Work-In-Progress

What brand and class SD are you running on? I know CM10 is much more particular about quality of cards than previous CM versions were.

Sent from my Nook HD+ running CM10 on Hybrid SD
[Nook Color Android Development] steven676:
This. A naive or poorly-implemented flash translation layer combined with a crash or power loss can have catastrophic consequences for an ext4 filesystem (or any other reasonably modern FS, for that matter).
[Nook Color Android Development] bytte:
The SD card is less than 6 months old - and I've been using it for that time in this NC.

PQI MICRO SDHC CARD 8GB CLASS 10

On the reverse:

E408G1221

TFS | 120501121

I'm assuming that the booting into stock follows a panic.

When I ran the fsck it seemed that a 'block' of inodes had been trashed:

The adjacently numbered inodes had bad meta-data (invalid times, flags, cross linked data blocks - but not all the same blocks).

The OS would not run with the trashed filesystem, so I doubt that the corruption occurred much before the crash.

So the question is did the OS write that block to the flash, or was the flash returning data other than what was written. We will never know for certain.

To me it seems less likely, that a flash write failure would cause an old (unerased) block to replace the inode block, and that block would be read back and used (as opposed to the inode cache) and thus cause the crash.

My preferred alternative is that some kernel corruption occurred, which got flushed to disk, and the crash occurred when the corrupt in-memory structure was accessed.

Have any kernel memory corruption bugs been fixed in the last 3 months?

Are there any other instances of unexplained kernel panics?
[Nook Color Android Development] leapinlar:
Re: [DEV] CM9/CM10 Work-In-Progress

It does not matter how old or new the card is, it is the brand and class. Some cards don't do well with the new CM10 ROMs, especially the very newest. Definitely an off brand class 10 is suspect. Go to a SanDisk class 4 and you will have no issues.

Like Steven said, you just had a crash and it corrupted your card.

Edit: You can get an 8GB SanDisk class 4 for $8.99 at Staples right now.

Sent from my Nook HD+ running CM10 on Hybrid SD
[Nook Color Android Development] bytte:
Re: [DEV] CM9/CM10 Work-In-Progress

And that means there's a bug.

What's the point of booting into stock following a kernel panic?
[Nook Color Android Development] leapinlar:
Re: [DEV] CM9/CM10 Work-In-Progress

Or it could have meant that it could not write to the SD because of an io error and cause a kernel panic. Kernel panics lead to bootloader restarts and booting to emmc. It could be lots of things. Or it could be a bug. My bet is your SD.

Sent from my Nook HD+ running CM10 on Hybrid SD
[Nook Color Android Development] bytte:
Thanks for your detailed response.

The reason I am not using an SanDisk SDCard is because back in Oct 2011 when my NC life began, they would not work with my NC. I bought and still have 2 SanDisk 4GB Class 4 cards, bought from different US retail sources. I know there is talk of counterfeit SanDisk cards - mine are stamped (I don't know whether or not they genuine):

BH MADE IN TAIWAN

They have different batch numbers above that:

1119617348G671

1111917434G949

At the time others reported having similar problems using SanDisk Class 4 cards but not SanDisk Class 2 cards. (The discussion then was all about getting good small block write performance).

The symptoms were that when the Verygreen installer tried to populate the file systems on the card, there were numerous IO timeouts (error -110). Others saw similar behavior eg http://xdaforums.com/showthread.php?t=1297727. It wasn't just the Verygreen kernel that had problems. I installing to a Class 2 card (which was completely successful) and then dd-ed that to a SanDisk Class 4, The resultant card booted OK but experienced problems later.

I will try CM10 with the new installer on a SanDisk Class 4 and see if that is any better.

Update: The behavior is as before (fails with IO errors during mkfs). That was using leapinlar's generic-sdcard-v1.3...zip image. I have now gone through this process on 3 NC - so I suspect it is an incompatibility between the kernel and these cards, rather than the NC. (Is this using the original Verygreen kernel - is the source code available?).

That sounds disturbing: I can understand reorganization taking place as part of a write, but for the card to randomly do this without some external stimulus sound bizarre. How can you ever safely power down such a card?
[Nook Color Android Development] leapinlar:
Re: [DEV] CM9/CM10 Work-In-Progress

I agree about the counterfeit cards. I bought two bulk through Amazon mail order and they were horrible. Only buy from local retailers and then only if packaged. I recently bought some of those units from Staples that I mentioned earlier and they work great.

Edit: In my case, they were not counterfeit, they were real SanDisk, but were manufacturing rejects being resold bulk by a vendor as new. SanDisk did not stand behind them since they were rejects.

Sent from my Nook HD+ running CM10 on Hybrid SD
[Nook Color Android Development] leapinlar:
Re: [DEV] CM9/CM10 Work-In-Progress

Did you use real SanDisk cards or those suspect counterfeit cards? And I assume you meant my rev5 of the image.

And what do you mean Verygreen kernels? I was not aware that he made any for the NC. Steven is the one making the kernels for CM10.

Edit: Or did you mean the uImage kernel in the image file? I think verygreen has that on his github. Yes, here. I think I left his kernel in all of my revs. Yes, version 2.6.35, dated 5/11.

Edit2: And the mkfs command is part of the uRamDisk, not the kernel. I am not sure which version of mkfs he used. It was dated 11/30/2010.

Sent from my Nook HD+ running CM10 on Hybrid SD
[Nook Color Android Development] bytte:
Re: [DEV] CM9/CM10 Work-In-Progress

I don't know - that's why I quoted the batch nos. They came from real stores (Sears/Radio Shack) had real packaging - designed to keep you out, and had accessories - usb adaptor.

Yes

There is/was a fix around to address sdcard timing issues. drivers/mmc/host/omap_hsmmc.c around line 1533 in set_data_timeout() - override the calculation of dto with dto = 14;

I'll try giving it a go and let you know if it makes a difference.

I don't think that is the problem.
[Nook Color Android Development] steven676:
Good FTLs use journalling, a log-structured on-flash format, or other techniques which ensure that data is safe even if the card is powered down in the middle of the operation. Bad FTLs may not, or may have implemented said techniques incorrectly or incompletely.

It should be noted that many of the simpler controllers don't shuffle data for consolidation/wear-leveling purposes unless forced to, but this doesn't help you if you crash or lose power after having performed a write which triggers this process. (Also, not proactively shuffling data will cause uneven wear in the case where the storage contains both data that is rarely modified and data which is frequently updated.)

Again, you can generally get away with doing a lot of unsafe things in your SD card when subjecting it to a typical digital camera/file transfer workload, which is how the vast majority of these cards get used. It takes more complex usage patterns (lots of small files, interleaving writes to different files, mix of mostly-stable and volatile data) to expose the shortcomings of these cheapo controllers.
[Nook Color Android Development] bytte:
Are you sure - the strings in your uImage indicate that you used a 2.6.32.9 kernel.

(The wiki for verygreen 2.6.35 branch doesn't sound like its a production kernel).
[Nook Color Android Development] leapinlar:
Re: [DEV] CM9/CM10 Work-In-Progress

No I am not positive. All I know is the 5/11 version is what came with verygreen's original 1.3 zip and that is what I have included in all my revs. I assumed since his github was dated 3/11 that that is what he built from.

Sent from my Nook HD+ running CM10 on Hybrid SD
[Nook Color Android Development] nookle:
Kind of not really related to the current discussion, but is anyone else getting unbelievable battery life? I am running 03-10-2013 nightly. I am almost at 11 days and at 44%.

Also, the overclock still doesn't seem to be sticking, as it resets to 1Ghz all the time.
[Nook Color Android Development] bytte:
I incorporated the above change and (binary) patched the leapinlar (verygreen) SDcard installer. My Sandisk 4GB Class 4 card installed without error and has been running fine for the past week, whereas without this change it was unusable.

The most authoritative source I have found for this change is http://www.spinics.net/lists/linux-mmc/msg13173.html.
[Nook Color Android Development] leapinlar:
Did you just modify the kernel for the installer? If so, send me that kernel and I will test it and maybe put it in my image file for SD.

But it will make no difference for how a ROM operates once installed. The ROM puts its own kernel on the boot partition once the installer finishes.

Sent from my Nook HD+ running CM10 on Hybrid SD
[Nook Color Android Development] steven676:
Thanks for that. Looking at the discussion downthread, though, it looks like upstream decided to go another way -- http://git.kernel.org/cgit/linux/kernel/git/torvalds/linux.git/commit/?id=3bdc9ba does the same thing, except not just for OMAP MMC controllers.

Interestingly, it looks like TI ran into the same problem, because they committed a similar (but not identical) patch to the omapzoom kernels: http://omapzoom.org/?p=kernel/omap.git;a=commitdiff;h=6aabcb6. That's part of the CM10/10.1 3.0 kernel already, which would explain why your card runs fine once you get past the installer.

That said, I'll pick the upstream fix (which uses a longer timeout, which might help with really really crappy cards) into the NookieDevs kernel, and it'll probably show up in CM10 weeklies starting with next week's build. leapinlar, I don't know what kernel your installer's using, but I'd suggest taking the upstream patch as well.
[Nook Color Android Development] steven676:
The strange case of the hardware 720p video decoder

If you're running any recent CM10.1 build (or a CM10 build with http://review.cyanogenmod.org/#/c/26498/ applied -- this patch isn't in the nightlies is part of nightly builds starting with the 4/21 build), could you please try flashing the attached cm-10-encore-disable-720p-hw-decoder.zip and seeing whether Adobe Flash works? I'd also be interested in reports from other video applications for which people have been reporting problems (Netflix versions newer than 1.8.1, BBC iPlayer ...). Note: this will break decoding of higher-resolution (larger than about 640x360) H.264 videos.

Flashing cm-10-encore-enable-720p-hw-decoder.zip (also attached) will restore the original configuration (including playback of 720p H.264 video).

Some background: we have two different hardware video decoders for our devices. One is the standard TI decoder (OMX.TI.Video.Decoder), which uses only the OMAP3's DSP, and is limited (for H.264) to decoding baseline profile video at (IIRC) 640x360 resolution; the other is a decoder written for TI by Ittiam (OMX.TI.720P.Decoder) which divides decoding work between the DSP and the ARM CPU, and is able to decode H.264 baseline profile video at up to 1280x720 resolution.

Both codecs depend on (different) proprietary firmware for the DSP, but while we have source for the part of the TI decoder which runs on the CPU, the Ittiam 720p decoder we're using is a binary blob which (I think) was built for ICS. This means that we have little visibility over what happens inside the Ittiam decoder, which makes debugging problems rather difficult.

As far as I can tell, some (not all) video playing applications cause the Ittiam decoder to hang forever waiting for something to happen with a log signature similar to the one in this post. I've personally verified this is the case for Adobe Flash (even when it's attempting to load a Flash object which doesn't play video); for me, at least, removing the Ittiam decoder (forcing the TI decoder to be used) allows Flash to work. It's possible, though not certain, that Netflix and iPlayer are tickling the same bug.

I don't have much time to invest into looking further into why the Ittiam decoder is failing, but it'd be nice to know that I'm not chasing a wild goose, and even better to know whether or not the Flash, Netflix, and iPlayer issues can be traced to the same bug.
[Nook Color Android Development] bytte:
No I recompiled my kernel as well.
[Nook Color Android Development] leapinlar:
Thanks Steven, as soon as the new weekly comes out I will incorporate that kernel into a new SD installer image. The existing kernel is really old (5/11) and can stand updating.

And thanks bytte for revealing this.

Sent from my Nook HD+ running CM10 on Hybrid SD
[Nook Color Android Development] bytte:
I decided to take a look at the dto values being calculated - and found something weird going on. This is my code:

static void set_data_timeout(struct omap_hsmmc_host *host,
unsigned int timeout_ns,
unsigned int timeout_clks)
{
unsigned int timeout, cycle_ns;
uint32_t reg, clkd, dto = 0;
static int once[15] = {0};

reg = OMAP_HSMMC_READ(host->base, SYSCTL);
clkd = (reg & CLKD_MASK) >> CLKD_SHIFT;
if (clkd == 0)
clkd = 1;

cycle_ns = 1000000000 / (clk_get_rate(host->fclk) / clkd);
timeout = timeout_ns / cycle_ns;
timeout += timeout_clks;
if (timeout) {
while ((timeout & 0x80000000) == 0) {
dto += 1;
timeout <<= 1;
}
dto = 31 - dto;
timeout <<= 1;
if (timeout && dto)
dto += 1;
if (dto >= 13)
dto -= 13;
else
dto = 0;
if (dto > 14)
dto = 14;
}
if (!once[dto]) {
dev_warn(mmc_dev(host->mmc), "TIMING timeout_ns %d timeout_clks %d dto = %d\n",
timeout_ns, timeout_clks, dto);
once[dto] = 1;
}

dto = 14; // HACK

reg &= ~DTO_MASK;
reg |= dto << DTO_SHIFT;
OMAP_HSMMC_WRITE(host->base, SYSCTL, reg);
}

And this is the output:

root@android:/ # dmesg |grep TIMING

<4>omap_hsmmc omap_hsmmc.1: TIMING timeout_ns 100000000 timeout_clks 0 dto = 3

<4>omap_hsmmc omap_hsmmc.1: TIMING timeout_ns 100000000 timeout_clks 0 dto = 10

<4>omap_hsmmc omap_hsmmc.0: TIMING timeout_ns 100000000 timeout_clks 0 dto = 9

<4>omap_hsmmc omap_hsmmc.0: TIMING timeout_ns 500000000 timeout_clks 0 dto = 11

<4>omap_hsmmc omap_hsmmc.1: TIMING timeout_ns 1600000000 timeout_clks 0 dto = 14

<4>omap_hsmmc omap_hsmmc.2: TIMING timeout_ns 1000000000 timeout_clks 0 dto = 12

root@android:/ #

It makes no sense to me that the same timeout value can produce 3 different dto values!
[Nook Color Android Development] webyrd:
I don't use Flash, so can't comment on that aspect, but my observations on the video apps that I use:

with 720 enabled: Netflix 1.8.1 works fine; Netflix 2.2.0 (latest from PLAY) opens and then stops at a grey screen; Youtube works fine; MX Player works fine.

with 720 disabled: Netflix 1.8.1 works OK, had some buggy behavior (movies would play for a while but then it would kick back to the menu, others seemed to play fine); Netflix 2.2.0 opens and loads the menu, when I try to play a movie, it looks like it is beginning to buffer but then kicks back to the menu without actually ever playing any of the movie; Youtube works but will not play HD video; MX player works fine.

If logcat or DMESG outputs would help, let me know.
[Nook Color Android Development] leapinlar:
@steven676

Will you let me know when the new kernel makes it into the nightly or maybe post the kernel here so I can update my image when it is ready? Thanks.

Sent from my Nook HD+ running CM10 on Hybrid SD
[Nook Color Android Development] bytte:
ADDITIONAL DETAIL

<4>omap_hsmmc omap_hsmmc.1: TIMING clkd 240, cycle_ns 2500, timeout_ns 100000000, timeout_clks 0, dto = 3

<4>omap_hsmmc omap_hsmmc.1: TIMING clkd 2, cycle_ns 20, timeout_ns 100000000, timeout_clks 0, dto = 10

<4>omap_hsmmc omap_hsmmc.0: TIMING clkd 4, cycle_ns 41, timeout_ns 100000000, timeout_clks 0, dto = 9

<4>omap_hsmmc omap_hsmmc.0: TIMING clkd 4, cycle_ns 41, timeout_ns 500000000, timeout_clks 0, dto = 11

<4>omap_hsmmc omap_hsmmc.1: TIMING clkd 2, cycle_ns 20, timeout_ns 1600000000, timeout_clks 0, dto = 14

<4>omap_hsmmc omap_hsmmc.2: TIMING clkd 4, cycle_ns 41, timeout_ns 1000000000, timeout_clks 0, dto = 12
[Nook Color Android Development] steven676:
logcat output would be the most helpful to have here. Also, you may want to reboot between tries, as mediaserver crashes can leave the DSP in an inconsistent state.

Look at the code carefully -- you'll notice that, among other things, the value of dto depends on the bus clock speed. Also, note that the MMC controller drives three separate buses (eMMC, microSD, and the WLAN part of the wl1271), and you have messages coming from all three. Assuming it works, I wouldn't worry about it.

That said, if you haven't tested with the patch we're looking to merge in place of the one you're using right now, please do and report back with your results. Thanks!
[Nook Color Android Development] bytte:
Assuming I'm reading the Gerrit correctly, the proposed change sets the timeout to 3 sec for SDcard writes. I have replaced my global dto = 14 with that change (my source is still CM10.0 circa early Jan 2013). The 3 sec timeout generates dto = 14 and I have successfully written (and validated) 700MB to my card using the new kernel.

FWIIW the dto stats for the SDcard alone are:

<4>omap_hsmmc omap_hsmmc.0: TIMING base 0xfa09c000, clkd 240, cycle_ns 2500, timeout_ns 100000000, timeout_clks 0, dto = 3

<4>omap_hsmmc omap_hsmmc.0: TIMING base 0xfa09c000, clkd 4, cycle_ns 41, timeout_ns 100000000, timeout_clks 0, dto = 9

<4>omap_hsmmc omap_hsmmc.0: TIMING base 0xfa09c000, clkd 4, cycle_ns 41, timeout_ns 3000000000, timeout_clks 0, dto = 14

The only part that doesn't make sense is why it starts out reading with an increased clock devisor (clkd).

Thanks
[Nook Color Android Development] webyrd:
Netflix testing

Finally got around to being the guinea pig - for logcat and dmesg outputs see the attached zip file - also here:

https://dl.dropboxusercontent.com/u/20993724/4steven.zip

With 720P enabled: Netflix 1.8.1 works fine; Netflix 2.2.0 (latest from PLAY) opens and then stops at a grey screen.

With 720P disabled: Netflix 1.8.1 works OK, had some buggy behavior (buffering, freezes/dropped frames, but never did kick back to menu); Netflix 2.2.0 opens and loads the menu, when I try to play a movie, it looks like it is beginning to buffer but then kicks back to the menu without actually ever playing any of the movie.

Methodology: reboot NC; start logcat; start Netflix, if I could, play movie (Hatfields and McCoys if anyone cares, for 220d I also tried to play Soldier Blue because I don't think it's HD, did not work either); let Netflix run for a minute or two; end Netflix; end Logcat; take dmesg.

Output file naming:

lc or dm - logcat or dmesg output

181 or 220 - Netflix version 1.81 or 2.2.0 (latest from PLAY)

d or e - 720P Disabled or Enabled

Also to be considered, I did a dirty install of CM10.1 B1 (over CM10.1 B2) a few days ago trying to get SOD's again, no luck (or still lucky, no SOD's, going to try clean flash and change one setting at a time over a few days to see if I can reproduce), but have been getting the "Process System Not Responding" popup after booting that I had reported one time previously (clean flash fixed it then, but I remember someone else chiming in that they had these all the time) - all of the reboots during testing had this happen, maybe the output files show something about that problem also.

Hope this helps some.

MSBN QI0CM9PJ23L

DATE 12/25/2010

CODE OMAP3630 ES1.1

CYTTSP

OMAP3ISP REVISION 15.0
[Nook Color Android Development] steven676:
Thanks for this -- the dmesg is much more revealing than I'd anticipated it would be:

From the failure with Netflix 2.2.0 and the Ittiam decoder disabled:

<4>select 3232 (d.process.media), adj 15, size 4336, to kill
<4>select 3288 (ufou.android.su), adj 15, size 4430, to kill
<4>send sigkill to 3288 (ufou.android.su), adj 15, size 4430
<4>select 2843 (droid.deskclock), adj 15, size 3926, to kill
<4>select 3130 (gle.android.gms), adj 15, size 4240, to kill
<4>select 3205 (le.android.talk), adj 15, size 4300, to kill
<4>select 3232 (d.process.media), adj 15, size 4336, to kill
<4>send sigkill to 3232 (d.process.media), adj 15, size 4336
<4>select 2843 (droid.deskclock), adj 15, size 3926, to kill
<4>select 3130 (gle.android.gms), adj 15, size 4240, to kill
<4>select 3205 (le.android.talk), adj 15, size 4300, to kill
<4>send sigkill to 3205 (le.android.talk), adj 15, size 4300
<3>ion_share_mmap: trying to map larger area than handle has available
<3>ion_share_mmap: trying to map larger area than handle has available
<3>ion_share_mmap: trying to map larger area than handle has available
<3>ion_share_mmap: trying to map larger area than handle has available
<3>ion_share_mmap: trying to map larger area than handle has available
<3>ion_share_mmap: trying to map larger area than handle has available
<3>bridge_brd_mem_map: Failed to get VMA region for 0xfffff000 (352256)

That's strongly suggestive of memory pressure, which appears to be the reason why Netflix takes multiple seconds to start up. The ion_share_mmap messages (probably) tell us that the codec is trying to obtain a block of memory larger than the size of the ION heap (30 MB for OMAP3 devices by default). We could increase the size of this heap, but at a very steep price -- the memory has to be reserved at startup and is unavailable for any other use. (That said, crespo seems to be happy with ~350 MB on 4.1, and we have ~450 MB to work with, so there's a little room to play with here. It may be tricky to find room to expand the heap in the memory map, though.)

I don't see any new information on why the Ittiam decoder fails, though, which is annoying. If only we had source!
[Nook Color Android Development] steven676:
I landed the patches last night, so they'll be in Saturday/Sunday's CM10 weekly build.
[Nook Color Android Development] leapinlar:
bytte, you said you patched verygreen's installer with your changes and it worked with your card. How did you do that? Did you rebuild a 2.6.32.9 source with your changes in it?

Verygreen's ramdisk is not a standard ramdisk like CM10 uses, it is some sort of Linux image instead of a cpio image. And it does not seem to play well with the 3.0.8 kernel that was just built for CM10. I think it is expecting a 2.6.32.9 kernel, since it is depending on a lot of .so modules in the ramdisk.

So I was wondering what you used for your patch.

Sent from my Nook HD+ running CM10 on Hybrid SD