yashsingh1/xdaforums huggingface dataset:
[Nook Color Android Development] polymath257:
Thanks, that worked!

Now, I would like to understand the files that are in the encore directory after a compile. The kernel

can become the uRecImg and radmdisk-recovery.ub can become the uRecRam. How about MLO and

U-BOOT.BIN? Are those, perhaps bootloader and 2ndbootloader?
[Nook Color Android Development] leapinlar:
Exactly. And kernel is also uImage and ramdisk.img becomes uRamdisk.
[Nook Color Android Development] steven676:
Not quite -- in the regular flashing process, the script is interpreted by the installer binary shipped in the same package. That binary is built from source at the same time as the rest of Android, and supports the new SELinux-enabled commands.

The problem with older recoveries is that they use kernels which lack support for storing SELinux metadata (file context labels) on the filesystem. This causes the set_metadata_recursive() command to fail, which terminates the script, resulting in the permissions on /system being incorrect.

Is there source for this somewhere online? (I have disassembled the image to get at the scripts before, but I'm lazy, and it'd be good to have the scripts up somewhere in any event ...)
[Nook Color Android Development] leapinlar:
Yes, I knew the update-binary was doing the interpreting, I was just being simple in my explanation. But good point about the kernel. Thanks.

Verygreen did an excellent job of dynamically converting the Edify script in the zip to a bash script that can be executed in his script on the fly. His source is not posted, but I can post it here so you can see what he did and maybe you would have some ideas how to modify it. And based on what you said above I would probably have to put a SELinux aware kernel in also.

BRB with the script excerpt.

EDIT: Here it is:

echo creating symlinks and updating permissions
cat META-INF/com/google/android/updater-script | grep -v '^\(assert\)\|\(package_extract_.*\)\|\(mount(\)\|\(format(\)\|\(run_program(\)\|\(umount(\)\|\(show_progress(\)\|\(ui_print(\)\|\(retouch_binaries(\)' | sed 's/^set_perm(\([0123456789]\+\), \+\([0123456789]\+\), \([0123456789]\+\), "\(.*\)");/chown \1:\2 \4 ; chmod \3 \4/' | sed 's/^set_perm_recursive(\([0123456789]\+\), \+\([0123456789]\+\), \([0123456789]\+\), \([0123456789]\+\), "\(.*\)");/chown -R \1:\2 \5 ; chmod -R \3 \5 ; find \5 -type f -exec chmod \4 {} \\;/' | sed 's/,$/ \\/' | sed 's/^symlink(\("[\/a-zA-Z.0-9\-]*"\), /export LINK=\1 ; for i in /' | sed 's/\("\)\|\(,\)//g' | sed 's/);/ ; do ln -sf $LINK $i ; done/' | sed 's/\/system/system/g' > /tmp/updatescript

bash /tmp/updatescript
[Nook Color Android Development] tdm:
The sdcard fix is in 4.4.1 yay
[Nook Color Android Development] steven676:
Oh boy. I'd complain about the ridiculous use/abuse of sed if I hadn't been guilty of similar in the past ;)

Maybe we could replace add to the sed commands in the middle of the pipeline with (COMPLETELY UNTESTED)

sed 's/^set_metadata("\(.*\)", \+"uid", \+\([0123456789]\+\), \+"gid", \+\([0123456789]\+\), \+"mode", \+\([0123456789]\+\), .*);/chown \2:\3 \1 ; chmod \4 \1/' | sed 's/^set_metadata_recursive("\(.*\)", \+"uid", \+\([0123456789]\+\), \+"gid", \+\([0123456789]\+\), \+"dmode", \+\([0123456789]\+\), \+"fmode", \+\([0123456789]\+\), .*);/chown -R \2:\3 \1 ; chmod -R \4 \1 ; find \1 -type f -exec chmod \5 {} \\;/'
[Nook Color Android Development] leapinlar:
Ok, I'll give it a try. Just add that sequence after the set_perm pipe and before the symlink pipe? And then use the kernel from the CM11 zip?
[Nook Color Android Development] steven676:
You don't need to replace the kernel -- that little fragment strips out SELinux information completely (we don't have it enabled at the moment, so it's not needed). I'm not 100% sure how we'd deal with SELinux if we ever need to (for one thing, we'd probably need to add userspace tools to the installer image as well), but let's cross that bridge when we come to it.
[Nook Color Android Development] reynoben:
I am just running repo sync again after adding the roomservice.xml file to the local_manifests directory. Did you run the breakfast command again? I didnt' see if this question was answered.

Thanks!
[Nook Color Android Development] leapinlar:
Steven, I tested it and it gave me a syntax error in the bash script when running on SD. So I tested the script in Ubuntu and captured the bash script file.

Here is the code as I put it in the SD script:

cat META-INF/com/google/android/updater-script | grep -v '^\(assert\)\|\(package_extract_.*\)\|\(delete(\)\|\(mount(\)\|\(format(\)\|\(run_program(\)\|\(umount(\)\|\(show_progress(\)\|\(ui_print(\)\|\(retouch_binaries(\)' | sed 's/^set_perm(\([0123456789]\+\), \+\([0123456789]\+\), \([0123456789]\+\), "\(.*\)");/chown \1:\2 \4 ; chmod \3 \4/' | sed 's/^set_perm_recursive(\([0123456789]\+\), \+\([0123456789]\+\), \([0123456789]\+\), \([0123456789]\+\), "\(.*\)");/chown -R \1:\2 \5 ; chmod -R \3 \5 ; find \5 -type f -exec chmod \4 {} \\;/' | sed 's/,$/ \\/' | sed 's/^set_metadata("\(.*\)", \+"uid", \+\([0123456789]\+\), \+"gid", \+\([0123456789]\+\), \+"mode", \+\([0123456789]\+\), .*);/chown \2:\3 \1 ; chmod \4 \1/' | sed 's/^set_metadata_recursive("\(.*\)", \+"uid", \+\([0123456789]\+\), \+"gid", \+\([0123456789]\+\), \+"dmode", \+\([0123456789]\+\), \+"fmode", \+\([0123456789]\+\), .*);/chown -R \2:\3 \1 ; chmod -R \4 \1 ; find \1 -type f -exec chmod \5 {} \\;/' | sed 's/^symlink(\("[\/a-zA-Z.0-9\-]*"\), /export LINK=\1 ; for i in /' | sed 's/\("\)\|\(,\)//g' | sed 's/);/ ; do ln -sf $LINK $i ; done/' | sed 's/\/system/system/g' > /tmp/updatescript

Here is an excerpt of the updater-script that it operated on and appears to be the section where the syntax error is generated:

set_metadata_recursive("/system", "uid", 0, "gid", 0, "dmode", 0755, "fmode", 0644, "capabilities", 0x0, "selabel", "u:object_r:system_file:s0");
set_metadata_recursive("/system/addon.d", "uid", 0, "gid", 0, "dmode", 0755, "fmode", 0755, "capabilities", 0x0, "selabel", "u:object_r:system_file:s0");
set_metadata_recursive("/system/bin", "uid", 0, "gid", 2000, "dmode", 0755, "fmode", 0755, "capabilities", 0x0, "selabel", "u:object_r:system_file:s0");
set_metadata("/system/bin/app_process", "uid", 0, "gid", 2000, "mode", 0755, "capabilities", 0x0, "selabel", "u:object_r:zygote_exec:s0");

And here is the excerpt of the bash script it generated:

chown -R 0:0 system ; chmod -R 0755 system ; find system -type f -exec chmod 0644 {} \;
chown -R 0:0 system/addon.d ; chmod -R 0755 system/addon.d ; find system/addon.d -type f -exec chmod 0755 {} \;
chown -R 0:2000 system/bin ; chmod -R 0755 system/bin ; find system/bin -type f -exec chmod 0755 {} \;
chown 0:2000 system/bin/app_process ; chmod 0755 system/bin/app_process

As you can see, the set_metadata_recursive appears to be where the error is. The set_metadata seems to work fine. Can you help me debug this? I don't know enough of the syntax to make sense out of it. I will keep trying though.
[Nook Color Android Development] steven676:
That actually looks correct to me -- can you provide the entire output from the script and the error you get when running it?
[Nook Color Android Development] leapinlar:
You were right, your script works perfectly. It was a compiling error on my part causing the SD syntax error. It now works and I have CM11 running on SD. I just need to clean up a few things, like getting the vold set like CM10.2. Then a little more testing and I will publish.

Thank you so much.
[Nook Color Android Development] steven676:
Heads-up for anyone building kernels

Quick heads-up: a security bug in the tidspbridge driver was reported upstream (bug report, patch). The bug in question allows a user to read and write arbitrary memory, so it's very likely exploitable for privilege escalation (root) by a local attacker on any OMAP3 device using this driver(*) (basically anything with hardware-accelerated anything on the DSP).

My backport of this patch to 3.0 is https://github.com/CyanogenMod/andr...mmit/8f1bd0c0a8447f35b00130dd1a508dd95b5323ff (part of CM10.2 nightlies starting December 6). (EDIT: You'll also want the followup patch fixing a memory leak caused by that patch: https://github.com/CyanogenMod/andr...mmit/ae15456ce30a204942e1b92267313ffdcdebc62d.) If you build your own kernels, you probably want to apply this ASAP. I'm working on getting an updated CM10.1 stable release built for us at the moment, and I'll spin an updated kernel for CM10 tonight. EDIT: A CM10.1.3.1 stable release with this fix included is now available -- please upgrade.

Thanks to Nico Golde and Fabian Yamaguchi for discovering this bug and reporting it upstream, Ivaylo Dimitrov and Pavel Machek for developing the upstream patch, and dhiru1602 for pointing out a bug in that patch.

(*) The fact that we have the DspBridge device node locked down to just allow media and system access makes exploitation of this bug somewhat more difficult -- an attacker would need to subvert a process with media permissions first. But given that audio and video codecs -- not exactly famous for their stellar security track records -- run with media permissions, that's not a huge barrier to exploitation (you could craft a video file that exploits a bug in the codec and then uses the DspBridge bug to gain root, for example).
[Nook Color Android Development] tdm:
You will want to apply this for 4.4.1... https://android-review.googlesource.com/71730
[Nook Color Android Development] leapinlar:
Ok, I just published rev8 of the SD install image which makes it compatible with CM11. It is also still compatible with all CM versions back to CM7.

Thanks to bowguy for sharing his CM11 build with me and steven676 for actually preparing the basic code to that allowed the update.

You can get it on my NC updated SD install guide linked in my signature.
[Nook Color Android Development] polymath257:
I just installed the cm-11.0 I previously compiled to the SD drive. The only comment is that the boot partition of the installer is no longer large enough for both the ROM and the gapps zip. other than that, things went smoothly.

Next, to the ROM itself. My pervious ROM was 10.1.3 so I am mostly making comparisons with that one. These are only initial

impressions.

I have the android.media stop error (haven't applied the patch to the code yet) but stopped MediaStorage and that went away.

Gapps seems to be working well.

I like the fact that the lockscreen can go into portrait mode instead of always being in landscape.

SuperSU did not want to install. It failed after a reboot also.

Bluetooth did not turn on (it was good in 10.1.3).

Under Settings>>About Tablet>>Status, the MAC address does not show up. Wi-fi is working, though.

I don't know how to turn on ART, so that is not tested.
[Nook Color Android Development] leapinlar:
Thanks for the feedback. I just enlarged the boot partition by 100MB and posted it to my thread. That should be big enough for almost any set of ROMs and gapps. It is still called rev8, but the name has been changed to say "largest" instead of "larger".

BTW, where did you get gapps for 4.4? Or are you using the 4.3 version?
[Nook Color Android Development] polymath257:
http://d-h.st/mDt
[Nook Color Android Development] :
Haven't trued any 4.4 builds on the Nook, but it should be in Developer Settings.
[Nook Color Android Development] Arbelzapf:
I'm still getting these errors.:crying: What the hell is going on here?

I was trying to gather appropriate branches for my Evo3D and might have messed something up there, but I have since uncommented everything non-Nook from my local manifest and reinited and resynced. So why isn't this working :crying::crying::crying:
[Nook Color Android Development] polymath257:
Tried it. Went into a bootloop. No happiness.
[Nook Color Android Development] peramikic:
It might be the gapps package you flashed...as far as i know only PA or Banks gapps are ART compatible
[Nook Color Android Development] bowguy:
Don't think so. Mine did the same thing with no gapps installed. I am going to try to update the recovery file.
[Nook Color Android Development] drmarble:
ART bootlooping

This is from a post by FatTire in the Nexux 7 forum:

Link:

http://xdaforums.com/showthread.php?p=47241971#post47241971
[Nook Color Android Development] polymath257:
I saw that. To use this trick, though, you need to have adb permissions ahead of time. On my SD install,

I was able to run these commands on the files in the SD card while on my computer.

So, is there any way to get the ART going?
[Nook Color Android Development] moocow1452:
It kind of runs out of space when it's extracting. Is that a just me thing from dirty flashing, or something else of interest to look at?
[Nook Color Android Development] leapinlar:
Dirty flashing? You can't dirty flash if you make a new SD. It starts fresh. If you are using your old SD with just a uRecRam update, then that is dirty flashing and it is possible to run out of room. I recommend against that, especially over an installation with older gapps.
[Nook Color Android Development] bowguy:
Is the SD installer recovery compatible with CM11? The first install works fine but I am having issues with flashing an additional zip after the first (like installing gapps later).

Thanks for all the work!
[Nook Color Android Development] polymath257:
What if I simply put the new zip file in the boot partition and then go into recovery? It certainly installs the new ROM.
[Nook Color Android Development] dr_tracker:
Have you guys checked this Gapps package???

http://xdaforums.com/showthread.php?t=2012857
[Nook Color Android Development] leapinlar:
The SD installer recovery is the same exact script as the original script on initial install, so yes it is compatible with CM11.

But I did not have the gapps file to test earlier so did not. There is a problem with the new kk gapps. It uses the same set_metadata_recursive command as CM11. It extracts the files properly, but does not set the permissions properly. However, I had no problem with Play Store running. But there could be problems with other gapps.

I will issue an update to fix that issue. Thank you.

What issues were you having? You might try installing my alternate CWM for SD from my NC Tips thread linked in my signature and try fixing permissions to see if that helps.

---------- Post added at 09:58 AM ---------- Previous post was at 09:55 AM ----------

Yes. That is also dirty flashing, as it is an update. I thought he was referring to a first install of CM11.
[Nook Color Android Development] polymath257:
OK, I was attempting to track down the issue with bluetooth. It was working as of CM-10.1.3, but is now not working.

I found the code that fat-tire and dalingrin committed to github. It seems that their changes are not in the cm-11.0 code.

Now, admitting that I have had little experience with github, what would I need to do to put these changes back into my own code? What would be the dangers? Is there a way to find out why these changes are not in the current code?

---------- Post added at 12:19 PM ---------- Previous post was at 12:17 PM ----------
[Nook Color Android Development] leapinlar:
Yes it still should work for backup and restores. It is only for installing CM11 to emmc that there is a problem with older CWMs.
[Nook Color Android Development] leapinlar:
I just published rev8a of the SD image to my NC SD install guide linked in my signature. It fixes the KitKat gapps permissions issue discussed above.

If you have already made a rev8 SD, just use my uRecRam rev8a rar in the same thread above. Just extract the archive and replace the uRecRam file already on the boot partition.

It is still compatible with all previous CM versions but it is best to use rev7 or older if installing CM7.
[Nook Color Android Development] bowguy:
I'm running out of space as well. Clean image (8a) and it runs out during gapps install.
[Nook Color Android Development] leapinlar:
What is the size of your SD?
[Nook Color Android Development] bowguy:
8GB. I expanded your image after burning from 400 MB to 2GB. It is running out of room while inflating the gapps packages. Is it using one of the other partitions during installation?

It seems to be running just fine though. Everything seems to be working except a few apps so far. Play store is running perfectly
[Nook Color Android Development] leapinlar:
It uses p2 (/system) for expanding both the rom and gapps. P1 is just for the boot files and installation zips. Leave it at 400MB. On an 8GB SD, p2 is set at 465MB, the same size as the /system partition on internal. It is possible that that is not enough room since the two files together are 306MB compressed (that is why I had to increase the boot to 400MB). Uncompressed it would be more. If it is running out of room, it would do so on emmc too, but you would not know unless it misbehaved.

I just looked at my /system. I have a 32GB card so p2 is set to 920MB, but it is using 586 MB of it with CM11 and kitkat gapps. That tells me that emmc installs may be having a problem too and not know it. Can anyone running CM11 with kitkat gapps on emmc check their /system folder to see how much room is left, if any?

I suggest you move to a 16GB card. P2 increases to 920MB on it.
[Nook Color Android Development] bowguy:
When I get a chance I will move to a 16 GB. I expanded the p1 to 2GB so I can run CM10.2 and CM11 on the same nook. CM10.2 is on emmc and CM11 is on the SD card. The 2GB on p1 allows CM10.2 to use the card off of emmc. Gives me a nice dual boot.
[Nook Color Android Development] leapinlar:
steven676,

If you read my posts to bowguy above, you will see there might be a problem on emmc running CM11 and the kitkat gapps. The /system partition on emmc is only 465MB and on my SD install my device is using 586MB on /system when I have both CM11 and kitkat gapps installed.

Could you or someone else look at your emmc /system with CM11 and kitkat gapps installed? How much room is left, if any? Users may not know there is a problem unless some apps misbehave.

If it is full, then the solution might be to make a custom kitkat gapps that is pared down so it takes less space on emmc.
[Nook Color Android Development] leapinlar:
I just looked at my code. A 16GB card still stays at 465MB for p2. You have to go to 32GB to get the 920MB. I may modify the image again to set 8 GB to something bigger.
[Nook Color Android Development] leapinlar:
I updated my SD image again to rev 8b and posted it on my NC SD install guide. I modified it to increase the size of /system on 8GB cards to 680MB and for 16GB and larger cards to 902MB. On my SD CM11 and kitkat gapps used up 586MB of my /system.

CM11 with gapps will not run on 2 or 4GB SDs since the /system is too small.

Thanks again to bowguy for testing my revision 8b on his 8GB SD since I did not have a spare to test it on.
[Nook Color Android Development] polymath257:
I just did a fresh install with my recently compiled cm-11.0 and the gapps-kk-20131119. I have 34.38 MB left in internal storage.

Yikes!
[Nook Color Android Development] leapinlar:
Internal storage (/data) or /system? And on SD or emmc? And if SD, what size?
[Nook Color Android Development] polymath257:
It is a fresh emmc install. Running du /system gives 901712, and du /data gives 195960.

The previous was for internal storage from the settings menu. Sorry for the confusion.

I just did a df /system and got 8K free: size:447.8M Used 447.8M Free 8.0K.
[Nook Color Android Development] bowguy:
Bluetooth not working on CM11

From a logcat when trying to turn on bluetooth. This is the error:

W/ContextImpl(10374): Calling a method in the system process without a qualified user: android.app.ContextImpl.startService:1510 android.content.ContextWrapper.startService:494

android.content.ContextWrapper.startService:494

com.android.settings.bluetooth.DockEventReceiver.beginStartingService:134

com.android.settings.bluetooth.DockEventReceiver.onReceive:115

D/DockEventReceiver(10374): finishStartingService: stopping service

I haven't been able to understand why this has changed from 10.2 to 11

Edit: More detail http://pastebin.com/dFkAK9DG
[Nook Color Android Development] steven676:
Er, that looks suspiciously like it ran out of disk space during the install. Can you provide the contents of /cache/recovery/last_log? Do you notice missing Google apps or strange crashes?

Also, wich Google apps packages are you using? I wonder whether that's making a difference.

That's not the interesting error -- the framework generates a lot of noise along those lines on its own. Look farther back in the logs (dmesg may also be of interest here).
[Nook Color Android Development] polymath257:
I'm using the gapps linked to in post#3585 of this thread.

There were no error messages in recovery during the install. For both zip files, it said that the install was complete.

The only crashes are the previously noted ones from android.process.media.

I'm not quite sure how to post large files behind a tag, so I'll only give the last few lines of cache/recovery/last_log:

minzip: Extracted file "/tmp/face/vendor/pittpatt/models/recognition/face.face.y0-y0-22-b-N.bin"

about to run program [/tmp/install-optional.sh] with 2 args

about to run program [/sbin/busybox] with 3 args

Cleaning up and setting metadata...

script result was [Installation complete!]

Installation complete!

Install from sdcard complete.

Rebooting...

I:605 Volume sdcard0 /storage/sdcard0 state changed from 4 (Mounted) to 5 (Unmounting)

I:631 Volume sdcard1 /storage/sdcard1 disk removed (179:48)

I:605 Volume sdcard1 /storage/sdcard1 state changed from 1 (Idle-Unmounted) to 0 (No-Media)

I:605 Volume sdcard0 /storage/sdcard0 state changed from 5 (Unmounting) to 1 (Idle-Unmounted)

I:/storage/sdcard0: Idle-Unmounted

I:200 0 volume operation succeeded

If you want more, I can provide it.

df still shows 8.0K free in system.

In regard to Bluetooth functioning, I think the relevant error from dmesgs is

Bluetooth: st register failed -22

Here is the output of dmesg|grep luetooth :

<6>Bluetooth: Core ver 2.16

<6>Bluetooth: HCI device and connection manager initialized

<6>Bluetooth: HCI socket layer initialized

<6>Bluetooth: L2CAP socket layer initialized

<6>Bluetooth: SCO socket layer initialized

<6>Bluetooth: Bluetooth Driver for TI WiLink - Version 1.0

<6>Bluetooth: RFCOMM TTY layer initialized

<6>Bluetooth: RFCOMM socket layer initialized

<6>Bluetooth: RFCOMM ver 1.11

<6>Bluetooth: BNEP (Ethernet Emulation) ver 1.3

<6>Bluetooth: BNEP filters: protocol multicast

<6>Bluetooth: HIDP (Human Interface Emulation) ver 1.2

<3>Bluetooth: st_register failed -22
[Nook Color Android Development] polymath257:
I've been comparing the source code for bluetooth support in the kernel from fat-tire and dalingrin at

https://github.com/dalingrin/nook_kernel/commit/88ada0a44229c13bcc8109909a0e0bb9a39a4458

with the source code from the repo sync with cm-11.0

There are definitely some differences. For one, there is no longer a file arch/arm/mach-omap2/board-evt1a.c. It seems to have been replaced by board-encore.c and a couple of other encore specific files. It appears, however, that the changes are included in the new files. I am still working through the details to see if everything is there, though.
[Nook Color Android Development] anapospastos:
So, you have build your image and you are ready to flash it. You 've probably have read that our old cwm 3.0.2.1 recovery or twrp cannot flash cm11 so we need the new cwm 6.0.4.5. Here is how you 'll do it

Navigate to your build folder

cd ~/android/system/out/target/product/encore

Connect your nook color with your computer via usb, in recovery mode, mount /boot from options and push the new recovery and its kernel to the alternative boot slot

adb push ramdisk-recovery.ub /boot/uAltRam
adb push kernel /boot/uAltImg

Now reboot and select the third choice, alternative recovery. You are ready to flash your fresh rom with new cwm. If you want to flash an old rom or restore a backup (I didn't try restoring with new cwm but it will probably work fine) you can always find your old recovery in the bootloader, as the primary emmc recovery.