[Nook Color Android Development] steven676:
A sketch of the repartitioning process

As mentioned above, we're looking at repartitioning the eMMC on Nook Colors to a new layout that's more suitable for us going forward. In detail, there are at least three reasons why we'd do this:

At 440.5 MB, /system is simply getting to be too small to hold the latest bloatware from Google. (Anyone remember the days when your PC operating system could fit on a disk half that size? ... Anyone?) At the rate that Android and the Google apps are growing, it's likely that we'd have to slash and burn in the base system again if we wanted to get Android 4.4+1 (4.5? 5.0? if you work for Google and happen to know, feel free to chime in here :)) to fit into the existing space -- and after our aggressive cutting during the CM11 release cycle, there's not a lot of fat left to trim.

Neither the 1 GB/5 GB nor the 5 GB/1 GB userdata/media splits are optimal for today's Android releases. With the rapid growth of app install sizes, 1 GB for /data simply doesn't go all that far anymore, and if Google enables ART by default for future releases, we're going to need a lot of extra space to store ART's compiled code cache (the "oat" files). Meanwhile, 1 GB on /emmc doesn't fit all that much in the way of music, movies, or other content. Since Honeycomb, Google has strongly encouraged device manufacturers to use a combined userdata/media partition to solve exactly this problem.

Neither of the BN factory partition layouts is correctly aligned for flash storage. Back in the earliest days of hard disk storage, when dinosaurs roamed the Earth and the IBM PC XT was exciting new technology, hard drives were divided into "cylinders", "heads", and "sectors", which represented the on-disk geometry of the storage. The PC BIOS and operating systems expected disk partitions to be aligned to cylinder boundaries. While it's been decades since CHS geometry has had anything to do with how hard drives actually organize their data, through sheer inertia storage devices have continued to report fake CHS geometries and our partitioning tools have (until the last two or three years) continued to produce cylinder-aligned partitions by default.

NAND flash, meanwhile, is organized into "pages" (the smallest unit which can be individually read or written, usually something in the neighborhood of 8-16 KB) and "eraseblocks" (the smallest unit which can be individually erased, usually something like 4 MB). For optimal performance on flash, we want the start of each partition to be aligned to the start of an eraseblock, which reduces the number of operations spanning multiple pages/eraseblocks that need to be carried out. Trouble is, aligning the partitions to a fake CHS geometry almost always results in partitions that start in the middle of an eraseblock (and frequently in the middle of a flash page, too) -- and that's exactly what Barnes and Noble did when partitioning the Nook Color.

We can fix this when repartitioning by moving the start of each partition to line up with an eraseblock boundary, thereby exchanging a very small amount of wasted disk space in return for a large performance gain.

Repartitioning, of course, is a relatively painful process, so if we're going to do any of those things, we may as well do all of them at once. That means our plan is to end up with the following layout:

Untouched primary partitions at the beginning of the disk (/boot, /rom, recovery) -- these are infrequently accessed, even more infrequently written to, and highly risky to move, so the risks of modifying these partitions far outweigh any advantages.

A 512 MB /system partition (mmcblk0p5), aligned to a 4 MB boundary (the SanDisk eMMC parts used on older Nook Colors have 4 MB eraseblocks, and I suspect it's the same on the Samsung parts used in later devices). EDIT: verygreen pointed out we should probably leave more room for future bloat -- 640 MB? 800 MB? The size should be a multiple of 4 MB to ensure the next partition start is aligned.

A ~6 GB /data partition (mmcblk0p6) (round the size to a multiple of 4 MB to ensure the next partition is aligned)

A ~350 MB (same size as before) /cache partition (mmcblk0p7)

As for how we'd get there, here's a sketch of the process which verygreen proposed (with some contributions from Hashcode and me):

User downloads a special recovery-flashable ZIP, containing repartitioning tools and scripts

User boots into recovery

User inserts a microSD card (any kind, regular formatting) with enough free space to contain the combined contents of /system, /data and /emmc (up to 6.5 GB, though less if there's existing free space on those partitions)

User instructs recovery to flash the ZIP

Flash script makes some sanity checks: existing eMMC partition layout matches one of the known layouts, battery above a certain percentage or charger plugged in, SD card has enough free space to store backups of /system, /data, and /emmc

Flash script backs up /system, /data, and /emmc to SD card (probably using tar)

Flash script sets a flag on SD card to indicate that backup is complete

Flash script deletes existing partitions 5-8 (/system, /data, /cache, /emmc) and creates new partitions 5-7 (/system, /data, /cache) with new sizes and alignments

Flash script formats /system, /data, and /cache

Flash script restores /system and /data from backups

Flash script restores the contents of the /emmc backup to /data/media (the location used for media storage in combined userdata/media setups), setting file permissions as appropriate

Flash script removes backups-complete flag from SD card

Flash script removes backups from SD card

For initial installs and/or people who don't have a recovery on eMMC, we'd also provide an 8 GB bootable SD card image with the recovery and the flashable ZIP.

This process should be completely safe -- data is always stored somewhere safe throughout the process, and we should be able to recover from a crash at any point during the process. If the device crashes or runs out of battery in the middle of the process, the user should just reboot into recovery, insert the microSD with the backups, and flash the ZIP again; the crash recovery should work as follows:

If backups were not complete at the time of the crash, the flag indicating backups are complete won't exist. Flash script will start the process over from the beginning, wasting time but not losing any data.

If backups were complete at the time of the crash, but repartitioning wasn't complete: the backups-complete flag exists, but the partition table matches an old layout. Flash script will resume the process with deleting/recreating partitions. (The partition table is a single sector and only written to once during the process, so it should not be possible to end up with the partition table in an inconsistent state.)

If backups and repartitioning were complete at the time of the crash, but formatting and data restore wasn't complete: the backups-complete flag exists, the partition table matches the new layout. Flash script will resume the process at filesystem formatting -- again, possibly wasting time, but not losing any data.

If backups, repartitioning, and restore were complete at the time of the crash, but cleanup wasn't complete: the backups-complete flag is gone, the partition table matches the new layout. Flash script will refuse to do anything (the process is essentially complete), and the user can remove the remaining backups by hand.

It should be possible with a little scripting magic to detect which of the layouts is in use (easy to do -- check for the existence of /dev/block/mmcblk0p8) and set up the storage configuration appropriately on boot, so it would be possible to use one ROM with either layout and migrate to the new layout without reinstalling. That said, we expect to want more than 440 MB on /system for Android 4.4+1, so we would likely drop the compatibility code in that release and assume the new layout.

Thoughts? Have we missed anything?