[Android General] NYLimited:
Formatting and Troubleshooting

PARTITIONING AND FORMATTING

The many file systems available for these cards is beyond the scope of this post but a brief mention is needed, nonetheless. Likewise, deciding on read buffers and cluster sizes depend a great deal on your operating system, how the card is used, the speed of the card bus and other factors. You will have to read and experiment more if you want to get optimum performance from your card.

Because the host views the SD card as a block storage device, the card does not require MBR partitions or any specific file system. The card can be reformatted to use any file system the operating system supports such as UFS, Ext2, Ext3, Ext4, btrfs, HFS Plus, HFS Plus, NTFS, FAT16, FAT32, exFAT, etc.

Most consumer products that take an SD card will expect it to be partitioned and formatted in some way. The universal support for FAT16 and FAT32 allow the usage of SDSC and SDHC cards on most host devices. On such SD cards, standard utility programs can be used to repair a corrupted filing system and sometimes recover deleted files. Defragmentation tools for FAT file systems may be used on such cards but are generally not recommended. The resulting consolidation of files may provide a

marginal improvement

in the time required to read or write the file, but not an improvement comparable to defragmentation of hard drives, where storing a file in multiple fragments may involve a time penalty to move between physical areas of the drive. Moreover, defragmentation performs writes to the SD card that count against the card's rated lifespan.

An SD card should have a life span of roughly 10 years (which doesn't mean it always will). In theory, the more often you write/erase it the faster it will wear. Of course, replacing an SD card is fairly inexpensive these days.

The memory of a card is divided into minimum memory units. The device writes data onto memory units where no data is already stored. As available memory becomes divided into smaller units through normal use, this leads to an increase in non-linear, or fragmented storage. The amount of fragmentation can reduce write speeds, so faster SD memory card speed standards help compensate for fragmentation.

Reformatting an SD card with a different file system, or even with the same one, may make the card slower, or shorten its lifespan. Some cards use wear leveling, in which frequently modified blocks are mapped to different portions of memory at different times, and some wear-leveling algorithms are designed for the access patterns typical of the file allocation table on a FAT16 or FAT32 device. In addition, the preformatted file system may use a cluster size that matches the erase region of the physical memory on the card; reformatting may change the cluster size and make writes less efficient.

The SD Association provides free formatting software to overcome many problems described above. This software formats all SD memory cards, SDHC memory cards and SDXC memory cards. It was created specifically for memory cards using the SD/SDHC/SDXC standards. It is generally recommended to use the SD Formatter first if the card is to be reformatted by another method. Using generic formatting utilities may result in less than optimal performance for your memory cards.

Nota bene: cards greater than 32GB will be automatically formatted as exFat by this software! If you require another file system, such as FAT32, you will have to do a second format with another utility! If you need to format a 64 GB card to FAT32 you can use various utilities to get it done, including this

FAT32Format

program.

The SD/SDHC/SDXC memory cards have a "Protected Area" on the card for the SD standard's security function. The SD Formatter does not format the "Protected Area". This is genrally reserved for the use of the device, such as cameras and mobile phones.

To create other partitions, such as an "ext4" Linux formatted partition, grab a copy of MiniTool Partition Wizard. The Home Edition is a free partition manager software designed by MiniTool Solution Ltd. It supports 32/64 bit Windows Operating Systems. Functions include: resizing partitions, copying partitions, create partition, extend partition, split partition, delete partition, format partition, convert partition, explore partition, etc.

Similar to the MiniTool Partition Wizard is the Paragon Partition Manager which also has a free version you can download. This one will allow you to increase an ext4 partition's cluster size, if you wish to.

This is a very good

article on partition alignment

if you are interested in knowing more about that subject - thanks MarkAtHome!

​

The free SD Association formatting software is available

here

.

The free Minitool Partition Manager software is available

here

.

The free Paragon Partition Manager software is available

here

.

​

There are many ways you can format a card, including various recovery tools, some bootloaders, PC card readers, etc. I am not suggesting that other methods do not work or that they are bad. This is simply my preferred way of getting it done.

Let's take a 32 GB SanDisk card and set it up for some of the devices. I am using a card reader attached to my PC for these steps. If this is not a brand new card the first step I will take is format it with SD Formatter to make sure it is in optimal form. Loading the program on your PC will give you a screen like this one:

This program will create the "proper" format for your card. Note that this means a 64 GB card which was originally sold with the eXFat format will be so formatted. If your device requires FAT32 format (which many phones require) you will have to do that with another program.

Note the "Option" button! Clicking this button will let you select the format type (quick, full or erase). I generally select full although it takes quite a bit longer. I am seldom concerned with securing old data from other people. I do, however, like to have format adjustment turned on.

When SD Formatter is finished you should see something like this:

If you need only a FAT32 partition you can stop here. You are done. If your setup requires a Linux type partition also for apps or data or whatnot you keep going.

​

My ROM uses the extended partition for most everything except the Android system. The advantage of this separation is that reads and writes will deal with two different devices, hopefully making things a little faster than waiting for just one.

I prefer to use the ext4 file format which includes journaling (see writeup on file system for more info). I will accomplish the creating of this partition with the Minitool Partition Manager. When you first load the program you should see something like this image although the number of drives and their order will depend on your system.

Note that the FAT32 partition is marked as "Primary"! This was done automatically by SD Formatter but if you are using another program you must remember to do this yourself or chances are the partition will not been properly seen on your device!

Creating the second, ext4 type partition also requires it to be set as "Primary"!

WARNING!! This program requires you click the apply button in order for your selections to be completed. If you do everything and forget to click apply nothing will actually be done!!

​

As you can see, the FAT32 partition spans the entire card. The simplest way to create room for another partition is to first decrease the size of the FAT32 partition. You can select the "Partition" menu choice on top or right click the FAT32 partition and you will see a "Move/Resize" choice. Selecting it will bring you to this window:

Reduce the size of the FAT32 partition and you will see available space increase as you are doing it. I am aiming for a 3 GB ext4 partition:

Once you apply the changes you should see the unallocated space available for the second partition:

Right click the newly created space and select "Create" partition. Don't forget to set the new partition as "Primary"! Since I am not planning to create another partition (such as Linux swap) I am using all the free space on the card.

Select Apply again and you should see your second partition being created:

Once the process is completed both of your partitions should be clearly shown and, if you did it right, both will be flagged as "Primary" partitions! Now, that wasn't so difficult, was it?

A FEW WORDS ABOUT PARTITION TYPES

There is apparently a lot of information floating around on the topic of "partition types". While they are fairly well defined and documented I could find nothing specific about requiring FAT32 type "b" partitions for Android. That does not mean I could not have missed the info but one would think it should be quite easy to find if this was a blanket requirement.

I read through Tytung's git repositories (okay, kinda speed reading since there is way too much source code there for limited time reading) and I could not find any restrictions on FAT32 type, though again, admittedly I could have missed it.

As importantly, I created a FAT32 partition as type "b" on one card and another as type "c" on another card - they both worked equally well for me. Minitool Partition Wizard creates type "c" (the FAT32 default for the program) unless you change it yourself. I have apparently never used anything other than type "c" on my HD2, on Typhoon GB ROMs, and Tytung's ICS and JB ROMs. So, you got me. If it matters I have not seen that yet.

Briefly, the partition type (or partition ID) in a partition's entry in the partition table inside a Master Boot Record (MBR) is a byte value intended to specify the file system the partition contains and/or to flag special access methods used to access these partitions, such as CHS (Cylinder/Head/Sector) mappings, LBA (Logical Block Addressing) access, logical mapped geometries, special driver access, hidden partitions, secured or encrypted filesystems, etc.

It is up to an operating system's boot loader and/or kernel how to interpret the value.

Type "b" was the original (or older) WIN95 OSR2 FAT32 design. It is limited to partition sizes of 2047GB or less and it depended on the BIOS INT 13 of those PCs. Type "c" also dates back to WIN95 OSR2 FAT32 but is LBA-mapped and is considered an extended-INT 13 equivalent of 0b. Of course, both of these specs were designed for actual (hard) disk drives. According to some notes I have seen the "c" revision should be faster but there is no concrete evidence of this as far as I have seen.

By changing the partition type ID, users can prevent the system from using or initializing partitions - this is up to the operating system which will be accessing it.

According to Microsoft the following limitations exist

using the FAT32 file system with Windows

operating systems:

Clusters cannot be 64 kilobytes (KB) or larger. If clusters were 64 KB or larger, some programs (such as Setup programs) might calculate disk space incorrectly.

A volume must contain at least 65,527 clusters to use the FAT32 file system. You cannot increase the cluster size on a volume using the FAT32 file system so that it ends up with less than 65,527 clusters.

The maximum possible number of clusters on a volume using the FAT32 file system is 268,435,445. With a maximum of 32 KB per cluster with space for the file allocation table (FAT), this equates to a maximum disk size of approximately 8 terabytes (TB).

The ScanDisk tool included with Microsoft Windows 95 and Microsoft Windows 98 is a 16-bit program. Such programs have a single memory block maximum allocation size of 16 MB less 64 KB. Therefore, The Windows 95 or Windows 98 ScanDisk tool cannot process volumes using the FAT32 file system that have a FAT larger than 16 MB less 64 KB in size. A FAT entry on a volume using the FAT32 file system uses 4 bytes, so ScanDisk cannot process the FAT on a volume using the FAT32 file system that defines more than 4,177,920 clusters (including the two reserved clusters). Including the FATs themselves, this works out, at the maximum of 32 KB per cluster, to a volume size of 127.53 gigabytes (GB).

You cannot decrease the cluster size on a volume using the FAT32 file system so that the FAT ends up larger than 16 MB less 64 KB in size.

You cannot format a volume larger than 32 GB in size using the FAT32 file system in Windows 2000. The Windows 2000 FastFAT driver can mount and support volumes larger than 32 GB that use the FAT32 file system (subject to the other limits), but you cannot create one using the Format tool. This behavior is by design. If you need to create a volume larger than 32 GB, use the NTFS file system instead.

Keep in mind the above is Windows (Microsoft). Linux and Android handle things a little differently. All of the Linux filesystem drivers support all three FAT types, namely FAT12, FAT16 and FAT32. As far as I know Android should handle them the same way. Other common features that they all support are various Linux mounting options (specified with the -o option to the mount command), such as "uid", "gid", "umask", etc.

There really is so much more to partitioning and formatting that I simply cannot cover it all here, however, all the information is already available someplace on the internet. Search is a wonderful tool, everyone! :)

SOME OTHER SD CARD RELATED STUFF TO PONDER

Finding and Fixing (?) Bad Blocks

SD cards include controller circuitry to perform bad block management and wear leveling. Although there is software to find bad blocks considering the price of these cards today if you're starting to see bad blocks the device is just plain going bad. Time to get a new one before you have serious problems.

You could run diagnostics to try and find/fix it if you really want to. This can be done on your PC and a card reader or in Android Terminal:

su
fsck_msdos /dev/block/yourdevice

For ext2/3/4 partitions you can use e2fsck but it needs to run on an unmounted partition. If this is not clear you probably should not attempt it until you read a lot more, sorry.

You cannot fix bad blocks. Once a block is bad - it remains bad. Kind of like trying to fix a battery that cannot hold a charge any longer.

There are many supposed fixes claiming to do magic. None actually work. You may be able to tell the system a block is bad and it should not be used, thus avoid problems, but there is no fix.

Eventually, when your device develops too many bad blocks (around 50% of NAND) it will go to join that big phone company in the sky - time to get a new phone.

The best way to check bad blocks is to format

all

partitions in ClockWorkMod (warning: this will wipe your phone completely!) and check the recovery log when finished - after the format go to Advanced in ClockWorkMod Recovery and choose report error, this will save your log in your /sdcard/clockworkmod You can view this file with most text editors or copy/email it to your computer for viewing.

Alternately, you can try to reboot your device. When the boot process is completed (before you do anything else) load the TERMINAL program and enter the following:

su
dmesg > /sdcard/dmesg.txt

This will create a text file called dmesg.txt in the root of your card which can be viewed with a text editor. Look for the kernel name early in the file. Bad blocks, if any, should be listed not too far after the name. Or try

Lumberjack

.

Thanks to Robbie P for the reminder on a HD2/MAGLDR option:

Since we seem to be talking about bad blocks - it has been some time since this info was originally posted. For everyone who missed it (or forgot), here it is again.

The HD2 aka Leo is manufactured with 2 different NAND Flash ROM chipsets.

You can find the type by entering the tri-color boot mode.

If you see PB81120 SS-B3 on the first line that means your NAND Flash ROM is from Samsung (KBY00U00VM)

​

If you see PB1120 HX-B3 on the first line your NAND Flash ROM is from Hynix (H8BFS0WU0MCR)

​

After about one year of usage the Samsung chipset usually has a dozen or so bad clusters.

Hynix normally has none of them or just a couple.

A quick way to delete the Dalvik cache from your ext4 partition.

There are many, many ways to get this done should it be desired. One quick and easy way to do it in Android Terminal is:

su
cd /data/dalvik-cache
rm *
exit

Reboot your device and wait for the boot process to rebuild the cache.

Checking the system logs for problems

Problems with SD Cards as well as NAND and applications are often recorded in one of the Android system logs. Developers are often asking for logs so they can identify the source of a complaint. While not difficult to get, these logs are not in plain view. One the quick and easy ways to view and save the Android logs is a utility called Lumberjack.

The free Lumberjack software is available

here

.

If you prefer to use Android Terminal you can dump a lot of this info into text files which you can view later or email. The following will dump DMESG, KMSG and LOGCAT to your SD Card:

dmesg > /sdcard/dmesg.txt
cat /proc/kmsg > /sdcard/kmesg.txt
cat /dev/log/system > /sdcard/logcat.txt

Comments? Additions? Suggestions? They are all welcome.

Flame wars (relating to SD Cards or otherwise) are not. :-]