Reduced capacity on Sandisk MicroSD cards - Page 1
Forum section: electronics / general

author: Delta
author_rank: Super Contributor
timestamp: November 04, 2016, 01:37:00 pm
content: I've been using 32GB Sandisk cards (Sandisk Ultra microSDHC UHS-I, class 10, to be precise) for my Raspberry Pi projects, and once I had everything set up nicely I took a backup image of the SD card and use that same image for each new project.I had been buying the cards as and when needed off-the-shelf in my local Tesco (large supermarket chain) for £14 a pop, and hadn't had any problems.Then I bought one from eBay (just under a tenner) from an "Official Sandisk eBay partner" in the UK. I noticed the packaging was slightly different, which raised suspicions, and then when I tried to write the image to the card I found its capacity was 810MB less than the cards I had been using!I opened I request on eBay, and fired off a message to the seller explaining that this item was a fake.Next time I was in the supermarket, I bought a "proper" card, and the packaging seemed normal (the same as previous good cards bought there), yet when I got it home - exactly the same! 810MB smaller than the old cards!So either Sandisk themselves have reduced the capacity, or fakes have found their way into the mainstream supply lines...(I've shrunk my image so I can use these new cards, but I'm still a bit perplexed and pissed off with the whole thing)

author: kripton2035
author_rank: Super Contributor
timestamp: November 04, 2016, 01:44:16 pm
content: IMHO your supermarket also bought fakes ...

author: hans
author_rank: Super Contributor
timestamp: November 04, 2016, 02:38:35 pm
content: https://dashcamtalk.com/forum/threads/two-32gb-sd-cards-different-capacities-after-formatting.11690/QuoteThat could be a logical explanation. Still doesn't take away the fact that a new card should be 100% rated capacity and not have bad sectors affect the capacity in orders of magnitude like this.

author: rstofer
author_rank: Super Contributor
timestamp: November 04, 2016, 02:38:54 pm
content: Have you checked for a hidden partition? Maybe try fdisk?

author: Delta
author_rank: Super Contributor
timestamp: November 04, 2016, 02:52:50 pm
content: Quote from: rstofer on November 04, 2016, 02:38:54 pmI thought then when you wrote an image to a card, you are working at a level lower than the partition table, so this couldn't be an issue? (For example, my image contains two separate partitions, so the image must incorporate the partition table)

author: nfmax
author_rank: Super Contributor
timestamp: November 04, 2016, 02:57:53 pm
content: In my experience the exact capacity of flash thumb drives and SD cards is variable, even from new. The flash devices used aren't perfect and they expect a few bad blocks, which are mapped out by the device firmware. The stated capacity is the maximum value. On the other hand SSD drives seem to state the minimum capacity, keeping their spare blocks in reserve. As they are higher performance devices, they tend to have extra blocks dedicated to buffereing and caching anyway.Of course there are fakes, which may have sizes bearing no relation at all to the stated capacity

author: rstofer
author_rank: Super Contributor
timestamp: November 04, 2016, 03:37:24 pm
content: Quote from: Delta on November 04, 2016, 01:37:00 pmThat's true when using 'dd' but I'm not familiar with the program you seem to be using.I just looked at an fdsik of an SD card and when I use 'p' to print the partition table, I get a LOT of information on the physical disk size. It might be fun to compare two 'identical' SD cards and see what shows up.My 8GB card shows 7,948,206,080 bytes or 15,523,840 sectors of 512 bytes. fdisk shows this as 7.4 GiB which is short of even the disk industry's opinion on the size of a gigabyte.You are right about /dev/sdc being the entire SD card. My only experience with writing images to drives is via 'dd'. I do a lot of that kind of thing when I make CF images of IBM1130 drives, a completely unintelligible file system as far as a PC is concerned. But as long as I have the image in a file, 'dd' will write it.

author: Delta
author_rank: Super Contributor
timestamp: November 04, 2016, 03:45:18 pm
content: I'm using "Disks", the Gnome disk utility. I'd hazard a guess that it probably usesCode:ddto do its stuff.As I've now had two cards that were both 810MB too small, I think it's too much of a coincidence to be caused by random bad blocks / flash cells...I might contact Sandisk to see what they say...

author: Ampera
author_rank: Super Contributor
timestamp: November 04, 2016, 03:49:49 pm
content: Quote from: rstofer on November 04, 2016, 03:37:24 pmI rarely use FDisk unless I am working with MS-DOS on an 80486. GParted is honestly the only functioning partition manager. I don't think there is a partition it CAN'T manage (Except Windows Dynamic Volumes, the bane of my bloody existence).I've often seen stuff like this where I can have a 1TB drive, and get 9XX of storage on it, it's confusing but it has to do with drive formats and allocation and whatever it is.

author: Delta
author_rank: Super Contributor
timestamp: November 04, 2016, 03:58:09 pm
content: Here's a brand new card straight out the packet from eBay. It is 810MB too small to take the original image,Code::~$ sudo fdisk -l /dev/sdcDisk /dev/sdc: 29 GiB, 31104958464 bytes, 60751872 sectorsUnits: sectors of 1 * 512 = 512 bytesSector size (logical/physical): 512 bytes / 512 bytesI/O size (minimum/optimal): 512 bytes / 512 bytesDisklabel type: dosDisk identifier: 0x00000000Device Boot Start End Sectors Size Id Type/dev/sdc1 8192 60751871 60743680 29G c W95 FAT32 (LBA)Unfortunately I don't have any of the original "good" cards lying around. I'll see if can mount the image and get any info from that...

author: Delta
author_rank: Super Contributor
timestamp: November 04, 2016, 04:03:07 pm
content: Here is the original image, mounted as a device...Code:Disk /dev/loop0: 29.7 GiB, 31914983424 bytes, 62333952 sectorsUnits: sectors of 1 * 512 = 512 bytesSector size (logical/physical): 512 bytes / 512 bytesI/O size (minimum/optimal): 512 bytes / 512 bytesDisklabel type: dosDisk identifier: 0x5d98613bDevice Boot Start End Sectors Size Id Type/dev/loop0p1 * 8192 124927 116736 57M e W95 FAT16 (LBA)/dev/loop0p2 124928 62332927 62208000 29.7G 83 Linux

author: mtdoc
author_rank: Super Contributor
timestamp: November 04, 2016, 04:52:12 pm
content: An excellent tool for checking the integrity/authenticity of SD cards (or thumb drives) isH2testw

author: RGB255_0_0
author_rank: Frequent Contributor
timestamp: November 04, 2016, 05:14:54 pm
content: I have a 32GB Sandisk uSD from about 4 years ago and it's 31,914,983,424.Perhaps Sandisk have shaved off a few costs. Or there's a trojan in their supply line

author: suicidaleggroll
author_rank: Super Contributor
timestamp: November 04, 2016, 05:41:31 pm
content: I have a 32 GB Sandisk Ultra uSD from a few years ago, it's 31,914,983,424 as well.

author: Delta
author_rank: Super Contributor
timestamp: November 04, 2016, 06:29:48 pm
content: Quote from: mtdoc on November 04, 2016, 04:52:12 pmI've just ran the Linux version of that utility, and the reported capacity is genuine. That's something at least...Code:Data OK: 28.96 GB (60727232 sectors)Data LOST: 0.00 Byte (0 sectors)Corrupted: 0.00 Byte (0 sectors)Slightly changed: 0.00 Byte (0 sectors)Overwritten: 0.00 Byte (0 sectors)Average reading speed: 31.40 MB/sI'm still intrigued as to the why the capacity has been reduced; and whether these cards are genuine or not...If anyone buys a new Sandisk Ultra MicroSD card, please check its capacity and post here!

author: SeanB
author_rank: Super Contributor
timestamp: November 04, 2016, 06:36:44 pm
content: They probably changed supplier of the raw memories or the controller, and the new supplier is giving a smaller size flash, so they have to shave off near 1G to handle the spare cells needed for wear levelling, a wear level table, and probably a small chunk of flash to store the microcontroller on the unit firmware.

author: C
author_rank: Super Contributor
timestamp: November 04, 2016, 06:42:38 pm
content: DDIf you are working with sectors, what happens when you read a bad sector, go to write good data in a bad sector.At this level, you are below the bad sector information that a file system has. If you change where data is written then you are out of sync with file system.To clone you need a ZERO error read followed by a ZERO error write by DD.FDisk & GParted report the start and end blocks for a partition there is no separation of good blocks vs bad blocks.And at some point hidden replacement sectors will be used up.

author: CJay
author_rank: Super Contributor
timestamp: November 04, 2016, 11:05:52 pm
content: Quote from: TwoOfFive on November 04, 2016, 03:49:49 pmDynamic volumes are fairly easy to deal with but there's no easy official way to do some things.The lack of capacity on a 1TB drive is also to do with the drive manufacturers 'redefining' the GB

author: Monkeh
author_rank: Super Contributor
timestamp: November 05, 2016, 02:39:55 am
content: Drive manufacturers are using the correct units. 'everybody' (that is: Microsoft) uses the wrong units.

author: CJay
author_rank: Super Contributor
timestamp: November 05, 2016, 08:53:56 am
content: Quote from: Monkeh on November 05, 2016, 02:39:55 amTrue, but the only place I've seen GiB used on a PC was on a fairly recent Linux distro, Apple and pretty much every hardware manufacturer who isn't in the storage media business use GB to mean GiB, MS are far from the only abuser.

author: Kilrah
author_rank: Unknown
timestamp: November 05, 2016, 09:07:52 am
content: Apple actually uses GiB, but label it as GB to avoid confusion.Those differences also apply to hard drives, friend had 2 4TB drives of the same manufacturer but different "variant" and one had 13GB less.

author: Monkeh
author_rank: Super Contributor
timestamp: November 05, 2016, 10:34:56 am
content: Quote from: Kilrah on November 05, 2016, 09:07:52 amErrr.. no, they don't.

author: Kilrah
author_rank: Unknown
timestamp: November 05, 2016, 10:44:01 am
content: Quote from: Monkeh on November 05, 2016, 10:34:56 am?

author: Monkeh
author_rank: Super Contributor
timestamp: November 05, 2016, 10:50:11 am
content: Notice how Windows says '228GB' (ignoring your locale issues), and OS X says '245.21GB'. And how the real capacity is 245,202,157,568 bytes.Also:https://support.apple.com/en-us/HT201402Quote

author: Kilrah
author_rank: Unknown
timestamp: November 05, 2016, 11:26:32 am
content: Yeah, got GiB and GB swapped again.Shows how stupid the whole thing is.

author: CJay
author_rank: Super Contributor
timestamp: November 05, 2016, 11:27:43 am
content: Quote from: Kilrah on November 05, 2016, 09:07:52 amFirmware differences can often have that effect on hard drives, it was a fairly regular occurrence when ordering a certain make and model of Ultra 320 SCSI drive as a replacement part for a RAID array that the drive was rejected by the controller because some revisions, while having exactly the same model number, would be a minuscule amount lower capacity (8MB springs to mind).We eventually worked out how to identify the drives by serial number and would request 'lower than' or build new/replacement arrays with an amount of slack in them to allow them to be rebuilt with any capacity version.

author: Kilrah
author_rank: Unknown
timestamp: November 05, 2016, 11:29:52 am
content: Friend ran into it when wanting to back up one onto the other and the destination ran out of space. Yup it was that fullAnd no it wasn't cluster size difference/hidden files, the disk manager did show the matching difference on total drive size.

author: CJay
author_rank: Super Contributor
timestamp: November 05, 2016, 11:36:07 am
content: Quote from: Kilrah on November 05, 2016, 11:26:32 amExactly, while I'm all for the correct use of units, it should be just one standard, correct or not, and stuck to.The hard drive thing I am sure was just taking advantage of SI units to make their drives look larger for marketing as I remember one of the leading PC mags being rather outraged the first time they came across it.Pretty much all disk capacity until then was defined as binary, needless to say, all the other drive manufacturers caught on *real* quick.I and a lot of other techs/engineers/users have been brought up to think of GB in binary terms so 1024 is the measure I will always use to define a GB, MB and KB, even if it is correctly GiB,MiB and KiB.I've been around long enough to know where it fits from the context but so many non techs and newbies have no idea and I've had it 'explained' to me by kitchen table PC experts in some really amusing ways.

author: RGB255_0_0
author_rank: Frequent Contributor
timestamp: November 05, 2016, 11:41:32 am
content: I can't be bothered explaining Base 10 and Base 2 differences any more. So I follow what the OS and Manufacturers call it: "formatted" capacity. And Apple in IOS also use Base 2.

author: Delta
author_rank: Super Contributor
timestamp: November 05, 2016, 11:55:05 am
content: Quote from: Monkeh on November 05, 2016, 02:39:55 amI was thinking along those lines, but it doesn't add up...Old cards31,914,983,424 Bytes is (nearly) 32 GB, or 29.72 GiBNew cards31,104,958,464 Bytes is (about) 31 GB or 28.97 GiBSo which even way you spin it, the old cards were (nearly) 32 GB, the new ones are 31 GB. The old cards were 29.72 GiB, the new ones are 28.97 GiB

author: CJay
author_rank: Super Contributor
timestamp: November 05, 2016, 12:11:00 pm
content: Quote from: Delta on November 05, 2016, 11:55:05 amI was about to check my two Sandisk Extreme cards, one shows 28.79GiB in disk management, the other, in the same adapter, started to smell of burning and got too hot to touch. It's not recognised at all now.So, I guess I need to get that one replaced and try again later(Both appear identical under a 4x loupe and were bought from Costco)

author: Monkeh
author_rank: Super Contributor
timestamp: November 05, 2016, 12:22:49 pm
content: Quote from: Delta on November 05, 2016, 11:55:05 amNever said I thought it was a unit issue. They appear to be cutting corners.

author: stj
author_rank: Super Contributor
timestamp: November 05, 2016, 12:49:09 pm
content: the firmware in the card pre-allocates some of the flash for re-mapping bad blocks in the future.different firmware,controller or even trust in the flash lifespan could effect the size of the pre-allocated space.

author: amyk
author_rank: Super Contributor
timestamp: November 05, 2016, 04:20:45 pm
content: 31914983424, which is apparently the size of the "good" card, is not even a "HDD manufacturer's GB" or an "SI GB". If you bought a "32GB" HDD you would now expect at least 32000000000 bytes. Is there a "flash memory GB" which is exactly 997343232 bytes, and they're now trying to cut that down to 972029952?Note that 997343232 is exactly 16384*60873, and 972029952 is exactly 1048576*927 (927 "real" MB).I would demand a refund, tell Sandisk about it, and spread this news over the Internet. Selling "32GB" of storage which has <32000000000 bytes is not even correct according to SI.As for the "explanations" about spare area/extra space, that is just wrong. Flash comes in binary capacities and small integer multiples thereof, with an extra "spare area" in each page specifically for block mapping and error correction codes. A 32GB (256Gb) flashlike this onehas 16384 blocks * 256 pages * 8640 bytes/page = 36238786560 bytes. That would be 36GB in HDD manufacturer's units, or 33.75 binary GB. Even if you cut that down to 32000000000 actual accessible storage space, there's still 4238786560 bytes (4GB+) of spare area, or nearly 1/8th of the total bits on the device. If they really need more than that amount of spare area, then they should not call them 32GB, but 31GB.

author: ez24
author_rank: Super Contributor
timestamp: November 05, 2016, 04:36:35 pm
content: Quote from: Monkeh on November 05, 2016, 12:22:49 pmSome VP got a bonus for saving the company money (smaller die size). FYI I just got a new box of saltine crackers from Costco and they are 1/8 inch smaller than the box I got last month. One of their VP's got a bonus also. In other words making the product smaller makes VP bonuses. So someone made out.