what do you use to test harddrive, SSD, and NvME units? - Page 1
Forum section: electronics / general

author: DiTBho
author_rank: Super Contributor
timestamp: February 07, 2021, 02:12:46 pm
content: Last week I tested a lot of 160GB hard-drives with the program "badblocks" and it took 1038 minutesCode:# time badblocks -swv /dev/sdbChecking for bad blocks in read-write modeFrom block 0 to 156290903Testing with pattern 0xaa: done, Reading and comparing: doneTesting with pattern 0x55: done, Reading and comparing: doneTesting with pattern 0xff: done, Reading and comparing: doneTesting with pattern 0x00: done, Reading and comparing: donePass completed, 0 bad blocks found.1037m58.106sSo I wrote my own testing program, simpler than badblocks (only 2 passes) and it took 760 minutes, still "a lot of time", but less than badblocks.Code:# time ./disktest /dev/sdbdisk_size=160041885696testing 19536363 slices of 8192 byte eachCompleted, 0 defect found.759m59.631sThe USB2-FullSpeed-link is able to measure (time dd if=/dev/sdb of=/dev/null) 40Mbytes/s, but for some reason bad-blocks is slower. Ok, it has to write/read/write/read/write/read 3 times each block, but for 160Gbyte it takes 17 hours, ~ 2.5Mbyte/sWeird, and badIs there anything faster?

author: tunk
author_rank: Super Contributor
timestamp: February 07, 2021, 03:54:28 pm
content: It runs four write/read passes, so your average write/read speed is around 20MB/s.You could try the -b and -c options to see if that speeds it up, i.e. increasing the default.There's also the -t option if you want one (or several) pass(es).

author: CJay
author_rank: Super Contributor
timestamp: February 07, 2021, 04:55:42 pm
content: Manufacturer's diags and patience.

author: Halcyon
author_rank: Unknown
timestamp: February 07, 2021, 10:57:30 pm
content: I wouldn't bother doing that with SSDs. The blocks are completely transparent to the operating system. If there are any bad blocks, they will be marked unusable and a block is reallocated from the spare area. Check the SMART status to see if that has occurred. Not to mention you're also probably dealing with wear levelling. Whilst the sector reported to the OS remains the same, it doesn't mean that it's in the same physical place on a particular flash chip (or even on the same flash chip) as the previous write.

author: golden_labels
author_rank: Super Contributor
timestamp: February 08, 2021, 12:26:58 am
content: DiTBho:You haven’t provided source of your program or even a smallest part of information about how it works. However, assuming that it works as yourddcommand does(1), I suspect it’s faster because you have mostly tested your RAM, not the storage medium. You never or nearly never read anything from the flash memory itself — instead you write and read to/from in-RAM buffers of your kernel. And that is optimistic — a worse, though unlikely scenario is that you never left your CPU.Ensuring writes to the underlying medium is a bit tricky. But at very minimum you should flush buffers after a write and drop related caches before reading. You also must avoidTRIMbeing issued on all-zero writes, which may happen if you write to a file in a file system. Then you can at least hope you are testing your flash memory. Preferrably use direct I/O, as that solves the above issues — but, of course, is harder to use and then you see the real random-access write/read speed of the device. From that point you only need to solve the problem of the controller on the flash memory interfering with your test.____(1)Which is pointless in that invocation and puts you in unneccessary risk.

author: DiTBho
author_rank: Super Contributor
timestamp: February 08, 2021, 04:59:17 am
content: Quote from: golden_labels on February 08, 2021, 12:26:58 ambadblocks is opensource, mine is a simple piece of code that works like badblocks but with less passesCode:/** clean with fix pattern*/disk_block_clean(disk_block0, disk_block_size, 0xff);test_wr0 = IO_block_wr_sync(d_file, disk_addr, disk_block0, disk_block_size);/** write "fancy" pattern*/disk_block_fancy(disk_block0, disk_block_size);test_wr1 = IO_block_wr_sync(d_file, disk_addr, disk_block0, disk_block_size);/** read back*/test_rd0 = IO_block_rd_sync(d_file, disk_addr, disk_block1, disk_block_size);/** compare*/test_cmp = block_compare(disk_block0, disk_block1, disk_block_size);disk_block_size is calculated by a function that considers a lot of things and checks out if a proposed value satisfies constraints, so it chooses the best compromise (mostly, the program has limited internal buffers)Code:disk_size=160041852928152627 slices of 1048576 byte ~ 843776305255 slices of 524288 byte ~ 319488610511 slices of 262144 byte ~ 573441221022 slices of 131072 byte ~ 573442442044 slices of 65536 byte ~ 573444884089 slices of 32768 byte ~ 245769768179 slices of 16384 byte ~ 819219536359 slices of 8192 byte ~ 0Checking for bad 512-byte blocks in read-write modeFrom 0 to 160041852927, by 8192 bytes at timeQuote from: golden_labels on February 08, 2021, 12:26:58 amyou are talking only about flash-based storage devices; badblocks was designed for electromechanical hard drives and so my program. But you made good point: both are inadequate for the new technology

author: golden_labels
author_rank: Super Contributor
timestamp: February 09, 2021, 03:13:14 am
content: Quote from: DiTBho on February 08, 2021, 04:59:17 amFrom your post I understand you wrote something unrelated tobadblocks. Now it is clearer. So I suppose the anwer is indeed hidden in “less passes”.Quote from: DiTBho on February 08, 2021, 04:59:17 amYes, because this is what’s written in the subject of this thread. I assumed we’re talking about runnig such programs on flash-based media.Quote from: DiTBho on February 08, 2021, 04:59:17 amThere can be a problem with even the newest HDDs with more cache than my first PC had RAM. But the problem with the approach shown in theddinvocation (again:ddmake no sense there; simply usecat— yes, I will rant about that each time I see someone usingddjust because they deal with storage media) is not with the hardware. It fails at kernel level due to system caching. That’s the same situation that makes writes to slow USB sticks appear to go at 200MB/s just to drop suddenly to “ETA: next x-mas”. And the same reason they must be properly unmounted before removal.If you are seeking a sure method of testing: fill the whole storage with data from/dev/urandom,hashing that data on the fly, sync it (for removable media: eject to ensure power cycling), drop caches (unless reading in a manner that ignores them), hash the whole storage, compare hashes. If there is a mismatch, you know the device is damaged or counterfeit. That is in particular well suited for fake USB sticks that advertise sizes larger than the really available memory: it peforms a proof-of-space attack, which they can’t deal with. The same attack is what minimizes the risk of any type of on-device caching getting in the way of the test. The drawback is that you will not know where exactly the damage is and can’t test blocks that are not currently addressable. But I would argue that since 2010s the distinction between “working” and “broken” became much more binary, so the first issue is of minor importance. The second problem can only be solved by either physically operating on the chip or some manufacturer’s tools that use hidden functionality of the device to access them (but I know of no such tools and their existence is a bad sign).

author: DiTBho
author_rank: Super Contributor
timestamp: February 09, 2021, 11:51:00 am
content: Quote from: golden_labels on February 09, 2021, 03:13:14 amThat sounds interesting and last night I wrote a new C/89 testing program that does exactly thisIt access the storage-device at the low level with all the control handled by IOCTRL, then checks what IOCTRL reports about the amount of blocks (here the firmware can lie), then fills all the blocks with random data and calculate the the sh-a hash!Next step? It invokes data-flush, meta-data-flush, kernel-sync, read-back the whole blocks to re-calculate te sh-a hash, and finally it compares the "expected hash" (performed on blocks-write) with the "actual hash" (performed on block-read-back)test_passed = cmp(expected-hash, actual-hash)During these steps a lot of unexpected things happened, including some USB2-sATA adapters unable to have high intensive data-flow. I am not sure about what happened, but I see weird error messages in the kernel' dmesg; not yet investigated, I don't know, once replaced then with other USB3-sATA adapters, everything worked as expected.Quote from: golden_labels on February 09, 2021, 03:13:14 amYup. I own a 80Gbyte pATA hard-drive that has a couple of unrecoverable bad-blocks above the first 40Gbyte of space (if you look at it like a linear array of blocks), so I hacked the firmware in order to reduce the number of blocks it presents to IOCTRL request.It was "n-blocks", now it it's "n-blocks/2", just a couple of bytes in the firmware can do itI did it just for fun, and this way I saved this hard-drive, but it makes no sense if you think that you can buy a new one on eBay/Amazon/etc for something like 10 euro, while I spent something like two weeks to hack it.Quote from: golden_labels on February 09, 2021, 03:13:14 amSo ... it will be "Samsung-specific". I think. I am going to buy a lot of Samsung SSDs, and a couple of NVMe devices.Thank you for your amazing testing idea!

author: golden_labels
author_rank: Super Contributor
timestamp: February 09, 2021, 12:50:05 pm
content: Quote from: DiTBho on February 09, 2021, 11:51:00 amOh, have you ever published a description about that project? Not the modified firmware (that would be illegal), but a report of how you did that, what tools used &c.

author: DiTBho
author_rank: Super Contributor
timestamp: February 09, 2021, 01:15:00 pm
content: Quote from: golden_labels on February 09, 2021, 12:50:05 pmNo, it was done just for fun and to save a personal hard-drive for which I worked hard when I was a student. I remember I prepared hundreds of McDonald's sandwiches in order to get enough money to buy a new (second hand) laptop with a new hard-drive. To hack it I didn't use IDA or similar because too expensive to buy. I think I was lucky because I found a SH RISC chip, so I desolder the flash-chip, read it and deassebled its firmware with GNU tools. I understood very little about it (less than the 10%), but found the point where it was reporting the block-numbers to IOCTRL.Code:ioctl(fd, BLKGETSIZE, &numblocks);Lucky find! Exactly the number reported by fdisk, which internally uses IOCTRL to read the number of blocks! That's a specific question that the kernel drives asks to the unit and to which the units accordenly replies.Luckily there was no checksum for the "data area", so I burned a second flash, resoldered it, and that's it. Half the space, but working again with no more "bad-blocks" once I installed it back to the laptop I used to write my thesis years ago

author: DiTBho
author_rank: Super Contributor
timestamp: February 09, 2021, 01:15:56 pm
content: Code:usb 1-1: reset high speed USB device number 2 using ehci_hcdusb 1-1: device descriptor read/64, error -71usb 1-1: device descriptor read/64, error -71usb 1-1: reset high speed USB device number 2 using ehci_hcdusb 1-1: device descriptor read/64, error -71usb 1-1: device descriptor read/64, error -71usb 1-1: reset high speed USB device number 2 using ehci_hcdusb 1-1: device not accepting address 2, error -71usb 1-1: reset high speed USB device number 2 using ehci_hcdusb 1-1: device not accepting address 2, error -71usb 1-1: USB disconnect, device number 2sd 0:0:0:0: Device offlined - not ready after error recoverysd 0:0:0:0:  Unhandled error codesd 0:0:0:0:  Result: hostbyte=0x01 driverbyte=0x00sd 0:0:0:0:  CDB: cdb=0x2a: 2a 00 00 00 4b 40 00 00 80 00end_request: I/O error, dev sda, sector 19264Unable to handle kernel paging request for data at address 0x00000030Faulting instruction address: 0xc02f8b10Oops: Kernel access of bad area, sig: 11 [#1]PREEMPT PowerMaclast sysfs file: /sys/devices/virtual/net/br0/broadcastModules linked in:NIP: c02f8b10 LR: c02fdd64 CTR: 00000000REGS: db59d9b0 TRAP: 0300 Not tainted (2.6.39-apple-minimac-G4)MSR: 00009032 <EE,ME,IR,DR> CR: 24084424 XER: 20000000DAR: 00000030, DSISR: 40000000TASK = df44e000 'badblocks' THREAD: db59c000GPR00: c02fdd64 db59da60 df44e000 df88d908 db6a0000 00000010 db6a00d4 00000000GPR08: fffffffc 00000000 ffffffff 00000001 24084448 1001d0dc 00000040 00000001GPR16: 805e2000 00000008 df88d908 c0817344 c0817394 db59db50 00000000 00000010GPR24: 00000001 db59c000 db6a0000 00000005 00000000 00000001 df88d908 02200000NIP  elv_set_request+0x14/0x50LR  get_request+0x338/0x358Call Trace:  0x1 (unreliable)  get_request+0x338/0x358  get_request_wait+0x4c/0x1b8  __make_request+0x90/0x370  generic_make_request+0x36c/0x438  submit_bio+0xb0/0x174  submit_bh+0x150/0x190  ll_rw_block+0x100/0x144  __block_write_begin+0x1f8/0x46c  block_write_begin+0x74/0xc8  blkdev_write_begin+0x1c/0x2c  generic_perform_write+0x1bc/0x29c  generic_file_buffered_write+0x74/0xd0  __generic_file_aio_write+0x284/0x554  blkdev_aio_write+0x48/0xc4  do_sync_write+0xb8/0x144  vfs_write+0xcc/0x184  sys_write+0x58/0xc8  ret_from_syscall+0x0/0x38--- Exception: c01 at 0xfed6718LR = 0x10002ff8Instruction dump:4e800421 7c601b78 7c030378 80010014 38210010 7c0803a6 4e800020 9421fff07c0802a6 90010014 8123000c 812900002f800000 419e001c 7c0903a6---[ end trace 8e039fa70a761634 ]---usb 1-1: new high speed USB device number 3 using ehci_hcdusb 1-1: device descriptor read/64, error -71usb 1-1: device descriptor read/64, error -71usb 1-1: new high speed USB device number 4 using ehci_hcdusb 1-1: device descriptor read/64, error -71usb 1-1: device descriptor read/64, error -71usb 1-1: new high speed USB device number 5 using ehci_hcdusb 1-1: device not accepting address 5, error -71usb 1-1: new high speed USB device number 6 using ehci_hcdusb 1-1: device not accepting address 6, error -71hub 1-0:1.0: unable to enumerate USB device on port 1usb 3-1: new full speed USB device number 2 using ohci_hcdusb 3-1: device descriptor read/64, error -62usb 3-1: device descriptor read/64, error -62usb 3-1: new full speed USB device number 3 using ohci_hcdusb 3-1: device descriptor read/64, error -62usb 3-1: device descriptor read/64, error -62usb 3-1: new full speed USB device number 4 using ohci_hcdusb 3-1: device not accepting address 4, error -62usb 3-1: new full speed USB device number 5 using ohci_hcdusb 3-1: device not accepting address 5, error -62hub 3-0:1.0: unable to enumerate USB device on port 1This is what happens with some USB2-sATA adapters ...

author: DiTBho
author_rank: Super Contributor
timestamp: February 09, 2021, 01:20:53 pm
content: The above messages was triggered by an old 20Gbyte harddrive unit, and it passes all the badblocks tests when directly connected to a pATA controller. But it fails when connected to a usb2-sATA adapter.usb-sATA seems to add more troubles, or it's only a USB2-bulk/storage problem? Not yet clear ...Anyway, this afternoon I will test my new and modern SSD-2015 and NVMe-2019 units with the new SH-A1 testing program I wrote yesterday