Reliability of USB flash memory vs. SSD drives? - Page 1
Forum section: electronics / general

author: Halcyon
author_rank: Unknown
timestamp: January 28, 2018, 11:10:32 pm
content: I'm in the process of building up a FreeNAS box and rather than have the OS sit on the traditional "spinning rust" disks, I've decided to go down the flash memory path, but mirrored to offer some redundancy.Given that FreeNAS has a relatively small footprint and once loaded doesn't write to the boot disks an awful lot, I've been thinking about just using a couple of USB3 flash drives as opposed to spending $100+ on a couple of small SATA SSDs. Both drives I'm looking at use MLC flash chips.Thoughts?

author: Kjelt
author_rank: Super Contributor
timestamp: January 28, 2018, 11:18:13 pm
content: The USB sticks are often as cheap as can be had flash, often slow and low mtbf.I would not trust them for photo's or serious data, had more then a few with failures.There are exceptions but those usb sticks are almost as expensive as ssd's.I do not know which size you are looking at but the 128GB and smaller SSD's can be gotten very cheap nowadays since no-one has a use for them anymore. There are also around 20-32GB ssd cards that were used as cache infront of an old school hdd esp. in laptops. Those have sata / m1 or m2 slots and can be gotten almost for free in the second hand market.

author: wraper
author_rank: Unknown
timestamp: January 28, 2018, 11:43:38 pm
content: USB flash is very unreliable. It may corrupt data after just a year. If SSD has such crappy flash, at least it should have some algoritms which refresh the data, preventing loosing it over time. Speed difference is also very different, especially random access where most USB flash crap out.

author: Mr. Scram
author_rank: Super Contributor
timestamp: January 29, 2018, 12:21:52 am
content: Flash drives are generally not built for this kind of use. The memory used isn't as robust that that of SSDs. They're made for occasional writing, not continuous or regular writes. The controller of an SSD also does wear levelling, mitigating wear and spreading it across all the cells. Finally, an SSD generally is over-provisioned, which means that there's some room to play when memory cells wear out. Portable memory generally doesn't have this.There are USB drives that actually use better memory types and SSD controllers, but those aren't much cheaper than a comparable SSD. Somehow portable memory manufacturers also refuse to be specific about what they sell, so it's hard to figure out whether the money you spend is actually making an improvement. It could work if care is taken to pick the right hardware, but considering you are already doubling up for peace of mind I would personally go for the proven route.

author: Halcyon
author_rank: Unknown
timestamp: January 29, 2018, 12:27:42 am
content: Quote from: Mr. Scram on January 29, 2018, 12:21:52 amVery good point! That being said, do I go with the "tried and tested" MLC route (SLC is outside of my budget and delivers little real-world benefit) or go with the newer "3D NAND" technologies? Keeping in mind I'll be mirroring the boot drive across two devices. The last time I bought an SSD was about 5-6 years ago (one of the basic Intel SSDs) and it's been absolutely fine even with daily use. I dare say technology has improved since then.

author: mikeselectricstuff
author_rank: Super Contributor
timestamp: January 29, 2018, 12:33:29 am
content: Something to bear in mind is that NAND flash also has endurance issues for reads (read disturb errors), and the controllers of USB drives may be less prepared to deal with these.

author: Mr. Scram
author_rank: Super Contributor
timestamp: January 29, 2018, 12:41:53 am
content: Quote from: Halcyon on January 29, 2018, 12:27:42 am3D NAND and MLC aren't mutually exclusive. 3D NAND is a memory cell technology or type and SLC, MLC and TLC are ways of using the memory cells. It's a bit confusing, I have to admit. It took me a while to catch on to that too. Apparently, 3D NAND is more reliable than planar NAND due to larger node sizes. Larger node sizes are normally more expensive, but due to the 3D NAND you can stuff more in the same chip area.There used to be Crucial drives that would behave as SLC drives when less than half of the capacity was used, but their recent drives don't do this any more. Those were ideal for server duty without going for actual and expensive server hardware. Semi SLC, MLC for the rest and power loss protection was a sweet combination. Now most of it is TLC with very little power protection and the few MLC drives focus on speed instead of ultimate reliability. It's what consumers want, apparently.

author: amspire
author_rank: Super Contributor
timestamp: January 29, 2018, 01:34:08 am
content: The SSD wear issue is disappearing for most people as the drive capacity increases. A recent test on a number of different brands of 240 to 256G SSD's showed that even with TLC drives, they could take about 100G of writes every day and still last 10 years. If you need to write more, just get a bigger drive. The 4T SSD's should be able to handle over 1T of writes every day.That is probably good enough for almost every case. In the rare cases of exceptional write loads, you are probably going to be regularly replacing the drive anyway.The longevity is another issue. I have lots of old 15+ year old drives that are still readable today. Many of them are only about 1Gbyte in size because of their age. Will data in an unpowered TLC SSD last for 10-20 years or more? The drives use error correction. but at some point, there are too many errors to correct.

author: David Hess
author_rank: Super Contributor
timestamp: January 29, 2018, 03:29:58 am
content: USB and memory card flash memory has to live in an environment where power can be removed at any time and lacks the physical size to include internal backup power so it is not even designed with reliability features like idle time scrubbing. What *might* work is to use devices which support scrub on read while running a low priority task which reads the entire contents.

author: rdl
author_rank: Super Contributor
timestamp: January 29, 2018, 03:52:30 am
content: You can run FreeNAS from a USB flash drive no problem, though I use two, 8 GB each in a mirrored configuration, as recommended. They are basically just used to store the OS. Supposedly (I've never done it), once it's loaded and running you could remove the flash drives and it would continue to work because FreeNAS runs almost totally in RAM (try that with Windows). It can however write a lot to the system dataset (tons of logging by default) depending on how much data and how much it's used. I put the system dataset on a third flash drive and it wore out in about a year. When I replaced it, I set the logging to the minimum.

author: amspire
author_rank: Super Contributor
timestamp: January 29, 2018, 04:17:23 am
content: Quote from: rdl on January 29, 2018, 03:52:30 amThis is the way most appliances work that are based on Linux-like O/Ss and I have always thought it a pity that the method didn't always work like this in all O/Ss - have a small O/S core that is loaded into RAM and that never gets corrupted by a power fail and the larger application layer where problem caused by a power fail is most likely to just damage a file belonging to that application. If you look at the C:\Windows folder in particular, it is a total mess - program files, log files, installation files, debug files, application driver files, inf files all jumbled up together in a single folder.

author: IanMacdonald
author_rank: Frequent Contributor
timestamp: January 29, 2018, 08:31:57 am
content: Had one problem with an SSD so far out of a good few installed, which seemed to be a mobo compatibility issue. Otherwise, fast and reliable. Even with legacy OS's.SD cards and USB keys just never seem to be reliable. At one time wondered if I was being sold fakes, as so many went down. Reasonably sure they are genuine though. I have some very old CompactFlash cards from my previous SLR and they are still better than the latest SD cards for speed and reliability, so it IS possible to have reliable cards.

author: Kjelt
author_rank: Super Contributor
timestamp: January 29, 2018, 09:37:06 am
content: If you want cheap and reliable look at 2nd hand Samsung (pro) drives, 740/750/850 series.Ask the seller for the SMART data from Magician and you know exactly how many operating hours and more importantly how much data has been written.As i said earlier many hobbieists replace their ssd's because they need more space.If i look here locally I see 128GB ssd's with less than 2000 operating hours and around 1 to 2 TB writes for around €30 offered.For the discussion about if new generations flash are more reliable I want to point out that Samsung has lowered the guarantee period from ten years for their 850pro series to five years for their 750 series.There is a lot of debate now going on if that indicates that the reliability has gone down.

author: tom66
author_rank: Super Contributor
timestamp: January 29, 2018, 09:47:32 am
content: I've had the memory stick that hosted my FreeNAS instance go bad after about two years of continuous service.FreeNAS does a lot of logging and statistical collection. I think these get saved to the USB drive and because it was plugged into the rear of the system it was exposed to the warm air from the power supply/CPU.It began showing page read and write errors occasionally, causing kernel panics and various FreeNAS bugs, but I was able to back up the data before it totally failed, onto another memory stick.I then moved to a dedicated NAS which writes logging data to the hard disk as it should.

author: Jeroen3
author_rank: Super Contributor
timestamp: January 29, 2018, 10:59:58 am
content: 1. MLC flash on a thumb drive probably has poor wear leveling. I'd get industrial SLC thumbs drives or SD cards. But these are ridiculously expensive.2. Using hardware raid on consumer SSD's is tricky. The SSD's need to support this (eg: no random garbage collect delays) and the raid card must support TRIM.3. Cheap SSD's and thumb drives do not have power loss protection.Why not get proven technology in the shape of 2.5" drives?

author: GeorgeOfTheJungle
author_rank: Super Contributor
timestamp: January 29, 2018, 12:01:54 pm
content: I have nine linux sbcs (*) that have been running (headless) 24/7 for years, all are booting (armbian) from their sdcards, and for storage all have JBOD "raid"s made of usb pendrives (SanDisk Ultra Fit 128 GB are my favourites) or sdcards I had no better use for on usb readers. If I need more storage just add more pendrives to the jbod. Some are web servers and some are file servers, so the use is almost 100% of the time reads much more than writes. IDK what would happen if the use were mostly writes.My experience so far is good, very good. These pendrives are more than fast enough because the network in my case is the bottleneck.(*) 2*bananapi, 4*orangepi PC, 2*odroid c1, 2*odroid xu4, 1*odroid u3, none is a raspberry pi anymore because it can't serve from a USB port at ethernet speeds (ethernet and all its usb ports are behind a hub connected to a single usb port). My favourite cheap sbcs are the orange pi PC ($15!) and the odroid C1 and C2 (~$50).

author: rdl
author_rank: Super Contributor
timestamp: January 29, 2018, 04:58:51 pm
content: Quote from: tom66 on January 29, 2018, 09:47:32 amI believe the more recent versions of FreeNAS save logs and such to one of the storage drives if it's running from USB flash drives. The problem there is if you don't use the data often and want the storage disks to power down. The thing to do is move the location where these logs and such are saved to somewhere else. I use a third USB flash drive. There is monitoring available in FreeNAS and the drive(s) where the OS is stored get written to very rarely while it's running, whereas the drive where .sys is stored gets written to nearly constantly at the default level of logging. Mine failed after about a year. The two flash drives the OS is on are still fine.

author: Rick Law
author_rank: Super Contributor
timestamp: January 29, 2018, 05:47:29 pm
content: May be USB flash running in RAID 5 configuration?I've often wonder if that is a viable solution, but of course there is no good controller for that in the market. RAID5 with 128gb flash cards will still give a mere 192gb. One would need 10 of those (30 sticks of 32g) to get a respectable 2TB config...Software RAID supported since NT (in my opinion) is kind of a nutty idea. 32gb SD cards are now cheap enough to be a play thing, may be I will get a few USB sticks/adapter to try to run it as 192gb RAID5 just for the fun of it.

author: Mr. Scram
author_rank: Super Contributor
timestamp: January 29, 2018, 05:51:49 pm
content: Quote from: Kjelt on January 29, 2018, 09:37:06 amThe problem with the Samsungs is that they don't do power loss protection. Samsung purely focusses on performance in the consumer market.

author: Mr. Scram
author_rank: Super Contributor
timestamp: January 29, 2018, 05:54:40 pm
content: Quote from: Jeroen3 on January 29, 2018, 10:59:58 amAgreed. You can get cheap and you can get quite reliable, but any attempt at a combination is an experiment at best. Though going for 2,5" drives doesn't have to be expensive. Proper drives have become dirt cheap.

author: Mr. Scram
author_rank: Super Contributor
timestamp: January 29, 2018, 06:03:29 pm
content: Quote from: blueskull on January 29, 2018, 06:00:47 pmCorrect, and I don't think Samsung has either. Crucial is the only party that has been fairly consistently protecting SSDs with capacitors, but unfortunately has started moving away from that in their new series too. Apparently consumers accept a power loss lottery when it saves them a few cents.

author: Jeroen3
author_rank: Super Contributor
timestamp: January 29, 2018, 06:17:50 pm
content: Quote from: Mr. Scram on January 29, 2018, 05:54:40 pmYou can sell SSD upgrades to some relatives and harvest the 2.5" drives. 90% chance on getting the same drive if you do laptops from the same year.

author: rdl
author_rank: Super Contributor
timestamp: January 29, 2018, 06:40:34 pm
content: Keep in mind the original post was concerning what to use for FreeNAS, not data storage. SSD is overkill for this. So is a spinning disk really, since FreeNAS needs less than 8 GB of disk space. Once it loads, it runs from RAM. A USB flash drive is fine for this, though having two in mirror configuration could save time if there is a failure.

author: Jeroen3
author_rank: Super Contributor
timestamp: January 29, 2018, 06:54:12 pm
content: It would be nice if Freenas could do the Synology approach. It puts a small RAID 1 volume onall driveswith the operating system on it.edit: I just realized, does your board have capability forthose pci-e or msata drives?

author: David Hess
author_rank: Super Contributor
timestamp: January 29, 2018, 08:30:55 pm
content: Quote from: rdl on January 29, 2018, 06:40:34 pmI have actually been doing this for years now. I have a couple of FreeBSD based routers using standard PC hardware which boot from Compact Flash using CF to IDE adapters. But I deliberately paid for CF cards that I trust from Transcend and I keep image backups of the cards.They make "industrial" flash memory devices other than SSDs which may be more reliable. I am not sure they are any more economical than the smallest SSDs.

author: Kjelt
author_rank: Super Contributor
timestamp: January 29, 2018, 11:12:16 pm
content: Quote from: Mr. Scram on January 29, 2018, 05:51:49 pmThat is in practice hardly a problem.If mains fails it would be very coincidental that the write action already finished.So if the cpu gors down during the write action all hope is already lost no matter if the ssd can or can not finish the last write action.

author: Mr. Scram
author_rank: Super Contributor
timestamp: January 29, 2018, 11:38:22 pm
content: Quote from: Kjelt on January 29, 2018, 11:12:16 pmPower loss protection in consumer SSDs isn't about preservering the little bit of data being written while the power goes down. It's about preventing corruption of the whole SSD. That's a crapshoot without any protection in place. Even if you're careful you can't avoid these events completely. Check the SMART values of almost any SSD and see what the POR count is. Any one of those events is a potential disk corruption. Note that this data is sometimes presented in hexadecimal format, making it look like a smaller number than it actually is.It's an inherent trait of SSDs to be vulnerable to this and some manufacturers were addressing the issue with power less protection capacitors. They generally don't provide full protection, but prevent the worst of it. However, it seems these events are rare enough to remove the protection completely and just have the people unlucky enough to encounter them deal with it. I think in some cases you can even permanently damage your NAND too, though it might occur in subtle ways that may only produce intermittent issues.There are other ways to mitigate the issue too, but these are generally considered inferior and don't protect the drive in all use cases.

author: Halcyon
author_rank: Unknown
timestamp: January 30, 2018, 01:09:59 am
content: Quote from: rdl on January 29, 2018, 06:40:34 pmI'll think I'll go the USB stick approach. Worst case, if both of them die, it'll take me a few minutes to re-image a new one just to get back up and running.Quote from: Jeroen3 on January 29, 2018, 06:54:12 pmTraditional RAID is pretty much old technology now with arguably more downsides than positives. I'm ditching my hardware RAID entirely (currently running on Windows) in favor of ZFS.My board doesn't support M.2 drives.

author: David Hess
author_rank: Super Contributor
timestamp: January 30, 2018, 02:31:12 am
content: Quote from: Kjelt on January 29, 2018, 11:12:16 pmMr. Scram covered most of it but writes also occur during a scrub on read and scrub on idle operation so not writing to the drive is not sufficient by itself to prevent damage due to power loss on write.As Mr. Scram pointed out, it is not just the data being written which is at risk. If that was the case, then there would be no problem because file systems can be designed to be proof against that. The problem is when the data structures of the drive itself are written; if they are damaged, the result is usually total data loss and the drive may become unusable. There is also a further risk that data not associated with the current write operation will be damaged so *all* data on the drive is vulnerable to corruption.As far as which drives are susceptible to this failure, tests have shown that it is practically all of them whether they are described as having power loss protection or not. Drives which are proof against it are the exception and the only way to really know is through testing.Flash memory cards have all of the above problems and more.

author: amspire
author_rank: Super Contributor
timestamp: January 30, 2018, 02:37:01 am
content: Now that we have so many computer boards that run off 5V, all you need for a UPS so that the USB memory stick or SSD doesn't loose power suddenly is a USB battery pack. If you have a 5Ah- 10Ah battery pack followed by one of those $5 single cell ones, then you can setup a UPS shutdown. When the voltage into the $5 USB battery pack goes to zero, initiate the shutdown.

author: Naguissa
author_rank: Regular Contributor
timestamp: January 30, 2018, 07:44:40 am
content: Quote from: GeorgeOfTheJungle on January 29, 2018, 12:01:54 pmI also use armbian in a Banana Pi M1 as NAS, having root on an old sd-card (good sd hc one, but not new).But Armbian doen't have have statistics on by default, does not have debug logging on by default and system logs uses RAM storage. That results in low writing to sd.Enviado desde mi Jolla mediante Tapatalk

author: Ice-Tea
author_rank: Super Contributor
timestamp: January 30, 2018, 09:06:03 am
content: Quote from: amspire on January 29, 2018, 01:34:08 amOnly because of wear leveling. Which is absent or at least unknown for USB drives.

author: amspire
author_rank: Super Contributor
timestamp: January 30, 2018, 09:23:07 am
content: Quote from: Ice-Tea on January 30, 2018, 09:06:03 amMany USB memory sticks do use wear levelling, but not all. For example, to quote Kingston from their Flash Memory Guide:QuoteThis applies to Kingston USB and SD cards.All it really needs is a lookup table for addressing cells rather then direct addressing. It would not be hard to implement.Another company said that each time a cell is needed for a write, they pick a random cell. If this is accurate, it is probably a simplified form of wear levelling.Some USB and SD cards go a bit further - they even move unchanging data, so the cells they are in can be reused. Otherwise if a USB drive was 95% of unchanging files, all the wear levelling only happens with the last 5% and the drive can still fail early.

author: rrinker
author_rank: Super Contributor
timestamp: January 30, 2018, 02:30:28 pm
content: Interesting how the OP stated he wanted to use the USB for the OS drive, not his actual data storage, and immediately everyone assumed he wanted to use USB sticks in place of actual drives for the primary data storage. Some caught on, some who are doing exactly what the OP proposes to do even.Even non-NAS devices, like Cisco's UCS blade servers, have a pair of internal USB ports on the motherboard for this purpose. Put your OS (typically on these things, a bare metal VM hypervisor) on USB sticks, save all the physical hard drive slots for storage to be used by the VMs. The out of band management board offers some options for redundancy if you use 2 USB sticks, but since these are nearly 100% read and almost never written to, it's quite reliable.

author: Mr. Scram
author_rank: Super Contributor
timestamp: January 30, 2018, 05:11:25 pm
content: Quote from: rrinker on January 30, 2018, 02:30:28 pmI don't think there are many people here that didn't understand this. OP asked for opinions on reliability and that's what he got. Especially considering a RAID solution was mentioned, you have to ask yourself what you're trying to protect against in what way and that's what has been discussed. It's better to discuss things that might not be applicable than to be ambiguous and possibly leave someone with bad or misunderstood advice.

author: David Hess
author_rank: Super Contributor
timestamp: January 30, 2018, 06:01:45 pm
content: Quote from: rrinker on January 30, 2018, 02:30:28 pmI did it though with Compact Flash cards specifically selected for the application.I ran some informal tests on consumer grade USB drives a few years ago and they would never have worked to do this. Whether powered or unpowered, they started losing data before a year was up. I thought maybe the powered ones would do idle time scrubbing but they did not. Read scrubbing like I suggested might allow them to be used.There are application specific USB flash drives which might do much better but they are not something I would trust without testing or backups. Some use the common 9-pin header instead of a USB connector so they plug directly into the motherboard although this wastes a USB port.https://us.transcend-info.com/Embedded/cate-302#327

author: IanMacdonald
author_rank: Frequent Contributor
timestamp: January 30, 2018, 06:10:43 pm
content: The performance advantage offered by SSDs over spinning rust is far greater than that offered by a RAID array over a bunch of disks. The main point about RAID is that whilst some types offer data redundancy, all except simple mirroring (RAID1) also make it a lot HARDER to recover data following a serious malfunction.Thus, using RAID makes it MORE important to have a reliable backup of your data. It is also important that the backup can be restored without the original hardware. People overlook the possibility of a RAID controller built into a mobo going down instead of a disk. If the nearest replacement is in Korea, then you have a collection of SATA or SCA doorstops with your perfectly preserved but totally inaccessible data on them.That, and if integrity is important you can get enterprise grade SSDs.

author: rrinker
author_rank: Super Contributor
timestamp: January 30, 2018, 08:29:58 pm
content: One reason I don't use RAID. My server is backed up anyway, but even two drives failing at once won't wipe out all of my data, just some subset of it, and the server won't crash.On second thought, I think the slots in the Cisco UCS boxes are CF as well. But same idea, 99% read just to boot the hypervisor. They used to ship them all as boxes of parts, now they are all coming pre-configured to you just rack it and turn it on.

author: Mr. Scram
author_rank: Super Contributor
timestamp: January 30, 2018, 09:17:13 pm
content: Quote from: rrinker on January 30, 2018, 08:29:58 pmRAID is not backup anyway. RAID is intended to improve uptime, not to preserve data.

author: Kjelt
author_rank: Super Contributor
timestamp: January 30, 2018, 10:39:13 pm
content: Quote from: Mr. Scram on January 30, 2018, 09:17:13 pmIndeed and support higher datarates since all hdd's are being used simultaneously. My NAS with 7 WD gold hdd's can handle continuously write rate of 450MB/s and peaks over 700MB/s.

author: cdev
author_rank: Super Contributor
timestamp: January 31, 2018, 12:26:11 am
content: If you can just boot off of the flash drive and then disable writes by various means, disable logging to the boot drive, disable recording of file access times, it will be very reliable because you wont be writing to it.You can boot off a flash disk and do minimal writing. You also may be able to have the software store multiple updates and combine them into a single write at regular intervals.That said I have heard anecdotal accounts of NAND flash cards, sticks, etc. being surprisingly capable of writes due to steady improvement in wear leveling algorithms.None of them compete with SSDs for sustained write speed though.

author: rdl
author_rank: Super Contributor
timestamp: January 31, 2018, 02:32:24 pm
content: The version of FreeNAS I'm running does not write to the boot drive(s) much at all.There are claims that a constantly spinning HDD lasts longer than one which is powered down when not in use. This is probably situational. In my case, the data pool may only be accessed once a week or so. I'm pretty sure those drives will last longer if powered down when not in use. Even if that's not true, powered down drives are quieter and use less electricity. The problem is that FreeNAS wants to use those drives for the system dataset instead of the boot drive(s). This is the guide I followed to fix that problem.https://www.pretzellogix.net/2015/02/04/how-to-get-freenas-9-3-to-spin-down-your-idle-hard-drives/I put .sys on a USB flash drive. The first one lasted a bit more than a year. I've reduced the amount of logging to the minimum and expect the replacement to last much longer. Only time will tell. Both drives are/were Kingston. The OS resides on a pair of mirrored Sandisk 16 GB Ultra Fit which continue to work fine.

author: GeorgeOfTheJungle
author_rank: Super Contributor
timestamp: January 31, 2018, 02:40:17 pm
content: Quote from: rdl on January 31, 2018, 02:32:24 pmThose Sandisk have worked very very well for me, and I believe the larger they are the more they last, as it happens with SSDs too.