Low-cost SATA SSDs for ZFS - Page 1
Forum section: electronics / general

author: olkipukki
author_rank: Frequent Contributor
timestamp: December 01, 2020, 09:35:24 pm
content: I'm looking to buy 4 (four) SATA SSDs to be used for independent FreeBSD ZFS RAIDZ-2 pool with max storage up to 0.5TB (reality ~200GB)Nothing too serious, considering consumer grade drivers, so max 250GB per drive should be okayAny recommendations what (not) to buy?

author: wraper
author_rank: Unknown
timestamp: December 01, 2020, 09:37:19 pm
content: Don't buy DRAMless models. Also avoid QLC NAND.

author: olkipukki
author_rank: Frequent Contributor
timestamp: December 01, 2020, 09:52:01 pm
content: Any particular model you have in mind?For example, WDS240G2G0A (WD Green) - not clue if it fail into DRAMless and QLC categories

author: Haenk
author_rank: Super Contributor
timestamp: December 01, 2020, 09:54:37 pm
content: Keep in mind the standard SSDs have a rather low lifespan when used in a RAID (i.e. constantly reading/writing), there is a reason for a MTBF based on data amount written.We have one RAID 50 setup running for a couple of years now, using Crucial MX500 SSDs - those are very slow now, in fact, incredible slow. So you should replace them once every couple of years, at least.And yes, avoid QLC as the plague, TLC+DRAM Cache is the way to go, if you want it cheap. Don't fall for those very cheap Amazon/ebay/Aliexpress SSDs from dubious chinese sources. Those are made from B-grade NAND and therefor not very reliable.

author: David Hess
author_rank: Super Contributor
timestamp: December 02, 2020, 08:43:56 pm
content: Quote from: wraper on December 01, 2020, 09:37:19 pmI agree with both. Besides the performance difference, models with DRAM seem to suffer from less write amplification and the added DRAM is not expensive. QLC NAND has much worse endurance than TLC NAND. I do not know how other people are using their drives but at the rate I am going, I expect to burn through the endurance of my TLC drives in less than 5 years.Quote from: Haenk on December 01, 2020, 09:54:37 pmRAID does not have to mean constant writing. It is often used in standby and bulk storage applications with a majority of accesses being reads.QuoteWhat does Crucial's Storage Executive utility say about the health of the drives? Are they just worn out?The Crucial MX500 series is my preferred drive right now but you can sometimes find deals on enterprise drives like the Intel D3-S4510.

author: wraper
author_rank: Unknown
timestamp: December 02, 2020, 08:51:56 pm
content: Quote from: David Hess on December 02, 2020, 08:43:56 pmI suspect simply lack of TRIM and drives not having known free space which was pre-erased, thus now they need to erase sectors before any write happens. Which slows them down dramatically. Can be cured with secure erase within seconds. Also some old drives may become slow during reading data written a long time ago as their cells discharge, and drive has harder time to determine actual data written. Can be cured by rewriting the data. Upditing the firmware may help as well, as newer firmware may have added functionality to refresh old data.

author: David Hess
author_rank: Super Contributor
timestamp: December 02, 2020, 10:28:45 pm
content: Quote from: wraper on December 02, 2020, 08:51:56 pmI have not encountered either problem yet on my BX500s or MX500s, even the largely full ones, although they all take advantage of TRIM.The MX500 drives support Over-Provisioning accessed through the Crucial utility that I mentioned previously but I do not think it works with RAID. It seems that Flexcap which does the same thing and would work with RAID is only supported on Micron branded drives and not Crucial ones, so maybe the MICRON 1300 series would be more suitable.I have not yet had any reason to use RAID with SSDs.

author: Daixiwen
author_rank: Frequent Contributor
timestamp: December 03, 2020, 07:22:29 am
content: AFAIK when using ZFS RAID arrays you can turn on an autotrim option that will use the trim instruction on free spaces.

author: Kjelt
author_rank: Super Contributor
timestamp: December 03, 2020, 08:31:19 am
content: The most important question has not been asked, how many GB per day will you write on average ?If you mainly read and occasionally write that is a total different situation than when you use ssd's as write cache in a small business.

author: ejeffrey
author_rank: Super Contributor
timestamp: December 03, 2020, 06:37:28 pm
content: I wouldn't use RAIDZ(2) for 200-500 GB of space. I would either use a single 500 GB-1TB drive or if I really wanted the availability of online redundancy I would use two drives in a ZFS-mirror. I would only consider any form of parity raid when I needed more storage than I can get in a single drive at reasonable $/TB. This advice isn't true for 100% of workloads, but the fact that you say "nothing too serious, considering consumer grade drives" makes me think that the added complexity, cost, and overhead of any type of RAID is possibly not warranted.As for SSD choices, mostly what people say here: a non-QLC drive from a reputable manufacturer is what you want. Beyond that it depends on your workload. For light to medium duty read-heavy workload, a TLC drive from Samsung or Crucial is probably fine. For heavier use with lots of writes, MLC drives from Intel or Samsung or even Intel Optane SSDs might be a better choice if you can afford it. Enterprise targeted drives will have higher write endurance and more reliability under heavy use but may fare no better than good consumer drives under light use.QLC doesn't make much sense for any application. The capacity/$ advantage over TLC is marginal especially with the size SLC caches they use to get adequate performance. The reliability and performance are much worse even than TLC.

author: wraper
author_rank: Unknown
timestamp: December 03, 2020, 07:29:38 pm
content: Quote from: ejeffrey on December 03, 2020, 06:37:28 pmYeah, especially considering that 256GB drives cost more per GB and usually have worse performance. And more drives also means more points of failure.

author: ejeffrey
author_rank: Super Contributor
timestamp: December 03, 2020, 08:54:46 pm
content: Quote from: blueskull on December 03, 2020, 09:11:02 amUnfortunately Samsung is now doing that too. It used to be that the Evo and Pro lines were high performance desktop/workstation drives and the difference was TLC vs MLC. But the new 980 Pro is TLC just like the 970 Evo...

author: olkipukki
author_rank: Frequent Contributor
timestamp: December 03, 2020, 09:07:55 pm
content: Quote from: ejeffrey on December 03, 2020, 06:37:28 pmThought that four drives will buy some time if some drive will fail (max 2 per RAIDZ2 spec) and don't require to deal with an issue immediately.The mirroring still on cards, but I threw away the idea to have just one drive.

author: olkipukki
author_rank: Frequent Contributor
timestamp: December 03, 2020, 09:09:19 pm
content: Quote from: ejeffrey on December 03, 2020, 08:54:46 pmReally?*https://www.samsung.com/semiconductor/minisite/ssd/product/consumer/980pro/Quote* still awaiting 980 Pro 1TB availability to utilize PCIe4 on a desktop and never considered that they will downgrade Pro series

author: olkipukki
author_rank: Frequent Contributor
timestamp: December 03, 2020, 09:19:17 pm
content: Quote from: Kjelt on December 03, 2020, 08:31:19 amfrom zero write to max 50GB daily, but not more than approx 200GB weekly.

author: wraper
author_rank: Unknown
timestamp: December 03, 2020, 09:43:41 pm
content: Quote from: olkipukki on December 03, 2020, 09:09:19 pmMLC, if you are blind. Samsung calls everything as MLC because technically it is.

author: nightfire
author_rank: Frequent Contributor
timestamp: December 03, 2020, 10:17:57 pm
content: I agree with going for used Intel or Samsung Server/Enterprise SSDs- they are very good quality.On the consumer end, Crucial MX series deliver good quality performance and have some techniques embedded like thermal sensor ans some tiny capacitors to keep at least the filesystem intact in case of power loss...At Work, I have some legions of Crucial M500 up to MX500 SATA drives, and they are running rock-solid and without big issues.If real server-grade stuff is needed, I would look out for some old Intel S3500, if there are no big amounts of data to write.

author: tszaboo
author_rank: Super Contributor
timestamp: December 03, 2020, 11:08:14 pm
content: I was considering doing this, as my age old SSD dies on me this summer.But it is easier to do regular backups to the spinning rust. PCIe-4 is also a lot faster, and it is hardly possible to do raid on that. Unless you buy some of thes 16x to SSD adapters. Extra points of failure.

author: Kjelt
author_rank: Super Contributor
timestamp: December 03, 2020, 11:10:27 pm
content: Quote from: olkipukki on December 03, 2020, 09:19:17 pmOk so about 70TB/year means you need a big standard ssd or an industrial ssd.Check the datasheets of your ssd of interest and make sure the TBW within warranty matches your needed lifespan.For example a 256GB Samsung 270EVO SSD has a warranty upto 150TB TBW or 5 years which is the first reached. In your case the warranty ends after two years and your ssd might fail within 3 to 4 years.However the same ssd in a 1TB version has a 600TB TBW warranty which in your scenario the warranty of five years is the girst reached.Edit: mistakenly calculated your weekly usage as dayly, in any case you can calculate it yourself and make a decision.

author: ejeffrey
author_rank: Super Contributor
timestamp: December 03, 2020, 11:25:16 pm
content: Quote from: olkipukki on December 03, 2020, 09:09:19 pm3-bit MLC is a term Samsung uses for TLC, everyone else in the industry only calls 2-bit cells "MLC". Samsung is 100% right here BTW, as "TLC" has 8 levels, not three. But still: the 970 Pro uses 2-bit cells, the 980 pro uses 3-bit cells just like the 970 Evo.

author: David Hess
author_rank: Super Contributor
timestamp: December 04, 2020, 03:02:03 am
content: Quote from: blueskull on December 04, 2020, 02:04:04 amThe same applies to Samsung. They helpfully point you to their application note about power loss protection on SSDs, and imply that all of their SSDs have it, while if you check the actual datasheets, only specific ones do. At best, Samsung is deliberately misleading on the subject.

author: SarahSimm
author_rank: Newbie
timestamp: December 04, 2020, 05:42:25 am
content: Samsung is good, but it's kinda pricyCrucial is decent and affordableAlso, why not buying a M2 NVME SSD? They have a higher speed

author: olkipukki
author_rank: Frequent Contributor
timestamp: December 04, 2020, 08:20:33 am
content: Quote from: ejeffrey on December 03, 2020, 11:25:16 pmAh,3-bitMLC (aka TLC)Is it reason why 970 Pro cost more than 980 Pro?!If I'm not mistaken, all Samsung Enterprise SSDs (exceptSM833SM883) are TLC, right?

author: olkipukki
author_rank: Frequent Contributor
timestamp: December 04, 2020, 10:50:39 am
content: Is there way to verify Samsung drives by S/N or any text available on a label?Something like if exist,region etc.

author: nightfire
author_rank: Frequent Contributor
timestamp: December 05, 2020, 12:09:57 am
content: An ordinary mainboard provides usually not enough M-2 slots to go for real redundancy like the OP specified. In that price region is a software volume manager coupled with some affordable, reasonable quality SSDs a very good way to go. And 70TB/Year is also no big deal when it comes to bandwidth. If there are no additional needs like low (and guaranteed) latency, also usually no Enterprise SSD with optimized firmware are needed.

author: David Hess
author_rank: Super Contributor
timestamp: December 05, 2020, 03:18:08 am
content: There are PCIe cards available with M.2 connectors but then you have the find a motherboard with suitable PCIe slots.

author: nightfire
author_rank: Frequent Contributor
timestamp: December 05, 2020, 11:07:26 am
content: Correct. But in terms of reliability, they add up to some electrical overhead and are an additional single point of failure, like the backplane in a storage system...

author: wraper
author_rank: Unknown
timestamp: December 05, 2020, 02:44:20 pm
content: Quote from: nightfire on December 05, 2020, 11:07:26 amSATA already has connectors on both ends and wire in between. So even with additional adaptor board it's not worse than SATA.

author: Marco
author_rank: Super Contributor
timestamp: December 05, 2020, 04:18:46 pm
content: Why RAIDs the SSDs? Why not just do a persistent L2ARC with SSD (possibly RAID0) and use rotating storage for the RAID?

author: nightfire
author_rank: Frequent Contributor
timestamp: December 05, 2020, 10:01:16 pm
content: This is a design question the OP has to answer.But there are many reasons to use no rotational Storage when building a NAS, especially when it is used at home:- Noise- Power consumption- with lower power consumption lower heat to care about- smaller footprint, therefore higher availability of compact cases- with smaller footprint, possilby higher WAF (Wife acceptancy factor)- lower rebuild time in case of failureDepending on the actual usage scenario, it can happen that the system will not be used to transfer high amount of data, and may not be used all day long, but when it is needed, high IOPS are very important, especially in reading applications. (Writes can be cached to some extent, but reads not).Given the cost per Terabyte on nowadays consumer SSDs around the 1TB-2TB sizes, especially when bigger Volumes of Data are not needed, the advantages of an SSD only system can be quite compelling...

author: ejeffrey
author_rank: Super Contributor
timestamp: December 05, 2020, 10:03:57 pm
content: Quote from: Marco on December 05, 2020, 04:18:46 pmBecause they only need 250 GB and probably don't even need raid at all. Going to mechanical disks in raid plus an SSD l2arc is way more complicated and adds more points of failure and ways for a user to screw things up?It really sounds like the answer here is a 500GB or 1 TB TLC or MLC consumer SSD with regular backups.

author: olkipukki
author_rank: Frequent Contributor
timestamp: December 07, 2020, 11:04:04 pm
content: The budget allows to buy 1/2 TB enterprise drive for RRP price or four 1/4 TB consume drives.Thoughts 4x drives are better under ZFS than just one whatever is it.As per experience, HDD/SSD is most common hardware component to be failed over time.ZFS is biggest selling point - I can deal with a failed drive later rather than require immediate attention.There is space just for set of four 2.5" enclosure and no free PCI slots, although I would still prefer SATA in this case.

author: olkipukki
author_rank: Frequent Contributor
timestamp: December 07, 2020, 11:13:46 pm
content: Quote from: blueskull on December 03, 2020, 09:11:02 amThere are a lot of used or even new* (*aka opened never-used) enterprise drives for a fraction of RRP price.Did they find a way to reset firmware counters and relabel drives?""to be good to be true"

author: nightfire
author_rank: Frequent Contributor
timestamp: December 08, 2020, 12:02:03 am
content: Depending on what brand or such, it is an open secret that HP uses Intel Sata SSDs for their servers. One reason for the quite hefty markup is caused by the warranty replacement policy or support that the manufacturer has to provide.Some big Datacenter companies also routinely swap out their servers after 3 years, which means that lots of equipment and the spare parts on stock get unloaded routinely to refurbishers and second hand market. Of course, one has to have an eye what gets offered, if parts are announced as "brand new, open box only".Yes, it can happen in professional world, that a technician gets some spare parts for onsite repair and sends back some not used parts in an open box, but usually those get recycled with the next tech onsite...For the OP, some lightly used Intel S3500 (or S3510) could be a very good deal- have some here @work and they work like a charm.

author: olkipukki
author_rank: Frequent Contributor
timestamp: December 08, 2020, 11:55:22 am
content: Great, make sense now why soo-o many Samsungs on eBay...Intel S3nnn looks like a very attractive option

author: Wuerstchenhund
author_rank: Super Contributor
timestamp: December 08, 2020, 02:27:45 pm
content: Quote from: blueskull on December 03, 2020, 09:11:02 amCareful with intel, many drives are OEM drives which have special firmware which may or may not be available.Pro Series drives also don't have a particularly long endurance.Also, intel has now sold its flash memory business to Kioxia (Toshiba), so the future of firmware updates and support even for retail drives is somewhat uncertain (and Kioxia doesn't offer direct end user support or firmware downloads as intel did).My go-to brand for SSDs are Micron drives, so far they have been the most reliable ones, and they are (comparatively) moderately priced.Quote from: olkipukki on December 07, 2020, 11:13:46 pmA lot of these drives are specialty drives with custom firmware for big OEM customers like Facebook. Others are refurbs with reset counters sold as new, or simple fakes (especially M.2 drives).The specialty variants are particularly bad as its impossible to get firmware updates for them. But then so is it for genuine Western Digital Enterprise SSDsQuote from: nightfire on December 08, 2020, 12:02:03 amWhen HP was still making servers (which it hasn't for a few years now as the server business was spun out into a separate business called HPE some time ago) it used pretty much anything from intel, Samsung (Enterprise models, not the EVO/Pro crap), HGST, WDC, SanDisk, you name it.Today HPE does the same, as does Dell.QuoteNo, it's pure racketeering, nothing more. OEM drives in HP/Dell/Lenovo servers use the same drives with the same baseline firmware, just extended by a different ID (HP/HPE firmware changes also involve different smart parameters to feed the SSD Fuel Gauge). This is mostly to lock the customer in to pay over the top for what is essentially the same drive as the retail variant, just that the latter has 3 or 5 years warranty while many OEM drives only come with one or two (they used to take over the warranty of the server, not anymore).However, it's the price you pay if you want support from the server vendor.