SSDs in RAID 5 - Page 1
Forum section: electronics / general

author: grumpydoc
author_rank: Super Contributor
timestamp: February 01, 2025, 06:18:15 pm
content: I have a NAS/media server/web & email server/general PC consisting of a MSI Z690 ITX m/b about 68TB of raw disk, half spinning rust and half SSD. Mostly mirrored RAID pairs with a couple of individual drives used as scratch space/temporary stuff.Despite what seems like generous storage the media tree is near full - it consists of 4 8TB drives in RAID10 giving 16TB of useable space.So, I am converting the 4 drives to a 4 drive RAID5 array which will expand the useable storage to 24TB and keep me going for a few more years as I rip more stuff. I can't physically add more drives and I'm out of SATA ports anyway (4 on the M/B and 6 on a PCIe card).Here's the rub - the disks involved are SSDs - specifically 8TB Samsung 870 QVOs so this is a controversial move.I have a few thoughts on this but would be interested to hear the group views - would you do it, have you done it and what was your experience?

author: Halcyon
author_rank: Unknown
timestamp: February 02, 2025, 01:05:23 am
content: What operating system are you running on this machine? If it's TrueNAS, I wouldn't be using RAID at all, rather utilise ZFS.Your options for 4 disks are pretty limited, especially since you're preferring capacity over fault tolerance.Also, it goes without saying, RAID is not a backup, so ideally you'd be looking to have at least a full second copy of your data elsewhere.

author: Shock
author_rank: Super Contributor
timestamp: February 02, 2025, 06:24:46 am
content: I'd use pro grade data disks in pairs and keep one main disk online and it's partner a standby incremental backup. The first pair would be primarily for new and most used data.The other data disk pairs would be standby, each having a corresponding offline incremental backup partner.This allows you to bring them online if required and if expansion is needed add two drives in a data/backup pair to replace the primary disk set.The OS and App disk I'd keep SSD. You could add redundancy or multiple scratch drives if you wish.Edit:There are a bunch of benefits to this system, especially in the finer details as you can manage everything yourself at a granular level.The main one is the standby/offline (backup) disks can be on another low workload machine or NAS and maximizes local storage and space availability of the online/standby data disks.As with anything it depends on your usage case but if there is a clear separation of "use once" or "archival" type data it shouldn't be live with the best read performance money can buy.It should be sitting somewhere where it can stay safely and happily offline to spin up 10 years from now while being periodically tested and checked against a backup.

author: Siwastaja
author_rank: Super Contributor
timestamp: February 02, 2025, 08:42:55 am
content: Quote from: Halcyon on February 02, 2025, 01:05:23 amIt is of paramount importance to repeat this over and over again. Even people deeply involved in information technology often fail to grok it.RAID -> minimize downtime and even small loss of very recent data, but only protect against some types of hardware failuresBackups -> minimize data loss in case of hardware failures, software failures and human errorsSince software failures and human errors tend to dominate, backups are catch-all solution. The only benefit of RAID is shorter downtime or ideally, no downtime in case of simple hardware failure, and recovery of very recent stuff not backed up yet."Media server" and especially "ripping stuff" (implying there are originals - if you lose one month of stuff you can re-rip them), to my ears, sounds like a case where small downtime, or losing a bit of recent data, might not be a big issue at all. In such case, backups would give much better coverage against different types of failures.Ideal solution could be something as simple as manually making backups of irreplaceable media, and forgetting about those that are not that valuable, or that are simple to re-rip.

author: Halcyon
author_rank: Unknown
timestamp: February 02, 2025, 08:57:49 am
content: Quote from: Siwastaja on February 02, 2025, 08:42:55 amI've been guilty of it myself. It's easy just to set it aside and "do it later".

author: NE666
author_rank: Frequent Contributor
timestamp: February 02, 2025, 09:01:37 am
content: In addition to the good advice given above (RAID being aimed at availability, more than durability), also keep in mind that RAID controllers and the commonly used RAID algorithms were devised at a time when disk IO throughput and latency were one or more orders of magnitude worse than today's SSDs provide. You'll need quite some controller not to bottleneck the raw performance of your SSDs. Consider also the time taken to perform a rebuild on a large RAID 5 volume.For photo/media/ripping storage I'd be inclined not to use RAID 5 on my 'working set', but institute a good backup or replication mechanism to a secondary archival system based on something like Ceph.

author: BradC
author_rank: Super Contributor
timestamp: February 02, 2025, 10:02:18 am
content: Quote from: Siwastaja on February 02, 2025, 08:42:55 amRAID is *all* about availability. It doesn't and can't prevent against "small loss of very recent data". It's all about keeping the machine up and running. If the failures > redundancy then all bets are off. The loss may be small or big, but a loss is a loss.This is why "RAID0" is not RAID. There's no redundancy.I've only ever run SSDs in RAID1 or RAID10, and spinning disks in RAID6 or RAID10. There's no reason RAID5 can't work with an SSD however.Now, there's also no theoretical reason a QLC drive won't perform in a RAID5. The first build or any rebuild might be pretty ugly in performance once you overwhelm the SLC cache, but I can't see why it wouldn't work.

author: Nominal Animal
author_rank: Super Contributor
timestamp: February 02, 2025, 10:17:10 am
content: Quote from: NE666 on February 02, 2025, 09:01:37 amThis is especially important since the current one is RAID10, because that involves no checksumming and the softRAID0/1/10 support by Linux and *BSD kernels (almost certainly used on the NAS) is known to be particularly efficient. I mean, we broke the gigabyte per second barrier (sustained data rate on contiguous data on ext4) on Linux softRAID0 (eight striped drives) on consumer spinning-rust hardware some two decades ago.For a NAS box, and especially for this case, RAIDZ1 (ZFS with its internal RAID5 implementation) makes the most sense. Assuming you have enough temporary storage to reconstruct the entire array from scratch.

author: hans
author_rank: Super Contributor
timestamp: February 02, 2025, 10:19:42 am
content: I'd also vote for ZFS, personally. It has helped me with data rot on spinning rust before. I have 3x WD Red 4TB drives running for 7years now, and ~2 years ago there was 1 drive that got thrown out of the pool. The drive itself said it was 100% fine, there were no error in SMART data, self-test was OK, etc. I think ZFS did checksum inspections while doing a scrub, got bad data for a handful of blocks, and then took action.Since the SMART seemed fine, I tried to resilver the pool once to see what would happen (yes I have backups). It resilvered about 2MiB of data and was fine ever since. There is a lot of stationary data on that pool as it includes most of my historic data in an archived format.Ever since the drive still operates as normal.I'm not certain if the controversial QLC has the same problems. Certainly there are stories of SSDs having a max retention span if you keep them turned off, but maybe its still a concern when powered on but idling. ZFS does frequent scrubs to prevent this from happening.QLC also has issues with low cycle counts, but if a drive contains mostly stationary data with a decent amount of overprovisioning, this shouldn't be a large issue. Especially on 8TiB x 300 cycles = >2.4PiB written rating. Even taking half of that rating in RAID5 or RAIDZ1, you'd be looking at writing close to a TB to the pool each day for the next 10 years.

author: David Hess
author_rank: Super Contributor
timestamp: February 02, 2025, 11:36:23 am
content: ZFS is probably a better choice, but RAID 5 will work for what you want also. I have an old system that uses RAID 5 to do exactly what you describe but with 4 x 4GB hard drives to make a single 12GB volume. At some point this year, I will test using external USB connected SSDs to make a large redundant volume, but using Windows Storage Spaces instead of ZFS or RAID 5. I have had problems with ZFS, or more likely Samba, as a file server in a Windows environment.Quote from: Siwastaja on February 02, 2025, 08:42:55 amIn the proposed application, RAID 5 is being used to expand volume size without the increased risk that a single drive failure will destroy the volume.

author: Siwastaja
author_rank: Super Contributor
timestamp: February 02, 2025, 01:42:24 pm
content: Quote from: BradC on February 02, 2025, 10:02:18 amOf course it does and can - that's the whole point. In RAID1 or RAID5, if a disk fails, another disk(s) hold copy (directly, or such that data can be reconstructed). Compare this to RAID-less backups where, if you backed up a week ago, you lose week worth of data if the drive fails.Therefore, RAID offers availability of service, and can restore data lost by hardware failure even before it was backed up. It is enough that data was stored by the original filesystem write - there is no additional "back-up" delay at all. Therefore, RAID is capable of completely preventinganydata loss during disk failure.OTOH, backups protect against other types of errors RAID cannot. They are orthogonal and can be combined.Next time I suggest you read what you reply to more carefully before anger-replying, because I'm quite sure you know perfectly well how this works and actually will agree with what I said.

author: vad
author_rank: Frequent Contributor
timestamp: February 02, 2025, 02:31:23 pm
content: May I know the reason for using four 8TB SSDs in RAID-5/RAID-Z1 instead of a pair of 24TB mechanical drives in a mirror? The latter costs half as much and has a better chance of recovery if the host itself crashes or loses power unexpectedly.

author: Shock
author_rank: Super Contributor
timestamp: February 02, 2025, 02:42:31 pm
content: The is no "does" only "can", which is like "may" and also like "may not" and in certain cases "will not". Which is bad mmkay, well at least for the data. With luck the array may still work so you can check what data you lost.

author: Kjelt
author_rank: Super Contributor
timestamp: February 02, 2025, 02:51:42 pm
content: Quote from: grumpydoc on February 01, 2025, 06:18:15 pmSince febrary 2023 I have a Synology Ds1522 with four 4TB Samsung 870 QVOs in SHR (Synology Hybrid Raid) config.I had the same doubts in the beginning but have had zero issues and very happy with the choice.The reason I went for SSD was power consumption (24/7 usage) and speed.I get sustained 800 MB/s write (till the buffers are full after +/- 20GB write it is 400MB/s)and 990 MB/s read speed over a 10Gb home network.For the weekly backups I have a NAS with datacenter HDDs.So yes I would recommend it when speed is needed else esp. for large storage I would still go to hdds.

author: coppice
author_rank: Super Contributor
timestamp: February 02, 2025, 03:06:36 pm
content: Quote from: Siwastaja on February 02, 2025, 08:42:55 amRAID -> minimum loss when something wears out.Backups -> minimum loss when the building burns down, assuming you store them off site.

author: vad
author_rank: Frequent Contributor
timestamp: February 02, 2025, 03:24:49 pm
content: Quote from: coppice on February 02, 2025, 03:06:36 pm+ Multiple historical backup copies with a reasonable retention policy to minimize data loss from corruption, ransomware attacks, accidental deletion or modification.Also, follow the 3-2-1 rule: have at least 3 copies of your unique data, stored on two different types of media, with at least one copy kept offsite.

author: Kjelt
author_rank: Super Contributor
timestamp: February 02, 2025, 03:36:06 pm
content: Quote from: vad on February 02, 2025, 03:24:49 pmYes but really who does that ? And also then dayly, weekly, monthly, quarterly, yearly keep safe versions ? Companies and a few die hard consumers perhaps.Rule one: select the data that you will cry and tear your hair out of your head for if you loose it, then make sure multiple instances are regularly created and verified including par2 recovery sets.You don't want to know the people that store their precious data on usb sticks just to find out five years later that flash has leaked and some of their data was lost.

author: vad
author_rank: Frequent Contributor
timestamp: February 02, 2025, 03:48:13 pm
content: Quote from: Kjelt on February 02, 2025, 03:36:06 pmI think we’re talking about the same thing - the unique data. Everyone has their own unique data they would hate to lose, such as precious family memories, source code or KiCad projects for hobby work you’ve dedicated thousands of hours to, manuscripts of books, etc.You don’t need to keep copies of non-unique data, such as installers for the latest KiCad version or ISO images of your favorite Linux distributions, he-he.

author: Siwastaja
author_rank: Super Contributor
timestamp: February 02, 2025, 05:46:55 pm
content: Quote from: coppice on February 02, 2025, 03:06:36 pmSoftware (when ran as intended) misusing files (doing something unintended to them), and accidental commands on files, and malicious acts on files are all more probable than building burning down.

author: NE666
author_rank: Frequent Contributor
timestamp: February 02, 2025, 07:13:47 pm
content: Quote from: Siwastaja on February 02, 2025, 05:46:55 pmQuite. Which is why industry is now very into 'immutable' backups and long-term retention. It's a 'bit of an arse', if all your backups have been infected with encryption malware too, and have been for some months.

author: coppice
author_rank: Super Contributor
timestamp: February 02, 2025, 07:50:19 pm
content: Quote from: Siwastaja on February 02, 2025, 05:46:55 pmPerhaps, but I been involved in recovering data from off site backups multiple times when there had been a fire, flood or other serious damage. The flood was actually a fire detection failure.

author: fzabkar
author_rank: Super Contributor
timestamp: February 02, 2025, 07:59:50 pm
content: There is some merit in choosing dissimilar drives for your RAID array. The following case is just one instance where all 4 drives failed at once:https://goughlui.com/2022/08/20/notes-ssdraid-recovery-samsung-870-evo-not-to-be-trusted/I recall another case where all the SSDs in a RAID failed due to a 40,000 hour firmware bug (or some similar time frame).

author: coppice
author_rank: Super Contributor
timestamp: February 02, 2025, 08:08:11 pm
content: Quote from: fzabkar on February 02, 2025, 07:59:50 pmI've experienced a couple of servers that were bought and set up with several (non-RAID) hard drives, and run for several years with no problems. Then one day a disk started giving trouble, and within a few days all the others did too. If you have a RAID array set up with all new drives and a drive eventually fails don't hang around. Replace it quick, and rebuild the set. There are lots anecdotes around that corroborate my personal experience of a bunch of disks from a single production batch, set up and run together, failing within a very short time of each other.

author: coppice
author_rank: Super Contributor
timestamp: February 02, 2025, 08:11:16 pm
content: Quote from: NE666 on February 02, 2025, 07:13:47 pmThese days some multi-site organisations are using servers backing up to each other at multiple locations as a form of off site backup. I'm suspicious of the validity of this. If something nasty happens on their network, like a malicious attack, they could lose every site. Air gaps seem line a wonderful thing.

author: NE666
author_rank: Frequent Contributor
timestamp: February 02, 2025, 08:29:01 pm
content: Quote from: coppice on February 02, 2025, 08:11:16 pmAnd you're right to be. That's no longer viewed as a credible approach as-is, not if your data is mission critical and/or you're likely to get hit with massive regulatory sanctions if it all goes wrong.Quote from: coppice on February 02, 2025, 08:11:16 pmhttps://piql.com/"This binary code is converted to grey pixels that are then written to film in four levels of grey, with 8 million data points per frame. Each film has capacity for 120 GB."

author: David Hess
author_rank: Super Contributor
timestamp: February 02, 2025, 08:59:50 pm
content: Quote from: fzabkar on February 02, 2025, 07:59:50 pmI have been using Crucial drives, but borrowed a 4TB Samsung 870 EVO at one point which produced data corruption within days. I never figured out what happened, but others at the time reported the same issue. I never used Samsung originally because their documentation was so vague about data protection in the event of power loss.

author: grumpydoc
author_rank: Super Contributor
timestamp: February 02, 2025, 09:02:39 pm
content: Thanks everyone. Especially for not immediately telling me I'm mad for even thinking about it. There follows a long post, sorryQuote from: Halcyon on February 02, 2025, 01:05:23 amFedoraAs I said the box winds up being a general purpose server as well so a NAS operating system would not be a great fit.I've thought about alternative arrangements including separate NAS storage or using BTRFS or even ZFS but I've more or less always come back round to a relatively simple setup of mdraid and/or LVM and ext4.QuoteTotally agree - the data is backed up periodically to a separate system kept off site, though as it is all media it could be re-ripped if needed (not a task I'd relish though).Quote from: Shock on February 02, 2025, 06:24:46 amI'd love to but my name isn't Linus SebastianAn enterprise U.2 or U.3 SSD with similar capacity is 2-3x the price of the 870 QVOQuote from: NE666 on February 02, 2025, 09:01:37 amI have the m/b SATA ports and a 6-port PCIe card based on the ASMedia ASM1166. Generally I try to make sure each half of a mirror uses a different path to the CPU.Consumer grade PCIe SATA cards are generally awful of course and usually have way too many SATA ports for the available PCI bandwidth, a 6-port card built around the ASM1166 is one of the least awful solutions as it can claim 16Gbps of PCI bandwidth so would be able to talk to six HDDs at pretty much full speed. Itwillbe a bottleneck if I put six SSDs on it. Partly for that reason (and partly to temporarily have more drives so I can copy data around more easily) two of the SSDs are on USB3 adapters (ASM1051E), I'm debating leaving them there longer term.I've previously thought about getting a decent SAS HBA but the Asmedia card has been OK for what I want, or at any rate has been up to now.I get about 1.7GB/s read from the RAID5 array now it is built.Quote from: BradC on February 02, 2025, 10:02:18 amHold those thoughts, I'll get back to them.Quote from: vad on February 02, 2025, 02:31:23 pmYes, that might be an option but I don't have any 24TB drives so would have to buy two and individually they are about the same as an 8TB 870 QVO.Yes the price per GB is 1/3rd but I wanted to transition this server away from HDDs for reasons of noise and power consumption.Quote from: Kjelt on February 02, 2025, 02:51:42 pmGood to know you have a similar setup which is working well for you. That's really useful.So - here are my own thoughts.This array has a relatively specific use case - I rip, occasionally download, media files which I might tweak one way or another with ffmpeg and which are then served locally using minidlna (or ReadyMedia as it is now officially known). There is not a huge volume of traffic on the array in normal use.If one searches the 'net there are lots of reasons you mightnotwant to build a RAID5 array out of SSDs, and especially not the 870 QVO.Some worry about the differing failure modes of SSDs, compared with HDDs - specifically the worry that all drives in an array might reach their endurance target and fall off the array simultaneously. I actually think this is a) unlikely and b) can be mitigated against.Techreport did a review of SSD endurancehttps://techreport.com/review/the-ssd-endurance-experiment-theyre-all-dead/https://web.archive.org/web/20200705090223/https://techreport.com/review/26523/the-ssd-endurance-experiment-casualties-on-the-way-to-a-petabyte/Although the review is a bit old now and did not include QLC drives it is clear that the TBW endurance figures quoted my manufacturers are relatively conservative, they did have an issue with a Samsung 840 series drive just disappearing but this was when something like 8x the quoted TBW figure had been written to the drive.When I initially started replacing HDDs with SSDs in the server I was cautious and just replaced one disk in each mirror, looking at the stats for the hard drives remaining I see that the oldest has been in the machine for 6 years and has had 345TB written to it. At 50TB/year the 870 QVO (with an endurance spec of 2880TBW) will last 50 years before it hits its endurance target.In an environment which is more intense this can surely be mitigated by monitoring the drives for TBW and wear levelling stats - a drive approaching it's specified TBW can be "retired" from critical data to "light duties" and undergo planned replacement in the array. this does, perhaps, involve a slight change in thinking - from "RAID means I can run my drives to the point of failure and not worry" to "RAID means I can protect from unforeseen drive failures" - but I'd argue the latter view is the one more in the spirit of using RAID in the first place.After that we're looking at random failures which can happen to any drive and an all-SSD array should not be more prone to multiple simultaneous failures than an all-HDD array.Except that there have been instances of firmware bugs which have wiped out SSDs (a scenario fzabkar mentions) - to be honest I'm not sure that SSDs have a higher rate of fatal firmware bugs than HDDs but my main defence against this sort of thing which is to always build arrays from heterogeneous groups of disks is not available to me - if you want an 8TB consumer grade SATA SSD the 870 QVO is the only game in town. One of the drives is still on version 1 firmware - not sure if I should upgrade as Samsung, rather irritatingly, don't issue change logs for their drive firmware.Moving on, another criticism of RAID5 with SSDs is that RAID5 exhibits "write amplification" - two blocks (data+parity) are written for every block of data written to the array. Um, yes, OK - well my drives might wear out in 25 years rather than 50 so chances are they will still outlive me. YMMV and if you are hitting your disks hard you might want to use an appropriate (i.e enterprise) drive.Mirrors do the same - the data is written twice and people seem happy with SSDs in mirrors so I don't understand this objection. People also get hung up on the RAID "chunk" size thinking that this means the minimum amount of data written for any update, but it's a layout thing, not a read-modify-write block (OK, that does matter at a low level for the drive, but that's what drive cache is about). I honestly don't think this is an issue.People also worry about write speed - MLC, especially QLC is relatively slow to write data. Caches can help (and also ensure optimal use of erase blocks) but caches are not infinite and when they fill the write speed tanks, the 870 QVO is especially bad for this as one the relatively modest 78GB cache is full the sustained write speed is just 160MB/s - slower than many HDDs, but still faster than ffmpeg or the disk ripper, or the internet will be writing to the array.I can't get worked up about this. If I cared for my application I'd use different drives but we're back to the point that the 870 QVO is the only 8TB SSD available without having to take out a mortgage. In any event it is quite likely any sustained write I will make to the array will be at less than 160MB/s which makes it "fast enough" for my needs.The one scenario this does matter, of course, is array rebuilds in the event of a drive failure - but Linux software RAID throttles the I/O rate for rebuilds - I think I saw about 110MB/s during creation of the array. For reasons of keeping the new and old array continuously available I started with a two disk degraded array, added a 3rd disk, then finally grew the array to four disks - one of the HDDs in the original arrangement gives occasional soft errors which I think is a SATA cable which needs replacing and I didnotwant the array to fail during data transfer, so its companion mirror needed to be moved last.So far I'm comfortable with the result, I'll probably do that firmware upgrade and I'll almost certainly keep the old array available until I can get the backup server "home" and refresh its copy of the data (the internet connection where it lives is a 4G router so backups involve bringing it back here and copying the data on the LAN).ZFS, yeah, not yet - or, indeed, XFS or BTRFS. Checksumming data is attractive but incurs quite an overhead (and don't forget the default in ZFS is metadata only IIRC). However, the best file system to use is something which is very much for debate, I just don't feel I have the workload for the enterprise features.

author: Monkeh
author_rank: Super Contributor
timestamp: February 02, 2025, 09:14:17 pm
content: Quote from: grumpydoc on February 02, 2025, 09:02:39 pmbtrfs is essentially pointless - the only redundancy mode it's safe to use in is mirror, everything else is broken. Zero advantage to putting it over an underlying RAID stack, IMO.ZFS checksums all data. Not much point in doing otherwise. XFS is metadata only.Fundamentally you have a fairly simple choice - use a basic RAID (md) and a simple filesystem over the top, or use a modern filesystem (ZFS). ZFS has a few good toys to play with, like deduplication.

author: grumpydoc
author_rank: Super Contributor
timestamp: February 02, 2025, 09:22:33 pm
content: Quote from: Monkeh on February 02, 2025, 09:14:17 pmAh, was probably thinking of XFS then, thanks for the correction.I long ago decided to avoid BTRFS - I periodically check back but haven't got the impression it is ready for prime time yet. Keep an open mind though

author: Monkeh
author_rank: Super Contributor
timestamp: February 02, 2025, 09:25:23 pm
content: Quote from: grumpydoc on February 02, 2025, 09:22:33 pmI've used it quite a bit. It has some.. substantial performance problems with spinning rust, and is entirely non-robust. Obviously without any redundancy, lost data is lost.. but in practical terms you lose the entire filesystem as a viable image, in my experience. This is not agreeable over a block or two. They also appear to have made approximately zero progress in this area, or in non-mirror redundancy safety, over at least the last five or six years. Dead end, IMO.

author: Kjelt
author_rank: Super Contributor
timestamp: February 02, 2025, 09:26:47 pm
content: Quote from: coppice on February 02, 2025, 08:08:11 pmTwo friends of mine had the exact same experience and they think it had to do with rebuilding the set that stresses the left over drives (took 72 hours 100% usage to rebuild).Make sure when rebuilding to check the proper cooling of the drives that they don't become too hot or add active cooling.Rule of thumb for me is to preventively replace the entire set every four years or even just buy a new NAS with new disks and use the old one as yet another extra backup vault.

author: hans
author_rank: Super Contributor
timestamp: February 02, 2025, 10:47:27 pm
content: QuoteConventional HDDs also have ratings, which often are far worse.Take a prosumer "WD Red Pro NAS Hard Drive - 24TB" (500$) which is advertized with "WD Red® Pro drives are engineered to handle high-intensity workloads in 24×7 multi-user commercial and enterprise NAS environments. "Meanwhile,open its datasheetto find out that: Workload rate: 550TB/year (defined as transferred data, so I take that as either read or write). So ZFS scrubbing that drive each month while its full: 12 x 24TB = 288TiB. Don't try to filling the drive up in the same year, otherwise you might go over this spec.(Also don't take a look at less prosumer line-ups of drives.. some >20TiB drives are not rated to be read/written to more than a handful times per year)A 8TiB SSD that can handle 2.88PiB of data written to it => 2.88PiB/5 years = 576TB/year. And you get the reads for free. However, for a home environment, I think most people will simply drive their hardware into the ground. It either fails, gets outdated, or gets replaced some time after the warranty runs low and anxiety of impending failure runs high.Some businesses will simply retire hardware after the warranty ends and with 3 year on this SSD, it's certainly not enterprise grade.QuoteI think there are countless of these tests, and although each and everyone is anecdotal, I think the general picture is that drives can far outlast their P/E cycle rating.. but its always with a big asterisk in my opinion, as often these tests take ratings in a slightly different way.Typically FLASH is rated to hold data for a certain timespanafterit has undergone a certain number of P/E cycles. Just like aluminum capacitors must meet their rating (capacitance, ESR) after being exposed to a certain number of hours at a certain temperature.These FLASH chips must still have retained the (enough) data correctly. Similar to what I elluded to with 'bit rot' in spinning rust. These stress tests are often highly accelerated and do not take storage time into account, which could be why they report >3x higher numbers as the manufacturers claim. But these manufacturers also know how much errors their ECC can take, what probability error their "rating" cuts of at, etc.This video does try to take storage time for a fresh and worn SSD into account. Even though its still anecdotal (sample size of 1 for each test condition), I still think its interesting enough to share:https://youtu.be/rx3Y5x6uzKQBut, unpowered harddrives is not a similar operating condition as it is in a NAS/home server. As I referred to in my previous post.. one of the nice features of ZFS is that it performs data repair for these kinds of bitrot on the fly. Having monthly scrubs also means that these data repairs are operated in a high availability mode with little nuisance to the end user, as a drive may hang for a few seconds before it gives up on reading a block of data (or has reallocated it to a spare sector, which means ZFS didn't safeguard the data through the RAID array, but it still prevented the data from rotting too far I guess).

author: grumpydoc
author_rank: Super Contributor
timestamp: February 02, 2025, 11:10:23 pm
content: Quote from: hans on February 02, 2025, 10:47:27 pmI don't think anyone is claiming that the 870 QVO is enterprise grade.You want something like the Micron 7450 Pro 7.68TB which has TBW a 28,000TB - 10x that of the 870 and a 5 year warranty.It's $1400 though compared with $560 for the Samsung, and has an I/O interface which is not common (does it exist at all?) in consumer PCs.QuoteInterestingly I could find only a few which try to determine just how far you can push SSDsQuoteWhich is fair enough - I would say taking any SSD beyond it's rated TBW is something you should only do if you can cope with your drive failing and have good backups.People will do what people do though.QuoteYes, I agree the "self healing" aspect of ZFS is really neat.