You might lose all your data in 2016 - Page 1
Forum section: electronics / general

author: Shock
author_rank: Super Contributor
timestamp: December 11, 2015, 04:54:44 pm
content: This year I had 2 HDDs fail, both of which were not properly backed up. You would think I learned my lesson after the first failure but it wasn't a bad one, the second failure took out a years work with it. I was lucky and managed to get most of my important data back but it was a really big job.I also had another unrelated data corruption problem, so just a reminder to keep multiple backups of important data separately and secured or off premises if you need to.The CDs and DVDs you burned years ago are also starting to fail as well.

author: MarkF
author_rank: Super Contributor
timestamp: December 11, 2015, 05:15:10 pm
content: Yes. CDs and DVDs have a shelf life. I seem to remember it's around 10 years.I stopped using them for backups. Instead, I use an old computer with a large drive as my main backup and backup all my data to it. The computer for backup is ONLY used for backups and is NOT turned on other than to do a backup. In addition, I use a USB drive as a secondary backup.The risk of 3 hard drives failing all at the same time is slim and easier than a bunch of CDs or DVDs.

author: Thorondor
author_rank: Contributor
timestamp: December 11, 2015, 05:24:39 pm
content: I try to follow the rule of three: three copies of important stuff, two different kinds of media, at least one copy stored offsite.I wish LTO drives were more affordable. As it is relatively unreliable hard drives are the only option a home user has for backing up terabytes of data on a regular basis.

author: daqq
author_rank: Super Contributor
timestamp: December 11, 2015, 05:25:53 pm
content: In addition to a locally stored copy (on a USB key used only for backup) I also have a zipped and encrypted archive that is on google drive. Works for me so far, I've got about 10GB of critical data.

author: Shock
author_rank: Super Contributor
timestamp: December 11, 2015, 05:38:39 pm
content: Tape is pretty horrible as a medium, too many things to go wrong. At least with disk in most cases the media has to fully break down or have a bad head crash for it to be forever lost and otherwise it's cheap and fairly fast.

author: moya034
author_rank: Regular Contributor
timestamp: December 11, 2015, 06:07:47 pm
content: Setup a file server with 3 HDD's and RAID 5. Setup a backup server with another 3 HDD's and RAID 5 which replicates any changes made on the file server on a daily basis. If you are concerned about your house burning down, then setup a VPN to your buddy's house and put a file server there

author: Thorondor
author_rank: Contributor
timestamp: December 11, 2015, 06:17:03 pm
content: I just keep a stack of hard drives and some BD-RE disks under my desk at work. It's a lot faster to rotate hard drives than to use my pitiful DSL connection for any sort of backup.

author: nctnico
author_rank: Super Contributor
timestamp: December 11, 2015, 06:20:05 pm
content: Quote from: Thorondor on December 11, 2015, 06:17:03 pmRsync does quite a good job on keeping files in sync over a low bandwidth connection. Only the initial upload takes long.

author: MrSlack
author_rank: Frequent Contributor
timestamp: December 11, 2015, 06:24:27 pm
content: My backups:* laptop itself, encrypted disk.* rsynced to my desktop PC, encrypted disk.* synced with Google drive (apps, not the free one which has a different SLA)* manually mirrored encrypted USB stick that lives on my keyring (corsair survivor)* archived monthly to USB hard disk in fire safe at home.* archived every quarter to Amazon s3 with duplicity.And I'm still paranoid I'll lose everything.This paranoia got triggered after losing an entire raid array and finding the DVD backup was corrupt. Lost 500mb of important photos.

author: Augustus
author_rank: Regular Contributor
timestamp: December 11, 2015, 06:36:29 pm
content: What about M-Discs? Every now and then I see them mentioned somewhere but I don't know of a single person who has ever used one. Are they really as reliable as advertised? Or just another snake-oil type of product?http://www.mdisc.com/

author: suicidaleggroll
author_rank: Super Contributor
timestamp: December 11, 2015, 06:51:34 pm
content: QuoteI doubt itI have all centralized data/media (movies, music, pictures, etc.) stored on a 4-disk RAID 5 on my home server, which is then mirrored daily onto a backup system in the basement, and also mirrored daily onto an encrypted drive stored off-site. All computers (laptops, etc.) back up their entire filesystem to the backup system in the basement nightly, using incremental backups (rsync --link-dest), which are also mirrored to a second drive on the same system, and the most recent of each of those backups is also mirrored daily on the encrypted drive off-site.

author: PE1RKI
author_rank: Regular Contributor
timestamp: December 11, 2015, 06:58:56 pm
content: if you actively use seagate barracuda hd's, start backup now and thank me later.

author: moya034
author_rank: Regular Contributor
timestamp: December 11, 2015, 07:09:18 pm
content: Speaking of losing data, has anybody had an SSD crash on them?

author: John Coloccia
author_rank: Super Contributor
timestamp: December 11, 2015, 08:00:46 pm
content: I have a Synology RAID with 2x3TB drives, mirrored, for daily backup. I also have hard drives that I periodically do full backups to. It interfaces with a little SATA dock. So total, things are generally stored on 5 different drives. 6, actually, because most of my important files are in source control, and I typically download the lastest to my laptop as well.

author: Shock
author_rank: Super Contributor
timestamp: December 11, 2015, 08:03:04 pm
content: Quote from: PE1RKI on December 11, 2015, 06:58:56 pmI wouldn't believe everything you read, but they did have bad firmware issues a few years back which was addressed and fixed. Seagate has been on the trailing end of a fanboy war for years. It all started when Western Digital started marketing their Raptor drives, gamers were all over that. Many years before that probably about 15 years ago I recall a Western Digital scare, then before that a Quantum scare. Heard rumors about Samsung HDDs a while back as well.If you are referring to the Backblaze stats, the fact that they got people to mail in external disks and "shuck" them is proof in itself they are morons, without bringing up the rest of their shenanigans. There is a big problem with hardware bias that has been going on since the dawn of time, without doing a proper analysis and post mortem their results are meaningless.You only got to listen to people "I'll never buy another (brand) again" based on a failure. Of course they forget about this when the next brand fails, by the third failure they are buying a tablet or a new TV per year and have been successfully brainwashed.

author: hammy
author_rank: Unknown
timestamp: December 11, 2015, 08:15:09 pm
content: Quote from: moya034 on December 11, 2015, 07:09:18 pmYes, a corrupted and defective block in my Transcend SSD. At first I thought it is a filesystem error and I started fsck to fix it, the I realized the kernel wrote something about I/O error into the log ...At the end I managed it to replace the defective block with a spare one, but the file was defective. I had to replace the file from the backup.Cheershammy

author: suicidaleggroll
author_rank: Super Contributor
timestamp: December 11, 2015, 08:17:25 pm
content: Quote from: moya034 on December 11, 2015, 07:09:18 pmI have about 50 SSDs under my control running 24/7. The oldest one dates back about 6 years, the average age is around 2 years. I've only ever had one crash. After a lot of hand-holding, I was able to get it to respond again, but a handful of files were corrupt and so the drive was RMA'd. It was a Crucial M4, about a year old at the time.

author: Thorondor
author_rank: Contributor
timestamp: December 11, 2015, 08:22:51 pm
content: I've had a SSD fail but it was one of the earlier OCZs with the crappy JMicron chip.

author: ciccio
author_rank: Frequent Contributor
timestamp: December 11, 2015, 08:32:56 pm
content: Quote from: PE1RKI on December 11, 2015, 06:58:56 pmyes, Seagate Barracudas fail, and Western Digital Caviars too, and Hitachi-Toshiba too...In the years I've installed maybe 250 hard drives, and I have about 30 defective units on my shelves.In this precise moment I'm retrieving data from a FULL 2 TB Hitachi drive. Windows says it is not formatted, but EaseUS is trying to recover (4 hours to completion..)Hope everithing goes well: the files are the final edited photos of maybe 25 marriages, and my cousin, the professional photographer, has no backup.The backup is of the unedited photos.. He has about 30 external 2 or 3 TB drives, and in 4 years he had 4 total failures, plus 4 or 5 problems that were solved with no data losses.External drives are, to my experience, more prone to failure than the internal ones. Maybe it is a temperature problem, maybe is mishandling.Never trust an external drive.

author: kripton2035
author_rank: Super Contributor
timestamp: December 11, 2015, 08:34:03 pm
content: Quote from: moya034 onTodayat 07:09:18I install for my customers between 1 and 4 ssd each business day since more than a yearthey are mainly samsung evo or proI've had some 5 defective items, and most of the time the failure happens when I clone the original drive, so at the beginning of the ssd life.nice to have is a software that monitors the drive sub smart counters (not the smart flag but the 20 or so counters that trigger the smart hs flag)the synology or qnap nas monitor them for you and tell you if a drive is falling before it actually fails that's a very nice feature.should be incorporated in every modern operating system ...

author: madires
author_rank: Super Contributor
timestamp: December 11, 2015, 08:37:03 pm
content: Quote from: moya034 on December 11, 2015, 06:07:47 pmAnd when you are hit by some crypto malware you won't have a backup since everything is mirrored. It's a good idea to have some older backups too.

author: PE1RKI
author_rank: Regular Contributor
timestamp: December 11, 2015, 09:08:43 pm
content: no i speak from my own experience about seagate barracuda's.i had 5 seagate hd's if not more and only 1 is a little bit functioning, the rest all died.the last 2 gradualy died, i didnt understand exactly what was happening cause i got weird crashes until it was to late.i lost half a year of my work.and i have older hitachi and western digital hd's that still work ok.

author: eugenenine
author_rank: Frequent Contributor
timestamp: December 11, 2015, 09:17:47 pm
content: Quote from: madires on December 11, 2015, 08:37:03 pmThats why you don't use anything Microsoft for your server either.

author: German_EE
author_rank: Super Contributor
timestamp: December 11, 2015, 09:18:24 pm
content: In my case I have a computer savvy friend a couple of Km away and we both hold a backup drive of each other's data. Unless a VERY large meteor lands on Darmstadt we should be safe.And none of our drives are Seagate

author: rdl
author_rank: Super Contributor
timestamp: December 11, 2015, 09:52:44 pm
content: Quote from: German_EE on December 11, 2015, 09:18:24 pmAnd if it's a small meteor, only one of you has a problem.

author: Richard Crowley
author_rank: Super Contributor
timestamp: December 11, 2015, 10:14:58 pm
content: Quote from: Shock on December 11, 2015, 05:38:39 pmThere are FAR FAR FAR fewer "things to go wrong" with digital tape than with ANY kind of hard drive.That is why probably 99.999999% of all the most valuable data on this planet is archived on digital TAPE.NOBODY uses hard drives for archival storage. It is just insane.QuoteI have a growing stack of hard drives in the corner (and hundreds of similar experiences from others) that say that is completely incorrect.

author: moya034
author_rank: Regular Contributor
timestamp: December 11, 2015, 10:19:09 pm
content: Quote from: eugenenine on December 11, 2015, 09:17:47 pm+1 on the Microsoft comment... and that's also why you configure your non-windows "backup server" to keep historical copies going back to whatever date you select.(Edit: Yes I'm a Linux user, but I make my living supporting Microsoft products)

author: rsjsouza
author_rank: Super Contributor
timestamp: December 11, 2015, 10:32:16 pm
content: Similar threads:https://www.eevblog.com/forum/chat/ot-backing-up-video-files-long-term/https://www.eevblog.com/forum/chat/hard-drive-outside-storage-ideas-and-suggestions/https://www.eevblog.com/forum/chat/backup-strategies/

author: eugenenine
author_rank: Frequent Contributor
timestamp: December 11, 2015, 10:43:03 pm
content: Quote from: moya034 on December 11, 2015, 10:19:09 pmI got out of MS support and moved into ITRisk so I only use MS stuff when I'm recreating whatever Excel or Sharepoint ate last.My most important data is synced across two laptops, phone, tablet and backup server via owncloud running on a Raspberry Pi 2 then the lessor important data is backed up to the backup server which is another Raspberry Pi.

author: VK3DRB
author_rank: Super Contributor
timestamp: December 12, 2015, 01:57:15 am
content: The Kodak CD's had a 25 year guarantee of data retention. However this article says they should work for over 200 years...http://www.cd-info.com/archiving/kodak/Still not as good as the Dead Sea scrolls. And not near as good as Aboriginal cave paintings. I think the human race has too short a view about retaining data, and I suspect there won't we much evidence we existed in 2,000 years, except for the nuclear desolation we left in our wake.I have only used Seagate Barracudas in the past 12 years. Never had a problem. Terrific drives I trust. Unlike IBM with their appalling Deskstar ("Deathstar") hard disk rubbish.

author: Richard Crowley
author_rank: Super Contributor
timestamp: December 12, 2015, 02:30:16 am
content: Quote from: VK3DRB on December 12, 2015, 01:57:15 amBalderdash. I would be astounded if ANY of the companies offering such outrageous "guarantees" were even in the optical disk business by the end of the "guarantee term". For example Kodak who doesn't make optical disks anymore, etc.Besides. What good is the "guarantee". If you take the disk out after 25 years and you can't read it, will Kodak give you your data back? Or just the $1.49 you paid tor the disk?

author: SeanB
author_rank: Super Contributor
timestamp: December 12, 2015, 06:34:30 am
content: I got some used Deskstar SCSI drives ( a whole 1G each) so decided, as I had an Adaptec card installed, to use them as scratch drives, making one the Windows Swap drive. The one I was using as swap is still alive, though the other, just used as some easy to access drive, did die after another year. Stopped using the card after having a capacitor plague induced upgrade to Win7 - new box and simply copied the old HDD contents verbatim over to the new massive ( for a desktop running off a server) 320G drive, then spent a week deciding what was needed and putting in the right new tree.Still have the100G blob around in case. A few 3G OST files as well, something about a 4G limit really slows searching down.....

author: Shock
author_rank: Super Contributor
timestamp: December 12, 2015, 10:33:22 am
content: Quote from: PE1RKI on December 11, 2015, 09:08:43 pmAs I mentioned there was some bad firmware, it was about 4-5 years ago when the current series of drives were fairly new. If you didn't update the firmware this would eventually lead to a total failure of the disk, but recoverable for someone with enough skill.Not many people know to watch out for HDD and BIOs firmware causing problems, this is why it pays not to be an early adopter. It still sucks either way but that is electronics and software for you.

author: funkyant
author_rank: Unknown
timestamp: December 12, 2015, 10:46:52 am
content: I have to say that I'm surprised to see people vouching for the reliability of SSD's.In my own experience, the failure rate has been orders of magnitude higher than mechanical drives.Admittedly my sample size is not massive, however my business was an early adopter of SSD's due to the fact that we build and supply commercial media servers, and the HDD was always a troublesome bottleneck.In 2010 I must have installed around 50 (or more) SSD's into various sites, and of those 50 around 10 failed within 2 years. And they were not all the same brand. Also just recently I had a Crucial SSD fail in a 20 thousand Euro Lighting Console.As standard procedure now, we mirror the media stored on SSD's, on regular WD mechanical disks. So the SSD is used for playback and fast access, but the data is stored on a mechanical drive. Basically we use the SSD as a kind of playback media buffer.As a result of all this, I don't really trust them wholeheartedly.

author: VK5RC
author_rank: Unknown
timestamp: December 12, 2015, 11:16:28 am
content: Overall I am more worried about software issues (viruses and poor code) rather than hardware but both are not highly reliable.In my business I have had more trouble with software problems, 10% of billing data lost but backup saved the day. As not data intensive I use RAID 1 on SSD, 2 off site back ups, different media, alt daily. New box every 3 or so years.

author: Shock
author_rank: Super Contributor
timestamp: December 12, 2015, 11:18:06 am
content: Quote from: Richard Crowley on December 11, 2015, 10:14:58 pmI'm not talking about drive failures, I'm talking about unrecoverable storage media in the event of a failure. I welcome you to dive into the technical aspects rather than just saying I'm wrong (if you still disagree with me).

author: Shock
author_rank: Super Contributor
timestamp: December 12, 2015, 11:23:19 am
content: Quote from: VK3DRB on December 12, 2015, 01:57:15 amI knew I missed one I was racking my brain. Quantum fireball and Quantum bigfoot were two of the others.

author: kripton2035
author_rank: Super Contributor
timestamp: December 12, 2015, 12:36:14 pm
content: Quote from: VK3DRB on December 12, 2015, 01:57:15 amyou can't say "seagate barracudas" like that. there are tens of models of seagate barracudas ! some are good, some are badyou can refer to a specific model may bemy advice: always look at the warranty length... it's a good keypoint.some mechanical hard drives have a 5 years warranty, you can trust them more than the othersI saw some ssd with only one year warranty, you should really not trust them - the least must be 3 yearsthe pro models of ssd have a 10 years warranty.and as always MAKE BACKUPS EVEN FOR NAS ALWAYS MAKE BACKUPS AND BACKUPS OF THE BACKUPS.

author: Richard Crowley
author_rank: Super Contributor
timestamp: December 12, 2015, 03:08:48 pm
content: Quote from: Shock on December 12, 2015, 11:18:06 amI dont know whether I agree with you or not because I don't know what you mean by "unrecoverable storage media in the event of a failure"? Can data be recovered from failed hard drives? Certainly, at tremendous cost. And that is one of the reasons that hard drives are not considered suitable for archival storage. Not to mention the cost of the "media" (the entire drive with all the mechanical and electronic assemblies, etc, etc.) It just doesn't make sense on ANY level. Which is why nobody does it. It has little to do with "the technical aspects", and mostly to do with the "economic aspects" that result from the "technical aspects".

author: graciaj
author_rank: Newbie
timestamp: December 12, 2015, 03:20:25 pm
content: the cloud is our friend for storage... unfortunatly it has a cost most of the time

author: madires
author_rank: Super Contributor
timestamp: December 12, 2015, 04:30:46 pm
content: Quote from: graciaj on December 12, 2015, 03:20:25 pm... but please don't rely on it! There are things like slow internet access, outages, bankrupt companies and more.

author: kripton2035
author_rank: Super Contributor
timestamp: December 12, 2015, 04:53:14 pm
content: Quote from: graciaj on December 12, 2015, 03:20:25 pmthe cloud is not really your friend : in the big text you dont read when you accept their conditions, they tell you they can do anything they want with the datas you provide... so better encrypt them before uploading them if you dont like surprises...also some clever pirate can find your password and delete anything they can't be responsible for thatand if you don't have a backup somewhere else than the cloud then ...?

author: Thorondor
author_rank: Contributor
timestamp: December 12, 2015, 05:03:02 pm
content: I don't trust the cloud for anything important. It doesn't matter what the terms of use or laws say - you no longer "own" the data once it's on someone else's servers.

author: MrSlack
author_rank: Frequent Contributor
timestamp: December 14, 2015, 09:26:03 am
content: This.Only encrypted stuff hits the cloud for me and the keys station my possession. Encrypted in transport and at rest for the technically inclined.

author: Rerouter
author_rank: Super Contributor
timestamp: December 14, 2015, 10:28:27 am
content: with the barracudas it was the 7200.11's that where effected, most notably the 2TB ones, essentially something went screwy when there power on hours exceeded some value then was power cycled, so an always on wouldn't know until something brought there system down, then magically one or more drives playing dead, for a friend that was all 6 drives in a raid 5 array,Luckily there was a method written up at the time to upload the new firmware very slowly through the diagnostic pins near the sata plug

author: Brumby
author_rank: Unknown
timestamp: December 14, 2015, 11:14:00 am
content: Quote from: Richard Crowley on December 12, 2015, 03:08:48 pmI read the "unrecoverable storage media in the event of a failure" as meaning "unable to read from the backup when the primary storage unit fails".This reminds me of a practice followed at my first employ to address the issue: "Is your backup readable?"After you write a backup, how many of you actually check to see that it is readable? How many of you have taken a flash drive that says it's 32GB and just used it - trusting that it IS capable of holding 32GB. The last time I purchased a couple of 32GB micro SD cards, I ran them through a write/verify utility that showed them to actually have only 8GB of memory. It was simply an 8GB chip reprogrammed to declare 32GB. You could write 32GB to it and it would all look rosy - until you tried to read it back.Then there's the case of magnetic media. As I understand it, magnetic domains are subject to weakening by thermal processes as well as changing magnetic fields. which - in our daily environment - means they will reach a point where data integrity will be lost. Admittedly, we could be talking years or maybe even decades, but the risk is there. The solution is simple - just copy the data and the magnetic domains will be as good as new.And if you want to really test out you 'backups' - try this exercise ....1. Get yourself a computer system ready to use - but with completely blank drives. That is, hardware only.2. Consider the location your system is normally run from as having been totally destroyed by, say, fire. (So you can't grab that 'cheat sheet' pinned to your wall.)3. Get all your 'backups' and 'restore' to this system.Can you do it?Do you have all the software? O/S? Licences? Do you need/have the installation discs? Product keys? Updates? If you need to partition drives in a particular way, do you have the means to do it?For bonus points, send the person who knows all the ins and outs away for a couple of days (without a mobile phone) while you do this. In real life, they may have been hospitalised as a result of the 'fire'.How would you go?

author: kripton2035
author_rank: Super Contributor
timestamp: December 14, 2015, 04:21:15 pm
content: using apple time machine, everything is verified periodically automatically for you...

author: rdl
author_rank: Super Contributor
timestamp: December 14, 2015, 05:21:38 pm
content: What is so special about 2016?You can lose data at anytime.

author: SpidersWeb
author_rank: Unknown
timestamp: December 14, 2015, 07:05:20 pm
content: It's not relevant to modern drives so much, but for anyone who is interested I have a lot of hard drive units from the 80's 90's and 2000's - and honestly couldn't find any large link between brand and failure rate. Seagate, WD, Quantum, etc I only have ~100-150 drives, so it's a small sample size, but I haven't been able to develop any stereotypes in the IDE generation.In the current market, I wont trust Samsung Spinpoint mechanical drives (SSD seem fine) nor Seagate 7200.11's (but you don't see those much anymore).For backups, I'm often too lazy to do it properly, so now I just install drives in pairs, and have one automatically backup to the other each night. It's not as good as off-site or even disconnected from the machine itself - but it's saved my ass on many occasions. I use this method rather than mirroring because it prevents even wear on the drives - I don't want to increase the probability of my backup drive failing at the same time the primary does.For someone with a laptop, external HDD is what I'd use.

author: free_electron
author_rank: Super Contributor
timestamp: December 14, 2015, 07:31:21 pm
content: Quote from: MarkF on December 11, 2015, 05:15:10 pmDepends. i have traxdata brand CD's (gold coating , not that greenish coating) that were written using a Kodak PCD 225 burner at double speed. These disks were written in 1996 .. still read flawlessly today. That is 20 years ago !usb drives are a disaster. one esd zap and kiss all goodbye.backups : one original + one copy is NOT a backup. one original + two copies is half a backup. one original + three copies, of which one is stored in a physically different location , is a backup.and for people using those NAS boxes. what if the NAS itself dies. what are your chances of popping the drive into a different nas and being able to read ? ( in case your particular nas machine is discontinued )

author: hendorog
author_rank: Super Contributor
timestamp: December 14, 2015, 08:05:19 pm
content: There are a few people mentioning RAID 5 - just beware as there is a nasty trap for young players with RAID:* Replacing a drive after a drive fails can make things much worse. Google Punctured RAID array if you are interested.* Make sure you backup your backupsRegarding backup devices, I have had good experiences with a couple of QNAP NAS devices.They backup themselves to Amazon Glacier which is a very cheap cloud service for large quantities of data.Other services which might be of interest:SpiderOak for important filesGmail for emailTimemachine for OSXGithub for source code

author: kripton2035
author_rank: Super Contributor
timestamp: December 14, 2015, 08:15:26 pm
content: Quote from: free_electron on December 14, 2015, 07:31:21 pmalmost none except if you have open nas OS but I don't know of one for "boxes" only for pc tranformed into NAS

author: Kjelt
author_rank: Super Contributor
timestamp: December 14, 2015, 08:33:13 pm
content: Quote from: hendorog on December 14, 2015, 08:05:19 pmI am far from a RAID specialist but reading this made me think about what a friend happened this summer:he suffered something similar with RAID 5 but in his case it was that he had 2 RAID controllers in his server each having 4 disks so totalling 8 disks.Now what happened (according to him) is that one disk went bad and that cause the write action that had to be distributed over all 8 disks to only go well on the 4 disks of the other controller. The controller where the disk went bad stopped all writing actions to all the disks due to the error on the one disk. So after one disk died, he had 4 disks that were up to date, 3 disks that were not up to date and 1 dead disk. He replaced the dead disk but the system could not be rebuild because there were 4 disks out of sync. Unbelievable because this is exactly what RAID 5 should protect you against

author: SpidersWeb
author_rank: Unknown
timestamp: December 14, 2015, 08:48:39 pm
content: RAID is fantastic for keeping your commercial server operating 24/7, but never use it as an alternative to backups.What's really fun is when one drive actually fails, and then a few minutes later one of your cheap desktop class hard drives falls off the array because of a pointless error (which it could've resolved by itself), and then you spend 5-6 hours trying to rebuild your data with forensic/recovery software! Good times!Good video/experiments on the subject (well, I liked it):

author: hendorog
author_rank: Super Contributor
timestamp: December 14, 2015, 09:03:28 pm
content: Hehe yep good war storiesAnother consideration with RAID is that you have x times the hardware and so a failure of some sort is more likely.

author: eugenenine
author_rank: Frequent Contributor
timestamp: December 14, 2015, 09:16:23 pm
content: Quote from: Brumby on December 14, 2015, 11:14:00 amThis is one of the reasons I can't use cloud backup, it would take weeks to restore all my data.All open source software so no licenses to worry about.

author: Shock
author_rank: Super Contributor
timestamp: December 14, 2015, 09:20:28 pm
content: It's a common misconception that RAID is for data redundancy, aside from it's original use of cheap storage it only offers availability (meaning uptime). The misconception is because RAID is used in backups and data storage and has redundancy features built in to recover volumes. So it can store backups, but is not a backup system in itself and has it's own set of quirks when it comes to recovering a hardware failure.

author: Shock
author_rank: Super Contributor
timestamp: December 14, 2015, 09:34:34 pm
content: Quote from: hendorog on December 14, 2015, 09:03:28 pmYes and if the array is affected all depends on the RAID level and scale you choose. Generally the larger the array and more parity, mirrors, online/offline disks waiting in the background increases the arrays uptime and life expectancy. So while greater chance an individual disk will fail less chance of losing the whole show, which is why they are used.

author: MrSlack
author_rank: Frequent Contributor
timestamp: December 14, 2015, 11:12:04 pm
content: Perhaps ironically as this is an electronics forum, we get more power supply failures than disk failures these days. Equally as destructive as they never seem to go out quietly.

author: suicidaleggroll
author_rank: Super Contributor
timestamp: December 14, 2015, 11:36:31 pm
content: Quote from: blueskull on December 14, 2015, 11:26:16 pmOne of my old HDDs begs to differhttp://thesuicidaleggroll.com/hosting/DSCN0521_resize.jpghttp://thesuicidaleggroll.com/hosting/DSCN0522_resize.jpgI had a system running fine, then the PSU failed. The system wouldn't turn on, so I swapped the PSU for a new one, still nothing. So I built a new machine from scratch, got it up and running, then plugged one of the old HDDs from the failed system into the new one to try to recover the data. The system locked up on POST, so I peaked my head around the corner of the chassis to see ~5" flames coming off of the HDD. I yanked the power cord out of the wall and it died out pretty quickly, the pictures above were the result.

author: MrSlack
author_rank: Frequent Contributor
timestamp: December 14, 2015, 11:42:14 pm
content: That looks toastyQuote from: blueskull on December 14, 2015, 11:26:16 pmI've got a photo somewhere of an entirely burned out HP DL380g5 we had. PSU went up, there was an alert saying failed PSU, then the internal temp alert shut it down and proceeded to just basically melt everything plastic inside. Burned up half the cables in the rack as well. We had to remove the front (!) fan tray at the opposite end of the unit with a flat blade and a hammer and replace about half a 42U of kit that was suspect.Then there's the nice big old 3-phase UPS we had. The 415v input exploded and set fire to half the rack as well.Power is where it's at and where it ends.Also the amount of MacBooks I've seen where either the left IO board catches fire, the magsafe connector catches fire or the switch mode explodes is getting ridiculous now. Seems to be 5-7 years then instant flame thrower for a lot of people.

author: John_ITIC
author_rank: Frequent Contributor
timestamp: December 14, 2015, 11:47:16 pm
content: Quote from: moya034 on December 11, 2015, 06:07:47 pmAny recommendation for such a solution? Ive been looking at WDs solution but I'm sure there are better / more professional options?

author: moya034
author_rank: Regular Contributor
timestamp: December 14, 2015, 11:58:32 pm
content: Quote from: John_ITIC on December 14, 2015, 11:47:16 pmRAID: Hardware raid, whether your motherboard has it built in, or you install a 3rd party RAID controller in an expansion slot.Backup Software: Can't give you much advice here. I need to start researching a good Linux solution. At work we use Symantec NetBackup, which I really like.VPN: Site to Site OpenVPN implemented with PfSense routers at both endpoints.

author: hendorog
author_rank: Super Contributor
timestamp: December 15, 2015, 12:11:24 am
content: Quote from: John_ITIC on December 14, 2015, 11:47:16 pmI've done the live sync part of this (apart from the VPN) with two QNAP NAS devices. The docs say they can also run both a VPN server and client so it should be possible to do it just with the two off the shelf boxes. Obviously a couple of PC's can do it too, it just depends on what you feel comfortable maintaining.Personally I would avoid RAID 5 in that situation and aim to keep it as simple and reliable as possible.

author: eugenenine
author_rank: Frequent Contributor
timestamp: December 15, 2015, 01:37:59 am
content: I think most just rysnc across to the backup server, thats what I do.

author: Shock
author_rank: Super Contributor
timestamp: December 15, 2015, 02:28:58 am
content: Quote from: eugenenine on December 15, 2015, 01:37:59 amI'll take this any day over a corrupted 100GB proprietary backup file format that for some reason or another won't open any more.I hope people took the hint by now that RAID is really cool, until you need to get it back.

author: Brumby
author_rank: Unknown
timestamp: December 15, 2015, 05:06:27 am
content: A casual note for anyone with a Raid setup - make sure you have a clear communication path between yourself and the raid controller, so when a drive fails, you know about it.I had someone with a 2 drive external Raid box under a desk and it performed quite well. So well, in fact, that they never knew the first drive had failed until the second one did.It seems there was no remote notification set up and the "drive fail" light couldn't be seen unless you got down on your hands and knees.Major problems ensued because of the archaic software design.

author: Brumby
author_rank: Unknown
timestamp: December 15, 2015, 05:09:51 am
content: Quote from: eugenenine on December 14, 2015, 09:16:23 pmThat's one of the "discoveries" such an exercise will reveal.

author: hendorog
author_rank: Super Contributor
timestamp: December 15, 2015, 06:58:31 am
content: Quote from: Shock on December 15, 2015, 02:28:58 amYeah exactly why I like the linux based NAS's. They are just linux and use rsync - the poster above was looking for something 'professional'.

author: Brumby
author_rank: Unknown
timestamp: December 15, 2015, 07:00:51 am
content: Quote from: blueskull on December 15, 2015, 05:19:06 amThat is a far too simplistic view - and idealistic.Quote from: blueskull on December 15, 2015, 05:19:06 amWhat if the company is out of business and there's no-one to respond to your request? What if they do not re-issue software/product keys? What if they do not support the old version that you have been using?You also assume that a computer system (that's used for a business, say) can run properly on a small amount of data - which ignores the situation where an extensive database is in place. Can you imagine DigiKey or RS Components running their online systems from a subset of their database?In truth, the questions I have raised are valid for EVERY situation, but the ANSWERS will vary. What seems obvious to you and works for you will almost certainly be inappropriate for the majority of others.

author: Halcyon
author_rank: Unknown
timestamp: December 15, 2015, 07:09:49 am
content: Quote from: moya034 on December 14, 2015, 11:58:32 pmThere are but I wouldn't consider on-board RAID a "proper" RAID unless it's on a proper server... Dell... HP... Supermicro etc...RAID on consumer boards is pretty much like software masquerading as hardware RAID. As for drives, go with drives suited to the enterprise market as they are designed to be installed in hardware RAID systems. Trying to run many consumer drives on a hardware RAID will cause you issues.As for software, I actually just use Microsoft Backup that comes free with Windows (and I backup to LTO tape). It's free and does the job, albeit with very limited features. There are some free programs around but they are pretty awful. If you expect business-grade backups, you'll have to pay.For your everyday home user, use multiple types of media, hard disks, DVD/BD discs and also upload important data to somewhere like Dropbox or Google Drive. Check/test physical media regularly.

author: MrSlack
author_rank: Frequent Contributor
timestamp: December 15, 2015, 10:18:23 am
content: Quote from: blueskull on December 15, 2015, 05:04:53 amYes and the game of data persistence is to avoid relying on luck

author: Rerouter
author_rank: Super Contributor
timestamp: December 18, 2015, 01:05:00 pm
content: The only other thing i will add that is less relavant now that motherboards are seeming to just get spammed by sata ports is, keep your array sizes smaller than the maximum amount of free ports you have incase a controller card fails, had just that happen, all the data was there nice and safe on the drives, but only had 2 ports, and a 4 drive array, so had to hijack a friends computer to plug all the drives into and start the very long process of extracting the data to a new storage array,To clarify, no longer sold controller card, and if you look hard enough you can find recovery software that can make sense of just about any way a raid array can be dreamed of,

author: madires
author_rank: Super Contributor
timestamp: December 18, 2015, 02:41:45 pm
content: CD/DVD: If you want durability look for MO drives. You can get MO disks with a guaranteed life time of 30 years. Those are used for archiving medical data for example if there's a legal requirement for such long periods.NAS with software RAID: A dead NAS is no big problem since all run linux. That means that you can connect the disks to a linux PC and recover the files. The RAID tools write some very useful management information on the disks (unless you got a NAS older than 8-10 years).Hardware RAID: Great as long as you got support, i.e. spares.Old stuff: Copy data from old media to current media before there aren't any drives for the old media anymore. Make sure that you can access old data with your software. Convert data into a new format if necessary.

author: TerraHertz
author_rank: Super Contributor
timestamp: December 18, 2015, 10:43:00 pm
content: I'm more concerned that I may lose my life in 2016. The way things are going, with the lunatics of a certain government apparently being hell-bent on starting WWIII. Before enough citizens wise up and lynch them for past and ongoing crimes.But anyway... RAID... Rapid Agony of Incompatible Drives.I now have a rule: If my data isn't on a media that I can read back as a plain, complete filesystem, from any single drive, on any standard PC with no special software and no net connection, then I consider it lost.Discovering that the files on RAID array hard disks, even just plain mirroring, generally can't be read back on a standard PC or USB external HD dock, was a bit of a shock. If you are not an enterprise, with cash to lay in stock of spare identical drives when you set up the RAID machine, then trying to find a same-drive to replace a failed RAID array drive later is generally not going to go well.As for obtaining replacement RAID array electronics - controller card, etc, in a hurry (or at all) , forget it.So for me, backup medium is CDs, DVDs, and (mainly) bare hard disks plugged into cheap USB HD docks. Keeping a bunch of dated copies on separate drives. Drives stored in anti-static bags, in padded boxes.Other measures:* Only use NAS boxes if I've verified that I can read back the bare NAS drives in a standard USB external dock.* Keeping different types of data in cleanly organized separate trees in my working machines, so I only have to backup datasets that need it, when they've changed significantly.* I no longer backup the installed OS and 'Windows installed' utils, due to it never being feasible to restore the whole bundle cleanly to new hardware. Thanks MS.* Trying to switch all my software utils to 'portable' (no Windows install) forms, as this allows me to have all my utils in one folder tree that CAN be cloned to multiple PCs and work, and can be easily backed up.* Using ZTreeWin for folder tree transfers, because it doesn't change file dates, and just does the copy. As opposed to Windows which will complain annoyingly. I suppose that counts as 'special software'. But at least it is possible to fall back to standard OS file copy.Still looking for a good freeware filesystem compare/sync utility. Or even a payware one that does the job properly. ViceVersa PRO has fatal flaws.What compare/copy/sync tools do people here like?

author: SL4P
author_rank: Super Contributor
timestamp: December 18, 2015, 10:44:32 pm
content: You might get a functional Batteriser in 2016 as well... !

author: eugenenine
author_rank: Frequent Contributor
timestamp: December 20, 2015, 09:02:10 pm
content: Quote from: TerraHertz on December 18, 2015, 10:43:00 pmrsyncrun it under cygwin if you must use windows

author: MT
author_rank: Super Contributor
timestamp: December 20, 2015, 10:09:49 pm
content: Weeell, i use one rune stone for data, one for programs and 1000 for web downloads! They tend to last, so far about 1600 years or sowe dont know exactly!https://en.wikipedia.org/wiki/Runestone

author: Kjelt
author_rank: Super Contributor
timestamp: December 20, 2015, 10:22:09 pm
content: Yup stone is a good storage medium, i wonder if the m-disk made it?http://www.extremetech.com/computing/92286-m-disc-is-a-dvd-made-out-of-stone-that-lasts-1000-years

author: timb
author_rank: Super Contributor
timestamp: December 20, 2015, 10:41:15 pm
content: I think I'm going to start backing up to 5.25" floppy media again.I recently found a 20+ year old box of floppies with games, source code and other stuff on them from my first computer when I was a kid. I broke out my old Compaq Portable III and sure enough, every one of them read just fine! Surprisingly, so did the Connor 20MB hard drive in the system, which was actually one of the first IDE drives made!

author: Kjelt
author_rank: Super Contributor
timestamp: December 21, 2015, 07:07:00 am
content: I got 40+ TB of data to backup, and you advise 360kB floppies?

author: Kjelt
author_rank: Super Contributor
timestamp: December 21, 2015, 07:15:19 am
content: No, I collect data. Most of that data I sometimes project on a 94" whitescreen while eating popcornMost of the good data also resides on discs in blue plastick covers, the bad data is erased after review

author: Brumby
author_rank: Unknown
timestamp: December 21, 2015, 07:47:20 am
content: Popcorn data - some of the best (and worst) data around.

author: stmdude
author_rank: Frequent Contributor
timestamp: December 21, 2015, 09:15:35 am
content: Quote from: Kjelt on December 21, 2015, 07:15:19 amI have a similar situation. However, I consider the blue plastic covers and its contents to be the backup, so the HDD based versions never get backuped. Sure, in case of a failure, I have some work to do, but nothing is lost.Either way, it always gets done "better" the next time I have to restore. ( .mp4 -> .mkv w/ subtitles and extra audio tracks last time ).I'm guessing that the next time it happens, it'll all be H.265./J

author: Kjelt
author_rank: Super Contributor
timestamp: December 21, 2015, 09:19:34 am
content: Quote from: stmdude on December 21, 2015, 09:15:35 amYeah well the data is also getting bigger with 4k UHD coming up this year, heard about 140GB per fileSo perhaps with the new 265 we can half that, still huge amounts and if you noticed the HDD developments seem to come to a halt yet again. 6TB/8TB seems to be the max, with one exotic helium filled 10TB with an extreme price the exception.The 6TB prices are constant now for well over a year.With Moore's law on HDD's we already should have seen standard 12TB drives for the same prices or 6TB for half those prices. So yeah challenges in the data storage and backup coming up