Backup software and strategy for small office/home office - Page 1
Forum section: electronics / general

author: rx8pilot
author_rank: Super Contributor
timestamp: December 02, 2016, 05:34:26 pm
content: I have been struggling to find a backup solution for my small biz and personal data - hoping that someone in the EEVblog world has some insight. The usual consumer suspects are generally too basic, the commercial options require a proper server, cost a ton, and require a lot of knowledge I don't have. I have yet to find a mid-level SOHO, semi-pro solution.SETUP:Backup and somewhat isolate active and archive data in a SOHO environment. Around 10TB, but growing about 1-2TB per year. No servers, just (7)workstations and (2)NAS storage devices. All windows 7 and 10, Linux is ok if that is where the solution is.GOALS:Local and remote backupsIsolate backed up data from any accidental deletion or crypto-locker malwareAll users/workstations only use and save data on the NAS - no active user data on system drives.Images of each PC are saved on the NAS to recover them when system drives fail.Need to backup the NAS to local drives for immediate availability if NAS failsNeed to pseudo-archive NAS Data to encrypted and compressed ZIP (or similar) files to another local driveability to push the NAS Data to a remote location/cloud/rackspace etcExcellent error reporting so that I will be notified when something gets stuck or otherwise failsSo far, I have downloaded trials of 5-6 consumer solutions - all fail for a number of reasons. Cannot deal with NAS, saving data to proprietary file format, buggy/crashy, etc. Any ideas from those more knowledgeable than I?

author: Ampera
author_rank: Super Contributor
timestamp: December 02, 2016, 06:41:09 pm
content: As a suggestion, get a professional version of Macrium Reflect for each system, and send the VERIFIED (I have lost entire installs twice over for forgetting that) images to a file server with hot swap screw-less drive enclosures. My idea would be to have a remove drive policy every week, where you have three sets of drives, each enough for what you need. Put one set in the server for daily backups, keeping at least 2 separate backups, and place week old backup sets in something like a fire proof safe, and every month, swap out a set of drives for drives stored at an off-site location (Bank storage locker, in a safe somewhere hidden in your house) the only real expense would be drives, and for those I suggest archive drives. They are cheaper, but perform slower than other drives, so backups could be performed at night when nobody is working.Another option, however slightly more labor intensive, is to get backup tapes, and maybe a few drives (The drives are expensivehttp://www.newegg.com/Backup-Drives/SubCategory/ID-46?Tid=7545) and just do a classic tape backup routine, and storing them somewhere safe like my suggestion.The CHEAP solution, but with slight risks is to pay somebody like google or dropbox for cloud storage, and to just keep your workstation images online.I'm not a professional with backups, and I am probably the idiot you don't want to take your advice from, but my suggestions stand for their own, look them over, tell me what you think.

author: rx8pilot
author_rank: Super Contributor
timestamp: December 02, 2016, 07:00:03 pm
content: Macrium is good at making images of systems, but has no real functionality for backing up network shares at a file/folder level with revisions and such. I currently use Paragon software to image machines which is effectively 'system' data. The user generated data like CAD, financial, spreadsheets, etc all are saved on a separate NAS system. The NAS has a little redundancy with RAID but needs to be backed up locally and offsite and that data need to be protected from changes or deletions from accidents or malware - that is where the biggest challenge lies. Dealing with a 10TB growing data set trying to keep the data protected at multiple levels, media types, and locations. I don't mind doing some work and oversight, but I cannot spend massive money that it takes to get a real enterprise solution.

author: R005T3r
author_rank: Frequent Contributor
timestamp: December 02, 2016, 11:19:17 pm
content: I have two drives: the first is for the OS and regular stuff, the second is for important stuff and it's always backed up regularly.

author: imidis
author_rank: Frequent Contributor
timestamp: December 02, 2016, 11:40:23 pm
content: I did physical remote backup they take home every night with these you can get them cheaper from amazon or where you can find em, and in kit/standalonehttps://www.startech.com/ca/HDD/Mobile-Racks/Removable-Hard-Drive-Enclosure-Backup-System~SAT2510U3REMDidn't pay attention to the link, the ones I had purchased cannot accommodate the larger than 1 tb drives, they run sabackup open source every night take the other drive home. A few in rotation, works ok, but may not fit your criteria.Works good and the nice thing about it, no power cords and hot pluggable. As said not what you are looking for, but if anyone else may it could be a good solution.

author: nctnico
author_rank: Super Contributor
timestamp: December 02, 2016, 11:45:04 pm
content: Buffalo has nice NAS systems (linkstation series) which support rsync. Rsync is a protocol for doing incremental backups so you don't need to transfer all files for every backup. Only what is new gets transferred. In order to retrieve files the NAS also support Windows network protocol with various user access levels (like a fileserver). Protection against ransomware is tricky though. I guess the bastards made it so it will also affect backups so you will lose several days of work even if you have backups which go back further than the infection.For easy recovery of Windows machines a virtual machine is very effective. Last week my primary Windows XP started acting up. Shut it down, roll-back the image to a known good one and back to work within 5 minutes.Edit: some more thinking about ransomware protection: if you setup a NAS with (for example) 7 different partitions and setup Rsync scripts for each day you will always have a backup which goes back for 7 days. You could take this even further and do a weekly backup as well but the 7 days in between will be lost. Note that this will probably take several up to tens of terabytes!

author: aandrew
author_rank: Frequent Contributor
timestamp: December 03, 2016, 01:54:39 am
content: I am a big fan of "doing it myself" -- cloud services are bullshit, commercial products, especially "pay once" services are only slightly better. To that end, I wanted a system that backed up my data with incremental backups and shoved them out to some cloud storage service without any reliance on a specific service. My primary storage is of course RAIDed, but RAID is not backup.To that end, I came up with a solution with two open-source software packages: duplicity and syncthing.duplicity (http://duplicity.nongnu.org/) is basically a wrapper for rdiff/librsync. It allows me to set up incremental backups from a server (right now Linux servers, but it works for OSX and has Windows wrappers as well), encrypts them with GPG and can throw them up practically anywhere. A local drive, a remote computer or any number of cloud services. I've personally been using Amazon S3.With the actual offline backups taken care of, another problem I had was having a "live" working set of data that allowed me to work from either my workstation or my laptop, and never have to worry about copying stuff or having stale data where I'm working. Syncthing (https://syncthing.net/) is an open-source version of BTSync. I have my "Projects" folder synchronized between my workstation, my laptop, a home storage server (where duplicity backs up from) and my "on the net" server. Syncthing watches the directory for changes (and tools like syncthing-inotify help accelerate this to be pretty much instant notification) and changes are pushed to the other machines. It's got a lot of configurability with respect to bandwidth controls and sync ordering. Once a change has been propagated then the other nodes can be updated from any up-to-date node, which helps my bandwidth use.It's been working well for the last few years and has survived drive failures and hardware upgrades. I'm very pleased with this system; the best part is perhaps that it relies on nothing proprietary or commercial. I can change my remote storage provider in an instant, and it's also operating system agnostic.

author: John Coloccia
author_rank: Super Contributor
timestamp: December 03, 2016, 02:09:47 am
content: I use Acronis software (not the home version...the more professional one. I think they call it "Workstation" now). I backup to a Synology RAID. Currently, I have the 2 drive version with 4TB disks in it. My particular one allows you add an extension with several more disks (5 more I think).What's nice about the Synology system is that you can divide up the storage however you like. I happen to have it setup as a simple mirror, but changing things around is trivial, adding storage is trivial...it's just a really nice system.I also have an external SATA dock. Every now and then, I kick off a full, offsite backup to a drive.My backups are setup to do a full disk backup every month, and staggered incremental and differentials every couple of days. All of my documents are setup to do a full document backup every week, with a differential backup every evening (except when the disk backups are going), and incremental backups every 6 hours. The idea is that the most I can ever loose is 6 hours worth of work, even if I completely screw up and delete something important by accident. I differentiate between disk and doc backups for technical reasons I won't get into here. Suffice it to say that the way Acronis works, incremental/differential document backups can be much smaller than an equivalent "disk" backup (their terminology...not necessarily the entire disk) if there isn't a lot of data changed.All that said, with 10TB of data, figure that you'll probably need 20-30TB to do a reasonable backup scheme. I have about 700Gb at any one time and that fills about 1.5TB to 2 TB using the scheme I just described. Figure at a minimum 2X storage since you need at least 1 full backup, and you don't want to delete that until you've performed the next backup and verification, and that's just the bare minimum. Even with compression, you're looking at some serious storage requirements. If your business depends on this, it might be time to spend a couple of bucks and ask a real pro for a suggestion that will actually work and will grow with you.

author: rx8pilot
author_rank: Super Contributor
timestamp: December 03, 2016, 02:39:22 am
content: I have not used rsync before although many of my colleagues use it all the time. What I am hoping for is a one-way backup. If I make file it gets baked up, if I delete it it is still backed up. I also want it to be inaccessible to any normal user and any crypto-locker malware. That way if someone deletes a whole folder, a disk array crashes, or the NAS gets hijacked by a hacker - we still have a full and untouched data set that is protected. I have no idea if rsync is appropriate for that goal or how challenging it would be to set it up. A bonus would be to compress the data into ZIP files. The ZIP (hopefully) could be archived to M-DISCs or LTO tapes and taken off the live NAS system to free up space and lock those data sets down.If all goes well......ACTIVE DATA: LOCAL NAS 10GbE 10+TB RAID6 or ZF3This is the everyday file server, dual power supply, shock isolated for earthquakes. Pretty reliable.MIRROR of ACTIVE DATA: BACKUP LOCAL NAS - slower/cheaper RAID5 to be a failover option.This should be in constant sync with the primary NAS but not normally accessible by anyone. No versioning needing, just an exact mirror that is on separate power and on the opposite side of the building.LOCAL OFFLINE ARCHIVES: 8TB RAW HDD's used in a DOCKThese would be for the NEVER DELETE data. As data is generated, it goes to these drives as read only and hopefully packed into M-DISC sized 100GB chunks for the most important stuff. Not sure how to manage these to keep track of what is on them and what is not. It needs some sort of library solution to keep track of what is on what drive. In theory as the primary NAS gets full, I would use this system to triplicate data so that I can free up space on the more expensive NAS - purging old data without destroying it.OFFLINE ARCHIVES OFFSITE:This is essentially a copy of the archive HDD's pushed out to some offsite place - possibly my parents, in-laws, not a service. Should be on the internet and I could push data to it slowly.In an ideal world, this would be as automated as possible and send me reports at a reasonable interval. If I create a new file and save it to the NAS - it should without any effort be mirrored, archived, and sent offsite within a few hours. If all of this actually happened, A total NAS fail could be recovered in minutes assuming the backup NAS survives. If they both die, it's probably because there was an earthquake and the archives and offsite backups should allow me to recover the data set.

author: rx8pilot
author_rank: Super Contributor
timestamp: December 03, 2016, 02:47:47 am
content: Quote from: John Coloccia on December 03, 2016, 02:09:47 amI may be at that point.....my DIY solutions are harder and harder to manage everyday. With the number of PC's and data, probably out of my league now.Acronis, BTW, totally failed to recover when my primary design workstation crashed. The support they offered was comical. The product never backed up the data without corrupting it even though it was verified. It was a huge setback for me - took months to get back to normal. The software on the machine was old, expensive, and very difficult to install and configure. Most of it was out of maintenance and needs support to deal with licensing issues. I had Acronis setup to do local images and those were never made and never threw a warning that something was wrong. The cloud backups were present, but many critical files including the windows registry were corrupted. I will not be using them ever again.

author: John Coloccia
author_rank: Super Contributor
timestamp: December 03, 2016, 03:52:44 am
content: Quote from: rx8pilot on December 03, 2016, 02:47:47 amWhich software was it? You have me worried, now.

author: tautech
author_rank: Super Contributor
timestamp: December 03, 2016, 04:28:33 am
content: Carlos, for the little bit of this that I've done........home and office PC, wired LAN and NAS box.....all simple stuff, a utility called Second Copy has done all I've needed. I was put onto it by a IT mate, it's very configurable for local stuff but I've not used it to send anything "off site".http://www.secondcopy.com/

author: rrinker
author_rank: Super Contributor
timestamp: December 03, 2016, 06:02:26 pm
content: I am still old school, I am running Windows Home Server for my central storage (it uses disk duplication, not RAID, so I can have a mix of whatever size disks I want and it still makes sure any file is on 2 different physical disks). That system backs up all connected workstations, using an excellent deduplication system which is totally invisible - if I need to recover a file or folder, I can just any given backup (you can configure retention periods) as a drive and then copy the older version of the file back to my workstation. No messing with backup and restore software.The server is then backs up with a cloud backup service. I use Crashplan because it is not expensive and when they say "unlimited" the mean unlimited without any asterisk.To replicate it in modern versions, most of the features are available in Server 2012 Essentials.I will also add, even though Home Server predates Windows 10 - it fully supports it, as all my machines have now been updated. Also, don't let the "Server" name fool you. I run that box headless and the last time I think it was powered off was when I moved here 2 1/2 years ago. It has rebooted since then, when installing updates, but it has never crashed. And never did from the time I built it until I shut it down to move. The last time it was shut down prior to moving was when I swapped the components to a different case, to get one with more drive bays.--Randy

author: nctnico
author_rank: Super Contributor
timestamp: December 03, 2016, 06:31:42 pm
content: Quote from: rx8pilot on December 03, 2016, 02:39:22 amrsync can do that (it has an option to keep or remove deleted files). Ofcourse it does take some time to set it up but it isn't black magic. If you are a small business it wouldn't hurt to have an IT guy over every now and then to set things like these up.

author: rx8pilot
author_rank: Super Contributor
timestamp: December 03, 2016, 06:36:34 pm
content: On the hunt for a freelance IT person. Its tough to find a solid and trustworthy person. Each decision they make is critical to my living. Asking around locally for referrals.Sent from my horrible mobile....

author: Howardlong
author_rank: Super Contributor
timestamp: December 03, 2016, 07:08:43 pm
content: My backup strategy uses StorageCraft ShadowProtect. I reviewed a number of products about three or four years ago and ShadowProtect wasn't the cheapest but it was the fastest and least intrusive. It uses Windows' VSS to do incremental backups, and I run it every half hour. It takes a few seconds typically to incrementally back up my largest system (two 1TB SSDs and a 15TB HDD hardware RAID archive) to a Windows server share.I also do a full backup every week, that takes about 12 hours on my largest machine. I keep the last three weeks' backups.The Windows server has a 15TB useable HDD hardware RAID and a 256GB SSD boot drive.You can mount the backups as drives so it's easy to retrieve stuff at specific points in time. Only last week I lost my entire 15TB RAID archive due to a software fault corrupting the data, I did an image restore of the entire drive to within half an hour, but it took almost a day to do it due to the disk size.The Windows Server with its RAID array is kept physically away from my work environment and is connected on 1Gbps ethernet. The Windows Server uses different access credentials to avoid ransomware style attacks.More recently I now backup document and current dev related stuff to the cloud using OneDrive as it's reasonably transparent, but that's more to enable sharing between machines than for backup purposes.I don't see how WAN or Cloud based full image backups can work practically due to the volume of data unless you have unmetered gigabit bandwidth in both directions. If you lose a drive but it's going to take two weeks to restore it then you'll probably have come up with other solutions before then.

author: rx8pilot
author_rank: Super Contributor
timestamp: December 09, 2016, 05:44:09 pm
content: I have taken a look at the suggested options.Quote from: nctnico on December 03, 2016, 06:31:42 pmrsync seems like it can technically accomplish all or most of my goals. The downside is that I have to develop the scripts and I prefer not to experiment on live data. I am afraid of making a mistake and learning something the hard way. What I did find is an rsync based package called Acrosync that essentially is a script builder that uses the rsync engine.https://acrosync.com/home.htmlLooks pretty cool and flexible. The cost is very low too. It would be nice to have a 'dashboard' or sorts to keep track of what is happening, but at the moment - my system is small enough to manage.Quote from: Howardlong on December 03, 2016, 07:08:43 pmThis product seems like it will grow with me without being so expensive that I could not buy it now. The website seems to encourage me to reach out to them to get the juiciest details, but that is ok. It is an interesting choice to pay more for something that has headroom to grow, or go with a lower cost system that may need to be replaced relatively soon as our needs become more sophisticated. It sure would be nice if all backup operations can be coordinated from a single point. The downside, is that is technically a single point of failure.