Veeam on Linux - Page 1
Forum section: electronics / general

author: PlainName
author_rank: Super Contributor
timestamp: July 13, 2021, 12:18:50 pm
content: Need to sort out Linux backups - this is a Windows house, so I am notau faitwith the Linux situation other than whenever I ask about it no-one has a solution...Veeam catches my eye. Looks reasonable (except... 3GB Ram necessary to do a bare metal restore? Blimey), but having browsed as much documentation as I can find and cope with, I can't find an answer to my incrementals question: can it cope with a changed backup set?Specifically, suppose you run off a full and then an incremental. That's a coherent backup set and should restore without a problem. Now change the destination media and run off a full and incremental. Still coherent and restorable. But change back to the first set and run off an incremental: is that restorable?There is a change tracker process which determines what needs to be backed up since the last backup job, but it's not clear if the information is based on the backup set or on the non-specific job. In the first case then switching media wouldn't be a problem, and in the second it surely would.Anyone know the answer? It is important because I never backup to my last backup media (since a problem will trash not just the current backup but the last known good one as well), thus incrementals (or diffs, preferentially) must track changes compared to the set and not local history.

author: PKTKS
author_rank: Super Contributor
timestamp: July 13, 2021, 12:42:40 pm
content: There is not a single one serve all answer.As in *NIX there are literally dozens methods to backup.They wide vary from replicating, compressed, incremental...Historically TAR is the tool which preserves just about everythingso.. expect TAR be present several times..Modern storage alternatives present the second relevant questionDepending on these 2 answers.. type_of_backup and storage* SOME* solutions are far more easier than others..Expect to do real *NIX stuff like piping, having several filtersand command options as that usually happens in unattended jobswith proper scheduling (likely in large pools cgroups)Paul

author: PlainName
author_rank: Super Contributor
timestamp: July 13, 2021, 03:07:25 pm
content: I'm not after a 'serve all' solution, just a simple and robust one for my problem: restore to bare metal. It would be nice the backup could be browsed, but that's not essential.Veeam looks like it might do that, but it is seriously broken if you can only use a single run of intermediates. Actually, I'm surprised it won't do diffs. Does any Linux backup strategy do diffs, or is it just a Veeam thing?

author: PKTKS
author_rank: Super Contributor
timestamp: July 13, 2021, 05:24:49 pm
content: You have not posted enough about your type_of_backup.Binary only? User stuff? FULL file system? Server or Workstation?Compressed type?Requires time tags ?Or just a vanilla shadow from some branch?*NIX con mount and image and put tons of different stuff in a single branch...They can vary from different file system type to different DEVICE TYPE sub systemWithout that kind of rationale chances are you will have issues choosing a wrong toolwhere actually a combination of scripting and some built in tool is enough*NIX already have almost all you need ready - as long as you can bare to use them.For the casual use I would just suggest rsync.More specialized stuff may require "imaging" or TAR for safe reasonseven more specialized stuff may required combining imaging with TAGsyou can control tags from several different appsPaul

author: bd139
author_rank: Super Contributor
timestamp: July 13, 2021, 06:07:18 pm
content: I suggest you run a fucking mile from veeam. It’s a shit show of pain. I spent half of last year getting people to remove it from things.My preferred solution is separating configuration and data explicitly. Keep configuration in GitHub and deploy with ansible. Data in one of the following destinations:1. Amazon S3. Enable bucket versioning and lifecycles to push it into cheaper storage as it ages. Various things out there can push it up there depending on your requirements and data volume. Beware: it costs nothing other than storage to put it up there but getting the data back is expensive.2. rdiff-backup either into a local volume or over SSH to another node. That does full incrementals with the “last backup” being effectively a full one and previous ones are incremental history (rather than the traditional way round of full then incrementals). The “state” is held by the destination rather than the source so you can cycle targets fine and it’ll resolve what the differences are between the source and destination state. The last successful backup is always 100% consistent directory tree as well so you can recover it with “cp” only.Both scale to hundreds of gigabytes and millions of files. I’ve used both in production for over 5 years each on absolutely critical data.Sounds like rdiff-backup is what you want.

author: Benta
author_rank: Super Contributor
timestamp: July 13, 2021, 09:07:24 pm
content: Are we talking server farm, multiple user machines or or single machine here? The picture is unclear...

author: PlainName
author_rank: Super Contributor
timestamp: July 13, 2021, 10:33:02 pm
content: QuoteI don't really care. My preference would be a disk image, but whatever works easily and simply is fine.QuoteEr, I don't know how to be more explicit than "baremetal restore". Let me try: I want to be able to put a new blank disk (or disks, as the case may be) into my PC and shortly afterwards have it be exactly as it was when the last backup took place.I guess the implication would be: boot media, preferably on a USB stick, and the backup set on a plug-in hard drive or network share.Currently I am thinking workstation, but I do have a couple of servers which would also benefit - those rely on rebuilding, importing saved configs and then restoring any saved data (although really the data is ephemeral so not actually important).

author: PlainName
author_rank: Super Contributor
timestamp: July 13, 2021, 10:50:32 pm
content: QuoteAh, thanks. I did look at it some years ago and passed, but it popped up on my radar again recently.QuoteThat kind of thing is why I never used Microsoft Backup on Windows - you need a working machine in order to do the restore part. Perhaps I am just spoiled on Windows right now, using Macrium Reflect. That's actually how I would prefer to have the Linux thing work as well, but it seems an impossible job!Briefly, I do a full job on each media (a set of sequenced removable hard drives and a network share). Thereafter it does a diif, but the diff is relative to the full which is on the media. Thus I can slap in any disk and ensure that no media is used twice in succession. Because the backups are diffs, as the drive fills up the earlier diffs can be deleted to make space, so there is a rolling history. The downside is that diffs only get bigger, so at some point you need to start with a full again (usually when I change a partition size or something).The backups are sector images, and only used sectors are stored (and those are compressed). It's very fast to run, and doing a restore is about the fastest you can achieve. As a bonus, you can mount the backup and access individual files and folders as if it were a real drive, and if it's not actually a bare metal restore (that is, it's recovery from some borked install of a shit app) then only the changed sectors get restored making it even faster.And it does all that while I'm working on the PC - the volume shadow service ensures a coherent snapshot of the system at the point the backup started, regardless of what changes after that.That's what I really want for the Linux box, but I accept I have to go with, er, something else. I just want something that doesn't need and admin certification to get going, and that I'm not going to find out that I missed something small but immensely important when the shit hits the fan and the restore isn't working.

author: PlainName
author_rank: Super Contributor
timestamp: July 13, 2021, 10:51:13 pm
content: QuoteSingle user workstation.

author: Jr460
author_rank: Regular Contributor
timestamp: July 13, 2021, 11:43:36 pm
content: Quote from: bd139 on July 13, 2021, 06:07:18 pmAgree with you on Veeam.Also I can't say this enough. You Linux machine phys or a VM should have a min of two disks that it sees. Those can be hardware RAID, Virtual disks, etc. The first disk is the OS and the packages and config files from the vendor, period. Apps, data for apps, etc all go on the second disk.If you blow up the OS, then you can take a clone image of another your machines, (do it at the VM storage level, or most HW disk controllers you can clone a mirror set in a few minutes), boot to single user, change name and IP, maybe a few other things, like mounting your app disk, and you are back up and running.Smack vendors that insist their so special code and crap goes under /opt or /usr. Nope, make on a different filesystem /crap-app and softlink /opt/crap-app to /crap-app. That way when some log file fills up /opt or something else important, the OS is still happy and you can get in with no trouble and fix things rather than having do things reboot in emergency mode, etc.Need a new machine for testing, use your OS disk clone as a starting point.Learn and use LVM always. If used correctly, you can take LV snapshots. Even without that except for the/boot can't be in LVM on the OS disk, never ever, partition (disk commands) a disk like you did for WinBlows.At some point you are going to want to restore or compare just a single file. The whole disk image route is not going to be fun. Some products say, we take an image, but we let you see the files in the image..... Yes, only because they "think" they can understand the disk and filesystem layout. Products like that (Veeam, Azure backup services and others I'm look at you) assume you you have done DOS-ish partitions, and are using straight ext2 to ext4 layout. No lovumne managers, no LVM, no other filesystems, (yes there are plenty, much better than the ext line)We just had one last week, an "expert" setup a machine with 20 disks in one big RAID5 array. Presented as just one LUN. He put the OS and app all in one big filesystem. Machine tossed up two disks within hours, the end of that RAID5 set. Yes we have backups, with Veritas NetBackup, but now we have to get an OS running on it, and then the agent installed, then you can start pulling back the Terabytes of data. But wait, doesn't Netbackup have a Baremetal restore function. Yes, but it will never work with the networking, firewall and security restrictions we have. We did get it going again, but we setup the 20 disks very differently. If it was setup correct, odds were we wouldn't have lost everything all at once, maybe the OS, but the app would be good, so just an OS build, restore a few keys files and we are done. If the app died, then just fire up the restore of it, since the OS and backup agent where working.Think about what you have, and how you are going to get things running if certain things happen. If fact try it a few times and see how much fun you have with the different procedures. The you can figure what tools, or set of tools work best for you.

author: PlainName
author_rank: Super Contributor
timestamp: July 14, 2021, 12:10:50 am
content: QuoteApparently world+dog are moving to /usr for everything apps now:https://www.freedesktop.org/wiki/Software/systemd/TheCaseForTheUsrMerge/I would like to keep each app in its own (user-specified) directory, but that seems to be impossible without compiling from source.I wonder, is that because Linux depends a lot on PATH (that is, any executable has to be on the path else you're forever typing in long path+name strings). Whereas on Windows, most apps are started from the GUI so don't need to be findable. I realise that's something of a generalisation (some Windows stuff does need to be on the path, etc).

author: PlainName
author_rank: Super Contributor
timestamp: July 14, 2021, 12:22:17 am
content: QuoteWell, this is it. After learning lessons the hard way I can say that I don't know how things are going to go bad, and that when they do that is not the time to be learning weirdo stuff or traversing chains of responsibility. Icouldjust back up data (actually, for some data it is also mirrored on the fly with versioning, but that's icing) or do other things, but I know that the one way I didn't do it is the way I'm going to wish I had. Thus a disk image pretty much covers all bases. It can be blatted onto a new disk, into a VM, browsed with a file manager, even run apps from it.

author: Ed.Kloonk
author_rank: Super Contributor
timestamp: July 14, 2021, 02:20:07 am
content: I think the one thing that mars the ppl who straddle Windows and Linux and to some degree, mac is whether or not to embrace the full system backup. Quite often the user would prefer to load a snapshot after a oh-shit moment and simply carry on. Well, we all want that. Many Linux users however soon find that storing hourly whole-system backups or worse incremental system wide backups are tedious to sort out in a recovery.You can drive yourself mad trying to save every file in the event of every eventuality. The key to developing your strategy was pointed out above. Gauge what you value and how hard you want to work to recover it. Turn-key solutions work for server farms. For us mere mortals, the admin effort out-weighs the task of just doing a manual backup, unfortunately. Luckily, in Linux, you can script and automate things. And there plenty of support to help you finely craft it so it's lean and mean.

author: Jr460
author_rank: Regular Contributor
timestamp: July 14, 2021, 02:52:31 am
content: Quote from: dunkemhigh on July 14, 2021, 12:10:50 amThat's not what I was talking about. /bin -> /usr/bin is something different. Right now on many UNIX it is already like that with links in place, so either way works. This more a difference in the System-V based UNIXes and the BSD based ones.The worst thing an app can do is just place things in /usr or /usr/bin if like. Next best thing is /usr/Crappy-App, best is in the install it ask where you want it and maybe defaults to /usr/Crappy-App. A good app will have Environment variables to setup all the places things should be. Windows puts all of that stuff in the Registry.The GUI or otherwise parts of Wins vs Windows makes no difference. UNIX (and others) had X-Windows and GUIs long before the first Windows versions appeared. Everything has to be found, that long path to a Win-Crappy.exe is in the registry. In some ways no different to the location of the app being in /etc/PATH. Don't confuse what certain shell or a window manager/GUI does to hide the details from you with what the OS needs to run an executable image. Somewhere the device and full path to search is given.

author: PlainName
author_rank: Super Contributor
timestamp: July 14, 2021, 03:06:25 am
content: QuoteThat's good advice, but the problem is you don't really know what you value until you lose it. And, generally, I value everything because the only good archive is a live one, thus my system is my archive too (yup, been bit storing important stuff offline on CD for posterity, then doing the same on DLT, etc). As for how hard I want to work to recover it, the answer is 'not very hard at all'.As I keep noting, it really is trivially easy to do on Windows. I'm aghast that it seems to be a major hassle on Linux, and even then not really practical. And that it's accepted that one has to pick and choose, mess with this and that, etc.There is an almost similar system from Terabyte:https://www.terabyteunlimited.com/image-for-linux.htmMain issue is you can't backup a live system so you have to, effectively, close down to do the backup. IME that kind of hurdle means it tends not to happen, although perhaps all you folks that leave your PCs on 24/7 might not have a problem there. Umm... so why am I not using that? Can't remember - I have a valid license but gave up on it when I was setting up this workstation.

author: PlainName
author_rank: Super Contributor
timestamp: July 14, 2021, 03:28:48 am
content: QuoteWouldn't that be better in a config file? Having everything in the environment means that a) it gets polluted with tons of stuff, lots of which may be used only rarely, and b) you can't have two different setups at the same time (unless you login as a different user, perhaps).QuoteI would dispute that. The app knows where it started from thus where the rest of it is. In fact, where there is a globally accessed file (perhaps a shared DLL) you can override it just by putting the copy you want in the app directory. No paths stored anywhere are necessary1, so the issue is simply how to get that exe running in the first place.1For common or garden apps.

author: Ed.Kloonk
author_rank: Super Contributor
timestamp: July 14, 2021, 04:17:48 am
content: Quote from: dunkemhigh on July 14, 2021, 03:06:25 amNonsense. Of course you can.

author: Nominal Animal
author_rank: Super Contributor
timestamp: July 14, 2021, 06:24:59 am
content: Note: I'm not saying that what you want is wrong, I am only describing why you are having trouble finding a solution, and what solutions I (and others) go for instead.There are three different things on a Linux system to consider for backup strategies: User files, system executables and libraries, and configuration files.If you want a true snapshot of a running system, use LVM between hardware and actual filesystems. LVM lets you make a snapshot of filesystem contents you can copy at leisure, and then remove the snapshot, without affecting the running filesystem at all. The only downside is a bit of complexity in your backup scripts (it has to make the snapshot, mount it, copy it, unmount it, and remove the snapshot) and temporarily increased disk space requirement –– changes to files during the snapshotting requiring new disk space, which you can cater to by slightly overprovisioning the partitions by keeping the LVM volumes slightly larger than the partitions using them.LVM also makes it easier to migrate a running system across storage hardware.(Unlike Windows, Linux does not stop processes from accessing files, even executables, in use at the same time. Because of the Unix inode model, executables belonging to running processes can be deleted and renamed, just not modified – except via copy-modify-rename cycle, of course. The above LVM snapshotting is not a requirement, just one option that provides a true snapshot in time, as opposed to a per-file snapshot taken sometime during the backup process.)I partition my storage so that system executables and configuration files are on one partition (root), and user files are on another (home). I only backup user files (/home) and configuration files (/etc and some subtrees under /var, excluding for example /var/lib/snapd and /var/run). I do not backup system files, because the configuration files includes the list of packages currently installed. (On Debian derivatives, you can obtain that using e.g.apt list --installed; this list is extremely useful in bringing back a system from the dead.)Because of my backup idiosyncracies (DVD-Rs, MicroSDs), I happen to usetar -Jinstead ofrsync. The difference is that I have a rotating set of full backups instead of incremental ones.Because I backup user files, I rarely restore a system exactly to how it was. Instead, I use the opportunity to upgrade to the latest stable versions, verify key applications work, then copy over the relevant user files (including email archives, some application configurations, and so on)selectively. I usually also delegate some of the rarely used user files to permanent storage, and leave them out from the new system.I also use the exact same mechanism to migrate to a new Linux distribution.This has worked well for me over two decades. I have had a motherboard b0rk (a CPU voltage regulator literally cratered), a couple of HDDs go bad, lose temporary data on USB and MicroSD Flash media; so I have needed to rely on this process at least three times (plus another three or four times voluntary migration using exact same mechanism). I have not checked how efficient this is time-wise, other than that I usually "lose" the whole day (mostly in deciding what exactly to switch to, and testing a bit before committing), but other than a couple of days work product, I've not lost anything important.Most importantly, it does not bother me when this happens: I know I won't lose much, and the balance can even be a positive change.Those relying on Linux as a tool, with the technical ability to do this, tend to do some variant of this.A tool that lets one go back to a previously snapshotted state is less useful than one that lets you migrate to a new system.This is why "single button backup and restore" solutions just aren't that popular in Linux – except for users who look to duplicate Windows functionality. The free/open source folks are mostly in the former group, and the latter group is serviced by paid proprietary software vendors.If you have neither the time nor inclination to set up such a backup system suitable to your work flow, I recommend you check the Linux distribution you use for backup tooling. You'll find quite a few tools that can automate this for you, with both command-line and GUI interfaces. Some only provide the backup tool and expect you to add the triggers yourself (usually in crontab, to be run at specific times), some provide the full kaboodle.rcloneis pretty commonly used, if you use cloud storage for your backups (but do consider using thecryptremote, so that the cloud backups are encrypted and decrypted on your local machine).If you have a virtual machine running Linux, I recommend making a snapshot, then installing one or more of the backup softwares, and seeing how their UIs feel like. Instead of uninstalling, just restore the machine to the previous snapshot; this saves time, and leaves no uninstaller crumbs lying around.I habitually do this – run another instance of Linux in a virtual machine on my Linux machine – for exactly this purpose: testing software I'm curious about, but unwilling to commit my work environment to.

author: PlainName
author_rank: Super Contributor
timestamp: July 14, 2021, 11:21:57 am
content: QuoteI appreciate you (and everyone else) posting thoughts on this and detailing possible solutions. As you note, part of my problem is trying to make a Windows solution work with Linux.I think part of my issue - a reluctance to reinstall the base and then restore configs, etc - is probably down to having once tried that and found it didn't work. There is always some config that isn't where it should be, or you didn't know it existed. That might be a Windows thing since it typically resolves to searching the registry to find likely entries.One colleague used to swear by just saving data and then reinstalling Windows and apps if necessary. Which sounded good but was tedious (some apps I forget where the install files are, and the company may no longer exist), and which assumes nothing is ever modified from default settings, because you'll drive yourself mad trying to duplicate every tweak and adjustment again. Hence just simpler to take everything, kitchen sink and all.With this workstation I did set out with the idea that my stuff would be completely separate from the OS because I wasn't sure which version of Linux I was going to go with. So the plan was that I could have dual boot into whatever distro and use the single data/config with whatever happens to be running. And, of course, I would only need to back up my data/config partition because the particular OS version wouldn't matter too much. However, it didn't work out like that. Maybe I'm not clever enough, even with the might of Google to fall back on!But, back to this thing. I think my question regarding Veeam is clearly answered, so thanks for that. I will think about the other suggestions and reflect on how my workflow might be suitable changed to accommodate some.

author: DimitriP
author_rank: Super Contributor
timestamp: July 14, 2021, 12:44:12 pm
content: backupEDGE from Microlitemicrolite.comYou are welcome!

author: PKTKS
author_rank: Super Contributor
timestamp: July 14, 2021, 02:53:27 pm
content: Have not tried BORG myself...But used partimage sometimes to save some space.It does what it says so. WO much hassle.Otherwise I just go sane w/RSYNC and TARed compressed images.Being open helps a tidbithttps://github.com/borgbackup/borg/releaseshttps://www.borgbackup.org/# best imaging tagged toolhttp://www.partimage.org/Paul

author: DiTBho
author_rank: Super Contributor
timestamp: July 14, 2021, 03:46:26 pm
content: I am in the process of designing and implementing a very crazy idea: a p2p-based distributed backup for my project-sources. For weird reasons, it's what I really need.The final media will be a, hard-drives (250/500GB per unit), DVD-RAM (4GB per cartridge) and LTO6 (40GB per slice).

author: PlainName
author_rank: Super Contributor
timestamp: July 14, 2021, 04:37:53 pm
content: Quote from: DimitriP on July 14, 2021, 12:44:12 pmThanks. Looks interesting, but not $500 interesting!

author: PlainName
author_rank: Super Contributor
timestamp: July 14, 2021, 04:40:21 pm
content: QuoteThanks. However, it's a centralised server-based affair (AFAICS), which just punts the problem to a higher level. I don't want to end up with a single point of failure to protect with it's own backup issues.

author: PlainName
author_rank: Super Contributor
timestamp: July 14, 2021, 04:41:52 pm
content: QuoteThat's getting there, but needs the system to be offline while it does the biz. I already have tools which will do that, and they're not used...

author: PlainName
author_rank: Super Contributor
timestamp: July 14, 2021, 05:02:37 pm
content: QuotePresumably that applies only to things installed via the repo's package manager? What about Windows apps installed into Wine? Or user-installed stuff (say, esp-idf)? The package manager wouldn't know about those, so they would have to be reinstalled manually, wouldn't they?QuoteGetting back to this, because it's a biggie and affects my entire strategy.Recently my Nextcloud server ran out of disk space, despite user files only taking up 10% of what should be available. I had this problem some time back and tracked it down to the Nextcloud cron not being run - it needs to be hooked into the system crontab or set to run whenever anyone uses the web interface (no-one does). I fixed that with a bit of customisation, and it was fine. Until it wasn't. Turns out that some update to the OS (not Nextcloud - I am too clever to let that update without notice) vaped or changed something so that the cron update no longer worked. Indeed, the thing no longer exists.Next, happily programming away when one day I found subversion couldn't check in to repos on the network. Took me a long time to figure out it was a silent update to Kaspersky AV that caused the issue (the primary reason for switching to Bitdefender - Kaspersky is unfixable for this). The thing to note is that in tracking it down I was able to restore before and after backupsvery quicklyand thus be able to zero in on what the differences were. A simple config or data backup would never have allowed this since the before backup would be just as broken as the after one. Also, until I needed to restore whatever it was that fixed things, I didn't know that I needed it.These sort of things happen and, as you say, periodically one has to figure stuff out and take the opportunity to update (or block from updating!), etc. But midway through a disaster recovery isabsolutely notthe time to do that. No, I am not going to fart around trying to work out why this apparently compatible install doesn't work, or breaks something. Typically, a disaster isreallya disaster, and there is no need to make it a bigger one.So that's where I am coming from. I want to know I can recoveras quickly and perfectly as possiblefrom an actual disaster, and then as a lower priority I can think of icing like how to undelete a fat finger moment, or compare this version to last weeks, etc.

author: rsjsouza
author_rank: Super Contributor
timestamp: July 14, 2021, 05:35:25 pm
content: Interesting discussion; I have been involved with this and my only solution at the time was to simply partition the various development environments (multiple SDKs hosted in multiple versions of Ubuntu) in separate virtual machines.The concept of re-creating an environment is very interesting when you are actually interested in updating the tools around it; in my case, however, I needed to keep the environments as similar as possible. I also had to distribute them across teams, so the VM was the easiest (although huge in size).In Windows-based environments, I used to create a system image of the main hard drive with all updates and bare minimum utilities that allowed to move forward with reinstalling everything - however, these images have a drawback of not working if the underlying machine is different enough (drivers would start to fail).Nowadays in my home system (Windows-based) I pretty much do a data backup to a NAS and also keep the original installers of the utilities I use - after I got burned, I don't trust the internet anymore to have the exact versions of what I need. I still need to trust and provision ($$$) for a cloud service.

author: Nominal Animal
author_rank: Super Contributor
timestamp: July 14, 2021, 06:49:56 pm
content: Quote from: dunkemhigh on July 14, 2021, 05:02:37 pmYes.QuoteThose are under user home directory,$HOME/.wine/.QuoteYes. I do not install such things (say, Arduino and Teensyduino, me being a Teensy user) in the system, but keep them under my own user home directory. Their binaries are launched by symlinks or scripts in$HOME/bin/, which is the first directory in my PATH. Same for stuff I pull from git. In that sense, they are "user files", just like stuff installed under Wine.QuoteHell, that should not happen. System updates should not overwrite customized configuration files without asking, and package managers should complain loudly if trying to remove a package whose configuration directory has user-customized files.The likeliest explanation is accidentally uninstalling anacron (and losing /etc/anacrontab), but otherwise it sounds like a serious bug.On servers with nginx or Apache, I do use a completely custom configuration directory for the service, instead of the /etc/apache2/ or /etc/nginx/ default in Debian derivatives. This is to ensure that system updates do not affect the service configuration at all. It is an exception, though.QuoteIn my experience, very rarely; but out workflows and the tools we use are completely different, which I think explains the difference.(Me using "disposable" virtual machines to experiment, is one way I keep my main working environment "clean", and reduces the likelihood of such events significantly.)QuoteIt sounds likeLVM snapshotswould work best for you, with both / (root) and /home (home) on the same partition.An LVM snapshot is an instant snapshot of an LVM volume. It can be mounted as a separate device, without affecting the volume it is a snapshot of, as long as the LVM Volume Group has sufficient unused disk space for it. (It is copy-on-write.) You can have as many of them as you like, you can mount them and explore and compare them, and so on.You can revert the volume to any previous snapshot with just a single command.Using LVM2 snapshots can be mounted read-write can be made read-write, so you can make a snapshot, then restore the contents of that snapshot from offline storage, and then revert to that.Unfortunately, I don't know of any GUI backup tools that do that.It also only works if one tells the installer to use LVM at system installation time, and gives the LVM volume groups much more disk space than the actual partitions (so that there is always disk space for the snapshots available).However, based on your description, I suspectTimeshiftandBack in Timemight work for you. If you have a NAS running Linux you store the backups to, you might wish to try outBacula.(I do not use them myself, and have not installed them. I've only seen them mentioned and read their descriptions.)

author: Jr460
author_rank: Regular Contributor
timestamp: July 14, 2021, 07:17:15 pm
content: Quote from: Nominal Animal on July 14, 2021, 06:49:56 pmCurrent workplace has "standards" that say you will use LVM, otherwise your system fails "checks". Had almost the same thing at other places many years before, you will use LVM, or you will rebuild the server from scratch.Let's see 20+ years ago, HP's Omniback, which changed names to "Data Protector", and I have no idea what it is called now, or if HPE sold it off to someone else.Had big Oracle database server, other than boot disk, data was all on a fiber channel attached SAN. Backup master server had connections to SAN and had a tape library. With no scripts, it would do a split mirror backup. When backup started, it would run an optional script of your writing on the database server, then tell the SAN device to split the mirror it had. It would then run a post-split script on the database server. then it would activate the VGs in read-only mode, and mount the file systems and back them up. Once all done rip things done on the backup server and tell the SAN to update the mirror. The SAN kept an update list, so it was never a full re-mirror. Now we did have a pre and post split script and in the first one it flushed redo logs, checkpoints the DB, etc. and then suspends IO. The post script let the IO fly again. A backup in the middle of the night and the database would juts seem pause for a few seconds.I know this not exactly what you said. Also Veritas Netbackup will snapshot backups with LV, or SAN mirror, or filesystem snaps, whatever you want in the GUI.

author: bd139
author_rank: Super Contributor
timestamp: July 14, 2021, 07:20:40 pm
content: Just a point on LVM snapshots. They’re not 100% consistent. They’re file system consistent but they are not application level consistent. Not everything fsyncs properly.

author: PlainName
author_rank: Super Contributor
timestamp: July 14, 2021, 11:10:28 pm
content: QuoteUnexpected, yes, since OS updates are usually 'safe'. On Windows too pre-10. May have been something I did (or, more likely, didn't do), but water under the bridge now.QuoteThat's generally what I do, and if it's a fairly safe app I'll run it under Sandboxie (a kind of chroot thing). But sometimes one slips through, and there are some normally OK apps that just won't work in a VM - FilterForge, I think, is one such (and they lost a sale because of that).QuoteMight do! It's not quite the same as VSS though - the system has to be effectively stopped to take the snapshot, and you need at least as much snapshot space as the volume you're snapshotting. It has potential, though.QuoteThat's a rather big fly in the ointment. Still, it might be both doable and worth itQuoteWorth taking a closer browse after a quick look, thanks. I'd briefly looked at bacula before, but it is again pushing the problem further up the road. And the interesting bits (snapshotting) are only in the enterprise version, for which they are too embarrassed to say upfront how much it costs.

author: PlainName
author_rank: Super Contributor
timestamp: July 14, 2021, 11:14:40 pm
content: Quote from: bd139 on July 14, 2021, 07:20:40 pmIndeed, and when it comes to stuff like databases they might not even be current. Applies to any kind of snapshotting (so not just LVM but also my Windows stuff).

author: bitwelder
author_rank: Super Contributor
timestamp: July 23, 2021, 01:02:13 pm
content: For bare metal recovery and various levels of backup on my Linux machines generally I use a combination of:- Clonezilla (to take once maybe a year a cold image of the disks, and e.g. NFS it to my NAS). I use it also with encrypted disks, Windows images, etc.- Timeshift (to save daily snapshots of the live filesystems)- plus additional rsync, syncthing and similar (for frequently changing work directories)The tools are all free

author: Benta
author_rank: Super Contributor
timestamp: July 24, 2021, 07:13:58 pm
content: On my (several) Lubuntu laptops, I use two types of backup. My machines all have a main user (admin/sudo) of course, but I often also add guest accounts for friends needing a laptop temporarily.For my main account, I use:1: Timeshift for system backup. This backs up all system files, plus the hidden files/directories in /home. I use this very often when I'm experimenting with new installations, or mucking around with system configurations. It does NOT back up /home generally.2: BackInTime for /home backups. This backs up all user files except the hidden files/directories (I've added a rule to exclude those).This split works well for me. The guest accounts can also use BackInTime freely for their own stuff. Timeshift is exclusively for me.Both are incremental and take up little space except for the initial backup.

author: PlainName
author_rank: Super Contributor
timestamp: July 24, 2021, 10:00:18 pm
content: Thanks, both. The thing that gets me about these is the single backup - you have your original data, and your backup, and that's it. So for me I would need to duplicate that backup destination somehow, which means another job.Thinking it through, I couldn't use an external drive for the destination (which is what I do now) because I could only use one drive. Yes, I could then copy that, but that's an extra manual step which means it's not going to happen often. Or I could have a network drive as destination, and an automated copying system for that. But now we've added at least one more box, plus the automated whatever now needs to be backed up, etc.One thing I forgot to ask: have you used your processes in anger? If so, often or just a one-off?

author: Foxxz
author_rank: Regular Contributor
timestamp: July 25, 2021, 03:35:32 am
content: I evaluated Veeam in a corporate environment close to 10 years ago. The security of the product was abysmal. It was sending VMWare creds in plaintext to the Veeam agents to backup VMs. And it seemed like the agents that ran on individual systems at the time had no security at all allowing full RW access to the filesystem. There were CVEs for the latter that I remember. Take a look here to see what current security issues it hashttps://www.cvedetails.com/vulnerability-list/vendor_id-15994/Veeam.html

author: PlainName
author_rank: Super Contributor
timestamp: July 25, 2021, 04:23:22 am
content: They do seem to have dropped the ball a couple of times.