Backup automated using rdiff-backup
Tags: backup, linux, bash
Published: 2019-02-22 23:47:41

---
title: Backup automated using rdiff-backup
published: true
description: One day your blog, code or pretty much anything may crash, and sadly, your most valuable information could be irredeemably lost ! Consider the consequences if this ever happens (touch wood!). Pictured them? Scary, right? Now, just imagine how relaxed you would have been instead, if only you’d bothered to make a backup.
tags: backup, linux, bash
---

*****

Originally published at [www.carloscaballero.io](https://carloscaballero.io/backup-using-rdiff-backup/) on January 25, 2019.

*****

One day your blog, code or pretty much anything may crash, and sadly, your most
valuable information could be irredeemably lost ! Consider the consequences if
this ever happens (touch wood!). Pictured them? Scary, right? Now, just imagine
how relaxed you would have been instead, if only you’d bothered to make a
backup.

Today I’m going to show you my personal backup method. I use the awesome
`rdiff-backup` tool which combines an incremental backup with a mirror.<br>  You
can read more about this tool on the [official
page.](https://www.nongnu.org/rdiff-backup/)

> *What is it? ***rdiff-backup*** backs up one directory to another, possibly over
> a network. The target directory ends up a copy of the source directory, but
extra reverse diffs are stored in a special subdirectory of that target
directory, so you can still recover files lost some time ago. The idea is to
combine the best features of a mirror and an incremental backup.*

### Installation

rdiff-backup is available in the most important linux distribution. In my case,
I’m using an ArchLinux distributions ([Manjaro](https://manjaro.org/)) and the
[yay package](https://github.com/Jguer/yay) (Yet another Yogurt — An AUR Helper
written in Go) to install the tool.


If you use another distribution, this software can also be installed:



### Using rdiff-backup

Making backups is very easy when you using `rdiff-backup`. You may picture this
tool as similar to the `cp` command. In other words, `rdiff-backup` has two
arguments:

* source directory.
* target directory.

Both directories can be local or remote disk. For example, if you want to use
`rdiff-backup` in a local directory you would use the following command:

![](https://cdn-images-1.medium.com/max/1000/1*zVTgymEetqnY6I3nwnnckA.png)

In the same way, if any of the directories are in a remote server, you need only
to indicate the path using the classic way: `user@server::PATH`. The following
commands show how either the remote or local servers can be used in both the
source and target directories:

![](https://cdn-images-1.medium.com/max/1000/1*aUrO1bpW2AqpSMyvO6UVHQ.png)

When using these commands, the remote machine will probably request the user’s
password (for the previous commands, `carloscaballero` and `luisgarcia`
respectively). You can omit this step by configuring an [SSH Key-Based
Authentication on a Linux
Server](https://www.digitalocean.com/community/tutorials/how-to-configure-ssh-key-based-authentication-on-a-linux-server).

The real power of this tool is truly appreciated when wanting to restore the
information. If you list the contents of the directory in which you made your
copy, you will see the contents that you’d previously copied, and futhermore,
you will find a directory named `rdiff-backup-data`. This directory is very
important, since it stores the [incremental
backups](https://en.wikipedia.org/wiki/Incremental_backup) of our data.

In this directory, the contents shown consist of the last version of our backup,
plus the incremental copies, which are stored in the
`rdiff-backup-data/increments` directory.

Now imagine that I’ve created a file called `file1.txt` which contains a single
sentence. A copy is done using `rdiff-backup` and, a few minutes after another
copy is done. Now, we shown the list of files in our system wich is the
following:

![](https://cdn-images-1.medium.com/max/1000/1*rXVJ-N6ctwWTQ5yUGOD-3g.png)

You may note that the file `file1.txt`has an incremental copy in the
`increments` directory.

### Restoring backups

We can restore a copy with the `rdiff-backup` comand, or by directly using the
`cp` command, since the copy is neither compressed, nor has any of its metadata
altered. Therefore, the files are in the same state as when they were copied.
Although, you may use the `cp` command, the `rdiff-backup` tool is better to
use, due to the data restoration being more flexible.

The use of the command for restoring backups is similar to the one to make the
backup, with the added the option of (restore-as-of, `-r`) , as well as the
timestamp to restore. The timestamp is very flexible, since the acceptible time
strings are intervals, like "3D64s"; w3-datetime strings, like
"2002-04-26T04:22:01-07:00" (strings like "2002-04-26T04:22:01" are also
acceptable - rdiff-backup will use the current time zone); or ordinary dates
like 2/4/1997 or 2001-04-23 (various combinations are acceptable, bearing in
mind that the month must always precede the day).

For example, the following command restores the copy made on 23 January 2010.

![](https://cdn-images-1.medium.com/max/1000/1*BZ2fELaLWzWmrQJQgxj-lA.png)

As you already know, the `rdiff-backup` command makes an incremental backup,
which entails a large amount of space disk being consumed. Therefore, it is
highly recommended to remove old backups (as long as you have other, more recent
backups, of course).

The `rdiff-backup` tool has the `remove-older-than` option, which removes any
backups older than that the date used in the argument. A good example is
removing any backups older than 1 year:

![](https://cdn-images-1.medium.com/max/1000/1*kT75Rt9loiExqYihqjtKQw.png)

### Filter Options

Most of the time, we are required to include o exclude files to our backup. The
most common options which can be used in the rdiff-backup are:

**- include.

* include-file-list
* exclude.
* exclude-file-list**

As well as these, there are plenty more filter options to make our backups, such
as:

![](https://cdn-images-1.medium.com/max/1000/1*V3iZR5g4ZvzBxSiG_ilO7g.png)

In this example we exclude /mnt/backup to avoid an infinite loop, even though
rdiff-backup can automatically detect simple loops like the one above. This is
just an example, in reality it would be important to exclude /proc as well.

There may be a time when we need information about the backup (metadata).
`rdiff-backup` allows us to obtain this information. The most common options for
this are the following:

* list-increments
* list-changed-since
* list-at-time
* compare
* compare-at-time

Since they are quite descriptive, it isn’t hard to imagine what the goal of each
of the different options is. Despite this, I will show several examples applying
each of them:

![](https://cdn-images-1.medium.com/max/1000/1*ynVmCLE9ualIW8UhAPhE4A.png)

### Using in cron

A good practice is automating the backups in our system. To do this, we may use
the cron service.

Prior to using cron, we must remember to make sure that the script used in cron
doesn’t output anything, otherwise:

* cron will assume there is an error
* if there is any error, you will not be able to see it

The command which we used in our script is the following:

![](https://cdn-images-1.medium.com/max/1000/1*KxWvPyaoOLXYiyRXwmiCFg.png)

The content of the `files_backup.txt` file is the following:


It is important to know that both success and error logs are saved in the same
logfile, named `rdiff-backup.log`. Another interesting point is that I've used
the filter option `include-globbing-filelist` which allows the use of a file as
argument. This file contains the directories which will be backed up by using
the string `+` or `-` to express that said directory must be either included or
excluded. Note that the backups older than 1 year are deleted to perserve disk
space.

Finally, edit the cron file using the `crontab -e` command.


### Conclusions

In this post I’ve explained the `rdiff-backup` tool, which allows us to make
incremental backups. I've also shown you the script I use to backup my projects,
which is executed by cron one time a day.

* [More examples from the official
page.](https://www.nongnu.org/rdiff-backup/examples.html)

*****

Originally published at [www.carloscaballero.io](https://carloscaballero.io/backup-using-rdiff-backup/) on January 25, 2019.

*****

Hi! My name is Carlos Caballero and I’m PhD. in Computer Science from Málaga,
Spain. Teaching developers and degree/master computer science how to be experts!