Restic

Related articles
System backup
Disk cloning
System maintenance#Backup
File recovery
This page discusses the
restic
backup tool, provides a 'quick start' guide, and suggests best practices in the context of Arch Linux.
Restic's main features and benefits are:
encrypted backups
remote backups
built-in support for compression
efficient storage (chunk-based increments, data is not duplicated)
gives flexibility to use a custom scheduler like cron or systemd
written in Go, providing stand-alone binaries
Installation
Install
restic
, then initialise a repository in an empty directory (for local backups) with:
$ restic init --repo
/path/to/backup/directory/
See the
official tutorial
.
Security
Restic stipulates
threat-model
assumptions for usage.
It uses symmetric encryption for repositories. This introduces some issues with scheduled backups
[1]
[2]
, as the key would generally have to be stored in plain text for an automated process to be able to create the backup. Ideally there would be asymmetric encryption that allows to create snapshots with a public key (used in an automated script by restic) yet only decrypt the snapshot with a private key, but this is not supported.
As an advantage of restic's symmetric encryption, it features
restic-key(1)
commands to manage multiple keys for a repository. Hence, it is possible to add a primary key and secondary keys. Further, the
--password-command
option can be used to fetch a key from a vault. In combination it is possible to safeguard a primary key and use a secondary key:
for an
append-only
backup repository,
securing it via a vault like
systemd-creds
, or with a
second-factor token
.
The above methods can be configured both for a system (root) and a regular user, limiting the risk associated with plaintext password credentials for the backup repository.
This article or section needs expansion.
Reason:
Mention TLS features
(Discuss in
Talk:Restic
)
Note
Restic repositories (backups) do not need to be encrypted with LUKS. Restic encrypts data by default.
Scheduling
systemd timers
Unlike other tools like timeshift
[3]
, restic does not include a scheduling capability. You are expected to either use
cron
or
systemd timers
.
You can use a ready project like
restic-automatic-backup-scheduler
or follow this example that creates local (full) system backups and should be run as the
root user
.
The example assumes there is an existing directory where an existing restic repository has been initialised (see
#Installation
).
Create separate volume
This step is optional, but it is a good idea to mount a separate volume there to prevent the automated backup process from potentially consuming all space available to the OS.
The first restic backup will have to clone the entire OS, so the minimum amount of space required is equal to the space taken up by the OS (subject to path exclusions described below) or any other directory that you decide to backup.
Of course, you need space for any additional incremental changes in the future so it is a good idea to create a volume with 2 or even 3 times the size of the data being backed up. E.g. if
/
takes up
55G
then you can create
/mnt/restic
with
110G
.
systemd service
You will need a service unit.
Create
one:
/etc/systemd/system/restic-backup.service
[Unit]
Description=Backup system

[Service]
ExecStart=systemd-inhibit /usr/local/bin/restic-backup
Note
The service will be invoked by the timer below but can also be
started
on demand for any unplanned snapshots.
systemd-inhibit(1)
is optional but recommended to prevent accidental shutdown of the system while a backup is being created. Only available when running in user session
To run the service with minimal required privileges while still allowing read access to all user files, run the service as a restricted user with AmbientCapabilities=CAP_DAC_READ_SEARCH
/etc/systemd/system/restic-backup.service
[Unit]
Description=Backup system

[Service]
Type=simple
User=backupuser

AmbientCapabilities=CAP_DAC_READ_SEARCH
CapabilityBoundingSet=CAP_DAC_READ_SEARCH
DevicePolicy=closed
LockPersonality=yes
MemoryDenyWriteExecute=yes
NoNewPrivileges=yes
PrivateDevices=yes
PrivateTmp=yes
ProtectClock=yes
ProtectControlGroups=yes
ProtectHome=read-only
ProtectHostname=yes
ProtectKernelLogs=yes
ProtectKernelModules=yes
ProtectKernelTunables=yes
ProtectProc=invisible
ProtectSystem=strict
RemoveIPC=yes
RestrictAddressFamilies=AF_UNIX # AF_INET AF_INET6 (for remote storage)
RestrictNamespaces=yes
RestrictRealtime=yes
RestrictSUIDSGID=yes
SystemCallFilter=@system-service
UMask=7007

ExecStart=/usr/local/bin/restic-backup

# this makes sure the service prevents shutdown while running (for 90 seconds by default)
ExecStop=bash -c 'if [[ -n "$MAINPID" ]] ; then tail --pid="$MAINPID" -f /dev/null; fi'
Configuring resource constraints
Using systemd gives you the option to put resource constraints on the backup process. E.g. you can limit the amount of memory and / or CPU. This is something you would configure in the systemd service unit. See
systemd.resource-control(5)
.
Another way to constrain the resources used by restic is to use the
GOMAXPROCS
environment variable as described in the
official documentation
.
systemd timer
You will also need a timer unit (this one runs every 15 min):
/etc/systemd/system/restic-backup.timer
[Unit]
Description=Timer for full system backups

[Timer]
OnBootSec=5min
OnUnitActiveSec=15min
Unit=restic-backup.service

[Install]
WantedBy=timers.target
Backup script
You will also want to create a small shell script to pass in all the required options, for example:
/usr/local/bin/restic-backup
#!/bin/bash

if pgrep -f 'restic backup' > /dev/null; then
  echo 'restic is already running...' 1>&2
  exit 0
fi

set -e
set -v

export RESTIC_REPOSITORY='/mnt/restic'
export RESTIC_PASSWORD_COMMAND='/usr/local/bin/get-restic-password'
export RESTIC_COMPRESSION='off'
export RESTIC_CACHE_DIR=~/.cache/restic

mkdir -p "${RESTIC_CACHE_DIR}"

restic unlock 
restic backup / --exclude-file=/etc/restic/excludes.txt --tag scheduled 
restic check --with-cache --read-data-subset=5G
restic forget --prune --keep-hourly 24 --keep-daily 30 --keep-monthly 6 --keep-weekly 4 --keep-yearly 3
/usr/local/bin/get-restic-password
#!/bin/bash
echo VerySecurePassword123
# chmod 744 /usr/local/bin/restic-backup
# chmod 700 /usr/local/bin/get-restic-password
Compression
You might want to consider
enabling compression in restic
to save space if you are backing up data that is not already compressed (like large text files).
Snapshot retention
Adjust the
restic forget --prune --keep-hourly 24 --keep-daily 30 --keep-monthly 6 --keep-weekly 4 --keep-yearly 3
values in the script above if wanted.
Configuring niceness
You may also wish to tweak niceness of the backup process. If you are running backups often you will likely want to reduce the resource usage to prevent it from affecting interactive use. However, you should check how long the backups are taking and make sure they are not overlapping (i.e. a new backup being started when the previous one has not finished).
You can do that with
nice(1)
. You may want to adjust the backup script by adding nice to the beginning of the resource intensive commands e.g.:
# nice -n 19 restic backup ...
# nice -n 19 restic check ...
Alternatively if you are using
ananicy-cpp
you may want to ensure that the niceness is configured in its configuration file(s) under
/etc/ananicy.d/
.
/etc/ananicy.d/00-types.types
{"type":"file-sync","nice":19,"ionice":7}
/etc/ananicy.d/99-custom/file-sync/restic.rules
{"name": "restic", "type": "file-sync"}
Refer to the
restic FAQ
for more information.
Excludes list
Add the excludes file under
/etc/restic
e.g.:
/etc/restic/excludes.txt
/data/**
/dev/**
/home/*/**/*.pyc
/home/*/**/__pycache__/**
/home/*/**/node_modules/**
/home/*/.cache/**
/home/*/.local/lib/python*/site-packages/**
/home/*/.mozilla/firefox/*/Cache/**
/lost+found/**
/media/**
/mnt/**
/proc/**
/root/**
/run/**
/swapfile
/sys/**
/tmp/**
/var/cache/**
/var/cache/pacman/pkg/**
/var/lib/docker/**
/var/lib/libvirt/**
/var/lock/**
/var/log/**
/var/run/**
Note
Virtual filesystems such as
/proc
or
/run
and partitions mounted under
/mnt
can be automatically excluded from the backup with the
--one-file-system
flag, instead of adding them to an exclude file. This has the advantage of preserving e.g. the directories under
/mnt
to not lose filesystems mountpoints. See
Backing up
for details.
Enable
Do not forget to
enable
the
restic-backup.timer
.
Remote append-only backup repository
As with most backup solutions, restic backups can usually be modified by the user that executes the backup. This makes the backup data vulnerable to manipulation and threats like ransomware. While the above example runs the backup as root and thereby prevents trivial modification by local unprivileged users, a compromise of the system will allow malware to delete/overwrite the backup data as well, even if that data is in a remote repository (since the local user authenticates to the remote repository). In order to setup a secure backup solution that cannot be modified, restic can be used in combination with
rclone
to make use of its append-only feature run on a restricted remote system.
Tip
If you control the remote system, the project's
restic-rest-server
AUR
is an alternative with its
append-only
mode.
Note
Be aware also an append-only repository can be vulnerable, for example if an attacker can manipulate a backup schedule or system date. Restic
advises to use
special use of its
--keep-within
option to execute any backup retention policy to forget/delete snapshots.
Setup restricted 'rclone serve' over ssh
The following will restrict the user to the specified command, effectively allowing only the addition of new data to the path
/home/myuser/backup
while preventing the modification of existing files.
/etc/ssh/sshd_config
/etc/ssh/sshd_config
Match user myuser
  ForceCommand rclone serve restic --stdio --append-only ./backup
Run restic through rclone
Initialize the backup repository on the remote ssh server:
restic -o rclone.program='ssh myuser@backup.tld' -r rclone: init
Execute restic backup with rclone as transport:
restic -o rclone.program='ssh myuser@backup.tld' -r rclone: backup /home/myuser/importantData
Append-only script
Note that it is no longer possible to use the prune option in order to delete old backups since no data on the remote repository can be modified now. The user can only write-append new files and read existing backup data to restore.
/usr/local/bin/restic-backup
#!/bin/bash

if pgrep -f 'restic backup' > /dev/null; then
  echo 'restic is already running...' 1>&2
  exit 0
fi

set -e
set -v

export RESTIC_REPOSITORY="rclone:"
export RESTIC_PASSWORD_COMMAND='/usr/local/bin/get-restic-password'
export RESTIC_COMPRESSION='off'
export RESTIC_CACHE_DIR=~/.cache/restic

RCLONE_EXEC="rclone.program=ssh myuser@backup.tld forced-command"
DATA_PATH="/home/myuser/importantData"

mkdir -p "${RESTIC_CACHE_DIR}"

restic -o "${RCLONE_EXEC}" unlock
restic -o "${RCLONE_EXEC}" backup ${DATA_PATH} --exclude-file="$HOME/.local/scripts/excludes" --tag scheduled
restic -o "${RCLONE_EXEC}" check --with-cache --read-data-subset=5G
Restoring backup data
restic -o rclone.program='ssh myuser@backup.tld forced-command' -r rclone: restore latest --target /tmp/restoredData
Tips and tricks
Password with token as second factor
The following employs a
Universal 2nd Factor
token, a
YubiKey
in this example, to derive a password used to unlock a restic repository with its
--password-command
option. A prerequisite is an already set up
challenge-response
slot, since it can be used for multiple purposes.
First, a random user key is generated and fed into the token's challenge-response function to obtain its token-unique hash:
$ dd if=/dev/urandom of=/home/
username
/
resticblob
bs=512 count=1
$ chmod 400 /home/
username
/
resticblob
$ ykchalresp -2 -i/home/
username
/
resticblob
71832e30cf9d5adb8672154d7a83fa1684f544d3
Second, the hash is copy-pasted to be added as a user access key to the restic repository.
Note
You
must not
save the static hash anywhere, its purpose is to unlock the repository when the token is available. To access the repository without the token, you can rely on the repository password used to add the new key.
$ restic -r /home/
username
/
restic
/ key add
enter password for repository: 
repository 045a06ef opened (version 2, compression level auto)
enter new password: 
enter password again: 
saved new key with ID 1991e876106f203c245e45a401b59dedec4aae6656f89152b66eca180385c1b
Now, the token can be used to access the repository transparently:
$ restic -r /home/
username
/
restic
unlock --password-command "ykchalresp -2 -i/home/
username
/
resticblob
"
repository 045a06ef opened (version 2, compression level auto)
Note for the non-interactive use this method requires a token configuration without user-presence (touch) verification for the challenge-response method and some tokens do not allow to configure the feature. The same restriction applies for other methods, unless the
--password-command
option is used to execute a shell script to prepare its output accordingly. Also possible is to use the
--password-file
option, see
for example
.