Package: distbackup
DistbackupDistbackup is a tool that can back up a large set of files to a number of offline
disks. Only one backup disk needs to be connected at a time for distbackup to operate on
it. Distbackup will attempt to distribute redundant copies of files across the backup
disks, as long as there is room.InstallationDistbackup can be installed using PIP:python3 -m pip install distbackupIf it fails, make sure you have Python, PIP, and OpenSSL development libraries
installed. On Ubuntu, runsudo apt install python3-pip python3-dev python3-wheel libssl-devDistbackup usesargcompleteto assist with tab completion in your shell. To enable it
(forbash), run:mkdir -p ~/.local/share/bash-completion/completions/
register-python-argcomplete --shell bash dsb > ~/.local/share/bash-completion/completions/dsbArgcomplete also supportstcshandfish. Runregister-python-argcomplete --helpfor
more info.How it worksDistbackup maintains a database (SQLite) which contains metadata about every file in the
source set, including its name, last modified date, and a SHA256 hash of its
contents. The hash is then used to uniquely identify an "object", which is then copied to
one or more backup disks, along with a copy of the database itself.  The files on this
backup disk are stored only by their hash, so a restore requires reading the database to
reconstruct the original file structure.The file structure on the backup disk looks like this:├── distbackup-disk.json
├── distbackup.sqlite
├── distbackup-objects
│   ├── 000
│   │   ├── 0005e3d2a9216a465148b424de67297ad5ce65b95289294f3ef53c856ca55088
│   │   └── 000c00bad31d126b054c6ec7f3e02b27c0f9a4d579f987d3c4f879cee1bacb81
│   ├── 004
│   │   ├── 0046066f500854ebc1eb5d679a7164235de42efdf4dfbacff70d9bdb5a2d65db
│   │   └── 004cf775fda2783974afc1599c33b77228f04f7c053760f4a9552927207a064e
│   ├── 007
│   │   ├── 00702164a628a9e65266f4aafec2e1faebc42f0cc2145408a74c3feae39bef6d
│   │   └── 0077c553ae28326ef59c06e3743a6ddf5e046d9482eb9becfa8e06ff5bd37e2e
│   ├── 008
│   │   └── 0083cc2e1d1d989795d02aa47d4dd42b9f90b644d025cece0ab3c953b3a4fa09
.
.
.Since objects are identified by their hash, their contents are immutable. This means that
if a source file changes, it will have a new hash and therefore refer to a new object.Distbackup works with one backup disk at a time. First, it will delete any orphaned
objects (i.e. files that have been deleted or changed on the source), then it will copy
any new objects to the disk, and then, if it still has room, it will try to make redundant
copies of objects that have already been copied to other disks. It may delete redundant
objects off the disk to make room, as long as the overall redundancy is not reduced. For
example, if there is an object which already has one copy on another disk, it may decide
to delete an object that has two copies on other disks to make room for a second copy of
the first object.Getting startedAll distbackup commands are accessed viadsb. You can rundsb --helpat any time to
get a list of commands.Distbackup keeps its data in an SQLite database stored in~/.config/distbackup/. If you
want to use a different path, you can set theDISTBACKUP_PATHenvironment variable or
override it with-d. The first time you run distbackup, it will create the database
automatically.First, you need to decide what files you want to back up. You can back up a single folder
or multiple folders spread out across different drives. You can "mount" a folder as a
specific path under the virtual tree with thedsb source mapcommand:# Make /media/photos/DCIM appear as /photos in the backup set
dsb source map /photos /media/photos/DCIM

# Another path, from your home folder
dsb source map /videos ~/Videos/recorded/

# You can "mount" directories within other virtual directories.
dsb source map /videos/stream-archive ~/livestream-archiveOnce you have your source map set up the way you want it, rundsb updateto scan the
source folders for files and record their metadata in the database. The first time you run
it, it will have to read every file to generate a hash. You can let this process run in
the background while you continue setup. If the hashing process is interrupted, it will
pick up where it left off the next time you rundsb updateand won't have to rehash any
files unless their contents have changed.Next, you need to find some disks to use as a backup. Each disk needs to have a unique
name, even if it's justdistbackup-01,distbackup-02, etc.  I highly recommend
physically labelling each disk with its name so it's easy to find. I also recommend
setting the volume label on the disk as well, though distbackup does not require that.Note: If your disks are formatted as ext4, you should set the "reserved blocks" to
zero. By default, ext4 reserves 5% for the root user, which for a 6TB disk is 300GB, an
insane amount for a data disk. You can usetune2fs -m 0to clear it.Once you have your disk connected, formatted, and mounted, it's time to make distbackup
aware of it:ktpanda@desktop:~$ dsb disk add distbackup-01 /media/distbackup-01/distbackup/
Added disk distbackup-01:
  UUID: db74b831-bd09-434f-ac9b-bc427dfc5628
  Nexus index: 0
  Size: 7,927,384,932,352
  Mount point (current): /media/distbackup-01
  Relative path from mountpoint: distbackupThe size, if not specified, defaults to 10GB less than the total size of the disk. If you
want to set a specific size (e.g. to reserve more space for other files), you can specify--sizewhen adding the disk or with thedsb disk setcommand:dsb disk set distbackup-01 --size 7.4TOnce you have all your disks set up, anddsb updatecompletes, you're ready to start
backing up! Just rundsb backup distbackup-01, and it will start copying files until it
hits the size limit of the disk, or it runs out of files to copy.