Repository: borgbackup/borg

Issue:
Store empty mountpoint-directories in archives when staying on single fs
When using borg -x currently the archive would not contain the mointpoints like /proc, /sys and whatever else.
This becomes an inconvenience later on when you want to do a full-system-restore since you need to remember to recreate all of them too.

I think it would be a lot better to store mountpoints as empty directories in the archive instead.

Similarly --exclude might need something like that if the target happens to be a directory, but I am less certain about that and there apparently might be workarounds for that too like specifying slashes at the end of the excluded names.



Discussion:
I usually do "mount --bind / /rootfs", so I do not have to exclude /proc, /sys, /dev. It will backup the directories then and it will even backup the on-disk files in /dev that are usually hidden by the mounted devfs.

This only mostly works when you do not have a gazillion of mountpoints you do want to backup.
Also in your case all your paths start with /rootfs/ tooo, don't they? so whole system restore is a lot less convenient as the result too.

It is trues that stuff hidden by mountpoints is not visible in normal case, but I think that's ok.

> Also in your case all your paths start with /rootfs/ tooo, don't they?

Yes. I have a similar issue with my sshfs backups; I thought about adding --strip-components to borg create, just for convenience.

@enkore there is strip-components for extract, that should be sufficient. for create, that feature might make quite some stuff more complicated (like known-files cache, esp. if the stripped levels change. also, we usually always have the full path in the archive, but not any more if you strip).

True, that's why there is no code for it :)

I'd really like to have strip-components on create, as my backup wrapper creates dated snapshots and mounts them in a dated directory for reasons outside of the borg portion of the backup. But that means that borg spends a lot of time chunking files that haven't changed, but it doesn't know because the files have a different path every day.

The problems you've mentioned explicitly, like different runs with different levels of strip-components, seem like things that could be documented as problematic.

I've been considering working on strip-components on create after I finish the implicit output PR, so if there are major problems that would make a PR adding that feature difficult or impossible to accept I'd really like to know.

A very fundamental problem with storing the mountpoint-directories as that we can not really access them while a filesystem is mounted there: what we see (e.g. in stat() results) is not the mountpoint directory, but the root directory of the mounted filesystem.

So, we can either ignore the difference and just archive that or close this issue as not fixable.

In the usual case the difference stat(mountpoint) vs stat(fs-root) doesn't really matter - root/the system will mount whatever goes there in the restored system.

OK, so we just ignore that.

Related: #1339


Resolving patch:
diff --git a/src/borg/archiver.py b/src/borg/archiver.py
--- a/src/borg/archiver.py
+++ b/src/borg/archiver.py
@@ -324,9 +324,10 @@ def _process(self, archive, cache, matcher, exclude_caches, exclude_if_present,
                 return
         if (st.st_ino, st.st_dev) in skip_inodes:
             return
-        # Entering a new filesystem?
-        if restrict_dev is not None and st.st_dev != restrict_dev:
-            return
+        # if restrict_dev is given, we do not want to recurse into a new filesystem,
+        # but we WILL save the mountpoint directory (or more precise: the root
+        # directory of the mounted filesystem that shadows the mountpoint dir).
+        recurse = restrict_dev is None or st.st_dev == restrict_dev
         status = None
         # Ignore if nodump flag is set
         try:
@@ -344,28 +345,30 @@ def _process(self, archive, cache, matcher, exclude_caches, exclude_if_present,
                     status = 'E'
                     self.print_warning('%s: %s', path, e)
         elif stat.S_ISDIR(st.st_mode):
-            tag_paths = dir_is_tagged(path, exclude_caches, exclude_if_present)
-            if tag_paths:
-                if keep_tag_files and not dry_run:
-                    archive.process_dir(path, st)
-                    for tag_path in tag_paths:
-                        self._process(archive, cache, matcher, exclude_caches, exclude_if_present,
-                                      keep_tag_files, skip_inodes, tag_path, restrict_dev,
-                                      read_special=read_special, dry_run=dry_run)
-                return
+            if recurse:
+                tag_paths = dir_is_tagged(path, exclude_caches, exclude_if_present)
+                if tag_paths:
+                    if keep_tag_files and not dry_run:
+                        archive.process_dir(path, st)
+                        for tag_path in tag_paths:
+                            self._process(archive, cache, matcher, exclude_caches, exclude_if_present,
+                                          keep_tag_files, skip_inodes, tag_path, restrict_dev,
+                                          read_special=read_special, dry_run=dry_run)
+                    return
             if not dry_run:
                 status = archive.process_dir(path, st)
-            try:
-                entries = helpers.scandir_inorder(path)
-            except OSError as e:
-                status = 'E'
-                self.print_warning('%s: %s', path, e)
-            else:
-                for dirent in entries:
-                    normpath = os.path.normpath(dirent.path)
-                    self._process(archive, cache, matcher, exclude_caches, exclude_if_present,
-                                  keep_tag_files, skip_inodes, normpath, restrict_dev,
-                                  read_special=read_special, dry_run=dry_run)
+            if recurse:
+                try:
+                    entries = helpers.scandir_inorder(path)
+                except OSError as e:
+                    status = 'E'
+                    self.print_warning('%s: %s', path, e)
+                else:
+                    for dirent in entries:
+                        normpath = os.path.normpath(dirent.path)
+                        self._process(archive, cache, matcher, exclude_caches, exclude_if_present,
+                                      keep_tag_files, skip_inodes, normpath, restrict_dev,
+                                      read_special=read_special, dry_run=dry_run)
         elif stat.S_ISLNK(st.st_mode):
             if not dry_run:
                 if not read_special:


Test patch: