Repository: borgbackup/borg

Issue:
--read-special crashes borgbackup upon broken symlink
When creating a backup and using --read-special borgbackup crashes if there is a broken symlink in the source data.

Tested with 1.1.0-beta1 and 1.0.7.

Reproduction case:

`
user@computer:/tmp/b$ LD_LIBRARY_PATH=/foo ./borg-linux64  init test1
Enter new passphrase: 
Enter same passphrase again: 
Do you want your passphrase to be displayed for verification? [yN]: 

user@computer:/tmp/b$ mkdir source
user@computer:/tmp/b$ cd source/
user@computer:/tmp/b/source$ touch foo
user@computer:/tmp/b/source$ ln -s foo bar
user@computer:/tmp/b/source$ rm foo
user@computer:/tmp/b/source$ cd ..

user@computer:/tmp/b$ LD_LIBRARY_PATH=/foo ./borg-linux64  create test1::a source
(OK)

user@computer:/tmp/b$ LD_LIBRARY_PATH=/foo ./borg-linux64  create --read-special test1::b source
Local Exception
Traceback (most recent call last):
  File "borg/archiver.py", line 2414, in main
  File "borg/archiver.py", line 2344, in run
  File "borg/archiver.py", line 98, in wrapper
  File "borg/archiver.py", line 313, in do_create
  File "borg/archiver.py", line 286, in create_inner
  File "borg/archiver.py", line 376, in _process
  File "borg/archiver.py", line 382, in _process
FileNotFoundError: [Errno 2] No such file or directory: 'source/bar'

Platform: Linux computer 4.4.0-34-generic #53-Ubuntu SMP Wed Jul 27 16:06:39 UTC 2016 x86_64 x86_64
Linux: debian stretch/sid 
Borg: 1.1.0b1  Python: CPython 3.5.2
PID: 4634  CWD: /tmp/b
sys.argv: ['./borg-linux64', 'create', '--read-special', 'test1::b', 'source']
SSH_ORIGINAL_COMMAND: None
`



Discussion:
oh, oops. thanks for reporting this.

should be fixed like in other borg code: if there is a problem with input file, log it as warning and continue, exit after completion with rc=1.

Are you sure that a broken symlink should be considered an error at all? I can think of a few use cases where I would see it as correct having a "broken" symlink.
- Pointing to file/dir in removable storage, which isn't attached right now, or network mounts.
- The link is only valid in some cases, for example when being chrooted.
- The link actually works, but due to lack of permissions it seems broken.

I am just glad if it's fixed, but taking my chance to give my two cents. I don't really see it as Borgbackup's job description to verify my symlinks for me.

Hmm, right. I'll have to try how it behaves without --read-special. Maybe it just silently skips?

It silently copies it. It doesn't follow the symlinks normally. The documentation says the following for --read-special
_open and read block and char device files as well as FIFOs as if they were regular files. Also follows symlinks pointing to these kinds of files._
As I understand it, it follows the symlink IF it points to a block/char device or FIFO. Otherwise it should just copy the symlink? So I guess it crashes when it tries to see if it points to anything of interest.

I have verified that working/connected symlinks pointing to regular files ends up as symlinks in the Borgbackup archive even when --read-special is used. Since I wasn't 100% sure after reading the documentation I tested it.

Personally I just needed Borgbackup to read my named pipe (FIFO).

The logic bug (broken symlink would've been skipped in read special mode) is older, probably since read special; the crash is a newer regression (since 5b453856).


Resolving patch:
diff --git a/borg/archive.py b/borg/archive.py
--- a/borg/archive.py
+++ b/borg/archive.py
@@ -610,7 +610,8 @@ def process_dev(self, path, st):
             return 'b'  # block device
 
     def process_symlink(self, path, st):
-        source = os.readlink(path)
+        with backup_io():
+            source = os.readlink(path)
         item = {b'path': make_path_safe(path), b'source': source}
         item.update(self.stat_attrs(st, path))
         self.add_item(item)
diff --git a/borg/archiver.py b/borg/archiver.py
--- a/borg/archiver.py
+++ b/borg/archiver.py
@@ -306,8 +306,13 @@ def _process(self, archive, cache, matcher, exclude_caches, exclude_if_present,
                 if not read_special:
                     status = archive.process_symlink(path, st)
                 else:
-                    st_target = os.stat(path)
-                    if is_special(st_target.st_mode):
+                    try:
+                        st_target = os.stat(path)
+                    except OSError:
+                        special = False
+                    else:
+                        special = is_special(st_target.st_mode)
+                    if special:
                         status = archive.process_file(path, st_target, cache)
                     else:
                         status = archive.process_symlink(path, st)
diff --git a/borg/testsuite/archiver.py b/borg/testsuite/archiver.py
--- a/borg/testsuite/archiver.py
+++ b/borg/testsuite/archiver.py
@@ -901,6 +901,14 @@ def test_create_topical(self):
         output = self.cmd('create', '-v', '--list', '--filter=AM', self.repository_location + '::test3', 'input')
         self.assert_in('file1', output)
 
+    def test_create_read_special_broken_symlink(self):
+        os.symlink('somewhere doesnt exist', os.path.join(self.input_path, 'link'))
+        self.cmd('init', self.repository_location)
+        archive = self.repository_location + '::test'
+        self.cmd('create', '--read-special', archive, 'input')
+        output = self.cmd('list', archive)
+        assert 'input/link -> somewhere doesnt exist' in output
+
     # def test_cmdline_compatibility(self):
     #    self.create_regular_file('file1', size=1024 * 80)
     #    self.cmd('init', self.repository_location)


Test patch: