Repository: borgbackup/borg

Issue:
LockTimeout: better error message
When I have a locking problem, I would like to have a "normal" error message like:

Error: The repository is locked, please try again later, or use the option --lock-wait.

Instead of the horrible stack trace:

```
Remote: Borg 1.0.6: exception in RPC call:
Remote: Traceback (most recent call last):
Remote:   File "/home/xyz/borg-env/lib/python3.4/site-packages/borg/locking.py", line 136, in acquire
Remote:     os.mkdir(self.path)
Remote: FileExistsError: [Errno 17] File exists: '/home/xyz/repos/host/trunk/lock.exclusive'
Remote:
Remote: During handling of the above exception, another exception occurred:
Remote:
Remote: Traceback (most recent call last):
Remote:   File "/home/xyz/borg-env/lib/python3.4/site-packages/borg/remote.py", line 97, in serve
Remote:     res = f(*args)
Remote:   File "/home/xyz/borg-env/lib/python3.4/site-packages/borg/remote.py", line 128, in open
Remote:     self.repository.__enter__()  # clean exit handled by serve() method
Remote:   File "/home/xyz/borg-env/lib/python3.4/site-packages/borg/repository.py", line 82, in __enter__
Remote:     self.open(self.path, self.exclusive, lock_wait=self.lock_wait, lock=self.do_lock)
Remote:   File "/home/xyz/borg-env/lib/python3.4/site-packages/borg/repository.py", line 171, in open
Remote:     self.lock = UpgradableLock(os.path.join(path, 'lock'), exclusive, timeout=lock_wait).acquire()
Remote:   File "/home/xyz/borg-env/lib/python3.4/site-packages/borg/locking.py", line 265, in acquire
Remote:     with self._lock:
Remote:   File "/home/xyz/borg-env/lib/python3.4/site-packages/borg/locking.py", line 120, in __enter__
Remote:     return self.acquire()
Remote:   File "/home/xyz/borg-env/lib/python3.4/site-packages/borg/locking.py", line 141, in acquire
Remote:     raise LockTimeout(self.path)
Remote: borg.locking.LockTimeout: /home/xyz/repos/host/trunk/lock.exclusive
Remote: Platform: Linux xyz 3.13.0-76-generic #120-Ubuntu SMP Mon Jan 18 15:59:10 UTC 2016 x86_64 x86_64
Remote: Linux: Ubuntu 14.04 trusty
Remote: Borg: 1.0.6  Python: CPython 3.4.3
Remote: PID: 37645  CWD: /home/xyz/repos/host
Remote: sys.argv: ['/home/xyz/bin/borg', 'serve', '--restrict-to-path', '/home/xyz/repos/host']
Remote: SSH_ORIGINAL_COMMAND: 'borg serve --umask=077'
Remote:
Remote Exception (see remote log for the traceback)
Platform: Linux host 3.2.0-4-amd64 #1 SMP Debian 3.2.81-1 x86_64
Linux: debian 7.11
Borg: 1.0.6  Python: CPython 3.5.2
PID: 45908  CWD: /root/bin
sys.argv: ['borg', 'mount', 'xyz@backup.example.com:trunk::20160804', '/mnt']
SSH_ORIGINAL_COMMAND: None
```



Discussion:
For expected cases of "this is locked", I agree, it should be prettier.
The problem might be if there is some unexpected case or bug, then we might need more information and if we suppress the stacktrace, we don't get it.

Yep, I think there must be a sweet spot to find, where a "this is locked" is the message and otherwise you get the stacktrace.

Hmm, maybe we could emit the traceback at DEBUG level for cases that are somehow expected to happen often under normal conditions.

So if something weird comes up, we can still give `--debug` and get the trace, if the problem can be reproduced.

For local repos with current borg 1.0.7pre, it is already cleaned up:

```
(borg-env)tw@tux:~/w/borg$ borg create repo::3 borg
Failed to create/acquire the lock /home/tw/w/borg/repo/lock (timeout).
```

The traceback still happens for remote repos though.
Guess we can improve that in 1.1 with the better remote logging system.

Yes, in 1.0.x the log level information isn't transported; moving the messages to a different log level would not help there. For 1.1 this should be relatively simple.


Resolving patch:
diff --git a/src/borg/archiver.py b/src/borg/archiver.py
--- a/src/borg/archiver.py
+++ b/src/borg/archiver.py
@@ -2354,37 +2354,51 @@ def main():  # pragma: no cover
     sys.stderr = ErrorIgnoringTextIOWrapper(sys.stderr.buffer, sys.stderr.encoding, 'replace', line_buffering=True)
     setup_signal_handlers()
     archiver = Archiver()
-    msg = None
+    msg = tb = None
+    tb_log_level = logging.ERROR
     try:
         args = archiver.get_args(sys.argv, os.environ.get('SSH_ORIGINAL_COMMAND'))
     except Error as e:
         msg = e.get_message()
-        if e.traceback:
-            msg += "\n%s\n%s" % (traceback.format_exc(), sysinfo())
+        tb_log_level = logging.ERROR if e.traceback else logging.DEBUG
+        tb = '%s\n%s' % (traceback.format_exc(), sysinfo())
         # we might not have logging setup yet, so get out quickly
         print(msg, file=sys.stderr)
+        if tb_log_level == logging.ERROR:
+            print(tb, file=sys.stderr)
         sys.exit(e.exit_code)
     try:
         exit_code = archiver.run(args)
     except Error as e:
         msg = e.get_message()
-        if e.traceback:
-            msg += "\n%s\n%s" % (traceback.format_exc(), sysinfo())
+        tb_log_level = logging.ERROR if e.traceback else logging.DEBUG
+        tb = "%s\n%s" % (traceback.format_exc(), sysinfo())
         exit_code = e.exit_code
     except RemoteRepository.RPCError as e:
-        msg = '%s\n%s' % (str(e), sysinfo())
+        msg = "%s %s" % (e.remote_type, e.name)
+        important = e.remote_type not in ('LockTimeout', )
+        tb_log_level = logging.ERROR if important else logging.DEBUG
+        tb = sysinfo()
         exit_code = EXIT_ERROR
     except Exception:
-        msg = 'Local Exception.\n%s\n%s' % (traceback.format_exc(), sysinfo())
+        msg = 'Local Exception'
+        tb_log_level = logging.ERROR
+        tb = '%s\n%s' % (traceback.format_exc(), sysinfo())
         exit_code = EXIT_ERROR
     except KeyboardInterrupt:
-        msg = 'Keyboard interrupt.\n%s\n%s' % (traceback.format_exc(), sysinfo())
+        msg = 'Keyboard interrupt'
+        tb_log_level = logging.DEBUG
+        tb = '%s\n%s' % (traceback.format_exc(), sysinfo())
         exit_code = EXIT_ERROR
     except SIGTERMReceived:
-        msg = 'Received SIGTERM.'
+        msg = 'Received SIGTERM'
+        tb_log_level = logging.DEBUG
+        tb = '%s\n%s' % (traceback.format_exc(), sysinfo())
         exit_code = EXIT_ERROR
     if msg:
         logger.error(msg)
+    if tb:
+        logger.log(tb_log_level, tb)
     if args.show_rc:
         rc_logger = logging.getLogger('borg.output.show-rc')
         exit_msg = 'terminating with %s status, rc %d'
diff --git a/src/borg/remote.py b/src/borg/remote.py
--- a/src/borg/remote.py
+++ b/src/borg/remote.py
@@ -6,6 +6,7 @@
 import shlex
 import sys
 import tempfile
+import traceback
 from subprocess import Popen, PIPE
 
 import msgpack
@@ -101,12 +102,21 @@ def serve(self):
                             f = getattr(self.repository, method)
                         res = f(*args)
                     except BaseException as e:
-                        # These exceptions are reconstructed on the client end in RemoteRepository.call_many(),
-                        # and will be handled just like locally raised exceptions. Suppress the remote traceback
-                        # for these, except ErrorWithTraceback, which should always display a traceback.
-                        if not isinstance(e, (Repository.DoesNotExist, Repository.AlreadyExists, PathNotAllowed)):
-                            logging.exception('Borg %s: exception in RPC call:', __version__)
-                            logging.error(sysinfo())
+                        if isinstance(e, (Repository.DoesNotExist, Repository.AlreadyExists, PathNotAllowed)):
+                            # These exceptions are reconstructed on the client end in RemoteRepository.call_many(),
+                            # and will be handled just like locally raised exceptions. Suppress the remote traceback
+                            # for these, except ErrorWithTraceback, which should always display a traceback.
+                            pass
+                        else:
+                            if isinstance(e, Error):
+                                tb_log_level = logging.ERROR if e.traceback else logging.DEBUG
+                                msg = e.get_message()
+                            else:
+                                tb_log_level = logging.ERROR
+                                msg = '%s Exception in RPC call' % e.__class__.__name__
+                            tb = '%s\n%s' % (traceback.format_exc(), sysinfo())
+                            logging.error(msg)
+                            logging.log(tb_log_level, tb)
                         exc = "Remote Exception (see remote log for the traceback)"
                         os.write(stdout_fd, msgpack.packb((1, msgid, e.__class__.__name__, exc)))
                     else:


Test patch: