Repository: borgbackup/borg

Issue:
Borg Return Code is 2 when Killed by Signal (Expecting 128 + N)
## Summary:
Borg returns code 2 when killed by signal, but the expected return code per the documentation is 128 + N.
(https://borgbackup.readthedocs.io/en/stable/usage/general.html#return-codes)

## Have you checked borgbackup docs, FAQ, and open Github issues?
Yes.

## Is this a BUG / ISSUE report or a QUESTION?
Bug/issue report

## System information. For client/server mode post info for both machines.
Linux 5.5.0-2-amd64 #1 SMP Debian 5.5.17-1 (2020-04-15) x86_64 GNU/Linux
I am running locally to a test repo. However, the issue also occurred during production to a remote repo.

#### Your borg version (borg -V).
borg 1.1.11
using pre-built binary

#### Operating system (distribution) and version.
Debian 9 "Stretch" x64

#### Hardware / network configuration, and filesystems used.
Thinkpad T430
Intel(R) Core(TM) i5-3360M
16GB DDR3
500GB SSD
ext4

#### How much data is handled by borg?
6GB test directory

#### Full borg commandline that lead to the problem (leave away excludes and passwords)
usr/bin/borg create /home/me/borgtmp::me-thinkpad-test /home/me/downloads --show-rc

## Describe the problem you're observing.
I am expecting borg to return an exit code of 128 + N, where N equals the signal number, when killed by a signal such as SIGINT or SIGTERM. Instead, I am always receiving a return code of 2.

#### Can you reproduce the problem? If so, describe how. If not, describe troubleshooting steps you took before opening the issue.
Yes, the issue can be reproduced.
1. Be me.
2. Run borg create command
3. Wait 30 seconds
4. Send CTRL + C to the terminal, or run 'pkill -TERM borg' in a separate terminal

The expected return code is 130 for CTRL + C (128 + 2), or 143 for SIGTERM (128 + 15).
Instead, I invariably get rc = 2.

#### Include any warning/errors/backtraces from the system logs
```
me@me-thinkpad:~/scripts/borg-backup-script$ /usr/bin/borg create /home/me/borgtmp::me-thinkpad-test /home/me/downloads --show-rc
Enter passphrase for key /home/me/borgtmp: 
^CKeyboard interrupt
terminating with error status, rc 2
```

```
me@me-thinkpad:~/scripts/borg-backup-script$ /usr/bin/borg create /home/me/borgtmp::me-thinkpad-test /home/me/downloads --show-rc
Enter passphrase for key /home/me/borgtmp: 
Received SIGTERM
terminating with error status, rc 2
```

Thoughts? Thank you for reading.




Discussion:
I can confirm that:

`borg create --show-rc ::archive /some/path || echo $?`

Killing it via `kill <x>` results in output:

    Received SIGTERM
    terminating with error status, rc 2
    2

Tested with GitHub releases Borg 1.1.11 and 1.2.0a8 (both borg-linux64), both give rc 2.
IIRC I got this rc == 128+N info from some (python?) docs, that this is the rc if the process is killed by a signal.

rc 2 is borg's generic "sudden death" return code (== generated by borg). So somehow also true, but less specific.

@greenmtnit @fantasya-pbem please edit your post and add what kind of borg binary you used (dist package or pyinstaller-made fat binary from github releases or ...).
Hmm, the reason could also be that 128+N is the general rule, but IIRC we catch some signals so they get special behaviour (e.g. finishing in a as-clean-as-it-gets way).

Guess in that "catching case", the general 128+N rule might not apply.
Updated original post to reflect I am using the pre-built binary.

It would be useful to return the 128 + N here, in my opinion. I would like to differentiate a signal-based kill from a crash.
For now I will try wrapping borg in a function in the script.

Thank you both for your dedicated work on the project!
I used the GitHub fat binary releases here; post edited.
Would be interesting if it also happens with the normal linux distribution or pip package to see whether this is caused by pyinstaller wrapping.
Just tested: My `borg-env/bin/borg` version from latest master also gives rc 2.
For anyone else facing this issue, here is a work-around script. This is a little ugly but it works. I noticed that borg will print "Received SIGTERM" consistently when killed by the TERM signal.

```
#!/bin/bash
output="$(borg create server::archive /some/path --show-rc 2>&1)"

if (echo $output | grep "Received SIGTERM")
then
  echo "Killed by SIGTERM"
  # do stuff to handle SIGTERM
elif (echo $output | grep "terminating with success status")
then
  echo "Job succeeded"
else 
  echo "Error! Borg did not finish, but no SIGTERM was detected"
  # Something went majorly wrong
fi
```
For me, the use case is sending SIGTERM to kill the borg process if it runs on "too long". If borg exits for any reason other than a normal finish or my SIGTERM, I want to send a notification.

The script does not handle the rc == 1 (warning) case correctly. It does not really fit into "something went majorly wrong", but could be also rather harmless.

Also, you could handle the rc == 0 and rc == 1 states by just checking the rc and you could save the expensive echo/grep operation for these cases.
Like:
```
if rc == 0:
  # all fine
elif rc == 1:
  # warning state, check the logs manually
elif rc == 2:
  # error state or killed by some signals due to issue #5161
else:
  # killed by some other signals
```

Resolving patch:
diff --git a/src/borg/archiver.py b/src/borg/archiver.py
--- a/src/borg/archiver.py
+++ b/src/borg/archiver.py
@@ -45,7 +45,7 @@
     from .compress import CompressionSpec
     from .crypto.key import key_creator, key_argument_names, tam_required_file, tam_required, RepoKey, PassphraseKey
     from .crypto.keymanager import KeyManager
-    from .helpers import EXIT_SUCCESS, EXIT_WARNING, EXIT_ERROR
+    from .helpers import EXIT_SUCCESS, EXIT_WARNING, EXIT_ERROR, EXIT_SIGNAL_BASE
     from .helpers import Error, NoManifestError, set_ec
     from .helpers import positive_int_validator, location_validator, archivename_validator, ChunkerParams, Location
     from .helpers import PrefixSpec, GlobSpec, CommentSpec, SortBySpec, HUMAN_SORT_KEYS, FilesCacheMode
@@ -4590,17 +4590,17 @@ def main():  # pragma: no cover
             msg = 'Keyboard interrupt'
             tb_log_level = logging.DEBUG
             tb = '%s\n%s' % (traceback.format_exc(), sysinfo())
-            exit_code = EXIT_ERROR
+            exit_code = EXIT_SIGNAL_BASE + 2
         except SigTerm:
             msg = 'Received SIGTERM'
             msgid = 'Signal.SIGTERM'
             tb_log_level = logging.DEBUG
             tb = '%s\n%s' % (traceback.format_exc(), sysinfo())
-            exit_code = EXIT_ERROR
+            exit_code = EXIT_SIGNAL_BASE + 15
         except SigHup:
             msg = 'Received SIGHUP.'
             msgid = 'Signal.SIGHUP'
-            exit_code = EXIT_ERROR
+            exit_code = EXIT_SIGNAL_BASE + 1
         if msg:
             logger.error(msg, msgid=msgid)
         if tb:
@@ -4614,6 +4614,8 @@ def main():  # pragma: no cover
                 rc_logger.warning(exit_msg % ('warning', exit_code))
             elif exit_code == EXIT_ERROR:
                 rc_logger.error(exit_msg % ('error', exit_code))
+            elif exit_code >= EXIT_SIGNAL_BASE:
+                rc_logger.error(exit_msg % ('signal', exit_code))
             else:
                 rc_logger.error(exit_msg % ('abnormal', exit_code or 666))
         sys.exit(exit_code)
diff --git a/src/borg/constants.py b/src/borg/constants.py
--- a/src/borg/constants.py
+++ b/src/borg/constants.py
@@ -76,6 +76,7 @@
 EXIT_SUCCESS = 0  # everything done, no problems
 EXIT_WARNING = 1  # reached normal end of operation, but there were issues
 EXIT_ERROR = 2  # terminated abruptly, did not reach end of operation
+EXIT_SIGNAL_BASE = 128  # terminated due to signal, rc = 128 + sig_no
 
 # never use datetime.isoformat(), it is evil. always use one of these:
 # datetime.strftime(ISO_FORMAT)  # output always includes .microseconds


Test patch: