Repository: borgbackup/borg

Issue:
Order of options and paths can cause errors when going from 1.0 to 1.1
After upgrading from 1.0.x to 1.1.0 our backup scripts fail with an error. The issue seems to be the order of options and paths to backup. This does not work:

`borg create '::{now:%Y-%m-%d}' -v --stats --compression lz4 --one-file-system --exclude /boot --exclude '/home/*/.cache' --exclude /root/.cache --exclude 're:^/usr/(?!local($|/))' --exclude 're:^/usr/local/(?!etc($|/))' --exclude /tmp --exclude /var/cache --exclude /var/tmp /home 
`

but moving the paths before all options does:

`borg create '::{now:%Y-%m-%d}' /home -v --stats --compression lz4 --one-file-system --exclude /boot --exclude '/home/*/.cache' --exclude /root/.cache --exclude 're:^/usr/(?!local($|/))' --exclude 're:^/usr/local/(?!etc($|/))' --exclude /tmp --exclude /var/cache --exclude /var/tmp`

The command line parsing should probably be fixed that the old order (which is also what `borg -h` shows) works again.


Discussion:
Which borg packages are you using? Is it the same python version?
RHEL/CentOS 7 python, borgbackup package from EPEL:

python34-3.4.5-4.el7.x86_64
borgbackup-1.1.0-1.el7.x86_64

We see it on three systems, after the upgrade from the previous 1.0.x EPEL package the command stops working.
@suuchan maybe don't use 1.1.0, it had some bugs and is superceded by 1.1.1 and 1.1.2 since a while.
https://centos.pkgs.org/7/epel-testing-x86_64/borgbackup-1.1.2-1.el7.x86_64.rpm.html
The question is whether you changed your python 3.x version while doing the upgrade from borg 1.0 to 1.1.
We also tried 1.1.2 on Ubuntu 17.10 using the package from 18.04. And Fedora 27 with 1.1.2.

Both have the same problem too.
A while ago, a similar (or same) problem has been seen already.

Back then we thought it is a bug in `argparse` stdlib module that's present in some python versions.

Simplifying it a bit:
```
$ borg create repo::test -v --stats scripts/
usage: borg [-V] [-h] [--critical] [--error] [--warning] [--info] [--debug]
            [--debug-topic TOPIC] [-p] [--log-json] [--lock-wait SECONDS]
            [--show-version] [--show-rc] [--umask M] [--remote-path PATH]
            [--remote-ratelimit RATE] [--consider-part-files]
            [--debug-profile FILE]
            <command> ...
borg: error: unrecognized arguments: scripts/
```

```
$ borg create -v --stats repo::test scripts/
(works)
```

So, it does not like options (like `-v --stats`) being put in between the positional args (like `repo::test` and `scripts/`). Seen it failing on Python 3.5.2 with latest 1.1-maint branch.

Please note that the docs tell you to use it in the working way:
```
usage: borg create [-h] [--critical] [--error] [--warning] [--info] [--debug]
                   [--debug-topic TOPIC] [-p] [--log-json]
                   ...
                   [--chunker-params PARAMS] [-C COMPRESSION]
                   ARCHIVE [PATH [PATH ...]]
```
You see there is nothing in between ARCHIVE and PATH.

Guess we have to check/fix the docs whether we show some other (not always working) order anywhere.
Another way to resolve it would be to try to find out whether there is some change in our argparse usage in 1.1 that triggers this and that can be solved otherwise.
https://stackoverflow.com/questions/33404227/python-argparse-optional-argument-only-works-when-its-entered-in-the-right-posi

http://bugs.python.org/issue15112
1.0: borg create uses `nargs='+'` for paths (but borg extract uses `nargs='*'`).

1.1: borg create uses `nargs='*'` for paths.
this is likely because `--pattern / --patterns-from` can also be used to give root paths, so the paths positional argument(s) became optional.

So, guess we have to live with it.

As the python bug is open since 2012, there's also no quick solution to be expected from there (and it would not help anyway if people use old pythons).

So, TODO:
- check the docs if we ever mention the bad order.
- mention it in compatibility section

Resolving patch:
diff --git a/src/borg/archiver.py b/src/borg/archiver.py
--- a/src/borg/archiver.py
+++ b/src/borg/archiver.py
@@ -2952,12 +2952,6 @@ def define_archive_filters_group(subparser, *, sort_by=True, first_last=True):
           it had before a content change happened. This can be used maliciously as well as
           well-meant, but in both cases mtime based cache modes can be problematic.
 
-        By default, borg tries to archive all metadata that it supports archiving.
-        If that is not what you want or need, there are some tuning options:
-
-        - --nobsdflags (getting bsdflags has a speed penalty under Linux)
-        - --noatime (if atime changes frequently, the metadata stream will dedup badly)
-
         The mount points of filesystems or filesystem snapshots should be the same for every
         creation of a new archive to ensure fast operation. This is because the file cache that
         is used to determine changed files quickly uses absolute filenames.


Test patch: