Repository: borgbackup/borg

Issue:
giving some options repeatedly does not give an error msg
## Have you checked borgbackup docs, FAQ, and open Github issues?

Yes

## Is this a BUG / ISSUE report or a QUESTION?

BUG

## System information. For client/server mode post info for both machines.
Local repo

#### Your borg version (borg -V).
borg 1.1.15

#### Operating system (distribution) and version.
Ubuntu Server 20.04

#### Hardware / network configuration, and filesystems used.
LizardFS

#### How much data is handled by borg?
```
------------------------------------------------------------------------------
                       Original size      Compressed size    Deduplicated size
All archives:               49.20 TB             26.39 TB              4.73 TB

                       Unique chunks         Total chunks
Chunk index:                11835460            117566592
```

#### Full borg commandline that lead to the problem (leave away excludes and passwords)
See below.

## Describe the problem you're observing.
I have a large archive that I wanted to do some cleaning on, so I was building a list of globs to remove what I wanted.  When I did a dry run it only reported two archive...  Thought that was odd so I switched to prefix as that covered 90% of what I wanted to remove.  I ended up with the same result.  After some trial and error, I found that after adding a third prifix filter option, it stopped processing past the first one (not always in that order).

Examples:
```
root@mediaserver:/$$$$$$/Backups# borg delete --dry -vsp -P "OS-TheMonolith-2019*" ./Borg
Ignoring --stats. It is not supported when using --dry-run.
Enter passphrase for key /LizardFS/Files/MediaStore/Backups/Borg: 
Would delete archive: OS-TheMonolith-2019-07-04_16-04      Thu, 2019-07-04 16:04:46 [e096112de753322d50bc438f70c93f69d0b415bd44096dc551624325a0be3f07] (1/10)
Would delete archive: OS-TheMonolith-2019-07-18_21-48      Thu, 2019-07-18 21:48:18 [198604fdca61a85be3157e886c5c80bcb0d73f0fe2f59467f3672e1734872717] (2/10)
Would delete archive: OS-TheMonolith-2019-08-07_13-50      Wed, 2019-08-07 13:50:11 [1a3ce6315325535c3249b42076623c6b377234a21df8385715b07eff107e7bff] (3/10)
Would delete archive: OS-TheMonolith-2019-08-10_12-01      Sat, 2019-08-10 12:01:11 [287541f3f5fde91dabfd31292f4b14d9071f247286db3bd5b29a9b0f46e581c6] (4/10)
Would delete archive: OS-TheMonolith-2019-09-02_14-19      Mon, 2019-09-02 14:21:53 [3ca76805f37df815508b94eddd913632e2a4b48dcb06838a552b924facb35886] (5/10)
Would delete archive: OS-TheMonolith-2019-09-03_14-41      Tue, 2019-09-03 14:41:20 [074bcc84a2cd2e177a4bf8795fe90cad861dac3b9b136306b013a62142dcc00a] (6/10)
Would delete archive: OS-TheMonolith-2019-09-11_18-33      Wed, 2019-09-11 18:41:46 [bed89fe7bc9c44d7039e02701f093486d85439a1e4d106443ccbc59b0af94860] (7/10)
Would delete archive: OS-TheMonolith-2019-09-28_15-32      Sat, 2019-09-28 15:32:09 [3507da2ad95973680c80189f809da31001d0b43b75fc899ef13ed81905b127c7] (8/10)
Would delete archive: OS-TheMonolith-2019-10-25_13-02      Fri, 2019-10-25 13:02:50 [471f4353c05846e2fc1c8cd4a9730d8dbc04e231bbdc650f48e64e7c5a97cffe] (9/10)
Would delete archive: OS-TheMonolith-2019-12-06_21-54      Fri, 2019-12-06 21:55:22 [67c591e5eabdfdc53ad9ded73159e964483006fb80d0be8b1960a20657b20a45] (10/10)

-----

root@mediaserver:/$$$$$$/Backups# borg delete --dry -vsp -P "OS-TheMonolith-2019*" -P "OS-TheMonolith-2020*" ./Borg
Ignoring --stats. It is not supported when using --dry-run.
Would delete archive: OS-TheMonolith-2020-02-09_19-02      Sun, 2020-02-09 19:03:02 [e9ec520302816ae2e36c87071e8c2ab7331201eafd65150f127690761303b4f0] (1/22)
Would delete archive: OS-TheMonolith-2020-03-15_18-33      Sun, 2020-03-15 18:33:47 [4d831a285d684872aeccc4ce8361636d17a832103aeb7ac0710d23b5175b3278] (2/22)
Would delete archive: OS-TheMonolith-2020-03-24_16-32      Tue, 2020-03-24 21:31:35 [511d1e9cda41927c2ea90a7b169707ff674dbf78df74f576f48a948143f954c1] (3/22)
Would delete archive: OS-TheMonolith-2020-04-01_07-25      Wed, 2020-04-01 07:25:13 [62c51645d1260c34616915a2bea98a0bcf8e3357f102b02038ab4b06414beaa4] (4/22)
Would delete archive: OS-TheMonolith-2020-04-01_07-35      Wed, 2020-04-01 07:35:17 [1ccf71c8ca68026f67792f7ba35625b904eabfd6a9ec33a509db6cf788235217] (5/22)
Would delete archive: OS-TheMonolith-2020-04-02_11-09      Thu, 2020-04-02 11:09:24 [4a981c566695f06bb2900b7eda3afc0b7d6daf25e5b1c6142c08bb104fccd9e4] (6/22)
Would delete archive: OS-TheMonolith-2020-04-02_12-58      Thu, 2020-04-02 12:58:42 [608ae5d2d9f8729339c4c0ce11cecf041ab7c38080bf3ccfe06db7958832d1b2] (7/22)
Would delete archive: OS-TheMonolith-2020-04-19_20-23      Sun, 2020-04-19 20:23:52 [c8971a2b5c9b1975ce0f95637eb797923a776a678db38cc3dc830799ad5bb961] (8/22)
Would delete archive: OS-TheMonolith-2020-05-05_07-54      Tue, 2020-05-05 07:54:28 [dac0ab07a7a4571dbb2d33c5a037b1b6b0011ae97ebee5813923387bb0fb5af6] (9/22)
Would delete archive: OS-TheMonolith-2020-05-08_11-27      Fri, 2020-05-08 12:07:09 [3fec23cdb746f932fd4da4ce5ac61312876abc3412fcf1ded2ac4a33113d03ef] (10/22)
Would delete archive: OS-TheMonolith-2020-05-08_12-38      Fri, 2020-05-08 12:38:25 [20c91d812766a5de94484c6efcf4a960f11bcfa2079e0d0d93e07ccdd3cb8d55] (11/22)
Would delete archive: OS-TheMonolith-2020-05-19_07-42      Tue, 2020-05-19 07:43:11 [23190deeb2bd632840d6d05e2e0c467292b461f9c00e9074385ace391f36add2] (12/22)
Would delete archive: OS-TheMonolith-2020-05-31_11-09      Sun, 2020-05-31 11:09:36 [6a71eda23196d14fe940dc2996f4b91a9b9a89f73756b5d7b116d9c4055bbf01] (13/22)
Would delete archive: OS-TheMonolith-2020-06-13_11-03      Sat, 2020-06-13 11:03:21 [9fb28a51d295042caa9ee8f7e5df5e8a50636f54093a26d78b0a1a8772cfd092] (14/22)
Would delete archive: OS-TheMonolith-2020-08-12_16-02      Wed, 2020-08-12 16:02:46 [9c2b538b9d90f3dba338a8cecaaf26ff357e762e33169c19111f1685d4b1cbcf] (15/22)
Would delete archive: OS-TheMonolith-2020-09-02_14-36      Wed, 2020-09-02 14:36:42 [e9b3bbe27312996b39a553e276de957814eb5a23afc7a4973faa4e61b2191b4a] (16/22)
Would delete archive: OS-TheMonolith-2020-09-18_12-35      Fri, 2020-09-18 12:35:59 [e74ab109d0a5c1aa7da67e25fd963246c39a13879cbd4ece20d531ca80a78ce6] (17/22)
Would delete archive: OS-TheMonolith-2020-10-02_16-22      Fri, 2020-10-02 16:22:19 [195ced5b99510eb854451c53c47301e5af5c33468a576bf6eefedbe8ae162c2f] (18/22)
Would delete archive: OS-TheMonolith-2020-11-03_11-45      Tue, 2020-11-03 11:45:28 [fe40dc6f86b79a12a5cb91bb6640b562d5e9dd3d11dc0e139aad32916fd708af] (19/22)
Would delete archive: OS-TheMonolith-2020-11-23_16-19      Mon, 2020-11-23 16:19:44 [fec70a0d715daae95b55cf57c62350f2eb5ed3db70beb15a0b7a8d1b0ad9b632] (20/22)
Would delete archive: OS-TheMonolith-2020-12-03_09-39      Thu, 2020-12-03 09:57:33 [ab0c80627e528f78882436a6da8f4be143871a6a73581007d2af30a2d0cd165a] (21/22)
Would delete archive: OS-TheMonolith-2020-12-16_12-05      Wed, 2020-12-16 12:07:47 [de0e0f91a6e01410cda9233ac33f705349e2e1b478df0b43c8db9270890ff620] (22/22)

-----

root@mediaserver:/$$$$$$/Backups# borg delete --dry -vsp -P "OS-TheMonolith-2019*" -P "OS-TheMonolith-2020*" -P "OS-MediaServer-2019-*" ./Borg
Ignoring --stats. It is not supported when using --dry-run.
Would delete archive: OS-MediaServer-2019-05-16_09-20      Thu, 2019-05-16 09:20:28 [d107b311ec03d6b5341b176500fd6a6fbc684c525f45987cd0809565173beee4] (1/10)
Would delete archive: OS-MediaServer-2019-05-22_05-53      Wed, 2019-05-22 05:53:54 [fa72a86dd2942e34883274345aab56370fcf3ebd700e526d7e1ba41d9f8cbdb9] (2/10)
Would delete archive: OS-MediaServer-2019-07-18_20-32      Thu, 2019-07-18 20:32:24 [afac386126dad5808c2d4f465aa9fc47f33edb0602e5fc4996c14e5a49c54371] (3/10)
Would delete archive: OS-MediaServer-2019-08-08_18-55      Thu, 2019-08-08 18:55:21 [e4b63d511fee5415ef1f70de64bf6e3d22f7fca8fdfc045dc77330879ede5949] (4/10)
Would delete archive: OS-MediaServer-2019-10-20_22-18      Sun, 2019-10-20 22:18:33 [a9cb807eeac1eb99703e8116a78c248c3d933d2551cff1a4cb65f252b08bf0bb] (5/10)
Would delete archive: OS-MediaServer-2019-10-21_12-25      Mon, 2019-10-21 12:26:50 [2610d7bad819e9e4bb9f44a28f3eaa851ed8a895be807166c078f8b34f161aa8] (6/10)
Would delete archive: OS-MediaServer-2019-11-09_19-16      Sat, 2019-11-09 19:17:05 [2a3b6fa2e2310a2d17d8752c4c68552bb7fa354e2167fe500e42126ca7abe408] (7/10)
Would delete archive: OS-MediaServer-2019-11-22_16-23      Fri, 2019-11-22 16:23:53 [6eb8e6bef73cb030fda25837ae77048b1334278335e3bf0306729ca67258bd7c] (8/10)
Would delete archive: OS-MediaServer-2019-11-23_17-31      Sat, 2019-11-23 17:31:56 [0c58b0098530b1c8e2ddecf4cbba2e65a8266b4ed37757dd24dee7cb637d44d9] (9/10)
Would delete archive: OS-MediaServer-2019-12-13_20-11      Fri, 2019-12-13 20:18:46 [151e0b689f4f9ddbfc4519c8a5cb0f005daa3eb7a3671b2669931ee01b21feb3] (10/10)

```

#### Can you reproduce the problem? If so, describe how. If not, describe troubleshooting steps you took before opening the issue.
Yes, see above.

#### Include any warning/errors/backtraces from the system logs
N/A


I am holding off on doing the delete to do any testing your would like.


Discussion:
There is no support for giving `--prefix` (short: `-P`) multiple times.

Borg just overwrites the value each time you give it and so it effectively only uses the last one given.

Same is true for `--glob-archives`(short: `-a`).
I thought that at first as well, so I was surprised when using it twice did work, but not three times...  

Should this support be improved (as I THINK this would make this time of operation MUCH faster, even if it is an edge case), or should some sort of lockout/error when attempted put in place? 

Should the documentation be updated to state that?
Why do you think using it twice worked? I only see it always used the last -P option.

For many options, you can only give them once, so guess that should be the default assumption if nothing else is documented.

Maybe the arg parsing (see `archiver.py`) could be improved to actually give an error msg if options are used incorrectly.

In the three examples I gave, in the first one with a single pass, it gives
about a dozen results, in the second with two it gives about two dozen
results and is accurate.

In the third example I am passing it three times and we are back down to
about a dozen results seemingly overwriting the first two.

Having multiple filters usually either works fine, overloads with an error
or only takes the first or last entry.  When I first tested it I only used
two, so assumed it would work fine with more.

On Sat, Oct 30, 2021, 10:57 AM TW ***@***.***> wrote:

> Why do you think using it twice worked? I only see it always used the last
> -P option.
>
> For many options, you can only give them once, so guess that should be the
> default assumption if nothing else is documented.
>
> Maybe the arg parsing (see archiver.py) could be improved to actually
> give an error msg if options are used incorrectly.
>
> —
> You are receiving this because you authored the thread.
> Reply to this email directly, view it on GitHub
> <https://github.com/borgbackup/borg/issues/6026#issuecomment-955570087>,
> or unsubscribe
> <https://github.com/notifications/unsubscribe-auth/AA6FDTUDRKNHEAPJXPDF6J3UJQ53PANCNFSM5HAJMSXA>
> .
>

about arg parsing: `action` and `nargs` influences this behaviour:
- https://docs.python.org/3/library/argparse.html#action default action is `store` which just stores 1 value.
- https://docs.python.org/3/library/argparse.html#nargs default is a single argument if action does not imply otherwise

@BloodBlight for every example you have given, it has always only used the last -P option value and ignored all other -P values (if any).

Related: https://stackoverflow.com/questions/18544468/how-to-add-optional-or-once-arguments
OK, so I think this is a bug.

The bug is **not** that it does not work as the user expected it (giving these options multiple times is just not implemented), but that it does not reject the option being given multiple times and does not give an appropriate error message.

Likely, this is the case for a lot of options, not just the ones mentioned above.

> for every example you have given, it has always only used the last -P option value and ignored all other -P values (if any).

Oh, yep, you are right!  My bad!  I will re-check, but I think I was just miss-reading.
Ya, can't reproduce.  My eyes are failing me.  Sorry about that @ThomasWaldmann
No problem, at least we found a usability issue!
Working on the stackoverflow suggested solution:

```
import argparse

class Highlander(argparse.Action):
    def __call__(self, parser, namespace, values, option_string=None):
        if getattr(namespace, self.dest, None) != self.default:
            raise argparse.ArgumentError(self, 'There can be only one.')
        setattr(namespace, self.dest, values)

parser = argparse.ArgumentParser()
parser.add_argument('-f', action=Highlander)
print (parser.parse_args('-f 1 -f 2'.split()))
```


Resolving patch:
diff --git a/src/borg/archiver.py b/src/borg/archiver.py
--- a/src/borg/archiver.py
+++ b/src/borg/archiver.py
@@ -218,6 +218,14 @@ def get_func(args):
     raise Exception('expected func attributes not found')
 
 
+class Highlander(argparse.Action):
+    """make sure some option is only given once"""
+    def __call__(self, parser, namespace, values, option_string=None):
+        if getattr(namespace, self.dest, None) != self.default:
+            raise argparse.ArgumentError(self, 'There can be only one.')
+        setattr(namespace, self.dest, values)
+
+
 class Archiver:
 
     def __init__(self, lock_wait=None, prog=None):
@@ -2841,10 +2849,10 @@ def define_archive_filters_group(subparser, *, sort_by=True, first_last=True):
             filters_group = subparser.add_argument_group('Archive filters',
                                                          'Archive filters can be applied to repository targets.')
             group = filters_group.add_mutually_exclusive_group()
-            group.add_argument('-P', '--prefix', metavar='PREFIX', dest='prefix', type=PrefixSpec, default=None,
+            group.add_argument('-P', '--prefix', metavar='PREFIX', dest='prefix', type=PrefixSpec, action=Highlander,
                                help='only consider archive names starting with this prefix.')
             group.add_argument('-a', '--glob-archives', metavar='GLOB', dest='glob_archives',
-                               type=GlobSpec, default=None,
+                               type=GlobSpec, action=Highlander,
                                help='only consider archive names matching the glob. '
                                     'sh: rules apply, see "borg help patterns". '
                                     '``--prefix`` and ``--glob-archives`` are mutually exclusive.')
@@ -2874,7 +2882,7 @@ def define_borg_mount(parser):
             parser.add_argument('-f', '--foreground', dest='foreground',
                                 action='store_true',
                                 help='stay in foreground, do not daemonize')
-            parser.add_argument('-o', dest='options', type=str,
+            parser.add_argument('-o', dest='options', type=str, action=Highlander,
                                 help='Extra mount options')
             parser.add_argument('--numeric-owner', dest='numeric_ids', action='store_true',
                                   help='deprecated, use ``--numeric-ids`` instead')
@@ -3423,7 +3431,7 @@ def define_borg_mount(parser):
 
         subparser.add_argument('--list', dest='output_list', action='store_true',
                                help='output verbose list of items (files, dirs, ...)')
-        subparser.add_argument('--filter', metavar='STATUSCHARS', dest='output_filter',
+        subparser.add_argument('--filter', metavar='STATUSCHARS', dest='output_filter', action=Highlander,
                                help='only display items with the given status characters (see description)')
         subparser.add_argument('--json', action='store_true',
                                help='output stats as JSON. Implies ``--stats``.')
@@ -3479,7 +3487,7 @@ def define_borg_mount(parser):
                               help='do not read and store xattrs into archive')
         fs_group.add_argument('--sparse', dest='sparse', action='store_true',
                                help='detect sparse holes in input (supported only by fixed chunker)')
-        fs_group.add_argument('--files-cache', metavar='MODE', dest='files_cache_mode',
+        fs_group.add_argument('--files-cache', metavar='MODE', dest='files_cache_mode', action=Highlander,
                               type=FilesCacheMode, default=DEFAULT_FILES_CACHE_MODE_UI,
                               help='operate files cache in MODE. default: %s' % DEFAULT_FILES_CACHE_MODE_UI)
         fs_group.add_argument('--read-special', dest='read_special', action='store_true',
@@ -3497,7 +3505,7 @@ def define_borg_mount(parser):
                                    type=int, default=1800,
                                    help='write checkpoint every SECONDS seconds (Default: 1800)')
         archive_group.add_argument('--chunker-params', metavar='PARAMS', dest='chunker_params',
-                                   type=ChunkerParams, default=CHUNKER_PARAMS,
+                                   type=ChunkerParams, default=CHUNKER_PARAMS, action=Highlander,
                                    help='specify the chunker parameters (ALGO, CHUNK_MIN_EXP, CHUNK_MAX_EXP, '
                                         'HASH_MASK_BITS, HASH_WINDOW_SIZE). default: %s,%d,%d,%d,%d' % CHUNKER_PARAMS)
         archive_group.add_argument('-C', '--compression', metavar='COMPRESSION', dest='compression',
@@ -4478,7 +4486,7 @@ def define_borg_mount(parser):
         subparser.set_defaults(func=self.do_recreate)
         subparser.add_argument('--list', dest='output_list', action='store_true',
                                help='output verbose list of items (files, dirs, ...)')
-        subparser.add_argument('--filter', metavar='STATUSCHARS', dest='output_filter',
+        subparser.add_argument('--filter', metavar='STATUSCHARS', dest='output_filter', action=Highlander,
                                help='only display items with the given status characters (listed in borg create --help)')
         subparser.add_argument('-n', '--dry-run', dest='dry_run', action='store_true',
                                help='do not change anything')
@@ -4517,7 +4525,7 @@ def define_borg_mount(parser):
                                         'recompression). '
                                         'If no MODE is given, `if-different` will be used. '
                                         'Not passing --recompress is equivalent to "--recompress never".')
-        archive_group.add_argument('--chunker-params', metavar='PARAMS', dest='chunker_params',
+        archive_group.add_argument('--chunker-params', metavar='PARAMS', dest='chunker_params', action=Highlander,
                                    type=ChunkerParams, default=CHUNKER_PARAMS,
                                    help='specify the chunker parameters (ALGO, CHUNK_MIN_EXP, CHUNK_MAX_EXP, '
                                         'HASH_MASK_BITS, HASH_WINDOW_SIZE) or `default` to use the current defaults. '
@@ -4768,7 +4776,7 @@ def define_borg_mount(parser):
                                           formatter_class=argparse.RawDescriptionHelpFormatter,
                                           help=self.do_import_tar.__doc__)
         subparser.set_defaults(func=self.do_import_tar)
-        subparser.add_argument('--tar-filter', dest='tar_filter', default='auto',
+        subparser.add_argument('--tar-filter', dest='tar_filter', default='auto', action=Highlander,
                                help='filter program to pipe data through')
         subparser.add_argument('-s', '--stats', dest='stats',
                                action='store_true', default=False,
@@ -4776,7 +4784,7 @@ def define_borg_mount(parser):
         subparser.add_argument('--list', dest='output_list',
                                action='store_true', default=False,
                                help='output verbose list of items (files, dirs, ...)')
-        subparser.add_argument('--filter', dest='output_filter', metavar='STATUSCHARS',
+        subparser.add_argument('--filter', dest='output_filter', metavar='STATUSCHARS', action=Highlander,
                                help='only display items with the given status characters')
         subparser.add_argument('--json', action='store_true',
                                help='output stats as JSON (implies --stats)')
@@ -4792,7 +4800,7 @@ def define_borg_mount(parser):
         archive_group.add_argument('-c', '--checkpoint-interval', dest='checkpoint_interval',
                                    type=int, default=1800, metavar='SECONDS',
                                    help='write checkpoint every SECONDS seconds (Default: 1800)')
-        archive_group.add_argument('--chunker-params', dest='chunker_params',
+        archive_group.add_argument('--chunker-params', dest='chunker_params', action=Highlander,
                                    type=ChunkerParams, default=CHUNKER_PARAMS,
                                    metavar='PARAMS',
                                    help='specify the chunker parameters (ALGO, CHUNK_MIN_EXP, CHUNK_MAX_EXP, '


Test patch: