Repository: borgbackup/borg
Pull request: docs: explain the difference between a path that ends with or without a slash
Resolves #6258: explains the difference between a path that ends with or without a slash.

In the doc I specifically mentioned the difference when used as an exclusion pattern, because if I'm not mistaken there's no difference for inclusion patterns (`some/path/` and `some/path` would result in the same backup set). But let me know if I should tweak this.

Problem statement:
Always getting "Permission denied" on a folder that has been excluded
## Have you checked borgbackup docs, FAQ, and open Github issues?

Yes

## Is this a BUG / ISSUE report or a QUESTION?

BUG

## System information. For client/server mode post info for both machines.

#### borg 1.1.17

#### macOS 11.5.2

## Describe the problem you're observing.

My borg command is as follow:

```
borg create \
    --exclude-from borg_excluded.txt \
    --progress \
    --one-file-system \
    --compression auto,zlib,6 \
    $BACKUP_DRIVE::$DATE \
    /Users/USERNAME/ \
    /Library
```

And my exclude file as follow:

```
...
/Library/Application Support/ApplePushService/
...
```

So the important part is that the "ApplePushService" folder is excluded. However, whenever I run the backup I'm getting the error:

> /Library/Application Support/ApplePushService: scandir: [Errno 13] Permission denied: '/Library/Application Support/ApplePushService'

So it seems Borg is still trying to access the folder even though I've excluded it. To be sure I've also tried excluding "**/ApplePushService" but that didn't make any difference and I still got the same error.

Any idea what might be the issue? Is that possibly a bug in Borg, or am I using the wrong exclude pattern?

Hints from issue discussion:
Can you try to remove the trailing slash?
Also, if you use patterns, always be careful about the kind of patterns and what's the default kind, see `borg help patterns`.
Yes thank you, removing the trailing slash fixed it! Would that be considered a bug, or is it working as expected? (although in any case it's a bit confusing to get an error on an excluded folder)
Well, borg's internal matcher tries to match the path it is currently processing against the patterns you have given.

If you give "path/" and borg is currently at "path", that does not (yet) match.

IIRC we had some trailing-slash related change, could be that this is fixed already in borg 1.2 (rc1 available already).

I would call this a feature.  "path/to/folder/" matches the *contents* of folder, while "path/to/folder" matches the folder itself.  You might want to exclude the contents but backup the folder...

That's true, like you might want to copy a cache folder but not its content. Should this behaviour be documented in the exclude pattern doc? Happy to create a pull request for this if that can help.
Guess we should check this with borg 1.2 first ...
Tried to reproduce with borg master branch (borg 1.2.0rc1+):
```
(borg-env) tw@mba2020 borg % borg create --show-rc --exclude '/Library/Application Support/ApplePushService/' repo::a1 '/Library/Application Support'
/Library/Application Support/ApplePushService: dir_open: [Errno 13] Permission denied: 'ApplePushService'
/Library/Application Support/Apple/AssetCache/Data: dir_open: [Errno 13] Permission denied: 'Data'
/Library/Application Support/Apple/ParentalControls/Users: dir_open: [Errno 13] Permission denied: 'Users'
terminating with warning status, rc 1

(borg-env) tw@mba2020 borg % borg create --show-rc --exclude '/Library/Application Support/ApplePushService' repo::a2 '/Library/Application Support' 
/Library/Application Support/Apple/AssetCache/Data: dir_open: [Errno 13] Permission denied: 'Data'
/Library/Application Support/Apple/ParentalControls/Users: dir_open: [Errno 13] Permission denied: 'Users'
terminating with warning status, rc 1
```

So, in 1.2.0, it depends on presence of the trailing slash.

Reproduce with borg 1.1-maint branch (borg 1.1.17+):
```
(borg-env) tw@mba2020 borg % borg create --show-rc --exclude '/Library/Application Support/ApplePushService/' repo::a11 '/Library/Application Support'
/Library/Application Support/ApplePushService: scandir: [Errno 13] Permission denied: '/Library/Application Support/ApplePushService'
/Library/Application Support/Apple/AssetCache/Data: scandir: [Errno 13] Permission denied: '/Library/Application Support/Apple/AssetCache/Data'
/Library/Application Support/Apple/ParentalControls/Users: scandir: [Errno 13] Permission denied: '/Library/Application Support/Apple/ParentalControls/Users'
terminating with warning status, rc 1

(borg-env) tw@mba2020 borg % borg create --show-rc --exclude '/Library/Application Support/ApplePushService' repo::a12 '/Library/Application Support' 
/Library/Application Support/Apple/AssetCache/Data: scandir: [Errno 13] Permission denied: '/Library/Application Support/Apple/AssetCache/Data'
/Library/Application Support/Apple/ParentalControls/Users: scandir: [Errno 13] Permission denied: '/Library/Application Support/Apple/ParentalControls/Users'
terminating with warning status, rc 1
```

So, same thing in borg 1.1.

I agree with @jdchristensen, it's a feature! :-)

For the other include/exclude mechanism (`--pattern`) I'ld like to note that there are 2 kinds of exclusion:
- `-` excludes, but still recurses to match any potential includes
- `!` excludes and does not recurse (good if access is forbidden / problematic)

1.1-maint:
```
(borg-env) tw@mba2020 borg % borg create --show-rc --pattern '- /Library/Application Support/ApplePushService/' repo::a21 '/Library/Application Support'
/Library/Application Support/ApplePushService: scandir: [Errno 13] Permission denied: '/Library/Application Support/ApplePushService'
/Library/Application Support/Apple/AssetCache/Data: scandir: [Errno 13] Permission denied: '/Library/Application Support/Apple/AssetCache/Data'
/Library/Application Support/Apple/ParentalControls/Users: scandir: [Errno 13] Permission denied: '/Library/Application Support/Apple/ParentalControls/Users'
terminating with warning status, rc 1

(borg-env) tw@mba2020 borg % borg create --show-rc --pattern '- /Library/Application Support/ApplePushService' repo::a22 '/Library/Application Support' 
/Library/Application Support/ApplePushService: scandir: [Errno 13] Permission denied: '/Library/Application Support/ApplePushService'
/Library/Application Support/Apple/AssetCache/Data: scandir: [Errno 13] Permission denied: '/Library/Application Support/Apple/AssetCache/Data'
/Library/Application Support/Apple/ParentalControls/Users: scandir: [Errno 13] Permission denied: '/Library/Application Support/Apple/ParentalControls/Users'
terminating with warning status, rc 1

(borg-env) tw@mba2020 borg % borg create --show-rc --pattern '! /Library/Application Support/ApplePushService/' repo::a31 '/Library/Application Support'
/Library/Application Support/ApplePushService: scandir: [Errno 13] Permission denied: '/Library/Application Support/ApplePushService'
/Library/Application Support/Apple/AssetCache/Data: scandir: [Errno 13] Permission denied: '/Library/Application Support/Apple/AssetCache/Data'
/Library/Application Support/Apple/ParentalControls/Users: scandir: [Errno 13] Permission denied: '/Library/Application Support/Apple/ParentalControls/Users'
terminating with warning status, rc 1

(borg-env) tw@mba2020 borg % borg create --show-rc --pattern '! /Library/Application Support/ApplePushService' repo::a32 '/Library/Application Support' 
/Library/Application Support/Apple/AssetCache/Data: scandir: [Errno 13] Permission denied: '/Library/Application Support/Apple/AssetCache/Data'
/Library/Application Support/Apple/ParentalControls/Users: scandir: [Errno 13] Permission denied: '/Library/Application Support/Apple/ParentalControls/Users'
terminating with warning status, rc 1
```

master: same behaviour.

Commit message: docs: explain the difference between a path that ends with or without a slash

Patch to review:
diff --git a/src/borg/archiver.py b/src/borg/archiver.py
index 7a87702bfe..26a7ece801 100644
--- a/src/borg/archiver.py
+++ b/src/borg/archiver.py
@@ -2234,6 +2234,11 @@ def do_break_lock(self, args, repository):
         /path/to/repo /home/user`` will store all files as
         `home/user/.../file.ext`.
 
+        A folder exclusion pattern can end either with or without a slash ('/').
+        If it ends with a slash, such as `some/path/`, the directory will be
+        included but not its content. If it does not end with a slash, such as
+        `some/path`, both the directory and content will be excluded.
+
         File patterns support these styles: fnmatch, shell, regular expressions,
         path prefixes and path full-matches. By default, fnmatch is used for
         ``--exclude`` patterns and shell-style is used for the ``--pattern``


Code review comments:
diff_hunk: @@ -2234,6 +2234,11 @@ def do_break_lock(self, args, repository):
         /path/to/repo /home/user`` will store all files as
         `home/user/.../file.ext`.
 
+        A folder exclusion pattern can end either with or without a slash ('/').
path: src/borg/archiver.py
text: @user1:
s/folder/directory/ please.

i checked the patterns help, it nowhere else uses "folder", but always talks about "directory".

The paths mention here are usually included in backticks, like:
```
... without a slash (`/`).
```

Maybe check if that renders ok here, too.


@author:
Should it actually be backticks? I don't mind changing it but I've used single quotes here because that's how it is in several instances below (also for ':', '*', '?'). It's not consistent as there are also a few instances of backticks for single characters, but less.

Ok for folder/directory.

@user1:
Hmm, right, when talking about single chars, it uses normal ticks. OK, so we keep that one as is.