Repository: borgbackup/borg

Issue:
Extract pattern not working
<!--
Thank you for reporting an issue.

*IMPORTANT* -  *before* creating a new issue please look around:
 - Borgbackup documentation: http://borgbackup.readthedocs.io/en/stable/index.html
 - FAQ: https://borgbackup.readthedocs.io/en/stable/faq.html
 and
 - open issues in Github tracker: https://github.com/borgbackup/borg/issues
  
If you cannot find a similar problem, then create a new issue.

Please fill in as much of the template as possible.
-->

## Have you checked borgbackup docs, FAQ, and open Github issues?

Yes

## Is this a BUG / ISSUE report or a QUESTION?

BUG

## System information. For client/server mode post info for both machines.

#### Your borg version (borg -V).

`borg 1.1.7`

#### Operating system (distribution) and version.

`Linux 4.18.12-arch1-1-ARCH`

#### Hardware / network configuration, and filesystems used.

Raspberry pi 2B+, External USB HDD

#### How much data is handled by borg?

`180GB`

#### Full borg commandline that lead to the problem (leave away excludes and passwords)

    ~/test » borg extract -n --list pi@IP:PATH::data-2018-10-04 "*"

## Describe the problem you're observing.

I can't get patterns to work. Originally, I wanted to list all files in my config folder ending with .conf: `/home/piegames/.config/*.conf`, but that didn't work. I reduced even further to just `"*"`, which clearly should include all files. But it never matches.

#### Can you reproduce the problem? If so, describe how. If not, describe troubleshooting steps you took before opening the issue.

Use patterns in the extract PATH.

#### Include any warning/errors/backtraces from the system logs

    Include pattern '*' never matched.
<!--

If this complaint relates to borg performance, please include CRUD benchmark
results and any steps you took to troubleshoot.
How to run benchmark: http://borgbackup.readthedocs.io/en/stable/usage/benchmark.html

*IMPORTANT* - Please mark logs and text output from terminal commands 
or else Github will not display them correctly. 
An example is provided below.

Example:
```
this is an example how log text should be marked (wrap it with ```)
```
-->



Discussion:
please provide command line for `borg create`
```
borg create --list --progress --compression auto,zstd,9 --one-file-system --noatime \
    --stats --filter=AME \
    --exclude-caches \
    --exclude-if-present ".nomedia" --exclude-if-present ".trackerignore" --exclude-if-present ".borgignore" \
    --exclude "/home/piegames/Videos" \
    --exclude "/home/piegames/Pictures" \
    --exclude "/home/piegames/Downloads" \
    --exclude "/home/piegames/.log" \
    --exclude "/home/piegames/.backups" \
    --exclude "/home/piegames/.debug" \
    --exclude "/home/piegames/.local/share/Trash/" \
    --exclude "*/.Trash*/" \
    --exclude "*/.metadata/" \
    --exclude "*/.recommenders/" \
    --exclude "*/.cache/" \
    --exclude "/home/piegames/.*Cache/" \
    --exclude "/home/piegames/.*/Cache*/" \
    --exclude "/home/piegames/.*cache/" \
    --exclude "/home/piegames/.*/caches/" \
    --exclude "/home/piegames/.*/logs/" \
    --exclude "/home/piegames/.*/log/" \
    --exclude "*.log" \
    --exclude "*.log.old" \
    --exclude "*.log.?" \
    --exclude "*.log.txt" \
    --exclude "*.log.gz" \
    --exclude "*.lock" \
    pi@IP:PATH::data-{now:%Y_%m_%d-%H:%M} "/home/piegames"
```
Minimal not working example:

```
    mkdir test2
    cd test2
    mkdir data
    # (Put some test data into ./data now)
    borg init ./repo -e=none
    borg create --list ./repo::test ./data
    # Does work:
    borg extract -n --list ./repo::test ./data
    borg extract -n --list ./repo::test "./data"
    borg extract -n --list ./repo::test "data"
    # Does not:
    borg extract -n --list ./repo::test "*"
    borg extract -n --list ./repo::test \*
    borg extract -n --list ./repo::test "dat*"
```
how about using `borg list` first and seeing how the stored pathes look like?
Either I don't understand how the paths and patterns work and got everything horrible wrong. Or the documentation is faulty and borg does not support patterns here. But I still haven't got any pattern to work in the `PATH` (`--exclude` patterns work fine).

```
~/test2 » borg list ./repo::test
drwxr-xr-x piegames users         0 Mon, 2018-10-15 16:02:20 data
-rw-r--r-- piegames users         0 Mon, 2018-10-15 15:59:28 data/Testdata.txt
------------------------------------------------------------
~/test2 » borg list ./repo::test "*.txt"
------------------------------------------------------------
~/test2 »
```
Extraction works if you prefix the pattern with "sh:", for example.  I'm guessing the default style is something like "pp:" or "pf:" for extraction?

@jdchristensen Thank you, that is the reason. Prepending `fm:` to all my pattern makes it work perfectly fine.

The documentation says:

> Fnmatch, selector fm:
>              This is the default style for --exclude and --exclude-from.

I was mistakenly assuming this would hold true to `PATH` values as well, but apparently you must manually enable it there. I can see why this was done, but this still caused me a lot of confusion and is a bit counter-intuitive. Especially since the error messages suggest that there was a pattern, even if that "pattern" was "look for a file that is literally called `*.conf`".

I'll leave this issue open since I think this should either be changed or documented.
I think I am running into this one too.

<details>
  <summary>Here's a script to reproduce the issue with a suitable exit code.</summary>

```sh
#!/bin/bash

set -eu

cd $(mktemp -d)
pwd

# setup
borg init -e none repo
mkdir data
echo x > data/foo
echo x > data/bar
borg create repo::archive data

echo
echo path is data/foo
borg list repo::archive

echo
echo delete both files
rm data/{foo,bar}
ls -l data/

echo
echo successfully extract file but exit non-zero?
set +e
borg extract repo::archive pf:data/foo
e=$?
set -e
echo EXIT $e

echo
echo only the in fact matched file was extracted
ls -l data/

exit $e
```
</details>

This prints
```
/tmp/tmp.thKgpmbIp1

path is data/foo
drwxr-xr-x user   user          0 Tue, 2023-04-18 00:33:43 data
-rw-r--r-- user   user          2 Tue, 2023-04-18 00:33:43 data/foo
-rw-r--r-- user   user          2 Tue, 2023-04-18 00:33:43 data/bar

delete both files
total 0

successfully extract file but exit non-zero?
Include pattern 'data/foo' never matched.
EXIT 1

only the in fact matched file was extracted
total 4
-rw-r--r-- 1 user user 2 Apr 18 00:33 foo
```

Tested with 1.2.3 and 1.1.16.
@hxtmdev that looks weird indeed. thanks for writing that script!
As I'm sure you are aware but I was not, 1 is EXIT_WARNING, not error, so it's probably just a side effect of wrongfully logging a warning in the first place.
1 is warning, sure, and that is due to it warning about "Include pattern 'data/foo' never matched.".

But as it extracted that file (and likely matched that pattern), it should not emit that warning.

Resolving patch:
diff --git a/src/borg/patterns.py b/src/borg/patterns.py
--- a/src/borg/patterns.py
+++ b/src/borg/patterns.py
@@ -131,8 +131,11 @@ def add_includepaths(self, include_paths):
         self.include_patterns = include_patterns
 
     def get_unmatched_include_patterns(self):
-        "Note that this only returns patterns added via *add_includepaths*."
-        return [p for p in self.include_patterns if p.match_count == 0]
+        """Note that this only returns patterns added via *add_includepaths* and it
+           won't return PathFullPattern patterns as we do not match_count for them.
+        """
+        return [p for p in self.include_patterns
+                if p.match_count == 0 and not isinstance(p, PathFullPattern)]
 
     def add_inclexcl(self, patterns):
         """Add list of patterns (of type CmdTuple) to internal list.


Test patch: