Repository: gorakhargosh/watchdog
Pull request #704 (2020-11-27 11:27:08)

Use the standard pathlib instead of pathtools
Now with docstrings and tests. To be honest, I have not actually ran the tests and am hoping CI will be green and I won't have to.
The tests are taken directly from [pathtools' `patterns.py`](https://github.com/gorakhargosh/pathtools/blob/master/pathtools/patterns.py#L72), so those should work.

One thing to note is the license. I have no idea if I'm supposed to copy/paste the complete license notice and if I'm supposed to attribute it to the original author. Just a thing to check...

Finally, to answer [this question](https://github.com/bstaletic/watchdog/commit/35feb63de1e10de0e07f19f341bfedbf28cba435#r44586707) (which I did, but never hit "post"), that was intentional, though the reason is not immediately obvious. `pathlib` doesn't have a `case_sensitive` parameter anywhere in its API - `PurePosixPath` and `PosixPath` are always case-sensitive and `PureWindowsPath` and `WindowsPath` are always case-insensitive. On top of that, only "pure" path objects can be instantiated anywhere - `WindowsPath` can't be instantiated on POSIX systems and vice versa.

This lead me to "emulating" the `case_sensitive` parameter, by converting the input paths (as strings) to either `PurePosixPath` or `PureWindowsPath`, to satisfy `case_sensitive`.
In short, it allows this:

```
>>> import pathlib
>>> p = pathlib.PureWindowsPath('/etc/hosts')
>>> p.match('*hOsTs')
True
>>> p
PureWindowsPath('/etc/hosts')
>>> p = pathlib.PurePosixPath('/etc/hosts')
>>> p.match('*hOsTs')
False
```

BoboTiG: And I did not say it, but this is a clever usage of subclasses to emulate case-sensitivity 💪 
bstaletic: Travis is done. The errors don't seem related to my changes...

- A `TypeError` in `eventlet` - there on the latest master.
- A `-choco install python3 --version=3.7.9` failure - also there on master.
- A supposed thread leak on Windows and Python 3.9 - I assume it's a flake?
BoboTiG: Yes, those are known issues.
bstaletic: Is there anything else I should do? Travis has finished running, it's just the github status that is stuck.

Patch:
diff --git a/README.rst b/README.rst
index 064f76a..c133702 100755
--- a/README.rst
+++ b/README.rst
@@ -227,7 +227,6 @@ appropriate observer like in the example above, do::
 Dependencies
 ------------
 1. Python 3.4 or above.
-2. pathtools_
 3. XCode_ (only on Mac OS X)
 4. PyYAML_ (only for ``watchmedo`` script)
 5. argh_ (only for ``watchmedo`` script)
@@ -277,7 +276,6 @@ to do:
 .. _PyYAML: http://www.pyyaml.org/
 .. _XCode: http://developer.apple.com/technologies/tools/xcode.html
 .. _LibYAML: http://pyyaml.org/wiki/LibYAML
-.. _pathtools: http://github.com/gorakhargosh/pathtools
 
 .. _pnotify: http://mark.heily.com/pnotify
 .. _unison fsmonitor: https://webdav.seas.upenn.edu/viewvc/unison/trunk/src/fsmonitor.py?view=markup&pathrev=471
diff --git a/changelog.rst b/changelog.rst
index c250554..2e53e3f 100644
--- a/changelog.rst
+++ b/changelog.rst
@@ -11,6 +11,7 @@ Changelog
 - Allow file paths on Unix that don't follow the file system encoding (`# <https://github.com/gorakhargosh/watchdog/pull/703>`_)
 - Drop support for Python 2.7 (`# <https://github.com/gorakhargosh/watchdog/pull/703>`_)
 - Thanks to our beloved contributors: @SamSchott
+- Use `pathlib` from the standard library, instead of pathtools: @bstaletic
 
 
 0.10.4
diff --git a/docs/requirements.txt b/docs/requirements.txt
deleted file mode 100644
index 7d74e74..0000000
--- a/docs/requirements.txt
+++ /dev/null
@@ -1,1 +0,0 @@
-pathtools>=0.1.2
\ No newline at end of file
diff --git a/docs/source/global.rst.inc b/docs/source/global.rst.inc
index d3f754c..c2edd56 100644
--- a/docs/source/global.rst.inc
+++ b/docs/source/global.rst.inc
@@ -42,4 +42,3 @@
 .. _unison fsmonitor: https://webdav.seas.upenn.edu/viewvc/unison/trunk/src/fsmonitor.py?view=markup&pathrev=471
 .. _XCode: http://developer.apple.com/technologies/tools/xcode.html
 .. _zc.buildout: http://www.buildout.org/
-.. _pathtools: http://github.com/gorakhargosh/pathtools
diff --git a/docs/source/installation.rst b/docs/source/installation.rst
index 962408d..1096f31 100644
--- a/docs/source/installation.rst
+++ b/docs/source/installation.rst
@@ -59,8 +59,6 @@ using.
 +=====================+=============+=============+=============+=============+
 | XCode_              |             |             |     Yes     |             |
 +---------------------+-------------+-------------+-------------+-------------+
-| pathtools_          |     Yes     |     Yes     |     Yes     |     Yes     |
-+---------------------+-------------+-------------+-------------+-------------+
 
 The following is a list of dependencies you need based on the operating system you are
 using the ``watchmedo`` utility.
diff --git a/setup.py b/setup.py
index a379f55..af63bb1 100644
--- a/setup.py
+++ b/setup.py
@@ -82,9 +82,6 @@
         ),
     ]
 
-install_requires = [
-    "pathtools>=0.1.1",
-]
 extras_require = {
     'watchmedo': ['PyYAML>=3.10', 'argh>=0.24.1'],
 }
@@ -145,7 +142,6 @@
       package_dir={'': SRC_DIR},
       packages=find_packages(SRC_DIR),
       include_package_data=True,
-      install_requires=install_requires,
       extras_require=extras_require,
       cmdclass={
           'build_ext': build_ext,
diff --git a/src/watchdog/events.py b/src/watchdog/events.py
index 38e0d8c..2600a2a 100755
--- a/src/watchdog/events.py
+++ b/src/watchdog/events.py
@@ -88,8 +88,8 @@
 import os.path
 import logging
 import re
-from pathtools.patterns import match_any_paths
 from watchdog.utils import has_attribute
+from watchdog.utils.patterns import match_any_paths
 
 
 EVENT_TYPE_MOVED = 'moved'
diff --git a/src/watchdog/observers/kqueue.py b/src/watchdog/observers/kqueue.py
index 20468e0..95b6acc 100644
--- a/src/watchdog/observers/kqueue.py
+++ b/src/watchdog/observers/kqueue.py
@@ -76,7 +76,7 @@
 import os.path
 import select
 
-from pathtools.path import absolute_path
+from pathlib import Path
 
 from watchdog.observers.api import (
     BaseObserver,
@@ -126,6 +126,10 @@
     | select.KQ_NOTE_REVOKE
 )
 
+
+def absolute_path(path):
+    return Path(path).resolve()
+
 # Flag tests.
 
 
diff --git a/src/watchdog/utils/patterns.py b/src/watchdog/utils/patterns.py
new file mode 100644
index 0000000..0dc495a
--- /dev/null
+++ b/src/watchdog/utils/patterns.py
@@ -0,0 +1,88 @@
+#!/usr/bin/env python
+# -*- coding: utf-8 -*-
+# patterns.py: Common wildcard searching/filtering functionality for files.
+#
+# Copyright (C) 2010 Yesudeep Mangalapilly <yesudeep@gmail.com>
+#
+# Written by Boris Staletic <boris.staletic@gmail.com>
+
+# Non-pure path objects are only allowed on their respective OS's.
+# Thus, these utilities require "pure" path objects that don't access the filesystem.
+# Since pathlib doesn't have a `case_sensitive` parameter, we have to approximate it
+# by converting input paths to `PureWindowsPath` and `PurePosixPath` where:
+#   - `PureWindowsPath` is always case-insensitive.
+#   - `PurePosixPath` is always case-sensitive.
+# Reference: https://docs.python.org/3/library/pathlib.html#pathlib.PurePath.match
+from pathlib import PureWindowsPath, PurePosixPath
+
+
+def _match_path(path, included_patterns, excluded_patterns, case_sensitive):
+    """Internal function same as :func:`match_path` but does not check arguments."""
+    if case_sensitive:
+        path = PurePosixPath(path)
+    else:
+        included_patterns = {pattern.lower() for pattern in included_patterns}
+        excluded_patterns = {pattern.lower() for pattern in excluded_patterns}
+        path = PureWindowsPath(path)
+
+    common_patterns = included_patterns & excluded_patterns
+    if common_patterns:
+        raise ValueError('conflicting patterns `{}` included and excluded'.format(common_patterns))
+    return (any(path.match(p) for p in included_patterns)
+            and not any(path.match(p) for p in excluded_patterns))
+
+
+def filter_paths(paths, included_patterns=None, excluded_patterns=None, case_sensitive=True):
+    """
+    Filters from a set of paths based on acceptable patterns and
+    ignorable patterns.
+    :param pathnames:
+        A list of path names that will be filtered based on matching and
+        ignored patterns.
+    :param included_patterns:
+        Allow filenames matching wildcard patterns specified in this list.
+        If no pattern list is specified, ["*"] is used as the default pattern,
+        which matches all files.
+    :param excluded_patterns:
+        Ignores filenames matching wildcard patterns specified in this list.
+        If no pattern list is specified, no files are ignored.
+    :param case_sensitive:
+        ``True`` if matching should be case-sensitive; ``False`` otherwise.
+    :returns:
+        A list of pathnames that matched the allowable patterns and passed
+        through the ignored patterns.
+    """
+    included = ["*"] if included_patterns is None else included_patterns
+    excluded = [] if excluded_patterns is None else excluded_patterns
+
+    for path in paths:
+        if _match_path(path, set(included), set(excluded), case_sensitive):
+            yield path
+
+
+def match_any_paths(paths, included_patterns=None, excluded_patterns=None, case_sensitive=True):
+    """
+    Matches from a set of paths based on acceptable patterns and
+    ignorable patterns.
+    :param pathnames:
+        A list of path names that will be filtered based on matching and
+        ignored patterns.
+    :param included_patterns:
+        Allow filenames matching wildcard patterns specified in this list.
+        If no pattern list is specified, ["*"] is used as the default pattern,
+        which matches all files.
+    :param excluded_patterns:
+        Ignores filenames matching wildcard patterns specified in this list.
+        If no pattern list is specified, no files are ignored.
+    :param case_sensitive:
+        ``True`` if matching should be case-sensitive; ``False`` otherwise.
+    :returns:
+        ``True`` if any of the paths matches; ``False`` otherwise.
+    """
+    included = ["*"] if included_patterns is None else included_patterns
+    excluded = [] if excluded_patterns is None else excluded_patterns
+
+    for path in paths:
+        if _match_path(path, set(included), set(excluded), case_sensitive):
+            return True
+    return False


Test patch:
diff --git a/tests/test_pattern_matching_event_handler.py b/tests/test_pattern_matching_event_handler.py
index e89bb4f..eda458d 100644
--- a/tests/test_pattern_matching_event_handler.py
+++ b/tests/test_pattern_matching_event_handler.py
@@ -16,7 +16,6 @@
 # See the License for the specific language governing permissions and
 # limitations under the License.
 
-from pathtools.patterns import filter_paths
 from watchdog.events import (
     FileDeletedEvent,
     FileModifiedEvent,
@@ -33,6 +32,7 @@
     EVENT_TYPE_MOVED,
 )
 from watchdog.utils import has_attribute
+from watchdog.utils.patterns import filter_paths
 
 path_1 = '/path/xyz'
 path_2 = '/path/abc'
diff --git a/tests/test_patterns.py b/tests/test_patterns.py
new file mode 100644
index 0000000..343a8ee
--- /dev/null
+++ b/tests/test_patterns.py
@@ -0,0 +1,44 @@
+# -*- coding: utf-8 -*-
+#
+# Copyright (C) 2010 Yesudeep Mangalapilly <yesudeep@gmail.com>
+# Copyright 2020 Boris Staletic <boris.staletic@gmail.com>
+
+import pytest
+from watchdog.utils.patterns import _match_path, filter_paths, match_any_paths
+
+
+@pytest.mark.parametrize("input, included_patterns, excluded_patterns, case_sensitive, expected", [
+    ("/users/gorakhargosh/foobar.py", {"*.py"}, {"*.PY"}, True, True),
+    ("/users/gorakhargosh/foobar.py", {"*.py"}, {"*.PY"}, True, True),
+    ("/users/gorakhargosh/", {"*.py"}, {"*.txt"}, False, False),
+    ("/users/gorakhargosh/foobar.py", {"*.py"}, {"*.PY"}, False, ValueError),
+])
+def test_match_path(input, included_patterns, excluded_patterns, case_sensitive, expected):
+    if expected == ValueError:
+        with pytest.raises(expected):
+            _match_path(input, included_patterns, excluded_patterns, case_sensitive)
+    else:
+        assert _match_path(input, included_patterns, excluded_patterns, case_sensitive) is expected
+
+
+@pytest.mark.parametrize("included_patterns, excluded_patterns, case_sensitive, expected", [
+    (None, None, True, None),
+    (None, None, False, None),
+    (["*.py", "*.conf"], ["*.status"], True, {"/users/gorakhargosh/foobar.py", "/etc/pdnsd.conf"}),
+])
+def test_filter_paths(included_patterns, excluded_patterns, case_sensitive, expected):
+    pathnames = {"/users/gorakhargosh/foobar.py", "/var/cache/pdnsd.status", "/etc/pdnsd.conf", "/usr/local/bin/python"}
+    actual = set(filter_paths(pathnames, included_patterns, excluded_patterns, case_sensitive))
+    assert actual == expected if expected else pathnames
+
+
+@pytest.mark.parametrize("included_patterns, excluded_patterns, case_sensitive, expected", [
+    (None, None, True, True),
+    (None, None, False, True),
+    (["*py", "*.conf"], ["*.status"], True, True),
+    (["*.txt"], None, False, False),
+    (["*.txt"], None, True, False),
+])
+def test_match_any_paths(included_patterns, excluded_patterns, case_sensitive, expected):
+    pathnames = {"/users/gorakhargosh/foobar.py", "/var/cache/pdnsd.status", "/etc/pdnsd.conf", "/usr/local/bin/python"}
+    assert match_any_paths(pathnames, included_patterns, excluded_patterns, case_sensitive) == expected