Repository: facelessuser/wcmatch

Issue:
Add option to support capturing
Similar to extended match patterns: it would be nice to be able to be able to have a flag to enable selective capturing of matched text when using wcmatch.translate. This could probably be done with `(...)` so that it's similar to an extended match, but without any prefix. Existing match patterns are non-capturing and should probably stay as such. `@(...)` also makes sense as non-capturing if you think of the `@` like it's used in make files to suppress echoing of commands.

Discussion:
facelessuser: @gir-bot lgtm

Patch:
diff --git a/docs/src/markdown/about/changelog.md b/docs/src/markdown/about/changelog.md
index 1bdcdaa..fda3911 100644
--- a/docs/src/markdown/about/changelog.md
+++ b/docs/src/markdown/about/changelog.md
@@ -1,5 +1,10 @@
 # Changelog
 
+## 7.1
+
+- **NEW**: Translate functions will now use capturing groups for `EXTGLOB`/`EXTMATCH` groups in the returned regular
+  expression patterns.
+
 ## 7.0.1
 
 - **FIX**: Ensure that when using `REALPATH` that all symlinks are evaluated.
diff --git a/docs/src/markdown/about/contributing.md b/docs/src/markdown/about/contributing.md
index e6f9eb8..4c289fc 100644
--- a/docs/src/markdown/about/contributing.md
+++ b/docs/src/markdown/about/contributing.md
@@ -3,13 +3,10 @@
 ## Become a Sponsor :octicons-heart-fill-16:{: .heart-throb}
 
 Open source projects take time and money. Help support the project by becoming a sponsor. You can add your support at
-any tier you feel comfortable with. No amount is too little.
+any tier you feel comfortable with. No amount is too little. We also accept one time contributions via PayPal.
 
-:octicons-mark-github-16: [GitHub Sponsors ](https://github.com/sponsors/facelessuser)
-
-If you'd like to do a one time contribution, you can do so via PayPal.
-
-:fontawesome-brands-paypal: [PayPal](https://www.paypal.me/facelessuser)
+[:octicons-mark-github-16: GitHub Sponsors](https://github.com/sponsors/facelessuser){: .md-button .md-button--primary }
+[:fontawesome-brands-paypal: PayPal](https://www.paypal.me/facelessuser){ .md-button}
 
 ## Bug Reports
 
diff --git a/docs/src/markdown/fnmatch.md b/docs/src/markdown/fnmatch.md
index aded127..310e61e 100644
--- a/docs/src/markdown/fnmatch.md
+++ b/docs/src/markdown/fnmatch.md
@@ -145,6 +145,20 @@ matched if it matches at least one inclusion pattern and matches **none** of the
 (['^(?s:(?=.)(?![.]).*?)$'], ['^(?s:(?=.).*?\\.a)$', '^(?s:(?=.).*?\\.b)$', '^(?s:(?=.).*?\\.c)$'])
 ```
 
+When using [`EXTMATCH`](#extmatch) patterns, patterns will be returned with capturing groups around the groups:
+
+While in regex patterns like `#!py3 r'(a)+'` would capture only the last character, even though multiple where matched,
+we wrap the entire group to be captured: `#!py3 '+(a)'` --> `#!py3 r'((a)+)'`.
+
+```pycon3
+>>> from wcmatch import fnmatch
+>>> import re
+>>> gpat = fnmatch.translate("@(file)+([[:digit:]])@(.*)", flags=fnmatch.EXTMATCH)
+>>> pat = re.compile(gpat[0][0])
+>>> pat.match('file33.test.txt').groups()
+('file', '33', '.test.txt')
+```
+
 !!! new "Changes 4.0"
     Translate now outputs exclusion patterns so that if they match, the file is excluded. This is opposite logic to how
     it used to be, but is more efficient.
@@ -152,6 +166,9 @@ matched if it matches at least one inclusion pattern and matches **none** of the
 !!! new "New 6.0"
     `limit` was added in 6.0.
 
+!!! new "New 7.0"
+    Translate patterns now provide capturing groups for [`EXTMATCH`](#extmatch) groups.
+
 ## Flags
 
 #### `fnmatch.CASE, fnmatch.C` {: #case}
diff --git a/docs/src/markdown/glob.md b/docs/src/markdown/glob.md
index 849dc73..f8f442f 100644
--- a/docs/src/markdown/glob.md
+++ b/docs/src/markdown/glob.md
@@ -465,6 +465,20 @@ matched if it matches at least one inclusion pattern and matches **none** of the
 (['^(?s:(?=.)(?!(?:\\.{1,2})(?:$|\\/))(?:(?!\\.)[^\\/]*?)?[\\/]*?)$'], ['^(?s:(?=.)(?!(?:\\.{1,2})(?:$|\\/))[^\\/]*?\\/+(?=.)(?!(?:\\.{1,2})(?:$|\\/))[^\\/]*?\\.\\{py\\,txt\\}[\\/]*?)$'])
 ```
 
+When using [`EXTGLOB`](#extglob) patterns, patterns will be returned with capturing groups around the groups:
+
+While in regex patterns like `#!py3 r'(a)+'` would capture only the last character, even though multiple where matched,
+we wrap the entire group to be captured: `#!py3 '+(a)'` --> `#!py3 r'((a)+)'`.
+
+```pycon3
+>>> from wcmatch import glob
+>>> import re
+>>> gpat = glob.translate("@(file)+([[:digit:]])@(.*)", flags=glob.EXTGLOB)
+>>> pat = re.compile(gpat[0][0])
+>>> pat.match('file33.test.txt').groups()
+('file', '33', '.test.txt')
+```
+
 !!! new "Changes 4.0"
     Translate now outputs exclusion patterns so that if they match, the file is excluded. This is opposite logic to how
     it used to be, but is more efficient.
@@ -472,6 +486,9 @@ matched if it matches at least one inclusion pattern and matches **none** of the
 !!! new "New 6.0"
     `limit` was added in 6.0.
 
+!!! new "New 7.0"
+    Translate patterns now provide capturing groups for [`EXTGLOB`](#extglob) groups.
+
 #### `glob.escape` {: #escape}
 
 ```py3
diff --git a/docs/theme/announce.html b/docs/theme/announce.html
index fceed04..c4b557f 100644
--- a/docs/theme/announce.html
+++ b/docs/theme/announce.html
@@ -1,7 +1,7 @@
 <span class="twemoji"><svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 24 24"><path d="M23 12l-2.44-2.78.34-3.68-3.61-.82-1.89-3.18L12 3 8.6 1.54 6.71 4.72l-3.61.81.34 3.68L1 12l2.44 2.78-.34 3.69 3.61.82 1.89 3.18L12 21l3.4 1.46 1.89-3.18 3.61-.82-.34-3.68L23 12m-10 5h-2v-2h2v2m0-4h-2V7h2v6z"></path></svg></span>
 7.0 has been released! Check out the <a href="about/release/#upgrade-to-70">Release Notes</a> for more information and migration tips.
 <br>
-<a href="{{ 'about/contributing/#contributing--support' | url }}">Sponsorship</a>
+<a href="{{ 'about/contributing/#become-a-sponsor' | url }}">Sponsorship</a>
 is now available!
 <span class="twemoji heart-throb">
   <svg xmlns="http://www.w3.org/2000/svg" width="16" height="16" viewBox="0 0 16 16"><path fill-rule="evenodd" d="M7.655 14.916L8 14.25l.345.666a.752.752 0 01-.69 0zm0 0L8 14.25l.345.666.002-.001.006-.003.018-.01a7.643 7.643 0 00.31-.17 22.08 22.08 0 003.433-2.414C13.956 10.731 16 8.35 16 5.5 16 2.836 13.914 1 11.75 1 10.203 1 8.847 1.802 8 3.02 7.153 1.802 5.797 1 4.25 1 2.086 1 0 2.836 0 5.5c0 2.85 2.045 5.231 3.885 6.818a22.075 22.075 0 003.744 2.584l.018.01.006.003h.002z"/></svg>
diff --git a/wcmatch/__meta__.py b/wcmatch/__meta__.py
index 79b26c6..1a949c4 100644
--- a/wcmatch/__meta__.py
+++ b/wcmatch/__meta__.py
@@ -186,5 +186,5 @@ def parse_version(ver, pre=False):
     return Version(major, minor, micro, release, pre, post, dev)
 
 
-__version_info__ = Version(7, 0, 1, "final")
+__version_info__ = Version(7, 1, 0, ".dev")
 __version__ = __version_info__._get_canonical()
diff --git a/wcmatch/_wcparse.py b/wcmatch/_wcparse.py
index 40d599c..adfab58 100644
--- a/wcmatch/_wcparse.py
+++ b/wcmatch/_wcparse.py
@@ -228,18 +228,23 @@ _NEED_CHAR = r'(?=.)'
 _NEED_SEP = r'(?=%s)'
 # Group that matches one or none
 _QMARK_GROUP = r'(?:%s)?'
+_QMARK_CAPTURE_GROUP = r'((?#)(?:%s)?)'
 # Group that matches Zero or more
 _STAR_GROUP = r'(?:%s)*'
+_STAR_CAPTURE_GROUP = r'((?#)(?:%s)*)'
 # Group that matches one or more
 _PLUS_GROUP = r'(?:%s)+'
+_PLUS_CAPTURE_GROUP = r'((?#)(?:%s)+)'
 # Group that matches exactly one
 _GROUP = r'(?:%s)'
+_CAPTURE_GROUP = r'((?#)%s)'
 # Inverse group that matches none
 # This is the start. Since Python can't
 # do variable look behinds, we have stuff
 # everything at the end that it needs to lookahead
 # for. So there is an opening and a closing.
 _EXCLA_GROUP = r'(?:(?!(?:%s)'
+_EXCLA_CAPTURE_GROUP = r'((?#)(?!(?:%s)'
 # Closing for inverse group
 _EXCLA_GROUP_CLOSE = r')%s)'
 # Restrict root
@@ -989,6 +994,7 @@ class WcParse(object):
         self.rtl = bool(flags & _RTL)
         self.anchor = bool(flags & _ANCHOR)
         self.nodotdir = bool(flags & NODOTDIR)
+        self.capture = self.translate
         self.case_sensitive = get_case(flags)
         self.in_list = False
         self.inv_nest = False
@@ -1433,7 +1439,10 @@ class WcParse(object):
                 content = current[index + 1:]
                 if not nested:
                     content.append(_EOP if not self.pathname else self.path_eop)
-                current[index] = (''.join(content)) + (_EXCLA_GROUP_CLOSE % str(current[index]))
+                current[index] = (
+                    (''.join(content).replace('(?#)', '?:') if self.capture else ''.join(content)) +
+                    (_EXCLA_GROUP_CLOSE % str(current[index]))
+                )
             index -= 1
         self.inv_ext = 0
 
@@ -1457,10 +1466,12 @@ class WcParse(object):
         index = i.index
         list_type = c
         extended = []
+
         try:
             c = next(i)
             if c != '(':
                 raise StopIteration
+
             while c != ')':
                 c = next(i)
 
@@ -1507,17 +1518,17 @@ class WcParse(object):
                 self.update_dir_state()
 
             if list_type == '?':
-                current.append(_QMARK_GROUP % ''.join(extended))
+                current.append((_QMARK_CAPTURE_GROUP if self.capture else _QMARK_GROUP) % ''.join(extended))
             elif list_type == '*':
-                current.append(_STAR_GROUP % ''.join(extended))
+                current.append((_STAR_CAPTURE_GROUP if self.capture else _STAR_GROUP) % ''.join(extended))
             elif list_type == '+':
-                current.append(_PLUS_GROUP % ''.join(extended))
+                current.append((_PLUS_CAPTURE_GROUP if self.capture else _PLUS_GROUP) % ''.join(extended))
             elif list_type == '@':
-                current.append(_GROUP % ''.join(extended))
+                current.append((_CAPTURE_GROUP if self.capture else _GROUP) % ''.join(extended))
             elif list_type == '!':
                 self.inv_ext += 1
                 # If pattern is at the end, anchor the match to the end.
-                current.append(_EXCLA_GROUP % ''.join(extended))
+                current.append((_EXCLA_CAPTURE_GROUP if self.capture else _EXCLA_GROUP) % ''.join(extended))
                 if self.pathname:
                     if not temp_after_start or self.match_dot_dir:
                         star = self.path_star
@@ -1550,6 +1561,7 @@ class WcParse(object):
             self.in_list = False
         if not temp_inv_nest:
             self.inv_nest = False
+
         if success:
             self.reset_dir_track()
         else:
@@ -1666,6 +1678,7 @@ class WcParse(object):
             self.update_dir_state()
 
         self.clean_up_inverse(current)
+
         if self.pathname:
             current.append(_PATH_TRAIL % self.get_path_sep())
 
@@ -1732,6 +1745,10 @@ class WcParse(object):
         else:
             pattern = r'(?s{})^(?:{})$'.format(case_flag, ''.join(result))
 
+        if self.capture:
+            # Strip out unnecessary regex comments
+            pattern = pattern.replace('(?#)', '')
+
         if self.is_bytes:
             pattern = pattern.encode('latin-1')
 
@@ -1766,8 +1783,7 @@ def _fs_match(pattern, filename, is_dir, sep, follow, symlinks, root):
         # Lets look at the captured `globstar` groups and see if that part of the path
         # contains symlinks.
         if not follow:
-            groups = m.groups()
-            last = len(groups)
+            last = len(m.groups())
             for i, star in enumerate(m.groups(), 1):
                 if star:
                     at_end = m.end(i) == end
@@ -1815,12 +1831,12 @@ def _match_real(filename, include, exclude, follow, symlinks, root):
             break
 
     if matched:
-        matched = True
         if exclude:
             for pattern in exclude:
                 if _fs_match(pattern, filename, is_dir, sep, True, symlinks, root):
                     matched = False
                     break
+
     return matched
 
 
@@ -1828,8 +1844,6 @@ def _match_pattern(filename, include, exclude, real, path, follow, root_dir=None
     """Match includes and excludes."""
 
     if real:
-
-        symlinks = {}
         if isinstance(filename, bytes):
             root = root_dir if root_dir else b'.'
             ptype = BYTES
@@ -1844,10 +1858,11 @@ def _match_pattern(filename, include, exclude, real, path, follow, root_dir=None
         else:
             exists = os.path.lexists(filename)
 
-        if not exists:
-            return False
-        if path:
+        if exists:
+            symlinks = {}
             return _match_real(filename, include, exclude, follow, symlinks, root)
+        else:
+            return False
 
     matched = False
     for pattern in include:


Test patch:
diff --git a/tests/test_fnmatch.py b/tests/test_fnmatch.py
index 998689b..d6f8c74 100644
--- a/tests/test_fnmatch.py
+++ b/tests/test_fnmatch.py
@@ -1,6 +1,7 @@
 # -*- coding: utf-8 -*-
 """Tests for `fnmatch`."""
 import unittest
+import re
 import pytest
 import wcmatch.fnmatch as fnmatch
 from unittest import mock
@@ -326,6 +327,30 @@ class TestFnMatchTranslate(unittest.TestCase):
 
         return fnmatch.translate(pattern, flags=flags | fnmatch.SPLIT)
 
+    def test_capture_groups(self):
+        """Test capture groups."""
+
+        gpat = fnmatch.translate("test @(this) +(many) ?(meh)*(!) !(not this)@(.md)", flags=fnmatch.E)
+        pat = re.compile(gpat[0][0])
+        match = pat.match('test this manymanymany meh!!!!! okay.md')
+        self.assertEqual(('this', 'manymanymany', 'meh', '!!!!!', 'okay', '.md'), match.groups())
+
+    def test_nested_capture_groups(self):
+        """Test nested capture groups."""
+
+        gpat = fnmatch.translate("@(file)@(+([[:digit:]]))@(.*)", flags=fnmatch.E)
+        pat = re.compile(gpat[0][0])
+        match = pat.match('file33.test.txt')
+        self.assertEqual(('file', '33', '33', '.test.txt'), match.groups())
+
+    def test_list_groups(self):
+        """Test capture groups with lists."""
+
+        gpat = fnmatch.translate("+(f|i|l|e)+([[:digit:]])@(.*)", flags=fnmatch.E)
+        pat = re.compile(gpat[0][0])
+        match = pat.match('file33.test.txt')
+        self.assertEqual(('file', '33', '.test.txt'), match.groups())
+
     def test_split_parsing(self):
         """Test wildcard parsing."""
 
diff --git a/tests/test_globmatch.py b/tests/test_globmatch.py
index 56fcfcf..c11e937 100644
--- a/tests/test_globmatch.py
+++ b/tests/test_globmatch.py
@@ -1100,6 +1100,30 @@ class TestGlobMatchSpecial(unittest.TestCase):
         self.assertEqual(1, len(pos))
         self.assertEqual(1, len(neg))
 
+    def test_capture_groups(self):
+        """Test capture groups."""
+
+        gpat = glob.translate("test/@(this)/+(many)/?(meh)*(!)/!(not this)@(.md)", flags=glob.E)
+        pat = re.compile(gpat[0][0])
+        match = pat.match(os.path.normpath('test/this/manymanymany/meh!!!!!/okay.md'))
+        self.assertEqual(('this', 'manymanymany', 'meh', '!!!!!', 'okay', '.md'), match.groups())
+
+    def test_nested_capture_groups(self):
+        """Test nested capture groups."""
+
+        gpat = glob.translate("@(file)@(+([[:digit:]]))@(.*)", flags=glob.E)
+        pat = re.compile(gpat[0][0])
+        match = pat.match('file33.test.txt')
+        self.assertEqual(('file', '33', '33', '.test.txt'), match.groups())
+
+    def test_list_groups(self):
+        """Test capture groups with lists."""
+
+        gpat = glob.translate("+(f|i|l|e)+([[:digit:]])@(.*)", flags=glob.E)
+        pat = re.compile(gpat[0][0])
+        match = pat.match('file33.test.txt')
+        self.assertEqual(('file', '33', '.test.txt'), match.groups())
+
     def test_glob_translate(self):
         """Test glob translation."""