Repository: django-wiki/django-wiki

Issue:
Python 3.11 support: global flags not at the start of the expression at position 9
hi, for all the pages in /wiki/ same error raises, "global flags not at the start of the expression at position 9"
do you have any advice?

```python
Internal Server Error: /wiki/manual/

error at /wiki/manual/
global flags not at the start of the expression at position 9

Request Method: GET
Request URL: https://forum.pc5bai.com/wiki/manual/
Django Version: 4.0.8
Python Executable: /home/forumapp/dsite/venv/bin/python3
Python Version: 3.11.0
```

below is the latter part of the stacktrace:
```python
  File "/home/myuser/myapp/venv/lib/python3.11/site-packages/wiki/models/article.py", line 218, in render
    article_markdown(
  File "/home/myuser/myapp/venv/lib/python3.11/site-packages/wiki/core/markdown/__init__.py", line 52, in article_markdown
    md = ArticleMarkdown(article, *args, **kwargs)
  File "/home/myuser/myapp/venv/lib/python3.11/site-packages/wiki/core/markdown/__init__.py", line 11, in __init__
    super().__init__(*args, **kwargs)
  File "/home/myuser/myapp/venv/lib/python3.11/site-packages/markdown/core.py", line 96, in __init__
    self.registerExtensions(extensions=kwargs.get('extensions', []),
  File "/home/myuser/myapp/venv/lib/python3.11/site-packages/markdown/core.py", line 125, in registerExtensions
    ext._extendMarkdown(self)
  File "/home/myuser/myapp/venv/lib/python3.11/site-packages/markdown/extensions/__init__.py", line 77, in _extendMarkdown
    self.extendMarkdown(md)
  File "/home/myuser/myapp/venv/lib/python3.11/site-packages/wiki/plugins/images/markdown_extensions.py", line 36, in extendMarkdown
    md.inlinePatterns, "dw-images", ImagePattern(IMAGE_RE, md), ">link"
  File "/home/myuser/myapp/venv/lib/python3.11/site-packages/markdown/inlinepatterns.py", line 209, in __init__
    self.compiled_re = re.compile(r"^(.*?)%s(.*)$" % pattern,
  File "/usr/local/lib/python3.11/re/__init__.py", line 227, in compile
    return _compile(pattern, flags)
  File "/usr/local/lib/python3.11/re/__init__.py", line 294, in _compile
    p = _compiler.compile(pattern, flags)
  File "/usr/local/lib/python3.11/re/_compiler.py", line 743, in compile
    p = _parser.parse(p, flags)
  File "/usr/local/lib/python3.11/re/_parser.py", line 980, in parse
    p = _parse_sub(source, state, flags & SRE_FLAG_VERBOSE, 0)
  File "/usr/local/lib/python3.11/re/_parser.py", line 455, in _parse_sub
    itemsappend(_parse(source, state, verbose, nested + 1,
  File "/usr/local/lib/python3.11/re/_parser.py", line 863, in _parse
    p = _parse_sub(source, state, sub_verbose, nested + 1)
  File "/usr/local/lib/python3.11/re/_parser.py", line 455, in _parse_sub
    itemsappend(_parse(source, state, verbose, nested + 1,
  File "/usr/local/lib/python3.11/re/_parser.py", line 841, in _parse
    raise source.error('global flags not at the start '

Exception Type: error at /wiki/manual/
Exception Value: global flags not at the start of the expression at position 9
```

recently I updated python from 3.10 to 3.11. 



Discussion:
maybe below is related to my issune.
https://stackoverflow.com/questions/74695525/django-wiki-installed-but-root-article-throws-global-flag-error

it seems that inlinepatterns.py line 209 below always fails.

re.compile(r"^(.*?)%s(.*)$" % pattern, re.DOTALL | re.UNICODE)

evaluation result for string 'r"^(.*?)%s(.*)$" % pattern' is below:
'^(.*?)(?:(?im)\\[image\\:(?P<id>[0-9]+)(?:\\s+align\\:(?P<align>right|left))?(?:\\s+size\\:(?P<size>default|small|medium|large|orig))?\\s*\\](?P<trailer>[^\\n]*)$(?P<caption>(?:\\n    [^\\n]*)*))(.*)$'


so I tried to check script below and got same result.  
```python
import re

_pattern = '^(.*?)(?:(?im)\\[image\\:(?P<id>[0-9]+)(?:\\s+align\\:(?P<align>right|left))?(?:\\s+size\\:(?P<size>default|small|medium|large|orig))?\\s*\\](?P<trailer>[^\\n]*)$(?P<caption>(?:\\n    [^\\n]*)*))(.*)$'
result = re.compile(_pattern)

>>> re.error: global flags not at the start of the expression at position 9
```

is this issue relate to new features in Python 3.11?
https://docs.python.org/3/whatsnew/3.11.html#re

```
Atomic grouping ((?>...)) and possessive quantifiers (*+, ++, ?+, {m,n}+) are now supported in regular expressions. (Contributed by Jeffrey C. Jacobs and Serhiy Storchaka in bpo-433030.)
```

Hey @k-brahma thanks for the detailed report!

At this moment we do not support Python 3.11 on Django-Wiki, I'll leave this issue open and will check what's going on with this regex.
Oh god :exploding_head: 

I was doing a search of the changelog on the pattern "regular" :smile:  https://docs.python.org/3/whatsnew/changelog.html - There are these possibly related changes in Python 3.11

* `https://github.com/python/cpython/issues/91760`
* `https://github.com/python/cpython/issues/91700`
* `https://github.com/python/cpython/issues/98740`
* `https://github.com/python/cpython/issues/91222`

I think it's the latter of the bunch... 91222?

> Convert a warning about flags not at the start of the regular expression into error

Python 3.11 seems to have elevated a previous Warning to an Exception. The error message seems to match what we are getting here. Didn't notice the warning from before, but guessing it must have been buried somewhere.

Resolving patch:
diff --git a/src/wiki/plugins/images/markdown_extensions.py b/src/wiki/plugins/images/markdown_extensions.py
--- a/src/wiki/plugins/images/markdown_extensions.py
+++ b/src/wiki/plugins/images/markdown_extensions.py
@@ -1,3 +1,5 @@
+import re
+
 import markdown
 from django.template.loader import render_to_string
 from wiki.core.markdown import add_to_registry
@@ -5,7 +7,7 @@
 from wiki.plugins.images import settings
 
 IMAGE_RE = (
-    r"(?:(?im)"
+    r"(?:"
     +
     # Match '[image:N'
     r"\[image\:(?P<id>[0-9]+)"
@@ -53,6 +55,14 @@ class ImagePattern(markdown.inlinepatterns.Pattern):
     So: Remember that the caption text is fully valid markdown!
     """
 
+    def __init__(self, pattern, md=None):
+        """Override init in order to add IGNORECASE and MULTILINE flags"""
+        super().__init__(pattern, md=md)
+        self.compiled_re = re.compile(
+            r"^(.*?)%s(.*)$" % pattern,
+            flags=re.DOTALL | re.UNICODE | re.IGNORECASE | re.MULTILINE,
+        )
+
     def handleMatch(self, m):
         image = None
         image_id = None
diff --git a/src/wiki/plugins/macros/mdx/macro.py b/src/wiki/plugins/macros/mdx/macro.py
--- a/src/wiki/plugins/macros/mdx/macro.py
+++ b/src/wiki/plugins/macros/mdx/macro.py
@@ -10,7 +10,7 @@
 # http://stackoverflow.com/questions/430759/regex-for-managing-escaped-characters-for-items-like-string-literals
 re_sq_short = r"'([^'\\]*(?:\\.[^'\\]*)*)'"
 
-MACRO_RE = r"((?i)\[(?P<macro>\w+)(?P<kwargs>\s\w+\:.+)*\])"
+MACRO_RE = r"(\[(?P<macro>\w+)(?P<kwargs>\s\w+\:.+)*\])"
 KWARG_RE = re.compile(
     r"\s*(?P<arg>\w+)(:(?P<value>([^\']+|%s)))?" % re_sq_short, re.IGNORECASE
 )
@@ -32,6 +32,13 @@ class MacroPattern(markdown.inlinepatterns.Pattern):
     """django-wiki macro preprocessor - parse text for various [some_macro] and
     [some_macro (kw:arg)*] references."""
 
+    def __init__(self, pattern, md=None):
+        """Override init in order to add IGNORECASE flag"""
+        super().__init__(pattern, md=md)
+        self.compiled_re = re.compile(
+            r"^(.*?)%s(.*)$" % pattern, flags=re.DOTALL | re.UNICODE | re.IGNORECASE
+        )
+
     def handleMatch(self, m):
         macro = m.group("macro").strip()
         if macro not in settings.METHODS or not hasattr(self, macro):


Test patch: