Repository: Python-Markdown/markdown
Pull request #1165 (2021-08-01 08:26:16)

Require a local-part to be a valid email address in AUTOMAIL_RE.
Valid email addresses require a _local-part_ before the `@` followed by _domain_.  This PR updates the `AUTOMAIL_RE` to use the `+` quantifier to require 1 or more characters in the _local-part_ instead of the `*` quantifier which allows 0 characters.

The current regex allows the invalid addresses in the form `<@domain>`.  My particular use case is formatting messages in Slack, where mentioned users take the form `<@Uxxxxx>`.  markdown is incorrectly processing the mention string form as an email address.  I still want markdown's automail feature for processing email addressing in the text.

The below code demonstrates the current and new behaviour:

```
>>> AUTOMAIL_RE = r'<([^<> !]*@[^@<> ]*)>'
>>> print(re.match(AUTOMAIL_RE, "<@localhost>"))
<re.Match object; span=(0, 12), match='<@localhost>'>
>>> print(re.match(AUTOMAIL_RE, "<bob@localhost>"))
<re.Match object; span=(0, 15), match='<bob@localhost>'>

>>> AUTOMAIL_RE = r'<([^<> !]+@[^@<> ]*)>'
>>> print(re.match(AUTOMAIL_RE, "<@localhost>"))
None
>>> print(re.match(AUTOMAIL_RE, "<bob@localhost>"))
<re.Match object; span=(0, 15), match='<bob@localhost>'>
```


mitya57: Make it makes sense to also replace `*` with `+` in the domain part? With your fix it still allows for addresses like `bob@`:

```python
>>> AUTOMAIL_RE = r'<([^<> !]+@[^@<> ]*)>'
>>> re.match(AUTOMAIL_RE, "<bob@>")
<re.Match object; span=(0, 6), match='<bob@>'>
```
nzlosh: I've updated the regex to require both `local-part` and `domain`.
nzlosh: I've added tests for the missing local-part and domain.  I noticed that the `<>` characters are encode in html form `&lt;&gt;` for the missing local-part, but not for the missing domain part.  The tests mirror this behaviour, but I don't know if it's expected behaviour or not. 
nzlosh: Thanks for the pointers, I've updated the PR with the recommended changes.

Patch:
diff --git a/docs/change_log/index.md b/docs/change_log/index.md
index 11c997b..fdf213b 100644
--- a/docs/change_log/index.md
+++ b/docs/change_log/index.md
@@ -6,9 +6,10 @@ Python-Markdown Change Log
 Under development: version 3.3.5 (a bug-fix release).
 
 * Make the `slugify_unicode` function not remove diacritical marks (#1118).
-* Fix `[toc]` detection when used with `nl2br` extension (#1160)
-* Re-use compiled regex for block level checks (#1169)
+* Fix `[toc]` detection when used with `nl2br` extension (#1160).
+* Re-use compiled regex for block level checks (#1169).
 * Don't process shebangs in fenced code blocks when using CodeHilite (#1156).
+* Improve email address validation for Automatic Links (#1165).
 
 Feb 24, 2021: version 3.3.4 (a bug-fix release).
 
diff --git a/markdown/inlinepatterns.py b/markdown/inlinepatterns.py
index b0621a8..f7d604e 100644
--- a/markdown/inlinepatterns.py
+++ b/markdown/inlinepatterns.py
@@ -160,10 +160,10 @@ NOT_STRONG_RE = r'((^|\s)(\*|_)(\s|$))'
 AUTOLINK_RE = r'<((?:[Ff]|[Hh][Tt])[Tt][Pp][Ss]?://[^<>]*)>'
 
 # <me@example.com>
-AUTOMAIL_RE = r'<([^<> !]*@[^@<> ]*)>'
+AUTOMAIL_RE = r'<([^<> !]+@[^@<> ]+)>'
 
 # <...>
-HTML_RE = r'(<([a-zA-Z/][^<>]*|!--(?:(?!<!--|-->).)*--)>)'
+HTML_RE = r'(<(\/?[a-zA-Z][^<>@ ]*( [^<>]*)?|!--(?:(?!<!--|-->).)*--)>)'
 
 # "&#38;" (decimal) or "&#x26;" (hex) or "&amp;" (named)
 ENTITY_RE = r'(&(?:\#[0-9]+|\#x[0-9a-fA-F]+|[a-zA-Z0-9]+);)'


Test patch:
diff --git a/tests/test_syntax/inline/test_autolinks.py b/tests/test_syntax/inline/test_autolinks.py
new file mode 100644
index 0000000..b6bd1cf
--- /dev/null
+++ b/tests/test_syntax/inline/test_autolinks.py
@@ -0,0 +1,63 @@
+"""
+Python Markdown
+
+A Python implementation of John Gruber's Markdown.
+
+Documentation: https://python-markdown.github.io/
+GitHub: https://github.com/Python-Markdown/markdown/
+PyPI: https://pypi.org/project/Markdown/
+
+Started by Manfred Stienstra (http://www.dwerg.net/).
+Maintained for a few years by Yuri Takhteyev (http://www.freewisdom.org).
+Currently maintained by Waylan Limberg (https://github.com/waylan),
+Dmitry Shachnev (https://github.com/mitya57) and Isaac Muse (https://github.com/facelessuser).
+
+Copyright 2007-2021 The Python Markdown Project (v. 1.7 and later)
+Copyright 2004, 2005, 2006 Yuri Takhteyev (v. 0.2-1.6b)
+Copyright 2004 Manfred Stienstra (the original version)
+
+License: BSD (see LICENSE.md for details).
+"""
+
+from markdown.test_tools import TestCase
+
+
+class TestAutomaticLinks(TestCase):
+
+    def test_email_address(self):
+        self.assertMarkdownRenders(
+            'asdfasdfadsfasd <yuri@freewisdom.org> or you can say ',
+            '<p>asdfasdfadsfasd <a href="&#109;&#97;&#105;&#108;&#116;&#111;&#58;&#121;&#117;&#114;'
+            '&#105;&#64;&#102;&#114;&#101;&#101;&#119;&#105;&#115;&#100;&#111;&#109;&#46;&#111;&#114;'
+            '&#103;">&#121;&#117;&#114;&#105;&#64;&#102;&#114;&#101;&#101;&#119;&#105;&#115;&#100;'
+            '&#111;&#109;&#46;&#111;&#114;&#103;</a> or you can say </p>'
+        )
+
+    def test_mailto_email_address(self):
+        self.assertMarkdownRenders(
+            'instead <mailto:yuri@freewisdom.org>',
+            '<p>instead <a href="&#109;&#97;&#105;&#108;&#116;&#111;&#58;&#121;&#117;&#114;&#105;&#64;'
+            '&#102;&#114;&#101;&#101;&#119;&#105;&#115;&#100;&#111;&#109;&#46;&#111;&#114;&#103;">'
+            '&#121;&#117;&#114;&#105;&#64;&#102;&#114;&#101;&#101;&#119;&#105;&#115;&#100;&#111;&#109;'
+            '&#46;&#111;&#114;&#103;</a></p>'
+        )
+
+    def test_email_address_with_ampersand(self):
+        self.assertMarkdownRenders(
+            '<bob&sue@example.com>',
+            '<p><a href="&#109;&#97;&#105;&#108;&#116;&#111;&#58;&#98;&#111;&#98;&#38;&#115;&#117;&#101;'
+            '&#64;&#101;&#120;&#97;&#109;&#112;&#108;&#101;&#46;&#99;&#111;&#109;">&#98;&#111;&#98;&amp;'
+            '&#115;&#117;&#101;&#64;&#101;&#120;&#97;&#109;&#112;&#108;&#101;&#46;&#99;&#111;&#109;</a></p>'
+        )
+
+    def test_invalid_email_address_local_part(self):
+        self.assertMarkdownRenders(
+            'Missing local-part <@domain>',
+            '<p>Missing local-part &lt;@domain&gt;</p>'
+        )
+
+    def test_invalid_email_address_domain(self):
+        self.assertMarkdownRenders(
+            'Missing domain <local-part@>',
+            '<p>Missing domain &lt;local-part@&gt;</p>'
+        )