Repository: Pycord-Development/pycord

Issue:
`discord.utils.escape_markdown` not escaping all links
### Summary

I have an input string with many markdown links that I would like to be escaped but only the first link is escaped.

### Reproduction Steps

Here's a sample input string

```
Tweaks: [List 1](https://canary.discord.com/channels/349243932447604736/688126500834246709/809439829116190750), [List 2](https://canary.discord.com/channels/349243932447604736/688126500834246709/835113300298956840), [List 3](https://canary.discord.com/channels/349243932447604736/688126500834246709/837451081779904532), [List 4](https://canary.discord.com/channels/349243932447604736/688126500834246709/841385662121115698), [List 5](https://canary.discord.com/channels/349243932447604736/688126500834246709/848978123077058660), [List 6](https://canary.discord.com/channels/349243932447604736/688126500834246709/855199876299227156), [List 7](https://canary.discord.com/channels/349243932447604736/688126500834246709/859784264723595285), [List 8](https://canary.discord.com/channels/349243932447604736/688126500834246709/866313717591048192), [List 9](https://canary.discord.com/channels/349243932447604736/688126500834246709/871379038789337108), [List 10](https://canary.discord.com/channels/349243932447604736/688126500834246709/873896972963164210), [List 11](https://canary.discord.com/channels/349243932447604736/688126500834246709/890662045933006869), [List 12](https://canary.discord.com/channels/349243932447604736/688126500834246709/895017520883462194), [List 13](https://canary.discord.com/channels/349243932447604736/688126500834246709/905283480626278420).
```

### Minimal Reproducible Code

_No response_

### Expected Results

Backslashes added before all links:

```
Tweaks: \[List 1](https://canary.discord.com/channels/349243932447604736/688126500834246709/809439829116190750), \[List 2](https://canary.discord.com/channels/349243932447604736/688126500834246709/835113300298956840), \[List 3](https://canary.discord.com/channels/349243932447604736/688126500834246709/837451081779904532), \[List 4](https://canary.discord.com/channels/349243932447604736/688126500834246709/841385662121115698), \[List 5](https://canary.discord.com/channels/349243932447604736/688126500834246709/848978123077058660), \[List 6](https://canary.discord.com/channels/349243932447604736/688126500834246709/855199876299227156), \[List 7](https://canary.discord.com/channels/349243932447604736/688126500834246709/859784264723595285), \[List 8](https://canary.discord.com/channels/349243932447604736/688126500834246709/866313717591048192), \[List 9](https://canary.discord.com/channels/349243932447604736/688126500834246709/871379038789337108), \[List 10](https://canary.discord.com/channels/349243932447604736/688126500834246709/873896972963164210), \[List 11](https://canary.discord.com/channels/349243932447604736/688126500834246709/890662045933006869), \[List 12](https://canary.discord.com/channels/349243932447604736/688126500834246709/895017520883462194), \[List 13](https://canary.discord.com/channels/349243932447604736/688126500834246709/905283480626278420).
```

### Actual Results

Backslash is only added before first link:

```
Tweaks: \[List 1](https://canary.discord.com/channels/349243932447604736/688126500834246709/809439829116190750), [List 2](https://canary.discord.com/channels/349243932447604736/688126500834246709/835113300298956840), [List 3](https://canary.discord.com/channels/349243932447604736/688126500834246709/837451081779904532), [List 4](https://canary.discord.com/channels/349243932447604736/688126500834246709/841385662121115698), [List 5](https://canary.discord.com/channels/349243932447604736/688126500834246709/848978123077058660), [List 6](https://canary.discord.com/channels/349243932447604736/688126500834246709/855199876299227156), [List 7](https://canary.discord.com/channels/349243932447604736/688126500834246709/859784264723595285), [List 8](https://canary.discord.com/channels/349243932447604736/688126500834246709/866313717591048192), [List 9](https://canary.discord.com/channels/349243932447604736/688126500834246709/871379038789337108), [List 10](https://canary.discord.com/channels/349243932447604736/688126500834246709/873896972963164210), [List 11](https://canary.discord.com/channels/349243932447604736/688126500834246709/890662045933006869), [List 12](https://canary.discord.com/channels/349243932447604736/688126500834246709/895017520883462194), [List 13](https://canary.discord.com/channels/349243932447604736/688126500834246709/905283480626278420).
```

### Intents

Default + members, messages, presences

### System Information

```
- Python v3.10.0-final
- py-cord v2.0.0-alpha
    - py-cord pkg_resources: v2.0.0a4596+g0fe9b0a2
- aiohttp v3.7.4.post0
- system info: Linux 5.10.47-linuxkit #1 SMP PREEMPT Sat Jul 3 21:50:16 UTC 2021
- 
```

### Checklist

- [X] I have searched the open issues for duplicates.
- [X] I have shown the entire traceback, if possible.
- [X] I have removed my token from display, if visible.

### Additional Context

_No response_


Discussion:
This bug appears to only affect links that are on the same line. E.g., the following string:

```
[link 1](https://example.org)
[link 2](https://example.org)
[link 3](https://example.org)
```
is escaped as:
```
\[link 1](https://example.org)
\[link 2](https://example.org)
\[link 3](https://example.org)
```

Whereas the following string:
```
[link 1](https://example.org)
[link 2](https://example.org)
[link 3](https://example.org), [link 4](https://example.org)
```
gets escaped as:
```
\[link 1](https://example.org)
\[link 2](https://example.org)
\[link 3](https://example.org), [link 4](https://example.org)
```

I used a debugger to get the actual value of `regex` in line 811 of `utils.py`:

https://github.com/Pycord-Development/pycord/blob/552dccfbf3ff8cc64d232471fb6b8a3cdd7ac294/discord/utils.py#L799-L814

The value of `regex` is:
```regex
/(?:(?P<url><[^: >]+:\/[^ >]+>|(?:https?|steam):\/\/[^\s<]+[^<.,:;\"\\'\]\s])|(?P<markdown>[_\\~|\*`]|^>(?:>>)?\s|\[.+\]\(.+\)))/gm
```
Which is kind of inscrutable, but I used regex101 to isolate the relevant portion to the "markdown" named capturing group. (Linebreaks and indentation added for clarification.)
```
(?P<markdown>
    [_\\~|\*`]
    |^>(?:>>)?\s
    |\[.+\]\(.+\)
)
```
Specifically, the pattern `\[.+\]\(.+\)` (fourth line) is causing trouble; as you can see from regex101, the greediness of the `+` quantifier means that the following example is considered one big link instead of two links.

<img width="691" alt="image" src="https://user-images.githubusercontent.com/2716970/148126180-1ead8719-d061-4e11-85b9-de034793aa36.png">

This can be fixed by changing the character class from `.` (any character) to something that won't match beyond the closing `]`.

One solution might be to disallow `]` (likewise with `)` in the link destination)  in the link text, so that `]` signals the closing of the original text and we know not to consider anything further as link text. (You could do this by replacing `.+` with the character class `[^\]]+`.) Unfortunately, the [GitHub-flavored Markdown spec](https://github.github.com/gfm/#links) notes that square brackets are allowed in links subject to certain rules.

Happy to take a crack at this PR myself later, but the bottom line is:
- our current Markdown escaping solution isn't really spec-compliant
- it will take a little more effort (recursive patterns? regex alternative?) to do this correctly

Resolving patch:
diff --git a/discord/utils.py b/discord/utils.py
--- a/discord/utils.py
+++ b/discord/utils.py
@@ -728,9 +728,22 @@ def resolve_template(code: Union[Template, str]) -> str:
 
 _MARKDOWN_ESCAPE_SUBREGEX = '|'.join(r'\{0}(?=([\s\S]*((?<!\{0})\{0})))'.format(c) for c in ('*', '`', '_', '~', '|'))
 
-_MARKDOWN_ESCAPE_COMMON = r'^>(?:>>)?\s|\[.+\]\(.+\)'
-
-_MARKDOWN_ESCAPE_REGEX = re.compile(fr'(?P<markdown>{_MARKDOWN_ESCAPE_SUBREGEX}|{_MARKDOWN_ESCAPE_COMMON})', re.MULTILINE)
+# regular expression for finding and escaping links in markdown
+# note: technically, brackets are allowed in link text.
+# perhaps more concerningly, parentheses are also allowed in link destination.
+# this regular expression matches neither of those.
+# this page provides a good reference: http://blog.michaelperrin.fr/2019/02/04/advanced-regular-expressions/
+_MARKDOWN_ESCAPE_LINKS = r"""
+\[  # matches link text
+    [^\[\]]* # link text can contain anything but brackets 
+\]
+\(  # matches link destination
+    [^\(\)]+ # link destination cannot contain parentheses
+\)""" # note 2: make sure this regex is consumed in re.X (extended mode) since it has whitespace and comments
+
+_MARKDOWN_ESCAPE_COMMON = fr'^>(?:>>)?\s|{_MARKDOWN_ESCAPE_LINKS}'
+
+_MARKDOWN_ESCAPE_REGEX = re.compile(fr'(?P<markdown>{_MARKDOWN_ESCAPE_SUBREGEX}|{_MARKDOWN_ESCAPE_COMMON})', re.MULTILINE | re.X)
 
 _URL_REGEX = r'(?P<url><[^: >]+:\/[^ >]+>|(?:https?|steam):\/\/[^\s<]+[^<.,:;\"\'\]\s])'
 
@@ -808,7 +821,7 @@ def replacement(match):
         regex = _MARKDOWN_STOCK_REGEX
         if ignore_links:
             regex = f'(?:{_URL_REGEX}|{regex})'
-        return re.sub(regex, replacement, text, 0, re.MULTILINE)
+        return re.sub(regex, replacement, text, 0, re.MULTILINE | re.X)
     else:
         text = re.sub(r'\\', r'\\\\', text)
         return _MARKDOWN_ESCAPE_REGEX.sub(r'\\\1', text)


Test patch: