Repository: sopel-irc/sopel

Issue:
"You[apos]re" result from Sopel's URL parser
<!-- Before reporting a bug, please search both open *and closed* issues to
see if it has already been reported. If you can, try to reproduce the problem
on an unmodified copy of the `master` branch first, as sometimes bugs are found
and fixed without a report. If the problem is unreported and persists in
`master`, please help us fix it quickly by filling out as much of this
information as you can. Thanks! -->

### Description
"You[apos]re" is shown for "You're" in a https://matrix.to/#/#element-community-testing:matrix.orghttps://matrix.to/#/#element-community-testing:matrix.org link.

### Reproduction steps
1. Copy and paste https://matrix.to/#/#element-community-testing:matrix.orghttps://matrix.to/#/#element-community-testing:matrix.org for Sopel bot to parse.
2. See Sopel's parsed URL results for "You[apos]re".

### Expected behavior
I should see "You're" instead of "You[apos]re".

### Logs
I don't see anything interesting in ~/.sopel/logs?

### Environment
- Sopel `.version`: 7.1.5
- Sopel installed via: pip
- Python version: 3.9.7
- Operating system: 64-bit Fedora v33
- IRCd `/version`: ircd-ratbox-3.0.8(20121231_3-27427)
- Relevant plugins: mostly defaults

### Notes
Chat log from my IRC client in my Debian Jessie v8 box:
...
06:14PM Mousey> but
          https://matrix.to/#/#element-community-testing:matrix.orghttps://matrix.to/#/#element-community-testing:matrix.org
06:14PM URL> [url] You[apos]re invited to talk on Matrix | matrix.to
06:14PM Mousey> but https://matrix.to/#/#element-community-testing:matrix.org is for
          community testing!
06:14PM URL> [url] You[apos]re invited to talk on Matrix | matrix.to
[06:14pm] * RecoveryAnty upvotes without watching.
[06:15pm] RecoveryAnty> weird
[06:15pm] RecoveryAnty> You[apos]re
[06:15pm] RecoveryAnty> Can't even show apostrophe?
06:15PM Mousey> that would be hexchat's fault
[06:15pm] RecoveryAnty> Uh...
[06:16pm] RecoveryAnty>
          https://matrix.to/#/#element-community-testing:matrix.orghttps://matrix.to/#/#element-community-testing:matrix.org
06:16PM URL> [url] You[apos]re invited to talk on Matrix | matrix.to
[06:16pm] RecoveryAnty> nope
[06:16pm] RecoveryAnty> BitchX here.
[06:16pm] RecoveryAnty> Sopel's fault I bet.
...


Discussion:

Resolving patch:
diff --git a/sopel/tools/web.py b/sopel/tools/web.py
--- a/sopel/tools/web.py
+++ b/sopel/tools/web.py
@@ -34,6 +34,11 @@
     unichr = chr
     unicode = str
 
+try:
+    import html as HTMLLib
+except ImportError:
+    HTMLLib = None
+
 __all__ = [
     'USER_AGENT',
     'DEFAULT_HEADERS',
@@ -121,6 +126,16 @@ def decode(html):
     :param str html: the HTML page or snippet to process
     :return str: ``html`` with all entity references replaced
     """
+    if HTMLLib is not None:
+        # Python's stdlib has our back in 3.4+
+        # TODO: This should be the only implementation in Sopel 8
+        try:
+            return HTMLLib.unescape(html)
+        except AttributeError:
+            # Must be py3.3; fall through to our half-assed version.
+            pass
+
+    # Not on py3.4+ yet? Then you get to deal with the jankiness.
     return r_entity.sub(entity, html)
 
 


Test patch: