Repository: sopel-irc/sopel

Issue:
"You[apos]re" result from Sopel's URL parser
<!-- Before reporting a bug, please search both open *and closed* issues to
see if it has already been reported. If you can, try to reproduce the problem
on an unmodified copy of the `master` branch first, as sometimes bugs are found
and fixed without a report. If the problem is unreported and persists in
`master`, please help us fix it quickly by filling out as much of this
information as you can. Thanks! -->

### Description
"You[apos]re" is shown for "You're" in a https://matrix.to/#/#element-community-testing:matrix.orghttps://matrix.to/#/#element-community-testing:matrix.org link.

### Reproduction steps
1. Copy and paste https://matrix.to/#/#element-community-testing:matrix.orghttps://matrix.to/#/#element-community-testing:matrix.org for Sopel bot to parse.
2. See Sopel's parsed URL results for "You[apos]re".

### Expected behavior
I should see "You're" instead of "You[apos]re".

### Logs
I don't see anything interesting in ~/.sopel/logs?

### Environment
- Sopel `.version`: 7.1.5
- Sopel installed via: pip
- Python version: 3.9.7
- Operating system: 64-bit Fedora v33
- IRCd `/version`: ircd-ratbox-3.0.8(20121231_3-27427)
- Relevant plugins: mostly defaults

### Notes
Chat log from my IRC client in my Debian Jessie v8 box:
...
06:14PM Mousey> but
          https://matrix.to/#/#element-community-testing:matrix.orghttps://matrix.to/#/#element-community-testing:matrix.org
06:14PM URL> [url] You[apos]re invited to talk on Matrix | matrix.to
06:14PM Mousey> but https://matrix.to/#/#element-community-testing:matrix.org is for
          community testing!
06:14PM URL> [url] You[apos]re invited to talk on Matrix | matrix.to
[06:14pm] * RecoveryAnty upvotes without watching.
[06:15pm] RecoveryAnty> weird
[06:15pm] RecoveryAnty> You[apos]re
[06:15pm] RecoveryAnty> Can't even show apostrophe?
06:15PM Mousey> that would be hexchat's fault
[06:15pm] RecoveryAnty> Uh...
[06:16pm] RecoveryAnty>
          https://matrix.to/#/#element-community-testing:matrix.orghttps://matrix.to/#/#element-community-testing:matrix.org
06:16PM URL> [url] You[apos]re invited to talk on Matrix | matrix.to
[06:16pm] RecoveryAnty> nope
[06:16pm] RecoveryAnty> BitchX here.
[06:16pm] RecoveryAnty> Sopel's fault I bet.
...


Discussion:

Resolving patch:
diff --git a/sopel/tools/__init__.py b/sopel/tools/__init__.py
--- a/sopel/tools/__init__.py
+++ b/sopel/tools/__init__.py
@@ -29,7 +29,6 @@
 from sopel import __version__
 
 from ._events import events  # NOQA
-from . import time, web  # NOQA
 
 
 # Kept for backward compatibility
@@ -199,6 +198,13 @@ def deprecated_func(*args, **kwargs):
     return deprecated_func
 
 
+# This has to be *after* the definition of `deprecated()` or we
+# get "AttributeError: partially initialized module 'sopel.tools' has no
+# attribute 'deprecated' (most likely due to a circular import)" when trying
+# to use the decorator in submodules.
+from . import time, web  # NOQA
+
+
 @deprecated('Shim for Python 2 cross-compatibility, no longer needed. '
             'Use built-in `input()` instead.',
             version='7.1',
diff --git a/sopel/tools/web.py b/sopel/tools/web.py
--- a/sopel/tools/web.py
+++ b/sopel/tools/web.py
@@ -3,11 +3,6 @@
 applications, APIs, or websites in your plugins.
 
 .. versionadded:: 7.0
-
-.. note::
-    Some parts of this module will remain accessible through ``sopel.web`` as
-    well until its final removal in Sopel 8. This is for backward
-    compatibility only; please update old code as soon as possible.
 """
 # Copyright © 2008, Sean B. Palmer, inamidst.com
 # Copyright © 2009, Michael Yanovich <yanovich.1@osu.edu>
@@ -18,12 +13,13 @@
 
 from __future__ import generator_stop
 
+import html
 from html.entities import name2codepoint
 import re
 import urllib
 from urllib.parse import urlparse, urlunparse
 
-from sopel import __version__
+from sopel import __version__, tools
 
 
 __all__ = [
@@ -85,9 +81,19 @@
 
 
 r_entity = re.compile(r'&([^;\s]+);')
-"""Regular expression to match HTML entities."""
+"""Regular expression to match HTML entities.
+
+.. deprecated:: 8.0
+
+    Will be removed in Sopel 9, along with :func:`entity`.
+"""
 
 
+@tools.deprecated(
+    version='8.0',
+    removed_in='9.0',
+    reason="No longer needed now that Python 3.4+ has `html.unescape()`",
+)
 def entity(match):
     """Convert an entity reference to the appropriate character.
 
@@ -96,6 +102,12 @@ def entity(match):
     :return str: the Unicode character corresponding to the given ``match``
         string, or a fallback representation if the reference cannot be
         resolved to a character
+
+    .. deprecated:: 8.0
+
+        Will be removed in Sopel 9. Use :func:`decode` directly or migrate to
+        Python's standard-library equivalent, :func:`html.unescape`.
+
     """
     value = match.group(1).lower()
     if value.startswith('#x'):
@@ -107,13 +119,20 @@ def entity(match):
     return '[' + value + ']'
 
 
-def decode(html):
+def decode(text):
     """Decode HTML entities into Unicode text.
 
-    :param str html: the HTML page or snippet to process
-    :return str: ``html`` with all entity references replaced
+    :param str text: the HTML page or snippet to process
+    :return str: ``text`` with all entity references replaced
+
+    .. versionchanged:: 8.0
+
+        Renamed ``html`` parameter to ``text``. (Python gained a standard
+        library module named :mod:`html` in version 3.4.)
+
     """
-    return r_entity.sub(entity, html)
+    # TODO deprecated?
+    return html.unescape(text)
 
 
 def quote(string, safe='/'):


Test patch: