Repository: Python-Markdown/markdown (python)

Issue:
Backslash escape not unescaped in ID attributes
I'm using the `TocExtension` to generate table of contens for my `.md. files (which could be relevant/irrelevant to the issue below).

In my original text, I'm escaping the single underscore `_` with a backslash:

```markdown
### select\_related
```

After running the text through `markdown`, this is the HTML result:

```HTML
<h3 id="select95related">select_related</h3>
```

And the generated TOC points to the ID above as well. I'm wondering if this is the expected behaviour? I would expect `markdown` to simply render:

```HTML
<h3 id="select_related">select_related</h3>
```

At least according to [one online converter](https://www.browserling.com/tools/markdown-to-html), the result should be: `<...id="select_related"...>`

 

Am I missing something?

Pull request:
Unescape IDs in TOC.
The slugify function will stript the STX and ETX characters from
placeholders for backslash excaped characters. Therefore, we need
to unescape any text before passing it to slugify. Fixes #864.

Patch:
diff --git a/markdown/extensions/toc.py b/markdown/extensions/toc.py
index 6d45239..7629ee9 100644
--- a/markdown/extensions/toc.py
+++ b/markdown/extensions/toc.py
@@ -16,6 +16,7 @@ License: [BSD](https://opensource.org/licenses/bsd-license.php)
 from . import Extension
 from ..treeprocessors import Treeprocessor
 from ..util import etree, parseBoolValue, AMP_SUBSTITUTE, HTML_PLACEHOLDER_RE
+from ..postprocessors import UnescapePostprocessor
 import re
 import unicodedata
 
@@ -56,6 +57,12 @@ def stashedHTML2text(text, md):
     return HTML_PLACEHOLDER_RE.sub(_html_sub, text)
 
 
+def unescape(text):
+    """ Unescape escaped text. """
+    c = UnescapePostprocessor()
+    return c.run(text)
+
+
 def nest_toc_tokens(toc_list):
     """Given an unsorted list with errors and skips, return a nested one.
     [{'level': 1}, {'level': 2}]
@@ -242,7 +249,7 @@ class TocTreeprocessor(Treeprocessor):
 
                 # Do not override pre-existing ids
                 if "id" not in el.attrib:
-                    innertext = stashedHTML2text(text, self.md)
+                    innertext = unescape(stashedHTML2text(text, self.md))
                     el.attrib["id"] = unique(self.slugify(innertext, self.sep), used_ids)
 
                 toc_tokens.append({


Test patch:
diff --git a/tests/test_syntax/extensions/test_toc.py b/tests/test_syntax/extensions/test_toc.py
new file mode 100644
index 0000000..65223aa
--- /dev/null
+++ b/tests/test_syntax/extensions/test_toc.py
@@ -0,0 +1,34 @@
+"""
+Python Markdown
+
+A Python implementation of John Gruber's Markdown.
+
+Documentation: https://python-markdown.github.io/
+GitHub: https://github.com/Python-Markdown/markdown/
+PyPI: https://pypi.org/project/Markdown/
+
+Started by Manfred Stienstra (http://www.dwerg.net/).
+Maintained for a few years by Yuri Takhteyev (http://www.freewisdom.org).
+Currently maintained by Waylan Limberg (https://github.com/waylan),
+Dmitry Shachnev (https://github.com/mitya57) and Isaac Muse (https://github.com/facelessuser).
+
+Copyright 2007-2019 The Python Markdown Project (v. 1.7 and later)
+Copyright 2004, 2005, 2006 Yuri Takhteyev (v. 0.2-1.6b)
+Copyright 2004 Manfred Stienstra (the original version)
+
+License: BSD (see LICENSE.md for details).
+"""
+
+from markdown.test_tools import TestCase
+
+
+class TestTOC(TestCase):
+
+    # TODO: Move the rest of the TOC tests here.
+
+    def test_escaped_char_in_id(self):
+        self.assertMarkdownRenders(
+            r'# escaped\_character',
+            '<h1 id="escaped_character">escaped_character</h1>',
+            extensions=['toc']
+        )