Repository: Python-Markdown/markdown (python)

Issue:
toc: Unusual characters in heading ids not well supported
I noticed that `toc` encodes characters like `*` as `\x0242\x03`, 42 being the index of `*` in the ASCII table. This causes a discrepancy between the permalink of a heading and the link in the table of contents.

```bash
mkdir /tmp/toc
cd /tmp/toc
python -m venv .venv
. .venv/bin/activate
python -m pip install mkdocs
python -m mkdocs new .
```

index file:

```md
# Welcome

Demonstrating an issue with HTML ids and `toc`.

## `*Foo*` { id="\*Foo\*" }

- Click on `*Foo*`'s permalink: `#*Foo*` in the URL.
- Click on `*Foo*` in the table of contents: `#%0242%03Foo%0242%03` in the URL.
```

mkdocs config:

```yaml
site_name: My Docs

markdown_extensions:
- attr_list
- toc:
    permalink: true
```

Serve and observe the behavior described in the index page.

I'm not saying this is a bug. I'm just curious if this is expected, and whether there would be a way improve support for headings with such "unusual" ids. This would help for the work I'm doing with mkdocstrings, where we try to expand our languages support, and some languages might use uncommon characters in object identifiers. Not only `toc` would have to work, but also mkdocs-autorefs, which picks up ids from the table of contents when registering URLs and anchors to objects.

I believe HTML5 supports any kind of characters in ids. Some of them just cause a bit of pain, like `.` or `#`, because they then need to be escaped in CSS selectors.

Pull request:
Backslash Unescape IDs set via `attr_list` on `toc`
Fixes #1493.

Patch:
diff --git a/docs/changelog.md b/docs/changelog.md
index 5ae2546..5ca092b 100644
--- a/docs/changelog.md
+++ b/docs/changelog.md
@@ -14,6 +14,10 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
 
 * DRY fix in `abbr` extension by introducing method `create_element` (#1483).
 
+### Fixed
+
+* Backslash Unescape IDs set via `attr_list` on `toc` (#1493).
+
 ## [3.7] -- 2024-08-16
 
 ### Changed
diff --git a/markdown/extensions/toc.py b/markdown/extensions/toc.py
index 5462a90..c92f928 100644
--- a/markdown/extensions/toc.py
+++ b/markdown/extensions/toc.py
@@ -391,7 +391,7 @@ class TocTreeprocessor(Treeprocessor):
                 if int(el.tag[-1]) >= self.toc_top and int(el.tag[-1]) <= self.toc_bottom:
                     toc_tokens.append({
                         'level': int(el.tag[-1]),
-                        'id': el.attrib["id"],
+                        'id': unescape(el.attrib["id"]),
                         'name': name,
                         'html': innerhtml,
                         'data-toc-label': data_toc_label


Test patch:
diff --git a/tests/test_syntax/extensions/test_attr_list.py b/tests/test_syntax/extensions/test_attr_list.py
index e9a1096..1a0ae12 100644
--- a/tests/test_syntax/extensions/test_attr_list.py
+++ b/tests/test_syntax/extensions/test_attr_list.py
@@ -72,6 +72,12 @@ class TestAttrList(TestCase):
             '<h1 foo="&quot;bar">Heading</h1>'
         )
 
+    def test_backslash_escape_value(self):
+        self.assertMarkdownRenders(
+            '# `*Foo*` { id="\\*Foo\\*" }',
+            '<h1 id="*Foo*"><code>*Foo*</code></h1>'
+        )
+
     def test_table_td(self):
         self.assertMarkdownRenders(
             self.dedent(
diff --git a/tests/test_syntax/extensions/test_toc.py b/tests/test_syntax/extensions/test_toc.py
index 9902072..372ba09 100644
--- a/tests/test_syntax/extensions/test_toc.py
+++ b/tests/test_syntax/extensions/test_toc.py
@@ -974,6 +974,32 @@ class TestTOC(TestCase):
             extensions=['toc']
         )
 
+    def test_escaped_char_in_attr_list(self):
+        self.assertMarkdownRenders(
+            r'# `*Foo*` { id="\*Foo\*" }',
+            '<h1 id="*Foo*"><code>*Foo*</code></h1>',
+            expected_attrs={
+                'toc': (
+                    '<div class="toc">\n'
+                      '<ul>\n'                                                           # noqa
+                        '<li><a href="#*Foo*">*Foo*</a></li>\n'                          # noqa
+                      '</ul>\n'                                                          # noqa
+                    '</div>\n'                                                           # noqa
+                ),
+                'toc_tokens': [
+                    {
+                        'level': 1,
+                        'id': '*Foo*',
+                        'name': '*Foo*',
+                        'html': '<code>*Foo*</code>',
+                        'data-toc-label': '',
+                        'children': []
+                    }
+                ]
+            },
+            extensions=['toc', 'attr_list']
+        )
+
     def testAutoLinkEmail(self):
         self.assertMarkdownRenders(
             '## <foo@example.org>',