Repository: executablebooks/MyST-Parser
Pull request #715 (2023-02-22 07:11:50)

🐛 FIX encoded URL ID references
For example `<#name with spaces>` will be encoded as `#name%20with%20spaces`, so we need to decode before resolving the reference.

codecov[bot]: # [Codecov](https://codecov.io/gh/executablebooks/MyST-Parser/pull/715?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=executablebooks) Report
Base: **89.36**% // Head: **89.40**% // Increases project coverage by **`+0.03%`** :tada:
> Coverage data is based on head [(`b75636b`)](https://codecov.io/gh/executablebooks/MyST-Parser/pull/715?src=pr&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=executablebooks) compared to base [(`4cac732`)](https://codecov.io/gh/executablebooks/MyST-Parser/commit/4cac73228303670f3466e2fa9bdde844cc58f8cd?el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=executablebooks).
> Patch coverage: 100.00% of modified lines in pull request are covered.

<details><summary>Additional details and impacted files</summary>


```diff
@@            Coverage Diff             @@
##           master     #715      +/-   ##
==========================================
+ Coverage   89.36%   89.40%   +0.03%     
==========================================
  Files          23       23              
  Lines        2867     2868       +1     
==========================================
+ Hits         2562     2564       +2     
+ Misses        305      304       -1     
```

| Flag | Coverage Δ | |
|---|---|---|
| pytests | `89.40% <100.00%> (+0.03%)` | :arrow_up: |

Flags with carried forward coverage won't be shown. [Click here](https://docs.codecov.io/docs/carryforward-flags?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=executablebooks#carryforward-flags-in-the-pull-request-comment) to find out more.

| [Impacted Files](https://codecov.io/gh/executablebooks/MyST-Parser/pull/715?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=executablebooks) | Coverage Δ | |
|---|---|---|
| [myst\_parser/mdit\_to\_docutils/base.py](https://codecov.io/gh/executablebooks/MyST-Parser/pull/715?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=executablebooks#diff-bXlzdF9wYXJzZXIvbWRpdF90b19kb2N1dGlscy9iYXNlLnB5) | `92.88% <100.00%> (ø)` | |
| [myst\_parser/mdit\_to\_docutils/transforms.py](https://codecov.io/gh/executablebooks/MyST-Parser/pull/715?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=executablebooks#diff-bXlzdF9wYXJzZXIvbWRpdF90b19kb2N1dGlscy90cmFuc2Zvcm1zLnB5) | `86.84% <100.00%> (+1.50%)` | :arrow_up: |

Help us with your feedback. Take ten seconds to tell us [how you rate us](https://about.codecov.io/nps?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=executablebooks). Have a feature suggestion? [Share it here.](https://app.codecov.io/gh/feedback/?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=executablebooks)

</details>

[:umbrella: View full report at Codecov](https://codecov.io/gh/executablebooks/MyST-Parser/pull/715?src=pr&el=continue&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=executablebooks).   
:loudspeaker: Do you have feedback about the report comment? [Let us know in this issue](https://about.codecov.io/codecov-pr-comment-feedback/?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=executablebooks).


Patch:
diff --git a/myst_parser/mdit_to_docutils/base.py b/myst_parser/mdit_to_docutils/base.py
index 2c30fe8..7f3d3c6 100644
--- a/myst_parser/mdit_to_docutils/base.py
+++ b/myst_parser/mdit_to_docutils/base.py
@@ -969,7 +969,7 @@ class DocutilsRenderer(RendererProtocol):
         ref_node = nodes.reference()
         self.add_line_and_source_path(ref_node, token)
         ref_node["id_link"] = True
-        ref_node["refuri"] = token.attrGet("href") or ""
+        ref_node["refuri"] = self.md.normalizeLinkText(str(token.attrGet("href") or ""))
         self.copy_attributes(
             token, ref_node, ("class", "id", "reftitle"), aliases={"title": "reftitle"}
         )
diff --git a/myst_parser/mdit_to_docutils/transforms.py b/myst_parser/mdit_to_docutils/transforms.py
index 78f8826..b193ebd 100644
--- a/myst_parser/mdit_to_docutils/transforms.py
+++ b/myst_parser/mdit_to_docutils/transforms.py
@@ -5,6 +5,7 @@ import typing as t
 
 from docutils import nodes
 from docutils.transforms import Transform
+from markdown_it.common.normalize_url import normalizeLink
 
 from myst_parser._compat import findall
 from myst_parser.mdit_to_docutils.base import clean_astext
@@ -138,7 +139,7 @@ class ResolveAnchorIds(Transform):
                 line=refnode.line,
                 append_to=refnode,
             )
-            refnode["refid"] = target
+            refnode["refid"] = normalizeLink(target)
             if not refnode.children:
                 refnode += nodes.inline(
                     "#" + target, "#" + target, classes=["std", "std-ref"]


Test patch:
diff --git a/tests/test_renderers/fixtures/docutil_link_resolution.md b/tests/test_renderers/fixtures/docutil_link_resolution.md
index ee93d2d..f5e39e7 100644
--- a/tests/test_renderers/fixtures/docutil_link_resolution.md
+++ b/tests/test_renderers/fixtures/docutil_link_resolution.md
@@ -15,11 +15,12 @@
             https://www.google.com
 .
 
-[missing]
+[missing] 
 .
 [](#test)
 [...](#test)
 [explicit](#test)
+[](<#name with spaces>)
 .
 <document source="<src>/index.md">
     <paragraph>
@@ -39,10 +40,16 @@
                 <paragraph>
                     'myst' reference target not found: 'test' [myst.xref_missing]
 
+        <reference id_link="True" refid="name%20with%20spaces">
+            <system_message level="2" line="1" source="<src>/index.md" type="WARNING">
+                <paragraph>
+                    'myst' reference target not found: 'name with spaces' [myst.xref_missing]
+
 
 <src>/index.md:1: (WARNING/2) 'myst' reference target not found: 'test' [myst.xref_missing]
 <src>/index.md:1: (WARNING/2) 'myst' reference target not found: 'test' [myst.xref_missing]
 <src>/index.md:1: (WARNING/2) 'myst' reference target not found: 'test' [myst.xref_missing]
+<src>/index.md:1: (WARNING/2) 'myst' reference target not found: 'name with spaces' [myst.xref_missing]
 .
 
 [implicit_anchor] --myst-heading-anchors=1
@@ -131,6 +138,23 @@
                 Other
 .
 
+[id-with-spaces] 
+.
+(name with spaces)=
+Paragraph
+
+[](<#name with spaces>)
+.
+<document source="<src>/index.md">
+    <target refid="name-with-spaces">
+    <paragraph ids="name-with-spaces" names="name\ with\ spaces">
+        Paragraph
+    <paragraph>
+        <reference id_link="True" refid="name-with-spaces">
+            <inline classes="std std-ref">
+                #name with spaces
+.
+
 [ref-table] 
 .
 ```{table} caption
diff --git a/tests/test_renderers/fixtures/sphinx_link_resolution.md b/tests/test_renderers/fixtures/sphinx_link_resolution.md
index bbf88a5..85577ff 100644
--- a/tests/test_renderers/fixtures/sphinx_link_resolution.md
+++ b/tests/test_renderers/fixtures/sphinx_link_resolution.md
@@ -78,6 +78,7 @@
 [](#target)
 [...](#target)
 [explicit](#target)
+[](<#name with spaces>)
 .
 <document source="<src>/index.md">
     <target refid="target">
@@ -95,6 +96,9 @@
 
             <reference id_link="True" refid="target">
                 explicit
+
+            <pending_xref refdoc="index" refdomain="True" refexplicit="False" reftarget="name with spaces" reftype="myst" refwarn="True">
+                <inline classes="xref myst">
 .
 
 [explicit>implicit] {"myst_heading_anchors": 1}
@@ -120,6 +124,23 @@
                         Other
 .
 
+[id-with-spaces] 
+.
+(name with spaces)=
+Paragraph
+
+[](<#name with spaces>)
+.
+<document source="<src>/index.md">
+    <target refid="name-with-spaces">
+    <paragraph ids="name-with-spaces" names="name\ with\ spaces">
+        Paragraph
+    <paragraph>
+        <reference id_link="True" refid="name-with-spaces">
+            <inline classes="std std-ref">
+                #name with spaces
+.
+
 [ref-table] 
 .
 ```{table} caption
@@ -189,7 +210,7 @@ c  | d
                 relative to source dir
 .
 
-[source-file]
+[source-file] 
 .
 [](other.rst)
 [...](./other.rst)