Repository: executablebooks/MyST-Parser
Pull request #654 (2022-12-13 15:17:40)

✨ NEW: Add `attrs_inline` extension
By adding `"attrs_inline"` to `myst_enable_extensions` (in the sphinx `conf.py`),
you can enable parsing of inline attributes after certain inline syntaxes.
This is adapted from [djot inline attributes](https://htmlpreview.github.io/?https://github.com/jgm/djot/blob/master/doc/syntax.html#inline-attributes),
and also related to [pandoc bracketed spans](https://pandoc.org/MANUAL.html#extension-bracketed_spans).

This extension replaces `"attrs_image"`, which is still present, but deprecated.


Patch:
diff --git a/docs/conf.py b/docs/conf.py
index 7db210c..e39c807 100644
--- a/docs/conf.py
+++ b/docs/conf.py
@@ -90,7 +90,7 @@ myst_enable_extensions = [
     "strikethrough",
     "substitution",
     "tasklist",
-    "attrs_image",
+    "attrs_inline",
 ]
 myst_number_code_blocks = ["typescript"]
 myst_heading_anchors = 2
diff --git a/docs/syntax/optional.md b/docs/syntax/optional.md
index b11ab08..c497d92 100644
--- a/docs/syntax/optional.md
+++ b/docs/syntax/optional.md
@@ -28,6 +28,7 @@ To enable all the syntaxes explained below:
 ```python
 myst_enable_extensions = [
     "amsmath",
+    "attrs_inline",
     "colon_fence",
     "deflist",
     "dollarmath",
@@ -43,7 +44,7 @@ myst_enable_extensions = [
 ]
 ```
 
-:::{important}
+:::{versionchanged} 0.13.0
 `myst_enable_extensions` replaces previous configuration options:
 `admonition_enable`, `figure_enable`, `dmath_enable`, `amsmath_enable`, `deflist_enable`, `html_img_enable`
 :::
@@ -101,7 +102,7 @@ Math is parsed by adding to the `myst_enable_extensions` list option, in the sph
 
 These options enable their respective Markdown parser plugins, as detailed in the [markdown-it plugin guide](markdown_it:md/plugins).
 
-:::{important}
+:::{versionchanged} 0.13.0
 `myst_dmath_enable=True` and `myst_amsmath_enable=True` are deprecated, and replaced by `myst_enable_extensions = ["dollarmath", "amsmath"]`
 :::
 
@@ -484,7 +485,7 @@ This text is **standard** _Markdown_
 
 ## Admonition directives
 
-:::{important}
+:::{versionchanged} 0.13.0
 `myst_admonition_enable` is deprecated and replaced by `myst_enable_extensions = ["colon_fence"]` (see above).
 Also, classes should now be set with the `:class: myclass` option.
 
@@ -727,9 +728,90 @@ Send a message to a recipient
 Currently `sphinx.ext.autodoc` does not support MyST, see [](howto/autodoc).
 :::
 
+(syntax/attributes)=
+## Inline attributes
+
+By adding `"attrs_inline"` to `myst_enable_extensions` (in the sphinx `conf.py` [configuration file](https://www.sphinx-doc.org/en/master/usage/configuration.html)),
+you can enable parsing of inline attributes after certain inline syntaxes.
+This is adapted from [djot inline attributes](https://htmlpreview.github.io/?https://github.com/jgm/djot/blob/master/doc/syntax.html#inline-attributes),
+and also related to [pandoc bracketed spans](https://pandoc.org/MANUAL.html#extension-bracketed_spans).
+
+:::{important}
+This feature is in *beta*, and may change in future versions.
+It replace the previous `attrs_image` extension, which is now deprecated.
+:::
+
+Attributes are specified in curly braces after the inline syntax.
+Inside the curly braces, the following syntax is recognised:
+
+- `.foo` specifies `foo` as a class.
+  Multiple classes may be given in this way; they will be combined.
+- `#foo` specifies `foo` as an identifier.
+  An element may have only one identifier;
+  if multiple identifiers are given, the last one is used.
+- `key="value"` or `key=value` specifies a key-value attribute.
+    Quotes are not needed when the value consists entirely of
+    ASCII alphanumeric characters or `_` or `:` or `-`.
+    Backslash escapes may be used inside quoted values.
+    **Note** only certain keys are supported, see below.
+- `%` begins a comment, which ends with the next `%` or the end of the attribute (`}`).
+
+For example, the following Markdown:
+
+```md
+
+- [A span of text with attributes]{#spanid .bg-warning},
+  {ref}`a reference to the span <spanid>`
+
+- `A literal with attributes`{#literalid .bg-warning},
+  {ref}`a reference to the literal <literalid>
+
+- An autolink with attributes: <https://example.com>{.bg-warning}
+
+- [A link with attributes](syntax/attributes){#linkid .bg-warning},
+  {ref}`a reference to the link <linkid>`
+
+- ![An image with attribute](img/fun-fish.png){#imgid .bg-warning w=100px align=center}
+  {ref}`a reference to the image <imgid>`
+
+```
+
+will be parsed as:
+
+- [A span of text with attributes]{#spanid .bg-warning},
+  {ref}`a reference to the span <spanid>`
+
+- `A literal with attributes`{#literalid .bg-warning},
+  {ref}`a reference to the literal <literalid>`
+
+- An autolink with attributes: <https://example.com>{.bg-warning}
+
+- [A link with attributes](syntax/attributes){#linkid .bg-warning},
+  {ref}`a reference to the link <linkid>`
+
+- ![An image with attribute](img/fun-fish.png){#imgid .bg-warning w="100px" align=center}
+  {ref}`a reference to the image <imgid>`
+
+### key-value attributes
+
+`id` and `class` are supported for all inline syntaxes,
+but only certain key-value attributes are supported for each syntax.
+
+For **literals**, the following attributes are supported:
+
+- `language`/`lexer`/`l` defines the syntax lexer,
+  e.g. `` `a = "b"`{l=python} `` is displayed as `a = "b"`{l=python}.
+  Note, this is only supported in `sphinx >= 5`.
+
+For **images**, the following attributes are supported (equivalent to the `image` directive):
+
+- `width`/`w` defines the width of the image (in `%`, `px`, `em`, `cm`, etc)
+- `height`/`h` defines the height of the image (in `px`, `em`, `cm`, etc)
+- `align`/`a` defines the scale of the image (`left`, `center`, or `right`)
+
 (syntax/images)=
 
-## Images
+## HTML Images
 
 MyST provides a few different syntaxes for including images in your documentation, as explained below.
 
@@ -786,42 +868,6 @@ HTML image can also be used inline!
 
 I'm an inline image: <img src="img/fun-fish.png" height="20px">
 
-### Inline attributes
-
-:::{warning}
-This extension is currently experimental, and may change in future versions.
-:::
-
-By adding `"attrs_image"` to `myst_enable_extensions` (in the sphinx `conf.py` [configuration file](https://www.sphinx-doc.org/en/master/usage/configuration.html)),
-you can enable parsing of inline attributes for images.
-
-For example, the following Markdown:
-
-```md
-![image attrs](img/fun-fish.png){#imgattr .bg-primary width="100px" align=center}
-
-{ref}`a reference to the image <imgattr>`
-```
-
-will be parsed as:
-
-![image attrs](img/fun-fish.png){#imgattr .bg-primary width="100px" align=center}
-
-{ref}`a reference to the image <imgattr>`
-
-Inside the curly braces, the following syntax is possible:
-
-- `.foo` specifies `foo` as a class.
-  Multiple classes may be given in this way; they will be combined.
-- `#foo` specifies `foo` as an identifier.
-  An element may have only one identifier;
-  if multiple identifiers are given, the last one is used.
-- `key="value"` or `key=value` specifies a key-value attribute.
-    Quotes are not needed when the value consists entirely of
-    ASCII alphanumeric characters or `_` or `:` or `-`.
-    Backslash escapes may be used inside quoted values.
-- `%` begins a comment, which ends with the next `%` or the end of the attribute (`}`).
-
 (syntax/figures)=
 
 ## Markdown Figures
@@ -830,7 +876,7 @@ By adding `"colon_fence"` to `myst_enable_extensions` (in the sphinx `conf.py` [
 we can combine the above two extended syntaxes,
 to create a fully Markdown compliant version of the `figure` directive named `figure-md`.
 
-:::{important}
+:::{versionchanged} 0.13.0
 `myst_figure_enable` with the `figure` directive is deprecated and replaced by `myst_enable_extensions = ["colon_fence"]` and `figure-md`.
 :::
 
diff --git a/myst_parser/config/main.py b/myst_parser/config/main.py
index 118c3d0..b32714a 100644
--- a/myst_parser/config/main.py
+++ b/myst_parser/config/main.py
@@ -33,6 +33,7 @@ def check_extensions(_, __, value):
         [
             "amsmath",
             "attrs_image",
+            "attrs_inline",
             "colon_fence",
             "deflist",
             "dollarmath",
diff --git a/myst_parser/mdit_to_docutils/base.py b/myst_parser/mdit_to_docutils/base.py
index f21e6d1..6b198ea 100644
--- a/myst_parser/mdit_to_docutils/base.py
+++ b/myst_parser/mdit_to_docutils/base.py
@@ -9,7 +9,15 @@ from collections import OrderedDict
 from contextlib import contextmanager
 from datetime import date, datetime
 from types import ModuleType
-from typing import TYPE_CHECKING, Any, Iterator, MutableMapping, Sequence, cast
+from typing import (
+    TYPE_CHECKING,
+    Any,
+    Callable,
+    Iterator,
+    MutableMapping,
+    Sequence,
+    cast,
+)
 from urllib.parse import urlparse
 
 import jinja2
@@ -362,6 +370,44 @@ class DocutilsRenderer(RendererProtocol):
             for child in findall(node)():
                 self.add_line_and_source_path(child, token)
 
+    def copy_attributes(
+        self,
+        token: SyntaxTreeNode,
+        node: nodes.Element,
+        keys: Sequence[str] = ("class",),
+        *,
+        converters: dict[str, Callable[[str], Any]] | None = None,
+        aliases: dict[str, str] | None = None,
+    ) -> None:
+        """Copy attributes on the token to the docutils node."""
+        if converters is None:
+            converters = {}
+        if aliases is None:
+            aliases = {}
+        for key, value in token.attrs.items():
+            key = aliases.get(key, key)
+            if key not in keys:
+                continue
+            if key == "class":
+                node["classes"].extend(str(value).split())
+            elif key == "id":
+                name = nodes.fully_normalize_name(str(value))
+                node["names"].append(name)
+                self.document.note_explicit_target(node, node)
+            else:
+                if key in converters:
+                    try:
+                        value = converters[key](str(value))
+                    except ValueError:
+                        self.create_warning(
+                            f"Invalid {key!r} attribute value: {token.attrs[key]!r}",
+                            MystWarnings.INVALID_ATTRIBUTE,
+                            line=token_line(token, default=0),
+                            append_to=node,
+                        )
+                        continue
+                node[key] = value
+
     def update_section_level_state(self, section: nodes.section, level: int) -> None:
         """Update the section level state, with the new current section and level."""
         # find the closest parent section
@@ -490,6 +536,14 @@ class DocutilsRenderer(RendererProtocol):
     def render_code_inline(self, token: SyntaxTreeNode) -> None:
         node = nodes.literal(token.content, token.content)
         self.add_line_and_source_path(node, token)
+        self.copy_attributes(
+            token,
+            node,
+            ("class", "id", "language"),
+            aliases={"lexer": "language", "l": "language"},
+        )
+        if "language" in node and "code" not in node["classes"]:
+            node["classes"].append("code")
         self.current_node.append(node)
 
     def create_highlighted_code_block(
@@ -697,10 +751,8 @@ class DocutilsRenderer(RendererProtocol):
         """
         ref_node = nodes.reference()
         self.add_line_and_source_path(ref_node, token)
+        self.copy_attributes(token, ref_node, ("class", "id", "title"))
         ref_node["refuri"] = cast(str, token.attrGet("href") or "")
-        title = token.attrGet("title")
-        if title:
-            ref_node["title"] = title
         with self.current_node_context(ref_node, append=True):
             self.render_children(token)
 
@@ -717,17 +769,16 @@ class DocutilsRenderer(RendererProtocol):
         """
         ref_node = nodes.reference()
         self.add_line_and_source_path(ref_node, token)
+        self.copy_attributes(token, ref_node, ("class", "id", "title"))
         ref_node["refname"] = cast(str, token.attrGet("href") or "")
         self.document.note_refname(ref_node)
-        title = token.attrGet("title")
-        if title:
-            ref_node["title"] = title
         with self.current_node_context(ref_node, append=True):
             self.render_children(token)
 
     def render_autolink(self, token: SyntaxTreeNode) -> None:
         refuri = escapeHtml(token.attrGet("href") or "")  # type: ignore[arg-type]
         ref_node = nodes.reference()
+        self.copy_attributes(token, ref_node, ("class", "id"))
         ref_node["refuri"] = refuri
         self.add_line_and_source_path(ref_node, token)
         with self.current_node_context(ref_node, append=True):
@@ -760,58 +811,31 @@ class DocutilsRenderer(RendererProtocol):
         img_node["uri"] = destination
 
         img_node["alt"] = self.renderInlineAsText(token.children or [])
-        title = token.attrGet("title")
-        if title:
-            img_node["title"] = token.attrGet("title")
-
-        # apply other attributes that can be set on the image
-        if "class" in token.attrs:
-            img_node["classes"].extend(str(token.attrs["class"]).split())
-        if "width" in token.attrs:
-            try:
-                width = directives.length_or_percentage_or_unitless(
-                    str(token.attrs["width"])
-                )
-            except ValueError:
-                self.create_warning(
-                    f"Invalid width value for image: {token.attrs['width']!r}",
-                    MystWarnings.INVALID_ATTRIBUTE,
-                    line=token_line(token, default=0),
-                    append_to=self.current_node,
-                )
-            else:
-                img_node["width"] = width
-        if "height" in token.attrs:
-            try:
-                height = directives.length_or_unitless(str(token.attrs["height"]))
-            except ValueError:
-                self.create_warning(
-                    f"Invalid height value for image: {token.attrs['height']!r}",
-                    MystWarnings.INVALID_ATTRIBUTE,
-                    line=token_line(token, default=0),
-                    append_to=self.current_node,
-                )
-            else:
-                img_node["height"] = height
-        if "align" in token.attrs:
-            if token.attrs["align"] not in ("left", "center", "right"):
-                self.create_warning(
-                    f"Invalid align value for image: {token.attrs['align']!r}",
-                    MystWarnings.INVALID_ATTRIBUTE,
-                    line=token_line(token, default=0),
-                    append_to=self.current_node,
-                )
-            else:
-                img_node["align"] = token.attrs["align"]
-        if "id" in token.attrs:
-            name = nodes.fully_normalize_name(str(token.attrs["id"]))
-            img_node["names"].append(name)
-            self.document.note_explicit_target(img_node, img_node)
+
+        self.copy_attributes(
+            token,
+            img_node,
+            ("class", "id", "title", "width", "height", "align"),
+            converters={
+                "width": directives.length_or_percentage_or_unitless,
+                "height": directives.length_or_unitless,
+                "align": lambda x: directives.choice(x, ("left", "center", "right")),
+            },
+            aliases={"w": "width", "h": "height", "a": "align"},
+        )
 
         self.current_node.append(img_node)
 
     # ### render methods for plugin tokens
 
+    def render_span(self, token: SyntaxTreeNode) -> None:
+        """Render an inline span token."""
+        node = nodes.inline()
+        self.add_line_and_source_path(node, token)
+        self.copy_attributes(token, node, ("class", "id"))
+        with self.current_node_context(node, append=True):
+            self.render_children(token)
+
     def render_front_matter(self, token: SyntaxTreeNode) -> None:
         """Pass document front matter data."""
         position = token_line(token, default=0)
diff --git a/myst_parser/mdit_to_docutils/sphinx_.py b/myst_parser/mdit_to_docutils/sphinx_.py
index b6c5572..b6607f3 100644
--- a/myst_parser/mdit_to_docutils/sphinx_.py
+++ b/myst_parser/mdit_to_docutils/sphinx_.py
@@ -84,9 +84,7 @@ class SphinxRenderer(DocutilsRenderer):
             text = ""
 
         self.add_line_and_source_path(wrap_node, token)
-        title = token.attrGet("title")
-        if title:
-            wrap_node["title"] = title
+        self.copy_attributes(token, wrap_node, ("class", "id", "title"))
         self.current_node.append(wrap_node)
 
         inner_node = nodes.inline("", text, classes=classes)
diff --git a/myst_parser/parsers/mdit.py b/myst_parser/parsers/mdit.py
index 8476495..dbb533d 100644
--- a/myst_parser/parsers/mdit.py
+++ b/myst_parser/parsers/mdit.py
@@ -101,7 +101,15 @@ def create_md_parser(
         md.use(tasklists_plugin)
     if "substitution" in config.enable_extensions:
         md.use(substitution_plugin, *config.sub_delimiters)
-    if "attrs_image" in config.enable_extensions:
+    if "attrs_inline" in config.enable_extensions:
+        md.use(
+            attrs_plugin,
+            after=("image", "code_inline", "link_close", "span_close"),
+            spans=True,
+            span_after="footnote_ref",
+        )
+    elif "attrs_image" in config.enable_extensions:
+        # TODO deprecate
         md.use(attrs_plugin, after=("image",))
     if config.heading_anchors is not None:
         md.use(
diff --git a/myst_parser/sphinx_ext/main.py b/myst_parser/sphinx_ext/main.py
index f5aeffc..c7e2a67 100644
--- a/myst_parser/sphinx_ext/main.py
+++ b/myst_parser/sphinx_ext/main.py
@@ -3,6 +3,8 @@ from typing import Any
 
 from sphinx.application import Sphinx
 
+from myst_parser.warnings_ import MystWarnings
+
 
 def setup_sphinx(app: Sphinx, load_parser=False):
     """Initialize all settings and transforms in Sphinx."""
@@ -58,3 +60,11 @@ def create_myst_config(app):
     except (TypeError, ValueError) as error:
         logger.error("myst configuration invalid: %s", error.args[0])
         app.env.myst_config = MdParserConfig()
+
+    if "attrs_image" in app.env.myst_config.enable_extensions:
+        logger.warning(
+            "The `attrs_image` extension is deprecated, "
+            "please use `attrs_inline` instead.",
+            type="myst",
+            subtype=MystWarnings.DEPRECATED.value,
+        )
diff --git a/myst_parser/warnings_.py b/myst_parser/warnings_.py
index 4e9bced..78b9491 100644
--- a/myst_parser/warnings_.py
+++ b/myst_parser/warnings_.py
@@ -10,6 +10,9 @@ from docutils import nodes
 class MystWarnings(Enum):
     """MyST warning types."""
 
+    DEPRECATED = "deprecated"
+    """Deprecated usage."""
+
     RENDER_METHOD = "render"
     """The render method is not implemented."""
 
diff --git a/pyproject.toml b/pyproject.toml
index 82cfb5e..1e71cf6 100644
--- a/pyproject.toml
+++ b/pyproject.toml
@@ -37,7 +37,7 @@ dependencies = [
     "docutils>=0.15,<0.20",
     "jinja2", # required for substitutions, but let sphinx choose version
     "markdown-it-py>=1.0.0,<3.0.0",
-    "mdit-py-plugins~=0.3.1",
+    "mdit-py-plugins~=0.3.3",
     "pyyaml",
     "sphinx>=4,<6",
     'typing-extensions; python_version < "3.8"',
@@ -54,7 +54,8 @@ linkify = ["linkify-it-py~=1.0"]
 # Note: This is only required for internal use
 rtd = [
     "ipython",
-    "sphinx-book-theme",
+    # currently required to get sphinx v5
+    "sphinx-book-theme @ git+https://github.com/executablebooks/sphinx-book-theme.git@8da268fce3159755041e8db93e132221a0b0def5#egg=sphinx-book-theme",
     "sphinx-design",
     "sphinxext-rediraffe~=0.2.7",
     "sphinxcontrib.mermaid~=0.7.1",


Test patch:
diff --git a/tests/test_renderers/fixtures/myst-config.txt b/tests/test_renderers/fixtures/myst-config.txt
index 48fd5e5..8a71971 100644
--- a/tests/test_renderers/fixtures/myst-config.txt
+++ b/tests/test_renderers/fixtures/myst-config.txt
@@ -156,7 +156,56 @@ www.commonmark.org/he<lp
         <lp
 .
 
-[attrs_image] --myst-enable-extensions=attrs_image
+[attrs_inline_span] --myst-enable-extensions=attrs_inline
+.
+[content]{#id .a .b}
+.
+<document source="<string>">
+    <paragraph>
+        <inline classes="a b" ids="id" names="id">
+            content
+.
+
+[attrs_inline_code] --myst-enable-extensions=attrs_inline
+.
+`content`{#id .a .b language=python}
+.
+<document source="<string>">
+    <paragraph>
+        <literal classes="a b code" ids="id" language="python" names="id">
+            content
+.
+
+[attrs_inline_links] --myst-enable-extensions=attrs_inline
+.
+<https://example.com>{.a .b}
+
+(other)=
+[text1](https://example.com){#id1 .a .b}
+
+[text2](other){#id2 .c .d}
+
+[ref]{#id3 .e .f}
+
+[ref]: https://example.com
+.
+<document source="<string>">
+    <paragraph>
+        <reference classes="a b" refuri="https://example.com">
+            https://example.com
+    <target refid="other">
+    <paragraph ids="other" names="other">
+        <reference classes="a b" ids="id1" names="id1" refuri="https://example.com">
+            text1
+    <paragraph>
+        <reference classes="c d" ids="id2" names="id2" refid="other">
+            text2
+    <paragraph>
+        <reference classes="e f" ids="id3" names="id3" refuri="https://example.com">
+            ref
+.
+
+[attrs_inline_image] --myst-enable-extensions=attrs_inline
 .
 ![a](b){#id .a width="100%" align=center height=20px}{.b}
 .
@@ -165,24 +214,24 @@ www.commonmark.org/he<lp
         <image align="center" alt="a" classes="a b" height="20px" ids="id" names="id" uri="b" width="100%">
 .
 
-[attrs_image_warnings] --myst-enable-extensions=attrs_image
+[attrs_inline_image_warnings] --myst-enable-extensions=attrs_inline
 .
 ![a](b){width=1x height=2x align=other }
 .
 <document source="<string>">
     <paragraph>
-        <system_message level="2" line="1" source="<string>" type="WARNING">
-            <paragraph>
-                Invalid width value for image: '1x' [myst.attribute]
-        <system_message level="2" line="1" source="<string>" type="WARNING">
-            <paragraph>
-                Invalid height value for image: '2x' [myst.attribute]
-        <system_message level="2" line="1" source="<string>" type="WARNING">
-            <paragraph>
-                Invalid align value for image: 'other' [myst.attribute]
         <image alt="a" uri="b">
+            <system_message level="2" line="1" source="<string>" type="WARNING">
+                <paragraph>
+                    Invalid 'width' attribute value: '1x' [myst.attribute]
+            <system_message level="2" line="1" source="<string>" type="WARNING">
+                <paragraph>
+                    Invalid 'height' attribute value: '2x' [myst.attribute]
+            <system_message level="2" line="1" source="<string>" type="WARNING">
+                <paragraph>
+                    Invalid 'align' attribute value: 'other' [myst.attribute]
 
-<string>:1: (WARNING/2) Invalid width value for image: '1x' [myst.attribute]
-<string>:1: (WARNING/2) Invalid height value for image: '2x' [myst.attribute]
-<string>:1: (WARNING/2) Invalid align value for image: 'other' [myst.attribute]
+<string>:1: (WARNING/2) Invalid 'width' attribute value: '1x' [myst.attribute]
+<string>:1: (WARNING/2) Invalid 'height' attribute value: '2x' [myst.attribute]
+<string>:1: (WARNING/2) Invalid 'align' attribute value: 'other' [myst.attribute]
 .