Repository: Python-Markdown/markdown

Issue:
Incorrect HTML output for code span containing `</>`
In markdown==3.8, a code span containing the string "\`</>\`" is incorrectly rendered. Instead of producing the expected HTML output with the characters escaped, it strips the content entirely:

```pycon
>>> import markdown
>>> markdown.__version__
'3.8'

>>> markdown.markdown('`</>`')
'<p>``</p>'
```

**Expected result:** 

```python
`'<p><code>&lt;/&gt;</code></p>'`
```

Other examples that work correctly:

```pycon
>>> markdown.markdown('`<`')
✅ '<p><code>&lt;</code></p>'
>>> markdown.markdown('`</`')
✅ '<p><code>&lt;/</code></p>'
>>> markdown.markdown('`</>`')
❌ '<p>``</p>'
>>> markdown.markdown('`/>`')
✅ '<p><code>/&gt;</code></p>'
>>> markdown.markdown('`>`')
✅ '<p><code>&gt;</code></p>'
```

This issue appears specific to the combination `</>`. Would appreciate any insight into whether this is a known bug, an edge case in the parser, or an intentional (though surprising) behavior. Thanks!

Hints from issue comments:
@facelessuser: It looks like a bug to me, but it'll need some investigation though to understand why this is happening.@facelessuser: As far as I can tell, the `</>` is processed and stripped _before_ the code block is handled, so ``` `` ``` is empty and not processed as a code block. I don't know why, but that will need more analysis.@facelessuser: So, I was digging deeper into this. This appears to be something that [HTMLParser](https://docs.python.org/3/library/html.parser.html#html.parser.HTMLParser) is doing.

Consider the following example:

```py
from html.parser import HTMLParser
from html.entities import name2codepoint

class MyHTMLParser(HTMLParser):
    def handle_starttag(self, tag, attrs):
        print("Start tag:", tag)
        for attr in attrs:
            print("     attr:", attr)
        super().handle_starttag(tag, attrs)

    def handle_endtag(self, tag):
        print("End tag  :", tag)
        super().handle_endtag(tag)

    def handle_data(self, data):
        print("Data     :", data)
        super().handle_data(data)

    def handle_comment(self, data):
        print("Comment  :", data)
        super().handle_comment(data)

    def handle_entityref(self, name):
        print('Entity:', name)
        super().handle_entityref(name)

    def handle_charref(self, name):
        print("Name:", name)
        super().handle_charref(name)

    def handle_decl(self, data):
        print("Decl     :", data)
        super().handle_decl(data)

    def handle_comment(self, data: str) -> None:
        print('Comment:', data)
        super().handle_comment(data)

    def handle_pi(self, data: str) -> None:
        print('PI:', data)
        super().handle_pi(data)

    def unknown_decl(self, data: str) -> None:
        print('Unknown:', data)
        super().unknown_decl(data)


parser = MyHTMLParser()

print('==== Test1')
print(parser.feed('<html><body><</body></html>'))

print('==== Test2')
print(parser.feed('<html><body></></body></html>'))
```

We can see that in the second test that `</>` is never seen:

```
==== Test1
Start tag: html
Start tag: body
Data     : <
End tag  : body
End tag  : html
None
==== Test2
Start tag: html
Start tag: body
End tag  : body
End tag  : html
None
```

I'm not sure what to do with this info currently. From what I can tell, the parser looks at it as a tag, but since it has no name, it is never reported or sent to the handler; it just gets ignored. While this is fine for an HTML parser (browsers do this) a Markdown parser wants to capture that content.@facelessuser: I've figured out a hacky way to capture and process `</>`. We have to sneak it in and capture it under something else (start tag, etc.), then we just check for it and send it to data instead.

It should be noted that putting a space anywhere in `</>` (`< />`, `</ >`, or `< / >`) also allows this to work with no code modification. The problem only arises when we have exactly `</>`.

There may be a less ugly way to handle these than my experimental hack, but as a starting point, the hack does seem to work.

```py
import importlib.util
import sys
import re

spec = importlib.util.find_spec('html.parser')
htmlparser = importlib.util.module_from_spec(spec)
spec.loader.exec_module(htmlparser)
sys.modules['htmlparser'] = htmlparser


htmlparser.starttagopen = re.compile('<[a-zA-Z]|</>')

class MyHTMLParser(htmlparser.HTMLParser):

    def parse_starttag(self, i):
        if self.rawdata[i:i + 3] == '</>':
            self.handle_data(self.rawdata[i:i + 3])
            return i + 3
        return super().parse_starttag(i)

    def handle_starttag(self, tag, attrs):
        print("Start tag:", tag)
        for attr in attrs:
            print("     attr:", attr)
        super().handle_starttag(tag, attrs)

    def handle_endtag(self, tag):
        print("End tag  :", tag)
        super().handle_endtag(tag)

    def handle_data(self, data):
        print("Data     :", data)
        super().handle_data(data)

    def handle_comment(self, data):
        print("Comment  :", data)
        super().handle_comment(data)

    def handle_entityref(self, name):
        print('Entity:', name)
        super().handle_entityref(name)

    def handle_charref(self, name):
        print("Name:", name)
        super().handle_charref(name)

    def handle_decl(self, data):
        print("Decl     :", data)
        super().handle_decl(data)

    def handle_comment(self, data: str) -> None:
        print('Comment:', data)
        super().handle_comment(data)

    def handle_pi(self, data: str) -> None:
        print('PI:', data)
        super().handle_pi(data)

    def unknown_decl(self, data: str) -> None:
        print('Unknown:', data)
        super().unknown_decl(data)


parser = MyHTMLParser()

print(parser.feed('<html><body></></body></html>'))
```

```
Start tag: html
Start tag: body
Data     : </>
End tag  : body
End tag  : html
None
```

This seems to work when patched into Markdown without breaking any tests:

```py
import markdown

text = '''
`</>`
'''

html = markdown.markdown(text)
print(HTML)
```

```html
<p><code>&lt;/&gt;</code></p>
```

So moving forward, we would just need to settle on the least egregious hack.@facelessuser: I've pushed a PR to get a discussion going. What I have works, and I've added tests that illustrate what happens in various scenarios. This is basically my current proposal. I am open to other creative approaches, but if no better solution can be determined, then I would vote just to move ahead with it as I don't think ignoring `</>` is the right approach.@waylan: @facelessuser thanks for the full analysis. This is a weird edge case, but I can also understand why the HTMLParser would act the way it does.

Solution patch:
diff --git a/docs/changelog.md b/docs/changelog.md
index 7c96a6f..9f972a3 100644
--- a/docs/changelog.md
+++ b/docs/changelog.md
@@ -14,6 +14,7 @@ See the [Contributing Guide](contributing.md) for details.
 ### Fixed
 
 * Fixed dropped content in `md_in_html` (#1526).
+* Fixed HTML handling corner case that prevented some content from not being rendered (#1528).
 
 ## [3.8.0] - 2025-04-09
 
diff --git a/markdown/htmlparser.py b/markdown/htmlparser.py
index 33b918d..1528325 100644
--- a/markdown/htmlparser.py
+++ b/markdown/htmlparser.py
@@ -41,6 +41,10 @@ htmlparser = importlib.util.module_from_spec(spec)
 spec.loader.exec_module(htmlparser)
 sys.modules['htmlparser'] = htmlparser
 
+# This is a hack. We are sneaking in `</>` so we can capture it without the HTML parser
+# throwing it away. When we see it, we will process it as data.
+htmlparser.starttagopen = re.compile('<[a-zA-Z]|</>')
+
 # Monkeypatch `HTMLParser` to only accept `?>` to close Processing Instructions.
 htmlparser.piclose = re.compile(r'\?>')
 # Monkeypatch `HTMLParser` to only recognize entity references with a closing semicolon.
@@ -297,6 +301,11 @@ class HTMLExtractor(htmlparser.HTMLParser):
         return self.__starttag_text
 
     def parse_starttag(self, i: int) -> int:  # pragma: no cover
+        # Treat `</>` as normal data as it is not a real tag.
+        if self.rawdata[i:i + 3] == '</>':
+            self.handle_data(self.rawdata[i:i + 3])
+            return i + 3
+
         self.__starttag_text = None
         endpos = self.check_for_whole_start_tag(i)
         if endpos < 0:


Test patch:
diff --git a/tests/test_syntax/blocks/test_html_blocks.py b/tests/test_syntax/blocks/test_html_blocks.py
index 33375d3..8ea3904 100644
--- a/tests/test_syntax/blocks/test_html_blocks.py
+++ b/tests/test_syntax/blocks/test_html_blocks.py
@@ -1643,3 +1643,21 @@ class TestHTMLBlocks(TestCase):
         placeholder = md.htmlStash.get_placeholder(md.htmlStash.html_counter + 1)
         result = md.postprocessors['raw_html'].run(placeholder)
         self.assertEqual(placeholder, result)
+
+    def test_noname_tag(self):
+        self.assertMarkdownRenders(
+            self.dedent(
+                """
+                <div>
+                </>
+                </div>
+                """
+            ),
+            self.dedent(
+                """
+                <div>
+                </>
+                </div>
+                """
+            )
+        )
diff --git a/tests/test_syntax/extensions/test_md_in_html.py b/tests/test_syntax/extensions/test_md_in_html.py
index 5ef860d..903766f 100644
--- a/tests/test_syntax/extensions/test_md_in_html.py
+++ b/tests/test_syntax/extensions/test_md_in_html.py
@@ -1538,6 +1538,24 @@ class TestMdInHTML(TestCase):
             extensions=['md_in_html']
         )
 
+    def test_noname_tag(self):
+        self.assertMarkdownRenders(
+            self.dedent(
+                """
+                <div markdown>
+                </>
+                </div>
+                """
+            ),
+            self.dedent(
+                """
+                <div>
+                <p>&lt;/&gt;</p>
+                </div>
+                """
+            )
+        )
+
 
 def load_tests(loader, tests, pattern):
     """ Ensure `TestHTMLBlocks` doesn't get run twice by excluding it here. """
diff --git a/tests/test_syntax/inline/test_code.py b/tests/test_syntax/inline/test_code.py
index 9fc3790..c7ccc4b 100644
--- a/tests/test_syntax/inline/test_code.py
+++ b/tests/test_syntax/inline/test_code.py
@@ -62,3 +62,20 @@ class TestCode(TestCase):
                 """
             )
         )
+
+    def test_noname_tag(self):
+        # Browsers ignore `</>`, but a Markdown parser should not, and should treat it as data
+        # but not a tag.
+
+        self.assertMarkdownRenders(
+            self.dedent(
+                """
+                `</>`
+                """
+            ),
+            self.dedent(
+                """
+                <p><code>&lt;/&gt;</code></p>
+                """
+            )
+        )
diff --git a/tests/test_syntax/inline/test_raw_html.py b/tests/test_syntax/inline/test_raw_html.py
index f165f75..57d725e 100644
--- a/tests/test_syntax/inline/test_raw_html.py
+++ b/tests/test_syntax/inline/test_raw_html.py
@@ -31,3 +31,6 @@ class TestRawHtml(TestCase):
 
     def test_inline_html_backslashes(self):
         self.assertMarkdownRenders('<img src="..\\..\\foo.png">', '<p><img src="..\\..\\foo.png"></p>')
+
+    def test_noname_tag(self):
+        self.assertMarkdownRenders('<span></></span>', '<p><span>&lt;/&gt;</span></p>')