Repository: Python-Markdown/markdown
Pull request #1436 (2024-01-22 21:29:58)

Heading in list
Here is a start at a fix for #1433. So far I have tests, one of which is failing.

waylan: This is ready to go. Turns out the first item in a list was being parsed differently because the list parser does not look ahead and it doesn't know it is in a loose list until it gets to the second list item. The fix here just detects that its in the second (or later) item and adjusts the content `after` the heading (by detabbing it) so that it gets parsed correctly. See the commit message for 3fca127 for more details.

Patch:
diff --git a/.spell-dict b/.spell-dict
index 13cdea5..ae12454 100644
--- a/.spell-dict
+++ b/.spell-dict
@@ -32,6 +32,7 @@ customizable
 dedent
 deliminators
 deregister
+detabbed
 Dmitry
 docdata
 ElementTree
diff --git a/LICENSE.md b/LICENSE.md
index 2652d97..6249d60 100644
--- a/LICENSE.md
+++ b/LICENSE.md
@@ -1,29 +1,30 @@
-Copyright 2007, 2008 The Python Markdown Project (v. 1.7 and later)
-Copyright 2004, 2005, 2006 Yuri Takhteyev (v. 0.2-1.6b)
-Copyright 2004 Manfred Stienstra (the original version)
+BSD 3-Clause License
 
-All rights reserved.
+Copyright 2007, 2008 The Python Markdown Project (v. 1.7 and later)  
+Copyright 2004, 2005, 2006 Yuri Takhteyev (v. 0.2-1.6b)  
+Copyright 2004 Manfred Stienstra (the original version)
 
 Redistribution and use in source and binary forms, with or without
 modification, are permitted provided that the following conditions are met:
 
-* Redistributions of source code must retain the above copyright
-  notice, this list of conditions and the following disclaimer.
-* Redistributions in binary form must reproduce the above copyright
-  notice, this list of conditions and the following disclaimer in the
-  documentation and/or other materials provided with the distribution.
-* Neither the name of the Python Markdown Project nor the
-  names of its contributors may be used to endorse or promote products
-  derived from this software without specific prior written permission.
+1. Redistributions of source code must retain the above copyright notice, this
+   list of conditions and the following disclaimer.
+
+2. Redistributions in binary form must reproduce the above copyright notice,
+   this list of conditions and the following disclaimer in the documentation
+   and/or other materials provided with the distribution.
+
+3. Neither the name of the copyright holder nor the names of its
+   contributors may be used to endorse or promote products derived from
+   this software without specific prior written permission.
 
-THIS SOFTWARE IS PROVIDED BY THE PYTHON MARKDOWN PROJECT ''AS IS'' AND ANY
-EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED
-WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE
-DISCLAIMED. IN NO EVENT SHALL ANY CONTRIBUTORS TO THE PYTHON MARKDOWN PROJECT
-BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR
-CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF
-SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS
-INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN
-CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE)
-ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE
-POSSIBILITY OF SUCH DAMAGE.
+THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS"
+AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE
+IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE
+DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE
+FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL
+DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR
+SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER
+CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY,
+OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
+OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
diff --git a/docs/changelog.md b/docs/changelog.md
index 6663606..884320f 100644
--- a/docs/changelog.md
+++ b/docs/changelog.md
@@ -13,6 +13,7 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
 ### Fixed
 
 * Include `scripts/*.py` in the generated source tarballs (#1430).
+* Ensure lines after heading in loose list are properly detabbed (#1443).
 
 ## [3.5.2] -- 2024-01-10
 
diff --git a/markdown/blockprocessors.py b/markdown/blockprocessors.py
index d2020b9..3ed4cf0 100644
--- a/markdown/blockprocessors.py
+++ b/markdown/blockprocessors.py
@@ -479,6 +479,11 @@ class HashHeaderProcessor(BlockProcessor):
             h.text = m.group('header').strip()
             if after:
                 # Insert remaining lines as first block for future parsing.
+                if self.parser.state.isstate('looselist'):
+                    # This is a weird edge case where a header is a child of a loose list
+                    # and there is no blank line after the header. To ensure proper
+                    # parsing, the line(s) after need to be detabbed. See #1443.
+                    after = self.looseDetab(after)
                 blocks.insert(0, after)
         else:  # pragma: no cover
             # This should never happen, but just in case...


Test patch:
diff --git a/tests/test_syntax/blocks/test_ul.py b/tests/test_syntax/blocks/test_ul.py
new file mode 100644
index 0000000..f90d43a
--- /dev/null
+++ b/tests/test_syntax/blocks/test_ul.py
@@ -0,0 +1,137 @@
+"""
+Python Markdown
+
+A Python implementation of John Gruber's Markdown.
+
+Documentation: https://python-markdown.github.io/
+GitHub: https://github.com/Python-Markdown/markdown/
+PyPI: https://pypi.org/project/Markdown/
+
+Started by Manfred Stienstra (http://www.dwerg.net/).
+Maintained for a few years by Yuri Takhteyev (http://www.freewisdom.org).
+Currently maintained by Waylan Limberg (https://github.com/waylan),
+Dmitry Shachnev (https://github.com/mitya57) and Isaac Muse (https://github.com/facelessuser).
+
+Copyright 2007-2024 The Python Markdown Project (v. 1.7 and later)
+Copyright 2004, 2005, 2006 Yuri Takhteyev (v. 0.2-1.6b)
+Copyright 2004 Manfred Stienstra (the original version)
+
+License: BSD (see LICENSE.md for details).
+"""
+
+import unittest
+from markdown.test_tools import TestCase
+
+
+class TestUnorderedLists(TestCase):
+
+    # TODO: Move legacy tests here
+
+    def test_header_and_paragraph_no_blank_line_loose_list(self):
+        self.assertMarkdownRenders(
+            self.dedent(
+                """
+                - ### List 1
+                    Entry 1.1
+
+                - ### List 2
+                    Entry 2.1
+                """
+            ),
+            self.dedent(
+                """
+                <ul>
+                <li>
+                <h3>List 1</h3>
+                <p>Entry 1.1</p>
+                </li>
+                <li>
+                <h3>List 2</h3>
+                <p>Entry 2.1</p>
+                </li>
+                </ul>
+                """
+            )
+        )
+
+    # Note: This is strange. Any other element on first line of list item would get very
+    # different behavior. However, as a heading can only ever be one line, this is the
+    # correct behavior. In fact, `markdown.pl` behaves this way with no indentation.
+    def test_header_and_paragraph_no_blank_line_loose_list_no_indent(self):
+        self.assertMarkdownRenders(
+            self.dedent(
+                """
+                - ### List 1
+                Entry 1.1
+
+                - ### List 2
+                Entry 2.1
+                """
+            ),
+            self.dedent(
+                """
+                <ul>
+                <li>
+                <h3>List 1</h3>
+                <p>Entry 1.1</p>
+                </li>
+                <li>
+                <h3>List 2</h3>
+                <p>Entry 2.1</p>
+                </li>
+                </ul>
+                """
+            )
+        )
+
+    # TODO: Possibly change this behavior. While this test follows the behavior
+    # of `markdown.pl`, it is likely surprising to most users. In fact, actual
+    # behavior is to return the same results as for a loose list.
+    @unittest.skip('This behaves as a loose list in Python-Markdown')
+    def test_header_and_paragraph_no_blank_line_tight_list(self):
+        self.assertMarkdownRenders(
+            self.dedent(
+                """
+                - ### List 1
+                    Entry 1.1
+                - ### List 2
+                    Entry 2.1
+                """
+            ),
+            self.dedent(
+                """
+                <ul>
+                <li>### List 1
+                Entry 1.1</li>
+                <li>### List 2
+                Entry 2.1</li>
+                </ul>
+                """
+            )
+        )
+
+    # TODO: Possibly change this behavior. While this test follows the behavior
+    # of `markdown.pl`, it is likely surprising to most users. In fact, actual
+    # behavior is to return the same results as for a loose list.
+    @unittest.skip('This behaves as a loose list in Python-Markdown')
+    def test_header_and_paragraph_no_blank_line_tight_list_no_indent(self):
+        self.assertMarkdownRenders(
+            self.dedent(
+                """
+                - ### List 1
+                Entry 1.1
+                - ### List 2
+                Entry 2.1
+                """
+            ),
+            self.dedent(
+                """
+                <ul>
+                <li>### List 1
+                Entry 1.1</li>
+                <li>### List 2
+                Entry 2.1</li>
+                </ul>
+                """
+            )
+        )