Repository: facelessuser/pymdown-extensions

Issue:
Parsing issue with nested lists in details block
Hi all 👋,

First, thanks for your work, it's awesome and very useful !

I think I spotted an issue when trying to add "complex" content in a details block, here is a description:

#### Without details block

I define the following structure:

```
Without details block
- Parent 1  
    Intro 1  

    - Child 1
    - Child 2
    - Child 3
    - Child 3

    Outro 1
```

which yields the following HTML:

```
<p>Without details block</p>
<ul>
    <li>
        <p>Parent 1<br />
            Intro 1 </p>
        <ul>
            <li>Child 1</li>
            <li>Child 2</li>
            <li>Child 3</li>
            <li>Child 3</li>
        </ul>
        <p>Outro 1</p>
    </li>
</ul>
```

which gives this in the browser:

![image](https://user-images.githubusercontent.com/11026636/107019424-ee47a380-67a1-11eb-9c13-6c89a2b7e247.png)

#### With details block

Now, when I try to add a details block around in order to make it collapsible:

```
???+ "With details block"
    - Parent 1  
        Intro 1  

        - Child 1
        - Child 2
        - Child 3
        - Child 3

        Outro 1
```

yielding this HTML:

```
<details open="open">
    <summary>With details block</summary>
    <ul>
        <li>
            <p>Parent 1<br />
                Intro 1 </p>
            <ul>
                <li>Child 1<ul>
                        <li>Child 2</li>
                        <li>Child 3</li>
                        <li>Child 3</li>
                    </ul>
                </li>
            </ul>
            <p>Outro 1</p>
        </li>
    </ul>
</details>
```

(you can already spot the issue here around `Child 1`), this gives:

![image](https://user-images.githubusercontent.com/11026636/107019583-23ec8c80-67a2-11eb-85ad-3e22820fc051.png)

The HTML is badly formatted in the children 2, 3, 4 got one extra level of indentation.

---

I tried a lot of whitespace / linebreaks / indentation combination attempting to solve the issue but I could not make it.

Is there any chance that this comes from the extension itself ?

Thanks a lot guys 😊

Discussion:
facelessuser: @gir-bot lgtm

Patch:
diff --git a/docs/src/markdown/about/changelog.md b/docs/src/markdown/about/changelog.md
index 4ee12abe..0a534252 100644
--- a/docs/src/markdown/about/changelog.md
+++ b/docs/src/markdown/about/changelog.md
@@ -1,5 +1,9 @@
 # Changelog
 
+## 8.1.2
+
+- **FIX**: Tabbed &amp; Details: Fix corner case with lists. (#1225)
+
 ## 8.1.1
 
 - **FIX**: Ensure content immediately before Details content or Tabbed content gets preserved.
@@ -334,7 +338,8 @@ Please see [Release Notes](./releases/6.0.md#6.0) for details on upgrading to 6.
 ## 3.0
 
 - **NEW**: Added Keys extension.
-- **NEW**: Generalized custom fences (#60). `flow` and `sequence` fence are now just custom fences and can be disabled simply by overwriting the `custom_fences` setting.
+- **NEW**: Generalized custom fences (#60). `flow` and `sequence` fence are now just custom fences and can be disabled
+  simply by overwriting the `custom_fences` setting.
 - **NEW**: Remove deprecated `no_nl2br` in GitHub extension. (#24)
 - **NEW**: Remove deprecated HeaderAnchor extension. (#24)
 - **NEW**: Remove deprecated PyMdown extension. (#24)
@@ -372,7 +377,9 @@ Please see [Release Notes](./releases/6.0.md#6.0) for details on upgrading to 6.
 
 - **NEW**: Arithmatex now supports `\(...\)`, `\[...\]`, and `\begin{}...\end{}`.
 - **NEW**: Arithmatex has an option to embed the math code in MathJax script tags.
-- **FIX**: Unfortunately the wrap option is now run through an HTML escaper and HTML tags can no longer be fed in this way.  Arithmatex also now wraps "wrapped" content with spans to containerize content and keep one equation from bleeding into the next.
+- **FIX**: Unfortunately the wrap option is now run through an HTML escaper and HTML tags can no longer be fed in this
+  way.  Arithmatex also now wraps "wrapped" content with spans to containerize content and keep one equation from
+  bleeding into the next.
 - **FIX**: Better handling of escaped Arithmatex inline tokens.
 - **FIX**: Better handling of escaped InlineHilite tokens.
 - **FIX**: Update InlineHilite and SuperFences so that the language option can accept things like `c#` and `.net` etc.
diff --git a/pymdownx/__meta__.py b/pymdownx/__meta__.py
index 0f1d85ef..f5193d6b 100644
--- a/pymdownx/__meta__.py
+++ b/pymdownx/__meta__.py
@@ -185,5 +185,5 @@ def parse_version(ver, pre=False):
     return Version(major, minor, micro, release, pre, post, dev)
 
 
-__version_info__ = Version(8, 1, 1, "final")
+__version_info__ = Version(8, 1, 2, ".dev")
 __version__ = __version_info__._get_canonical()
diff --git a/pymdownx/details.py b/pymdownx/details.py
index dfd57013..0683be98 100644
--- a/pymdownx/details.py
+++ b/pymdownx/details.py
@@ -43,7 +43,21 @@ class DetailsProcessor(BlockProcessor):
         self.current_sibling = None
         self.content_indention = 0
 
-    def get_sibling(self, parent, block):
+    def detab_by_length(self, text, length):
+        """Remove a tab from the front of each line of the given text."""
+
+        newtext = []
+        lines = text.split('\n')
+        for line in lines:
+            if line.startswith(' ' * length):
+                newtext.append(line[length:])
+            elif not line.strip():
+                newtext.append('')  # pragma: no cover
+            else:
+                break
+        return '\n'.join(newtext), '\n'.join(lines[len(newtext):])
+
+    def parse_content(self, parent, block):
         """Get sibling details.
 
         Retrieve the appropriate sibling element. This can get tricky when
@@ -51,13 +65,16 @@ class DetailsProcessor(BlockProcessor):
 
         """
 
+        old_block = block
+        non_details = ''
+
         # We already acquired the block via test
         if self.current_sibling is not None:
             sibling = self.current_sibling
-            block = block[self.content_indent:]
+            block, non_details = self.detab_by_length(block, self.content_indent)
             self.current_sibling = None
             self.content_indent = 0
-            return sibling, block
+            return sibling, block, non_details
 
         sibling = self.lastChild(parent)
 
@@ -91,10 +108,12 @@ class DetailsProcessor(BlockProcessor):
                 sibling = None
 
             if sibling is not None:
+                indent += self.tab_length
+                block, non_details = self.detab_by_length(old_block, indent)
                 self.current_sibling = sibling
                 self.content_indent = indent
 
-        return sibling, block
+        return sibling, block, non_details
 
     def test(self, parent, block):
         """Test block."""
@@ -102,7 +121,7 @@ class DetailsProcessor(BlockProcessor):
         if self.START.search(block):
             return True
         else:
-            return self.get_sibling(parent, block)[0] is not None
+            return self.parse_content(parent, block)[0] is not None
 
     def run(self, parent, blocks):
         """Convert to details/summary block."""
@@ -115,11 +134,9 @@ class DetailsProcessor(BlockProcessor):
             if m.start() > 0:
                 self.parser.parseBlocks(parent, [block[:m.start()]])
             block = block[m.end():]
+            block, non_details = self.detab(block)
         else:
-            sibling, block = self.get_sibling(parent, block)
-
-        # Get the details block and and the non-details content
-        block, non_details = self.detab(block)
+            sibling, block, non_details = self.parse_content(parent, block)
 
         if m:
             state = m.group(1)
diff --git a/pymdownx/tabbed.py b/pymdownx/tabbed.py
index d6e955eb..b6a99ec7 100644
--- a/pymdownx/tabbed.py
+++ b/pymdownx/tabbed.py
@@ -43,7 +43,21 @@ class TabbedProcessor(BlockProcessor):
         self.current_sibling = None
         self.content_indention = 0
 
-    def get_sibling(self, parent, block):
+    def detab_by_length(self, text, length):
+        """Remove a tab from the front of each line of the given text."""
+
+        newtext = []
+        lines = text.split('\n')
+        for line in lines:
+            if line.startswith(' ' * length):
+                newtext.append(line[length:])
+            elif not line.strip():
+                newtext.append('')  # pragma: no cover
+            else:
+                break
+        return '\n'.join(newtext), '\n'.join(lines[len(newtext):])
+
+    def parse_content(self, parent, block):
         """Get sibling tab.
 
         Retrieve the appropriate sibling element. This can get tricky when
@@ -51,13 +65,16 @@ class TabbedProcessor(BlockProcessor):
 
         """
 
+        old_block = block
+        non_tabs = ''
+
         # We already acquired the block via test
         if self.current_sibling is not None:
             sibling = self.current_sibling
-            block = block[self.content_indent:]
+            block, non_tabs = self.detab_by_length(block, self.content_indent)
             self.current_sibling = None
             self.content_indent = 0
-            return sibling, block
+            return sibling, block, non_tabs
 
         sibling = self.lastChild(parent)
 
@@ -107,10 +124,12 @@ class TabbedProcessor(BlockProcessor):
                 sibling = None
 
             if sibling is not None:
+                indent += self.tab_length
+                block, non_tabs = self.detab_by_length(old_block, indent)
                 self.current_sibling = sibling
                 self.content_indent = indent
 
-        return sibling, block
+        return sibling, block, non_tabs
 
     def test(self, parent, block):
         """Test block."""
@@ -118,7 +137,7 @@ class TabbedProcessor(BlockProcessor):
         if self.START.search(block):
             return True
         else:
-            return self.get_sibling(parent, block)[0] is not None
+            return self.parse_content(parent, block)[0] is not None
 
     def run(self, parent, blocks):
         """Convert to tabbed block."""
@@ -132,11 +151,9 @@ class TabbedProcessor(BlockProcessor):
                 self.parser.parseBlocks(parent, [block[:m.start()]])
             block = block[m.end():]
             sibling = self.lastChild(parent)
+            block, non_tabs = self.detab(block)
         else:
-            sibling, block = self.get_sibling(parent, block)
-
-        # Get the tabs block and the non-tab content
-        block, non_tabs = self.detab(block)
+            sibling, block, non_tabs = self.parse_content(parent, block)
 
         if m:
             special = m.group(1) if m.group(1) else ''


Test patch:
diff --git a/tests/test_extensions/test_details.py b/tests/test_extensions/test_details.py
index 6b24b19e..ed868218 100644
--- a/tests/test_extensions/test_details.py
+++ b/tests/test_extensions/test_details.py
@@ -324,3 +324,59 @@ class TestDetails(util.MdCase):
             ''',
             True
         )
+
+    def test_details_complex_list(self):
+        """Test details complex list scenario."""
+
+        self.check_markdown(
+            '''
+            ???+ "With details block with loose lists"
+                - Parent 1
+
+                    - Child 1
+                    - Child 2
+            ''',
+            '''
+            <details open="open"><summary>With details block with loose lists</summary><ul>
+            <li>
+            <p>Parent 1</p>
+            <ul>
+            <li>Child 1</li>
+            <li>Child 2</li>
+            </ul>
+            </li>
+            </ul>
+            </details>
+            ''',
+            True
+        )
+
+    def test_details_complex_list_unindentd_content(self):
+        """Test details complex list scenario with un-indented content."""
+
+        self.check_markdown(
+            '''
+            ???+ "With details block with loose lists"
+                - Parent 1
+
+                    - Child 1
+                    - Child 2
+            - Parent 2
+            ''',
+            '''
+            <details open="open"><summary>With details block with loose lists</summary><ul>
+            <li>
+            <p>Parent 1</p>
+            <ul>
+            <li>Child 1</li>
+            <li>Child 2</li>
+            </ul>
+            </li>
+            </ul>
+            </details>
+            <ul>
+            <li>Parent 2</li>
+            </ul>
+            ''',
+            True
+        )
diff --git a/tests/test_extensions/test_tabbed.py b/tests/test_extensions/test_tabbed.py
index 5803dc0e..27e2a7a6 100644
--- a/tests/test_extensions/test_tabbed.py
+++ b/tests/test_extensions/test_tabbed.py
@@ -384,3 +384,63 @@ class TestLegacyTab(util.MdCase):
             ''',  # noqa: E501
             True
         )
+
+    def test_tabbed_complex_list(self):
+        """Test tabbed complex list scenario."""
+
+        self.check_markdown(
+            '''
+            === "Tab with loose lists"
+                - Parent 1
+
+                    - Child 1
+                    - Child 2
+            ''',
+            '''
+            <div class="tabbed-set" data-tabs="1:1"><input checked="checked" id="__tabbed_1_1" name="__tabbed_1" type="radio" /><label for="__tabbed_1_1">Tab with loose lists</label><div class="tabbed-content">
+            <ul>
+            <li>
+            <p>Parent 1</p>
+            <ul>
+            <li>Child 1</li>
+            <li>Child 2</li>
+            </ul>
+            </li>
+            </ul>
+            </div>
+            </div>
+            ''',  # noqa: E501
+            True
+        )
+
+    def test_tabbed_complex_list_unindented_content(self):
+        """Test tabbed complex list scenario with un-indented content."""
+
+        self.check_markdown(
+            '''
+            === "Tab with loose lists"
+                - Parent 1
+
+                    - Child 1
+                    - Child 2
+            - Parent 2
+            ''',
+            '''
+            <div class="tabbed-set" data-tabs="1:1"><input checked="checked" id="__tabbed_1_1" name="__tabbed_1" type="radio" /><label for="__tabbed_1_1">Tab with loose lists</label><div class="tabbed-content">
+            <ul>
+            <li>
+            <p>Parent 1</p>
+            <ul>
+            <li>Child 1</li>
+            <li>Child 2</li>
+            </ul>
+            </li>
+            </ul>
+            </div>
+            </div>
+            <ul>
+            <li>Parent 2</li>
+            </ul>
+            ''',  # noqa: E501
+            True
+        )