Repository: jackdewinter/pymarkdown

Issue:
Parsing Error: Md031 test case causes block quote to be left open too long
## Prerequisites

* [X] Are you running the latest version of this application?
* [X] Have you checked the [Frequently Asked Questions](https://pymarkdown.readthedocs.io/en/latest/faq) document?
* [X] Have you simplified the bug report to the essential details?
  * [X] Do you have a distinct command line to report?
  * [X] Can you clearly state the configuration for this bug report?
  * [X] Do you have a minimal document that highlights this bug?
  * [X] Are any required files (configuration or Markdown document) attached to the issue?
* [X] Did you perform a cursory search of other issues to look for related issues?

## Bug Report

### Bug Type

* [X] Parsing Error

### Description

When parsing the test case test_extra_047f6 (extracted from one of the Md031 cases, bad_fenced_block_in_block_quote_with_previous_inner_blocks), the block quote is left open until after the fenced code block is opened.  This results in the fenced code block being closed as soon as it is opened.

### Specifics

What operating system and version are you running into this behavior on?

Win11

What version are you seeing this behavior in? (Run `pip list` or `pipenv run pip list` and look for the entry beside `pymarkdownlnt`.)

main, latest

Are there any extra steps that need to be taken before executing the application?

No

What is the command line you invoke to get this behavior?

test_extra_047f6

Are you using a configuration file? Either on the command line or one of the implicit configuration files? If so, attach that file to this issue.

No

What Markdown document causes this behavior to manifest?  Attach that file to this issue.

```
> > inner block
> > > innermost block
> > > innermost block
> > inner block
> ```block
> A code block
> ```
>This is a blank line and some text.
```

### Actual Behavior

Line 5 should close the block quote and open the fenced code block.

### Expected Behavior

Should close the block quote before the fenced code block.

Discussion:
sourcery-ai[bot]: <!-- Generated by sourcery-ai[bot]: start review_guide -->

🧙 Sourcery is reviewing your pull request!

---

<details>
<summary>Tips</summary>

- Trigger a new Sourcery review by commenting `@sourcery-ai review` on the pull request.
- Continue your discussion with Sourcery by replying directly to review comments.
- You can change your review settings at any time by accessing your [dashboard](https://app.sourcery.ai):
    - Enable or disable the Sourcery-generated pull request summary or reviewer's guide;
    - Change the review language;
- You can always [contact us](mailto:support@sourcery.ai) if you have any questions or feedback.

</details>

<!-- Generated by sourcery-ai[bot]: end review_guide -->
codecov[bot]: ## [Codecov](https://app.codecov.io/gh/jackdewinter/pymarkdown/pull/1215?dropdown=coverage&src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=None) Report
All modified and coverable lines are covered by tests :white_check_mark:
> Project coverage is 100.00%. Comparing base [(`0983b67`)](https://app.codecov.io/gh/jackdewinter/pymarkdown/commit/0983b67ce19c1986d7b6b35a1e697e53be12c91f?dropdown=coverage&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=None) to head [(`278589b`)](https://app.codecov.io/gh/jackdewinter/pymarkdown/commit/278589bae2a40c35fbb29a08e2fef6ae117997c7?dropdown=coverage&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=None).

<details><summary>Additional details and impacted files</summary>


```diff
@@            Coverage Diff            @@
##              main     #1215   +/-   ##
=========================================
  Coverage   100.00%   100.00%           
=========================================
  Files          190       190           
  Lines        20926     20941   +15     
  Branches      2677      2679    +2     
=========================================
+ Hits         20926     20941   +15     
```

</details>

[:umbrella: View full report in Codecov by Sentry](https://app.codecov.io/gh/jackdewinter/pymarkdown/pull/1215?dropdown=coverage&src=pr&el=continue&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=None).   
:loudspeaker: Have feedback on the report? [Share it here](https://about.codecov.io/codecov-pr-comment-feedback/?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=None).


Patch:
diff --git a/publish/coverage.json b/publish/coverage.json
index 3985ebae..66b479a8 100644
--- a/publish/coverage.json
+++ b/publish/coverage.json
@@ -2,12 +2,12 @@
     "projectName": "pymarkdown",
     "reportSource": "pytest",
     "branchLevel": {
-        "totalMeasured": 5355,
-        "totalCovered": 5355
+        "totalMeasured": 5359,
+        "totalCovered": 5359
     },
     "lineLevel": {
-        "totalMeasured": 20926,
-        "totalCovered": 20926
+        "totalMeasured": 20941,
+        "totalCovered": 20941
     }
 }
 
diff --git a/pymarkdown/container_blocks/container_helper.py b/pymarkdown/container_blocks/container_helper.py
index 4d9b85c1..1dd0c0db 100644
--- a/pymarkdown/container_blocks/container_helper.py
+++ b/pymarkdown/container_blocks/container_helper.py
@@ -79,6 +79,26 @@ class ContainerHelper:
                 extracted_whitespace = ""
         return did_once, extracted_whitespace, whitespace_prefix
 
+    @staticmethod
+    def __xx(parser_state: ParserState, extra_bqs: int) -> List[MarkdownToken]:
+        until_index = len(parser_state.token_stack) - 1
+        needed_bqs = extra_bqs
+        while (
+            until_index > 0
+            and needed_bqs > 0
+            and parser_state.token_stack[until_index].is_block_quote
+        ):
+            until_index -= 1
+            needed_bqs -= 1
+        x_tokens, _ = parser_state.close_open_blocks_fn(
+            parser_state,
+            include_block_quotes=True,
+            was_forced=True,
+            until_this_index=until_index + 1,
+        )
+        POGGER.debug("x_tokens>>:$:<", x_tokens)
+        return x_tokens
+
     # pylint: disable=too-many-arguments, too-many-locals
     @staticmethod
     def __reduce_containers_if_required_bq(
@@ -88,15 +108,13 @@ class ContainerHelper:
         split_tab: bool,
         extracted_whitespace: str,
         grab_bag: ContainerGrabBag,
+        extra_bqs: int,
     ) -> Tuple[bool, str, Optional[str]]:
-        x_tokens, _ = parser_state.close_open_blocks_fn(
-            parser_state,
-            include_block_quotes=True,
-            was_forced=True,
-            until_this_index=len(parser_state.token_stack) - 1,
-        )
-        POGGER.debug("x_tokens>>:$:<", x_tokens)
-        assert len(x_tokens) == 1, "Should have generated only one token."
+        x_tokens = ContainerHelper.__xx(parser_state, extra_bqs)
+        assert (
+            len(x_tokens) == extra_bqs
+        ), "Should have generated the requested number of tokens."
+
         first_new_token = cast(EndMarkdownToken, x_tokens[0])
 
         did_reduce_list, extracted_whitespace, whitespace_prefix = (
@@ -154,10 +172,10 @@ class ContainerHelper:
             first_new_token.set_extra_end_data(last_newline_part)
 
         new_tokens.extend(x_tokens)
-        xx = ContainerHelper.__handle_whitespace_prefix(
+        new_whitespace_prefix = ContainerHelper.__handle_whitespace_prefix(
             parser_state, whitespace_prefix, last_newline_part
         )
-        return split_tab, extracted_whitespace, xx
+        return split_tab, extracted_whitespace, new_whitespace_prefix
 
     # pylint: enable=too-many-arguments, too-many-locals
 
@@ -165,7 +183,7 @@ class ContainerHelper:
     def __handle_whitespace_prefix(
         parser_state: ParserState, whitespace_prefix: str, last_newline_part: str
     ) -> Optional[str]:
-        xx = None
+        new_whitespace_prefix = None
         if whitespace_prefix:
             indent_level = 0
             stack_index = len(parser_state.token_stack) - 1
@@ -185,8 +203,8 @@ class ContainerHelper:
                 last_split_bleading_spaces = len(split_bleading_spaces[-1])
                 indent_level += last_split_bleading_spaces
                 stack_index -= 1
-            xx = last_newline_part[indent_level:]
-        return xx
+            new_whitespace_prefix = last_newline_part[indent_level:]
+        return new_whitespace_prefix
 
     # pylint: disable=too-many-arguments
     @staticmethod
@@ -227,6 +245,7 @@ class ContainerHelper:
                     split_tab,
                     extracted_whitespace,
                     grab_bag,
+                    block_quote_data.stack_count - block_quote_data.current_count,
                 )
             )
 
diff --git a/pymarkdown/transform_markdown/transform_containers.py b/pymarkdown/transform_markdown/transform_containers.py
index 6432e28b..725d64d9 100644
--- a/pymarkdown/transform_markdown/transform_containers.py
+++ b/pymarkdown/transform_markdown/transform_containers.py
@@ -421,6 +421,19 @@ class TransformContainers:
         container_indices_copy.extend(removed_token_indices[::-1])
 
         if container_stack_copy[-1].is_block_quote_start:
+
+            # Temporary until we have more data. test_extra_047f6x
+            if (
+                len(removed_tokens) == 2
+                and container_stack_copy[-2].is_block_quote_start
+                and container_stack_copy[-3].is_block_quote_start
+            ):
+                del removed_tokens[0]
+                del removed_token_indices[0]
+                del removed_tokens[0]
+                del removed_token_indices[0]
+                return ""
+
             bq_spaces = cast(
                 BlockQuoteMarkdownToken, container_stack_copy[-1]
             ).bleading_spaces


Test patch:
diff --git a/publish/test-results.json b/publish/test-results.json
index 8a988312..6eae7fb0 100644
--- a/publish/test-results.json
+++ b/publish/test-results.json
@@ -1620,10 +1620,10 @@
         },
         {
             "name": "test.test_markdown_extra",
-            "totalTests": 333,
+            "totalTests": 336,
             "failedTests": 0,
             "errorTests": 0,
-            "skippedTests": 2,
+            "skippedTests": 1,
             "elapsedTimeInMilliseconds": 0
         },
         {
diff --git a/test/rules/test_md031.py b/test/rules/test_md031.py
index 6a8062d8..bcebb3d7 100644
--- a/test/rules/test_md031.py
+++ b/test/rules/test_md031.py
@@ -874,7 +874,7 @@ This is a blank line and some text.
 >This is a blank line and some text.
 """,
     ),
-    pluginRuleTest(
+    pluginRuleTest(  # test_extra_047f6b test_extra_047f6c
         "bad_fenced_block_in_block_quote_with_previous_inner_blocks_with_thematics",
         source_file_contents="""> > inner block
 > > > innermost block
@@ -888,7 +888,8 @@ This is a blank line and some text.
 >This is a blank line and some text.
 """,
         use_debug=True,
-        disable_rules="md022,md026",
+        disable_rules="md022,md026,md027",
+        # use_fix_debug=True,
         scan_expected_return_code=1,
         scan_expected_output="""{temp_source_path}:6:3: MD031: Fenced code blocks should be surrounded by blank lines (blanks-around-fences)
 {temp_source_path}:8:3: MD031: Fenced code blocks should be surrounded by blank lines (blanks-around-fences)
diff --git a/test/test_markdown_extra.py b/test/test_markdown_extra.py
index 9d4660d8..3aed14d5 100644
--- a/test/test_markdown_extra.py
+++ b/test/test_markdown_extra.py
@@ -16846,12 +16846,13 @@ abc
     act_and_assert(source_markdown, expected_gfm, expected_tokens)
 
 
-@pytest.mark.skip
 @pytest.mark.gfm
-def test_extra_047f6():  # https://github.com/jackdewinter/pymarkdown/issues/1213
+def test_extra_047f6x():  # https://github.com/jackdewinter/pymarkdown/issues/1213
     """
     TBD
     bad_fenced_block_in_block_quote_with_previous_inner_blocks
+
+    Note: temporary fix in __abcd_final( in transform containers.
     """
 
     # Arrange
@@ -16875,15 +16876,68 @@ def test_extra_047f6():  # https://github.com/jackdewinter/pymarkdown/issues/121
         "[text(2,7):innermost block\ninnermost block\ninner block::\n\n]",
         "[end-para:::False]",
         "[end-block-quote::> :True]",
-        "[fcode-block(5,3):`:3:block:::::]",
-        "[end-fcode-block::::True]",
         "[end-block-quote:::True]",
-        "[para(6,3):]",
+        "[fcode-block(5,3):`:3:block:::::]",
         "[text(6,3):A code block:]",
+        "[end-fcode-block:::3:False]",
+        "[para(8,2):]",
+        "[text(8,2):This is a blank line and some text.:]",
+        "[end-para:::True]",
+        "[end-block-quote:::True]",
+        "[BLANK(9,1):]",
+    ]
+    expected_gfm = """<blockquote>
+<blockquote>
+<p>inner block</p>
+<blockquote>
+<p>innermost block
+innermost block
+inner block</p>
+</blockquote>
+</blockquote>
+<pre><code class="language-block">A code block
+</code></pre>
+<p>This is a blank line and some text.</p>
+</blockquote>"""
+
+    # Act & Assert
+    act_and_assert(source_markdown, expected_gfm, expected_tokens)
+
+
+@pytest.mark.gfm
+def test_extra_047f6a():
+    """
+    TBD
+    """
+
+    # Arrange
+    source_markdown = """> > inner block
+> > > innermost block
+> > > innermost block
+> > inner block
+> > ```block
+> > A code block
+> > ```
+>This is a blank line and some text.
+"""
+    expected_tokens = [
+        "[block-quote(1,1)::>\n]",
+        "[block-quote(1,3)::> > \n> > \n> > ]",
+        "[para(1,5):]",
+        "[text(1,5):inner block:]",
+        "[end-para:::True]",
+        "[block-quote(2,1)::> > > \n> > > \n> > \n> > ]",
+        "[para(2,7):\n\n]",
+        "[text(2,7):innermost block\ninnermost block\ninner block::\n\n]",
         "[end-para:::False]",
-        "[fcode-block(7,3):`:3::::::]",
-        "[text(8,3):This is a blank line and some text.:]",
-        "[end-fcode-block::::True]",
+        "[end-block-quote::> > :True]",
+        "[fcode-block(5,5):`:3:block:::::]",
+        "[text(6,5):A code block:]",
+        "[end-fcode-block:::3:False]",
+        "[end-block-quote:::True]",
+        "[para(8,2):]",
+        "[text(8,2):This is a blank line and some text.:]",
+        "[end-para:::True]",
         "[end-block-quote:::True]",
         "[BLANK(9,1):]",
     ]
@@ -16895,9 +16949,137 @@ def test_extra_047f6():  # https://github.com/jackdewinter/pymarkdown/issues/121
 innermost block
 inner block</p>
 </blockquote>
+<pre><code class="language-block">A code block
+</code></pre>
 </blockquote>
+<p>This is a blank line and some text.</p>
+</blockquote>"""
+
+    # Act & Assert
+    act_and_assert(source_markdown, expected_gfm, expected_tokens)
+
+
+@pytest.mark.gfm
+def test_extra_047f6b():
+    """
+    TBD
+    bad_fenced_block_in_block_quote_with_previous_inner_blocks_with_thematics
+    """
+
+    # Arrange
+    source_markdown = """> > inner block
+> > > innermost block
+> > > innermost block
+> > inner block
+> ___
+> ```block
+> A code block
+> ```
+> ___
+>This is a blank line and some text.
+"""
+    expected_tokens = [
+        "[block-quote(1,1)::> \n> \n> \n> \n>\n]",
+        "[block-quote(1,3)::> > ]",
+        "[para(1,5):]",
+        "[text(1,5):inner block:]",
+        "[end-para:::True]",
+        "[block-quote(2,1)::> > > \n> > > \n> > \n> ]",
+        "[para(2,7):\n\n]",
+        "[text(2,7):innermost block\ninnermost block\ninner block::\n\n]",
+        "[end-para:::False]",
+        "[end-block-quote::> :True]",
+        "[end-block-quote:::True]",
+        "[tbreak(5,3):_::___]",
+        "[fcode-block(6,3):`:3:block:::::]",
+        "[text(7,3):A code block:]",
+        "[end-fcode-block:::3:False]",
+        "[tbreak(9,3):_::___]",
+        "[para(10,2):]",
+        "[text(10,2):This is a blank line and some text.:]",
+        "[end-para:::True]",
+        "[end-block-quote:::True]",
+        "[BLANK(11,1):]",
+    ]
+    expected_gfm = """<blockquote>
+<blockquote>
+<p>inner block</p>
+<blockquote>
+<p>innermost block
+innermost block
+inner block</p>
+</blockquote>
+</blockquote>
+<hr />
+<pre><code class="language-block">A code block
+</code></pre>
+<hr />
+<p>This is a blank line and some text.</p>
+</blockquote>"""
+
+    # Act & Assert
+    act_and_assert(source_markdown, expected_gfm, expected_tokens)
+
+
+@pytest.mark.gfm
+def test_extra_047f6c():
+    """
+    TBD
+    bad_fenced_block_in_block_quote_with_previous_inner_blocks_with_thematics
+    """
+
+    # Arrange
+    source_markdown = """> > inner block
+> > > innermost block
+> > > innermost block
+> > inner block
+> ___
+>
+> ```block
+> A code block
+> ```
+>
+> ___
+>This is a blank line and some text.
+"""
+    expected_tokens = [
+        "[block-quote(1,1)::>\n> \n> \n> \n>\n> \n>\n]",
+        "[block-quote(1,3)::> > ]",
+        "[para(1,5):]",
+        "[text(1,5):inner block:]",
+        "[end-para:::True]",
+        "[block-quote(2,1)::> > > \n> > > \n> > \n> ]",
+        "[para(2,7):\n\n]",
+        "[text(2,7):innermost block\ninnermost block\ninner block::\n\n]",
+        "[end-para:::False]",
+        "[end-block-quote::> :True]",
+        "[end-block-quote:::True]",
+        "[tbreak(5,3):_::___]",
+        "[BLANK(6,2):]",
+        "[fcode-block(7,3):`:3:block:::::]",
+        "[text(8,3):A code block:]",
+        "[end-fcode-block:::3:False]",
+        "[BLANK(10,2):]",
+        "[tbreak(11,3):_::___]",
+        "[para(12,2):]",
+        "[text(12,2):This is a blank line and some text.:]",
+        "[end-para:::True]",
+        "[end-block-quote:::True]",
+        "[BLANK(13,1):]",
+    ]
+    expected_gfm = """<blockquote>
+<blockquote>
+<p>inner block</p>
+<blockquote>
+<p>innermost block
+innermost block
+inner block</p>
+</blockquote>
+</blockquote>
+<hr />
 <pre><code class="language-block">A code block
 </code></pre>
+<hr />
 <p>This is a blank line and some text.</p>
 </blockquote>"""