Repository: KyleKing/mdformat-mkdocs (language: python)

Issue:
nested bullet points do not correctly implement integer word wrapping
This is the output when `wrap = 80`:
```
- Test Test Test Test Test Test Test Test Test Test Test Test Test Test Test Tes
    Test Test Test Test Test Test Test Test Test Test Test Test Test Test Tested
    Te
    - Test Test Test Test Test Test Test Test Test Test Test Test Test Test Test
        Test Test Test Test Test Test Test Test Test Test Test Test Test Test Te Tes
    - Test Test Test Test Test Test Test Test Test Test Test Test Test Test Test
        Test Test Test Test Test Test Test Test Test Test Test Test Test Test Tes Te
```

The second line of the non-nested bullet point gets formatted correctly. However, the second lines of both nested bullet points are 84 characters, with both "Tes" and "Te" beginning at 81 and 82.

For fun, I also checked with numbered lists (example slightly modified in length due to `-` being one less character than `1.`)

This also affects numbered lists if `align-semantic-breaks` is not enabled:
```
1. Test
    1. Test Test Test Test Test Test Test Test Test Test Test Test Test Test Tes
        Test Test Test Test Test Test Test Test Test Test Test Test Test Test Te
        Te
    1. Test Test Test Test Test Test Test Test Test Test Test Test Test Test Tes
        Test Test Test Test Test Test Test Test Test Test Test Test Test Test Te T
    1. Test Test Test Test Test Test Test Test Test Test Test Test Test Test Tes
        Test Test Test Test Test Test Test Test Test Test Test Test Test Test Test
```
The second line of the second and third nested list items not handled correctly, though the second line of the first bullet point is. So it seems to be affected, but has a slightly different error cutoff due to the extra character in the `1.` over the `-`.

However, this does not affect numbered lists if `align-semantic-breaks` is enabled:
```
1. Test
    1. Test Test Test Test Test Test Test Test Test Test Test Test Test Test Tes
        Test Test Test Test Test Test Test Test Test Test Test Test Test Test Te
        Tes
    1. Test Test Test Test Test Test Test Test Test Test Test Test Test Test Tes
        Test Test Test Test Test Test Test Test Test Test Test Test Test Test Tes
        Te
```

Thank you very much :)


Discussion / hints:
Thanks for reporting the issue and trying out the plugin

This might be a problem with `mdformat` rather than here, but I'll take a look this week and try to replicate the bug and add some tests
Okay, I can replicate the behavior you've described. I've been testing:

```txt
# Content

- Test Test Test Test T5st Test Test Test Test T5st Test Test Test Test T5st Test Test Test Test T5st Test Test Test Test T5st Test Test Test Test T5st Tested Test
    - Test Test Test Test T5st Test Test Test Test T5st Test Test Test Test T5st Test Test Test Test T5st Test Test Test Test T5st Test Test Test Test T5st Tested Test
    - Test Test Test Test T5st Test Test Test Test T5st Test Test Test Test T5st Test Test Test Test T5st Test Test Test Test T5st Test Test Test Test T5st Tested Test 

1. Test
    1. Test Test Test Test T5st Test Test Test Test T5st Test Test Test Test T5st Test Test Test Test T5st Test Test Test Test T5st Test Test Test Test T5st Tested Test
    1. Test Test Test Test T5st Test Test Test Test T5st Test Test Test Test T5st Test Test Test Test T5st Test Test Test Test T5st Test Test Test Test T5st Tested Test
    1. Test Test Test Test T5st Test Test Test Test T5st Test Test Test Test T5st Test Test Test Test T5st Test Test Test Test T5st Test Test Test Test T5st Tested Test
```

Which formats with `mdformat tmp.md --wrap=80` to:

```txt
# Content

- Test Test Test Test T5st Test Test Test Test T5st Test Test Test Test T5st
    Test Test Test Test T5st Test Test Test Test T5st Test Test Test Test T5st
    Tested Test
    - Test Test Test Test T5st Test Test Test Test T5st Test Test Test Test T5st
        Test Test Test Test T5st Test Test Test Test T5st Test Test Test Test T5st
        Tested Test
    - Test Test Test Test T5st Test Test Test Test T5st Test Test Test Test T5st
        Test Test Test Test T5st Test Test Test Test T5st Test Test Test Test T5st
        Tested Test

1. Test
    1. Test Test Test Test T5st Test Test Test Test T5st Test Test Test Test T5st
        Test Test Test Test T5st Test Test Test Test T5st Test Test Test Test T5st
        Tested Test
    1. Test Test Test Test T5st Test Test Test Test T5st Test Test Test Test T5st
        Test Test Test Test T5st Test Test Test Test T5st Test Test Test Test T5st
        Tested Test
    1. Test Test Test Test T5st Test Test Test Test T5st Test Test Test Test T5st
        Test Test Test Test T5st Test Test Test Test T5st Test Test Test Test T5st
        Tested Test
```

Or with `mdformat tmp.md --align-semantic-breaks-in-lists --wrap=80` to:

```txt
# Content

- Test Test Test Test T5st Test Test Test Test T5st Test Test Test Test T5st
  Test Test Test Test T5st Test Test Test Test T5st Test Test Test Test T5st
  Tested Test
    - Test Test Test Test T5st Test Test Test Test T5st Test Test Test Test T5st
      Test Test Test Test T5st Test Test Test Test T5st Test Test Test Test T5st
      Tested Test
    - Test Test Test Test T5st Test Test Test Test T5st Test Test Test Test T5st
      Test Test Test Test T5st Test Test Test Test T5st Test Test Test Test T5st
      Tested Test

1. Test
    1. Test Test Test Test T5st Test Test Test Test T5st Test Test Test Test T5st
       Test Test Test Test T5st Test Test Test Test T5st Test Test Test Test T5st
       Tested Test
    1. Test Test Test Test T5st Test Test Test Test T5st Test Test Test Test T5st
       Test Test Test Test T5st Test Test Test Test T5st Test Test Test Test T5st
       Tested Test
    1. Test Test Test Test T5st Test Test Test Test T5st Test Test Test Test T5st
       Test Test Test Test T5st Test Test Test Test T5st Test Test Test Test T5st
       Tested Test
```

Which ends up with lines longer than the specified wrap length

```sh
> mdformat --version
mdformat 0.7.16 (mdformat_tables: 0.4.1, mdformat_gfm: 0.3.5, mdformat_mkdocs: 1.0.3)
```
I can verify that installing `mdformat-mkdocs` breaks the word wrap logic:

```sh
> mdformat --version
mdformat 0.7.16 (mdformat_tables: 0.4.1, mdformat_gfm: 0.3.5)
```

```txt
# Content

- Test Test Test Test T5st Test Test Test Test T5st Test Test Test Test T5st
  Test Test Test Test T5st Test Test Test Test T5st Test Test Test Test T5st
  Tested Test
  - Test Test Test Test T5st Test Test Test Test T5st Test Test Test Test T5st
    Test Test Test Test T5st Test Test Test Test T5st Test Test Test Test T5st
    Tested Test
  - Test Test Test Test T5st Test Test Test Test T5st Test Test Test Test T5st
    Test Test Test Test T5st Test Test Test Test T5st Test Test Test Test T5st
    Tested Test

1. Test
   1. Test Test Test Test T5st Test Test Test Test T5st Test Test Test Test T5st
      Test Test Test Test T5st Test Test Test Test T5st Test Test Test Test T5st
      Tested Test
   1. Test Test Test Test T5st Test Test Test Test T5st Test Test Test Test T5st
      Test Test Test Test T5st Test Test Test Test T5st Test Test Test Test T5st
      Tested Test
   1. Test Test Test Test T5st Test Test Test Test T5st Test Test Test Test T5st
      Test Test Test Test T5st Test Test Test Test T5st Test Test Test Test T5st
      Tested Test
```

```sh
> pipx inject mdformat mdformat-mkdocs
  injected package mdformat-mkdocs into venv mdformat
done! ✨ 🌟 ✨
> mdformat --version
mdformat 0.7.16 (mdformat_tables: 0.4.1, mdformat_gfm: 0.3.5, mdformat_mkdocs: 1.0.3)
```

```txt
# Content

- Test Test Test Test T5st Test Test Test Test T5st Test Test Test Test T5st
    Test Test Test Test T5st Test Test Test Test T5st Test Test Test Test T5st
    Tested Test
    - Test Test Test Test T5st Test Test Test Test T5st Test Test Test Test T5st
        Test Test Test Test T5st Test Test Test Test T5st Test Test Test Test T5st
        Tested Test
    - Test Test Test Test T5st Test Test Test Test T5st Test Test Test Test T5st
        Test Test Test Test T5st Test Test Test Test T5st Test Test Test Test T5st
        Tested Test

1. Test
    1. Test Test Test Test T5st Test Test Test Test T5st Test Test Test Test T5st
        Test Test Test Test T5st Test Test Test Test T5st Test Test Test Test T5st
        Tested Test
    1. Test Test Test Test T5st Test Test Test Test T5st Test Test Test Test T5st
        Test Test Test Test T5st Test Test Test Test T5st Test Test Test Test T5st
        Tested Test
    1. Test Test Test Test T5st Test Test Test Test T5st Test Test Test Test T5st
        Test Test Test Test T5st Test Test Test Test T5st Test Test Test Test T5st
        Tested Test
```
When adding the additional indents, I'm not accounting for how that impacts word wrap. I'll take a look at the `mdformat` implementation and will open a PR to fix `mdformat-mkdocs`

- `mdformat` only uses `textwrap` for the CLI output: https://github.com/executablebooks/mdformat/blob/0cbd2054dedf98ec8366001c8a16eacfa85cebc1/src/mdformat/_cli.py#L235-L250
- https://github.com/executablebooks/mdformat/pull/85/files#diff-71a6e939642f3fd453c2fb3358e64b10d9166b0c2f3c02e0e724228c6134c01a

Fix patch:
diff --git a/mdformat_mkdocs/__init__.py b/mdformat_mkdocs/__init__.py
index 389df48..8f97b2e 100644
--- a/mdformat_mkdocs/__init__.py
+++ b/mdformat_mkdocs/__init__.py
@@ -1,6 +1,6 @@
 """An mdformat plugin for mkdocs."""
 
-__version__ = "1.0.4rc1"
+__version__ = "1.0.4rc2"
 
 from .plugin import (  # noqa: F401
     POSTPROCESSORS,
diff --git a/mdformat_mkdocs/plugin.py b/mdformat_mkdocs/plugin.py
index 3b84c96..a2da3ab 100644
--- a/mdformat_mkdocs/plugin.py
+++ b/mdformat_mkdocs/plugin.py
@@ -17,6 +17,12 @@
 
 """
 
+FILLER_CHAR = "𝕏"
+"""A spacer that is inserted and then removed to ensure proper word wrap."""
+
+FILLER = FILLER_CHAR * (_MKDOCS_INDENT_COUNT - 2)  # `mdformat` default is two spaces
+"""A spacer that is inserted and then removed to ensure proper word wrap."""
+
 
 def add_cli_options(parser: argparse.ArgumentParser) -> None:
     """Add options to the mdformat CLI, to be stored in `mdit.options["mdformat"]`."""
@@ -85,10 +91,62 @@ def _normalize_list(text: str, node: RenderTreeNode, context: RenderContext) ->
         if _ALIGN_SEMANTIC_BREAKS_IN_LISTS and not list_match and is_semantic_indent:
             removed_indents = -1 if is_numbered else -2
             new_indent = new_indent[:removed_indents]
+        new_line = new_line.replace(f"{FILLER_CHAR} ", "").replace(FILLER_CHAR, "")
         rendered += f"{new_indent}{new_line.strip()}{eol}"
     return rendered.rstrip()
 
 
+def _postprocess_inline(text: str, node: RenderTreeNode, context: RenderContext) -> str:
+    """Postprocess inline tokens.
+
+    Fix word wrap for lists to account for the change in indentation.
+    We fool word wrap by prefixing an unwrappable dummy string of the same length.
+    This prefix needs to be later removed (in `_list_item_renderer`).
+
+    Adapted from: https://github.com/hukkin/mdformat-gfm/blob/cf316a121b6cf35cbff7b0ad6e171f287803f8cb/src/mdformat_gfm/plugin.py#L86-L111
+
+    """
+    if not context.do_wrap:
+        return text
+    wrap_mode = context.options["mdformat"]["wrap"]
+    if (
+        not isinstance(wrap_mode, int)
+        or FILLER_CHAR in text  # noqa: W503
+        or (node.parent and node.parent.type != "paragraph")  # noqa: W503
+        or (node.parent.parent and node.parent.parent.type != "list_item")  # noqa: W503
+    ):
+        return text
+
+    _counter = -1
+    parent = node.parent
+    while parent and parent.type == "paragraph":
+        parent = parent.parent
+        _counter += 1
+    indent_count = max(_counter, 0)
+
+    soft_break = "\x00"
+    text = text.lstrip(soft_break).lstrip()
+    filler = (FILLER * indent_count)[:-1] if indent_count else ""
+    newline_filler = filler + FILLER if indent_count else FILLER[:-1]
+    if len(text) > wrap_mode:
+        indent_length = _MKDOCS_INDENT_COUNT * indent_count
+        wrapped_length = -123
+        words = []
+        for word in text.split(soft_break):
+            next_length = wrapped_length + len(word)
+            if not words:
+                words = [filler, word]
+                wrapped_length = indent_length + len(word)
+            elif next_length > wrap_mode:
+                words += [word, newline_filler]
+                wrapped_length = indent_length + len(word)
+            else:
+                words.append(word)
+                wrapped_length = next_length + 1
+        return soft_break.join(_w for _w in words if _w)
+    return f"{filler}{soft_break}{text}" if filler else text
+
+
 # A mapping from `RenderTreeNode.type` to a `Render` function that can
 # render the given `RenderTreeNode` type. These override the default
 # `Render` funcs defined in `mdformat.renderer.DEFAULT_RENDERERS`.
@@ -102,6 +160,7 @@ def _normalize_list(text: str, node: RenderTreeNode, context: RenderContext) ->
 POSTPROCESSORS: Mapping[str, Postprocess] = {
     "bullet_list": _normalize_list,
     "ordered_list": _normalize_list,
+    "inline": _postprocess_inline,
 }
 
 # See: https://github.com/executablebooks/mdformat/blob/5d9b573ce33bae219087984dd148894c774f41d4/src/mdformat/plugins.py


Test patch:
diff --git a/tests/test_align_semantic_breaks_in_lists.py b/tests/test_align_semantic_breaks_in_lists.py
index 7783a47..d202eda 100644
--- a/tests/test_align_semantic_breaks_in_lists.py
+++ b/tests/test_align_semantic_breaks_in_lists.py
@@ -14,7 +14,7 @@
 def test_align_semantic_breaks_in_lists(line, title, text, expected):
     output = mdformat.text(
         text,
-        options={"align_semantic_breaks_in_lists": True},
+        options={"align_semantic_breaks_in_lists": True, "wrap": "keep"},
         extensions={"mkdocs"},
     )
     assert output.rstrip() == expected.rstrip()
diff --git a/tests/test_fixtures.py b/tests/test_fixtures.py
index ec8886b..5daf3bb 100644
--- a/tests/test_fixtures.py
+++ b/tests/test_fixtures.py
@@ -12,5 +12,5 @@
     "line,title,text,expected", fixtures, ids=[f[1] for f in fixtures]
 )
 def test_fixtures(line, title, text, expected):
-    output = mdformat.text(text, extensions={"mkdocs"})
+    output = mdformat.text(text, extensions={"mkdocs"}, options={"wrap": "keep"})
     assert output.rstrip() == expected.rstrip()
diff --git a/tests/test_wrap.py b/tests/test_wrap.py
new file mode 100644
index 0000000..210841a
--- /dev/null
+++ b/tests/test_wrap.py
@@ -0,0 +1,116 @@
+import mdformat
+import pytest
+
+# Note: indented text that starts with a number is parsed as the start of a numbered list
+
+CASE_1 = """
+# Content
+
+- Test Testing Test Testing Test Testing Test Testing Test Testing
+    Test Testing
+  - Test Testing Test Testing Test Testing Test Testing Test Testing
+      Test Testing Test Testing Test Testing Test Testing Test Testing
+      Test Testing
+
+1. Test Testing Test Testing Test Testing Test Testing Test Testing
+    Test Testing
+  1. Test Testing Test Testing Test Testing Test Testing Test Testing
+      Test Testing Test Testing Test Testing Test Testing Test Testing
+      Test Testing
+"""
+
+CASE_1_FALSE_40 = """
+# Content
+
+- Test Testing Test Testing Test Testing
+    Test Testing Test Testing Test
+    Testing
+    - Test Testing Test Testing Test
+        Testing Test Testing Test Testing
+        Test Testing Test Testing Test
+        Testing Test Testing Test Testing
+        Test Testing
+
+1. Test Testing Test Testing Test
+    Testing Test Testing Test Testing
+    Test Testing
+1. Test Testing Test Testing Test
+    Testing Test Testing Test Testing
+    Test Testing Test Testing Test
+    Testing Test Testing Test Testing
+    Test Testing
+"""
+
+CASE_1_FALSE_80 = """
+# Content
+
+- Test Testing Test Testing Test Testing Test Testing Test Testing Test Testing
+    - Test Testing Test Testing Test Testing Test Testing Test Testing Test
+        Testing Test Testing Test Testing Test Testing Test Testing Test Testing
+
+1. Test Testing Test Testing Test Testing Test Testing Test Testing Test Testing
+1. Test Testing Test Testing Test Testing Test Testing Test Testing Test Testing
+    Test Testing Test Testing Test Testing Test Testing Test Testing
+"""
+
+CASE_1_TRUE_40 = """
+# Content
+
+- Test Testing Test Testing Test Testing
+  Test Testing Test Testing Test
+  Testing
+    - Test Testing Test Testing Test
+      Testing Test Testing Test Testing
+      Test Testing Test Testing Test
+      Testing Test Testing Test Testing
+      Test Testing
+
+1. Test Testing Test Testing Test
+   Testing Test Testing Test Testing
+   Test Testing
+1. Test Testing Test Testing Test
+   Testing Test Testing Test Testing
+   Test Testing Test Testing Test
+   Testing Test Testing Test Testing
+   Test Testing
+"""
+
+CASE_1_TRUE_80 = """
+# Content
+
+- Test Testing Test Testing Test Testing Test Testing Test Testing Test Testing
+    - Test Testing Test Testing Test Testing Test Testing Test Testing Test
+      Testing Test Testing Test Testing Test Testing Test Testing Test Testing
+
+1. Test Testing Test Testing Test Testing Test Testing Test Testing Test Testing
+1. Test Testing Test Testing Test Testing Test Testing Test Testing Test Testing
+   Test Testing Test Testing Test Testing Test Testing Test Testing
+"""
+
+
+@pytest.mark.parametrize(
+    "text,expected,align_lists,wrap",
+    [
+        (CASE_1, CASE_1_FALSE_40, False, 40),
+        (CASE_1, CASE_1_FALSE_80, False, 80),
+        (CASE_1, CASE_1_TRUE_40, True, 40),
+        (CASE_1, CASE_1_TRUE_80, True, 80),
+    ],
+    ids=[
+        "CASE_1_FALSE_40",
+        "CASE_1_FALSE_80",
+        "CASE_1_TRUE_40",
+        "CASE_1_TRUE_80",
+    ],
+)
+def test_wrap(text: str, expected: str, align_lists: bool, wrap: int):
+    output = mdformat.text(
+        text,
+        options={"align_semantic_breaks_in_lists": align_lists, "wrap": wrap},
+        extensions={"mkdocs"},
+    )
+
+    print(output.strip())
+    print("-- Expected --")
+    print(expected.strip())
+    assert output.strip() == expected.strip()