Repository: pelme/htpy

Issue:
html2htpy includes unneccessary white space
```
$ echo \<p\>     \t\t\nHi\n\n\t  \</p\> | html2htpy
from htpy import p

p[""" 		Hi	 """]
```

it would be nicer of the output was just

`p["Hi"]`

Discussion:
@OleJoik: I remember concidering to strip the strings, but decided against it for some reason. Cant quite recall why, but might have had to do with some cases where I actually preferred whitespace to stay as is. Possibly theres some quirks with this in multi line strings. 

I can take a look@pelme: I am not sure what the best exact rules would be, not all white space are safe to remove. I guess there are html minifiers and similar that could be used as inspiration for how to properly strip some of the whitespace@OleJoik: One special case at least would be `<pre>` tags. It should preserve most spaces and newlines.
 [https://developer.mozilla.org/en-US/docs/Web/HTML/Element/pre](https://developer.mozilla.org/en-US/docs/Web/HTML/Element/pre)@OleJoik: Another thing to consider here; other html tags also can have goofy behavior as relates to css properties such as `white-space` and `white-space-collapse`. After reviewing this I'm not sure ever stripping whitespace is a good idea, as in quite a few cases the white space does matter.

To play around a bit, I made a [codepen](https://codepen.io/olejoik/pen/ogvONYv) based on [this mdn example](https://developer.mozilla.org/en-US/docs/Web/CSS/white-space-collapse#examples) that showcases a couple of cases where we would __not__ want (all) whitespace trimmed. Below is a little teaser:

```html
<!-- Example: Newlines preserved -->
<h2 style="white-space-collapse: preserve-breaks;">
  
  In this case
  all  the   newlines   are   preserved
  in    the     heading, 
  but    spaces    are    collapsed     .
  
</h2>
```

One option might be to filter for these properties and handle them in a special way. I'm not sure if I'd recommend it though, as it might quickly get complicated. w3 spec defines a non trivial set of [white space processing rules](https://www.w3.org/TR/css-text-3/#white-space-rules) that we should probably try to abide by if we do decide to do some trimmin'.

Edit to add: These properties might also be inherited or assigned with a class as in the code-pen, so it would be impossible to accurately recognize when they're applied to a given string.@OleJoik: An option might be to disregard the css props and trim away, though leave `pre` elements as is. The benefit might outweigh the cost for those few (?) users going crazy with whitespace css attributes@pelme: What about:

- `text.replace("  ", " ").strip()` all nodes except `<pre>`, `<code>` and `<textarea>` (maybe there are a few others too)
- Add `--preserve-whitespace` to opt-out of this behaviour.

When copy/pasting HTML code from the internet, it is often filled with spaces/white space that makes the HTML document look nice but is just there because it makes the HTML source look good rather then required from a functionality point of view.
@pelme: To be clear: I do not think we should bother about `white-space-collapse: preserve-breaks`. I never used it myself AFAIK and think it is pretty rare. html2py is mostly meant as a starting point when converting a bulk of html to htpy but requires manual inspection/fixups.@OleJoik: I agree with your comments, and this should be pretty simple to implement I guess. I have some time and can bring a PR tomorrow?@pelme: That is very welcome! :)@OleJoik: #84

Solution patch:
diff --git a/docs/html2htpy.md b/docs/html2htpy.md
index f5b0fb1..f9f12e7 100644
--- a/docs/html2htpy.md
+++ b/docs/html2htpy.md
@@ -108,7 +108,7 @@ powershell Get-Clipboard | html2htpy > output.py
 `html2htpy` can format the output Python code using `black` or `ruff`.
 Select the preferred formatter with the `-f`/`--format` flag. Options are `auto`, `ruff`, `black` and `none`.
 
-By default, the selection will be `auto`, formatting if it finds a formatter on path, prefering `ruff` if it's available.
+By default, the selection will be `auto`, formatting if it finds a formatter on path, prefering `black` if it's available.
 If no formatters are available on path, the output will not be formatted.
 
 ## Import Options
@@ -181,7 +181,7 @@ See the example below:
   <h3>Instructions:</h3>
   <ol>
     {% for step in recipe.steps %}
-    <li>{{ step }}</li>
+      <li>{{ step }}</li>
     {% endfor %}
   </ol>
 </body>
@@ -196,10 +196,6 @@ body[
     h2[f"Recipe of the Day: { recipe.name }"],
     p[f"{ recipe.description }"],
     h3["Instructions:"],
-    ol[
-        """        {% for step in recipe.steps %}        """,
-        li[f"{ step }"],
-        """        {% endfor %}    """,
-    ],
+    ol[" {% for step in recipe.steps %}", li[f"{ step }"], " {% endfor %}"],
 ]
 ```
diff --git a/docs/usage.md b/docs/usage.md
index f007c43..6cfd9dd 100644
--- a/docs/usage.md
+++ b/docs/usage.md
@@ -6,6 +6,7 @@ Elements are imported directly from the `htpy` module as their name. HTML attrib
 >>> from htpy import div
 >>> print(div(id="hi")["Hello!"])
 <div id="hi">Hello!</div>
+
 ```
 
 ## Elements
@@ -18,6 +19,7 @@ Elements can be arbitrarily nested:
 >>> from htpy import article, section, p
 >>> print(section[article[p["Lorem ipsum"]]])
 <section><article><p>Lorem ipsum</p></article></section>
+
 ```
 
 ### Text/Strings
@@ -28,6 +30,7 @@ It is possible to pass a string directly:
 >>> from htpy import h1
 >>> print(h1["Welcome to my site!"])
 <h1>Welcome to my site!</h1>
+
 ```
 
 Strings are automatically escaped to avoid [XSS
@@ -39,6 +42,7 @@ and safe to directly insert variable data via f-strings:
 >>> user_supplied_name = "bobby </h1>"
 >>> print(h1[f"hello {user_supplied_name}"])
 <h1>hello bobby &lt;/h1&gt;</h1>
+
 ```
 
 ### Conditional Rendering
@@ -59,12 +63,13 @@ You can use this to conditionally render content with inline `and` and
 <div></div>
 
 >>> error = 'Enter a valid email address.'
->>> print(div[has_error and b[error_message]])
+>>> print(div[error and b[error]])
 <div><b>Enter a valid email address.</b></div>
 
 # Inline if/else can also be used:
 >>> print(div[b[error] if error else None])
 <div><b>Enter a valid email address.</b></div>
+
 ```
 
 ```pycon title="Conditional rendering based on a bool variable"
@@ -83,7 +88,8 @@ You can use this to conditionally render content with inline `and` and
 
 >>> is_allowed = False
 >>> print(div[is_allowed or "Access denied!"])
-<div>Access denied</div>
+<div>Access denied!</div>
+
 ```
 
 ### Loops / Iterating Over Children
@@ -94,6 +100,7 @@ You can pass a list, tuple or generator to generate multiple children:
 >>> from htpy import ul, li
 >>> print(ul[(li[letter] for letter in "abc")])
 <ul><li>a</li><li>b</li><li>c</li></ul>
+
 ```
 
 !!! note
@@ -109,6 +116,7 @@ A `list` can be used similar to a [JSX fragment](https://react.dev/reference/rea
 >>> my_images = [img(src="a.jpg"), img(src="b.jpg")]
 >>> print(div[my_images])
 <div><img src="a.jpg"><img src="b.jpg"></div>
+
 ```
 
 ### Custom Elements / Web Components
@@ -122,6 +130,7 @@ used in Python identifiers, use underscore (`_`) instead:
 >>> from htpy import my_custom_element
 >>> print(my_custom_element['hi!'])
 <my-custom-element>hi!</my-custom-element>
+
 ```
 
 ### Injecting Markup
@@ -135,6 +144,7 @@ library. markupsafe is a dependency of htpy and is automatically installed:
 >>> from markupsafe import Markup
 >>> print(div[Markup("<foo></foo>")])
 <div><foo></foo></div>
+
 ```
 
 If you are generating [Markdown](https://pypi.org/project/Markdown/) and want to insert it into an element, 
@@ -146,6 +156,7 @@ use `Markup` to mark it as safe:
 >>> from htpy import div
 >>> print(div[Markup(markdown('# Hi'))])
 <div><h1>Hi</h1></div>
+
 ```
 
 ### HTML Doctype
@@ -156,6 +167,7 @@ The [HTML doctype](https://developer.mozilla.org/en-US/docs/Glossary/Doctype) is
 >>> from htpy import html
 >>> print(html)
 <!doctype html><html></html>
+
 ```
 
 ### HTML Comments
@@ -169,6 +181,7 @@ If you want to emit HTML comments that will be visible in the browser, use the `
 >>> from htpy import div, comment
 >>> print(div[comment("This is a HTML comment, visible in the browser!")])
 <div><!-- This is a HTML comment, visible in the browser! --></div>
+
 ```
 
 It is safe to pass arbitrary text to the comment function. Double dashes (`--`)
@@ -190,6 +203,7 @@ Some elements do not have attributes, they can be specified by just the element
 >>> from htpy import hr
 >>> print(hr)
 <hr>
+
 ```
 
 ### Keyword Arguments
@@ -200,6 +214,7 @@ Attributes can be specified via keyword arguments:
 >>> from htpy import img
 >>> print(img(src="picture.jpg"))
 <img src="picture.jpg">
+
 ```
 
 In Python, `class` and `for` cannot be used as keyword arguments. Instead, they can be specified as `class_` or `for_` when using keyword arguments:
@@ -208,6 +223,7 @@ In Python, `class` and `for` cannot be used as keyword arguments. Instead, they
 >>> from htpy import label
 >>> print(label(for_="myfield"))
 <label for="myfield"></label>
+
 ```
 
 Attributes that contain dashes `-` can be specified using underscores:
@@ -216,6 +232,7 @@ Attributes that contain dashes `-` can be specified using underscores:
 >>> from htpy import form
 >>> print(form(hx_post="/foo"))
 <form hx-post="/foo"></form>
+
 ```
 
 ### Id/Class Shorthand
@@ -227,18 +244,21 @@ that looks like a CSS selector can be used to quickly define id and classes:
 >>> from htpy import div
 >>> print(div("#myid"))
 <div id="myid"></div>
+
 ```
 
 ```pycon title="Define multiple classes"
 >>> from htpy import div
 >>> print(div(".foo.bar"))
 <div class="foo bar"></div>
+
 ```
 
 ```pycon title="Combining both id and classes"
 >>> from htpy import div
 >>> print(div("#myid.foo.bar"))
 <div id="myid" class="foo bar"></div>
+
 ```
 
 ### Attributes as Dict
@@ -252,12 +272,14 @@ dynamically.
 >>> from htpy import button
 >>> print(button({"@click.shift": "addToSelection()"}))
 <button @click.shift="addToSelection()"></button>
+
 ```
 
 ```pycon title="Using an attribute with a reserved keyword"
 >>> from htpy import label
 >>> print(label({"for": "myfield"}))
 <label for="myfield"></label>
+
 ```
 
 ### Boolean/Empty Attributes
@@ -271,12 +293,14 @@ HTML.
 >>> from htpy import button
 >>> print(button(disabled=True))
 <button disabled></button>
+
 ```
 
 ```pycon title="False bool attribute"
 >>> from htpy import button
 >>> print(button(disabled=False))
 <button></button>
+
 ```
 
 ### Conditionally Mixing CSS Classes
@@ -293,6 +317,7 @@ accepts a list of class names or a dict. Falsey values will be ignored.
 >>> print(button(class_=["btn", {"btn-primary": is_primary}]))
 <button class="btn"></button>
 >>>
+
 ```
 
 ### Combining Modes
@@ -303,6 +328,7 @@ Attributes via id/class shorthand, keyword arguments and dictionary can be combi
 >>> from htpy import label
 >>> print(label("#myid.foo.bar", {'for': "somefield"}, name="myname",))
 <label id="myid" class="foo bar" for="somefield" name="myname"></label>
+
 ```
 
 ### Escaping of Attributes
@@ -313,8 +339,9 @@ all unsafe characters are escaped but the browser will interpret it correctly:
 
 ```pycon
 >>> from htpy import button
->>> print(button(id="example", onclick="let name = 'andreas'; alert('hi' + name);")["Say hi"])
+>>> print(button(onclick="let name = 'andreas'; alert('hi' + name);")["Say hi"])
 <button onclick="let name = &#39;andreas&#39;; alert(&#39;hi&#39; + name);">Say hi</button>
+
 ```
 
 In the browser, the parsed attribute as returned by
@@ -332,6 +359,7 @@ snippets as attributes:
 >>> some_markup = Markup("""<li class="bar"></li>""")
 >>> print(ul(data_li_template=some_markup))
 <ul data-li-template="&lt;li class=&#34;bar&#34;&gt;&lt;/li&gt;"></ul>
+
 ```
 
 ## Render elements without a parent (orphans)
@@ -345,6 +373,7 @@ You may use `render_node` to achieve this:
 >>> from htpy import render_node, tr
 >>> print(render_node([tr["a"], tr["b"]]))
 <tr>a</tr><tr>b</tr>
+
 ```
 
 `render_node()` accepts all kinds of [`Node`](static-typing.md#node) objects.
@@ -376,6 +405,7 @@ got a chunk: '<li>'
 got a chunk: 'b'
 got a chunk: '</li>'
 got a chunk: '</ul>'
+
 ```
 
 Just like [render_node()](#render-elements-without-a-parent-orphans), there is
@@ -393,6 +423,7 @@ got a chunk: '</li>'
 got a chunk: '<li>'
 got a chunk: 'b'
 got a chunk: '</li>'
+
 ```
 
 ## Passing Data with Context
diff --git a/htpy/html2htpy.py b/htpy/html2htpy.py
index a5e19f4..d2cd5c6 100644
--- a/htpy/html2htpy.py
+++ b/htpy/html2htpy.py
@@ -314,9 +314,39 @@ def html2htpy(
 def _convert_data_to_string(data: str) -> str:
     _data = str(data)
 
-    is_multiline = "\n" in _data
+    # Normalize leading whitespace
+    leading_whitespace_pattern = re.compile(r"^(\s+)(\S)")
+
+    def leading_whitespace_replacer(match: re.Match[str]) -> str:
+        whitespace = match.group(1)
+        first_char = match.group(2)
+
+        if whitespace.endswith(" "):
+            # keep single leading space (' ') before non-whitespace
+            # if it was there from before
+            return " " + first_char
+        else:
+            return first_char
+
+    _data = leading_whitespace_pattern.sub(leading_whitespace_replacer, _data)
+
+    # Normalize trailing whitespace
+    leading_whitespace_pattern = re.compile(r"(\S)(\s+)$")
 
-    _data = _data.replace("\n", "")
+    def trailing_whitespace_replacer(match: re.Match[str]) -> str:
+        last_char = match.group(1)
+        whitespace = match.group(2)
+
+        if whitespace.startswith(" "):
+            # keep single trailing space (' ') after non-whitespace
+            # if it was there from before
+            return last_char + " "
+        else:
+            return last_char
+
+    _data = leading_whitespace_pattern.sub(trailing_whitespace_replacer, _data)
+
+    is_multiline = "\n" in _data
 
     # escape unescaped dblquote: " -> \"
     _data = re.compile(r'(?<![\\])"').sub('\\"', _data)
diff --git a/pyproject.toml b/pyproject.toml
index 24cd8d2..6672c21 100644
--- a/pyproject.toml
+++ b/pyproject.toml
@@ -33,6 +33,8 @@ optional-dependencies.dev = [
     "jinja2",
     "starlette",
     "httpx",
+    "markdown",
+    "pytest-doctestplus"
 ]
 optional-dependencies.docs = [
     "mkdocs-material==9.5.12",
@@ -80,3 +82,6 @@ exclude = ["examples", "scripts"]
 [tool.pyright]
 include = ["htpy", "tests"]
 strict = ["htpy", "tests"]
+
+[tool.pytest.ini_options]
+addopts = "--doctest-glob='docs/*.md'"


Test patch:
diff --git a/tests/test_html2htpy.py b/tests/test_html2htpy.py
index d2e224c..6285b98 100644
--- a/tests/test_html2htpy.py
+++ b/tests/test_html2htpy.py
@@ -324,3 +324,34 @@ def test_del_tag_is_replaced_with_del_() -> None:
     )
 
     assert actual == 'from htpy import del_, div\ndiv[del_["deleted"]]'
+
+
+def test_convert_stripping_simple_whitespace() -> None:
+    actual = html2htpy(
+        "<p>      \t\t\nHi\n\n\t  </p>",
+        import_mode="no",
+    )
+
+    assert actual == 'p["Hi"]'
+
+
+def test_convert_pre_element_retains_all_whitespace() -> None:
+    actual = html2htpy(
+        textwrap.dedent(
+            """\
+            <pre>
+            hello,   fellow   programmer.
+
+            This   element   retains   newlines   and   whitespace.
+            </pre>
+            """
+        ),
+        import_mode="no",
+    )
+
+    assert actual == textwrap.dedent(
+        '''\
+        pre["""hello,   fellow   programmer.
+
+        This   element   retains   newlines   and   whitespace."""]'''
+    )