Repository: internetarchive/openlibrary

Issue:
Description on lists page should probably be truncated
<!-- What problem are we solving? What does the experience look like today? What are the symptoms? -->

### Evidence / Screenshot (if possible)
![Screen Shot 2021-06-21 at 18 18 23-fullpage](https://user-images.githubusercontent.com/921217/122862671-341e8780-d2bd-11eb-96ad-775d12eb5544.png)


### Relevant url?
https://openlibrary.org/lists

### Steps to Reproduce
<!-- What steps caused you to find the bug? -->
1. See screenshot

<!-- What actually happened after these steps? What did you expect to happen? -->
* Actual: Very long descriptions lead to a very long page
* Expected: descriptions are eventually truncated

### Details


### Proposal & Constraints
<!-- What is the proposed solution / implementation? Is there a precedent of this approach succeeding elsewhere? -->

### Related files
<!-- Files related to this issue; this is super useful for new contributors who might want to help! If you're not sure, leave this blank; a maintainer will add them. -->

### Stakeholders
<!-- @ tag stakeholders of this bug -->



Discussion:

Resolving patch:
diff --git a/openlibrary/plugins/upstream/utils.py b/openlibrary/plugins/upstream/utils.py
--- a/openlibrary/plugins/upstream/utils.py
+++ b/openlibrary/plugins/upstream/utils.py
@@ -11,6 +11,8 @@
 import xml.etree.ElementTree as etree
 import datetime
 import logging
+from html.parser import HTMLParser
+from typing import Optional
 
 import requests
 
@@ -26,7 +28,7 @@
 from infogami.utils.context import context
 from infogami.infobase.client import Thing, Changeset, storify
 
-from openlibrary.core.helpers import commify, parse_datetime
+from openlibrary.core.helpers import commify, parse_datetime, truncate
 from openlibrary.core.middleware import GZipMiddleware
 from openlibrary.core import cache, ab
 
@@ -862,6 +864,39 @@ def today():
     return datetime.datetime.today()
 
 
+class HTMLTagRemover(HTMLParser):
+
+    def __init__(self):
+        super().__init__()
+        self.data = []
+
+    def handle_data(self, data):
+        self.data.append(data.strip())
+
+    def handle_endtag(self, tag):
+        self.data.append('\n' if tag in ('p', 'li') else ' ')
+
+
+@public
+def reformat_html(html_str: str, max_length: Optional[int] = None) -> str:
+    """
+    Reformats an HTML string, removing all opening and closing tags.
+    Adds a line break element between each set of text content.
+    Optionally truncates contents that exceeds the given max length.
+
+    returns: A reformatted HTML string
+    """
+    parser = HTMLTagRemover()
+    # Must have a root node, otherwise the parser will fail
+    parser.feed(f'<div>{html_str}</div>')
+    content = [web.websafe(s) for s in parser.data if s]
+
+    if max_length:
+        return truncate(''.join(content), max_length).strip().replace('\n', '<br>')
+    else:
+        return ''.join(content).strip().replace('\n', '<br>')
+
+
 def setup():
     """Do required initialization"""
     # monkey-patch get_markdown to use OL Flavored Markdown


Test patch:
diff --git a/openlibrary/plugins/upstream/tests/test_utils.py b/openlibrary/plugins/upstream/tests/test_utils.py
--- a/openlibrary/plugins/upstream/tests/test_utils.py
+++ b/openlibrary/plugins/upstream/tests/test_utils.py
@@ -116,3 +116,22 @@ def test_get_coverstore_url(monkeypatch):
     # make sure trailing / is always stripped
     monkeypatch.setattr(config, "coverstore_url", "https://0.0.0.0:80/", raising=False)
     assert utils.get_coverstore_url() == "https://0.0.0.0:80"
+
+
+def test_reformat_html():
+    f = utils.reformat_html
+
+    input_string = '<p>This sentence has 32 characters.</p>'
+    assert f(input_string, 10) == 'This sente...'
+    assert f(input_string) == 'This sentence has 32 characters.'
+    assert f(input_string, 5000) == 'This sentence has 32 characters.'
+
+    multi_line_string = """<p>This sentence has 32 characters.</p>
+                           <p>This new sentence has 36 characters.</p>"""
+    assert f(multi_line_string) == 'This sentence has 32 ' \
+        'characters.<br>This new sentence has 36 characters.'
+    assert f(multi_line_string, 34) == 'This sentence has 32 ' \
+        'characters.<br>T...'
+
+    assert f("<script>alert('hello')</script>", 34) == "alert(&#39;hello&#39;)"
+    assert f("&lt;script&gt;") == "&lt;script&gt;"