Repository: boto/botocore
Pull request #2855 (2023-01-24 07:21:37)

Docs: collapse HTML whitespace for correct RST rendering
HTML in docstrings often contains extraneous white-space around tags, for readability. Browsers would collapse this white-space before rendering. However, the botocore docs are generated by converting the HTML from upstream into RST. If the whitespace is not collapsed before conversion to RST where white-space is part of the syntax, for example for indentation, it can result in incorrect output.

This screenshot from [our docs page for KMS](https://botocore.amazonaws.com/v1/documentation/api/latest/reference/services/kms.html) is an example of where this currently goes wrong. The text starting with "cross-account use" is not supposed to be part of the note.

<img width="841" alt="Screen Shot 2023-01-11 at 2 12 26 PM" src="https://user-images.githubusercontent.com/97921/214231860-6ed8ec5f-50af-49cd-8d96-f6bd5ba1e76d.png">

The HTML source for the example in the screenshot is [here](https://github.com/boto/botocore/blob/c2bc52dcf9d58d9699983b18d80f244511f07d36/botocore/data/kms/2014-11-01/service-2.json#L140).

This PR adds logic that mimics the whitespace collapsing logic implemented in browser, as described [here on MDN](https://developer.mozilla.org/en-US/docs/Web/API/Document_Object_Model/Whitespace) and [here by W3C](https://www.w3.org/TR/css-text-3/#white-space-rules).

After this change, the example screenshot above looks like this:

<img width="818" alt="Screen Shot 2023-01-11 at 2 12 49 PM" src="https://user-images.githubusercontent.com/97921/214231866-f8f7d9ef-ca27-4784-9d36-607798536f21.png">

Previously, we had a few instances of narrowly scoped code to remove extraneous whitespace, for example for list items or for hyperlinks `<code>` tags introduced recently in #2809. The new logic supersedes these.


codecov-commenter: # [Codecov](https://codecov.io/gh/boto/botocore/pull/2855?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=boto) Report
Base: **93.54**% // Head: **93.51**% // Decreases project coverage by **`-0.04%`** :warning:
> Coverage data is based on head [(`25642e6`)](https://codecov.io/gh/boto/botocore/pull/2855?src=pr&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=boto) compared to base [(`f14ab12`)](https://codecov.io/gh/boto/botocore/commit/f14ab129706a99198d42eed78d75350ea61c48e9?el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=boto).
> Patch coverage: 97.43% of modified lines in pull request are covered.

> :exclamation: Current head 25642e6 differs from pull request most recent head 7234468. Consider uploading reports for the commit 7234468 to get more accurate results

:mega: This organization is not using Codecov’s [GitHub App Integration](https://github.com/apps/codecov). We recommend you install it so Codecov can continue to function properly for your repositories. [Learn more](https://about.codecov.io/blog/codecov-is-updating-its-github-integration/?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=boto)

<details><summary>Additional details and impacted files</summary>


```diff
@@             Coverage Diff             @@
##           develop    #2855      +/-   ##
===========================================
- Coverage    93.54%   93.51%   -0.04%     
===========================================
  Files           63       63              
  Lines        13398    13438      +40     
===========================================
+ Hits         12533    12566      +33     
- Misses         865      872       +7     
```


| [Impacted Files](https://codecov.io/gh/boto/botocore/pull/2855?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=boto) | Coverage Δ | |
|---|---|---|
| [botocore/docs/bcdoc/style.py](https://codecov.io/gh/boto/botocore/pull/2855?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=boto#diff-Ym90b2NvcmUvZG9jcy9iY2RvYy9zdHlsZS5weQ==) | `95.46% <ø> (ø)` | |
| [botocore/docs/bcdoc/docstringparser.py](https://codecov.io/gh/boto/botocore/pull/2855?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=boto#diff-Ym90b2NvcmUvZG9jcy9iY2RvYy9kb2NzdHJpbmdwYXJzZXIucHk=) | `98.84% <97.43%> (-0.42%)` | :arrow_down: |
| [botocore/compat.py](https://codecov.io/gh/boto/botocore/pull/2855?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=boto#diff-Ym90b2NvcmUvY29tcGF0LnB5) | `91.33% <0.00%> (-1.34%)` | :arrow_down: |
| [botocore/httpsession.py](https://codecov.io/gh/boto/botocore/pull/2855?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=boto#diff-Ym90b2NvcmUvaHR0cHNlc3Npb24ucHk=) | `91.40% <0.00%> (-0.40%)` | :arrow_down: |
| [botocore/utils.py](https://codecov.io/gh/boto/botocore/pull/2855?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=boto#diff-Ym90b2NvcmUvdXRpbHMucHk=) | `79.01% <0.00%> (-0.20%)` | :arrow_down: |
| [botocore/credentials.py](https://codecov.io/gh/boto/botocore/pull/2855?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=boto#diff-Ym90b2NvcmUvY3JlZGVudGlhbHMucHk=) | `98.53% <0.00%> (-0.11%)` | :arrow_down: |
| [botocore/httpchecksum.py](https://codecov.io/gh/boto/botocore/pull/2855?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=boto#diff-Ym90b2NvcmUvaHR0cGNoZWNrc3VtLnB5) | `94.16% <0.00%> (+0.04%)` | :arrow_up: |
| [botocore/auth.py](https://codecov.io/gh/boto/botocore/pull/2855?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=boto#diff-Ym90b2NvcmUvYXV0aC5weQ==) | `98.04% <0.00%> (+0.19%)` | :arrow_up: |

Help us with your feedback. Take ten seconds to tell us [how you rate us](https://about.codecov.io/nps?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=boto). Have a feature suggestion? [Share it here.](https://app.codecov.io/gh/feedback/?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=boto)

</details>

[:umbrella: View full report at Codecov](https://codecov.io/gh/boto/botocore/pull/2855?src=pr&el=continue&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=boto).   
:loudspeaker: Do you have feedback about the report comment? [Let us know in this issue](https://about.codecov.io/codecov-pr-comment-feedback/?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=boto).


Patch:
diff --git a/botocore/docs/bcdoc/docstringparser.py b/botocore/docs/bcdoc/docstringparser.py
index bff33af11..6f1ff26be 100644
--- a/botocore/docs/bcdoc/docstringparser.py
+++ b/botocore/docs/bcdoc/docstringparser.py
@@ -11,10 +11,12 @@
 # ANY KIND, either express or implied. See the License for the specific
 # language governing permissions and limitations under the License.
 from html.parser import HTMLParser
+from itertools import zip_longest
 
 PRIORITY_PARENT_TAGS = ('code', 'a')
 OMIT_NESTED_TAGS = ('span', 'i', 'code', 'a')
 OMIT_SELF_TAGS = ('i', 'b')
+HTML_BLOCK_DISPLAY_TAGS = ('p', 'note', 'ul', 'li')
 
 
 class DocStringParser(HTMLParser):
@@ -27,20 +29,19 @@ class DocStringParser(HTMLParser):
     def __init__(self, doc):
         self.tree = None
         self.doc = doc
-        HTMLParser.__init__(self)
+        super().__init__()
 
     def reset(self):
         HTMLParser.reset(self)
         self.tree = HTMLTree(self.doc)
 
     def feed(self, data):
-        # HTMLParser is an old style class, so the super() method will not work.
-        HTMLParser.feed(self, data)
+        super().feed(data)
         self.tree.write()
         self.tree = HTMLTree(self.doc)
 
     def close(self):
-        HTMLParser.close(self)
+        super().close()
         # Write if there is anything remaining.
         self.tree.write()
         self.tree = HTMLTree(self.doc)
@@ -74,10 +75,7 @@ class HTMLTree:
             return
 
         if is_start:
-            if tag == 'li':
-                node = LineItemNode(attrs)
-            else:
-                node = TagNode(tag, attrs)
+            node = TagNode(tag, attrs)
             self.current_node.add_child(node)
             self.current_node = node
         else:
@@ -116,17 +114,51 @@ class StemNode(Node):
         self.children.append(child)
 
     def write(self, doc):
+        self.collapse_whitespace()
         self._write_children(doc)
 
     def _write_children(self, doc):
-        for index, child in enumerate(self.children):
-            if isinstance(child, TagNode) and index + 1 < len(self.children):
-                # Provide a look ahead for TagNodes when one exists
-                next_child = self.children[index + 1]
+        for child, next_child in zip_longest(self.children, self.children[1:]):
+            if isinstance(child, TagNode) and next_child is not None:
                 child.write(doc, next_child)
             else:
                 child.write(doc)
 
+    def is_whitespace(self):
+        return all(child.is_whitespace() for child in self.children)
+
+    def startswith_whitespace(self):
+        return self.children and self.children[0].startswith_whitespace()
+
+    def endswith_whitespace(self):
+        return self.children and self.children[-1].endswith_whitespace()
+
+    def lstrip(self):
+        while self.children and self.children[0].is_whitespace():
+            self.children = self.children[1:]
+        if self.children:
+            self.children[0].lstrip()
+
+    def rstrip(self):
+        while self.children and self.children[-1].is_whitespace():
+            self.children = self.children[:-1]
+        if self.children:
+            self.children[-1].rstrip()
+
+    def collapse_whitespace(self):
+        """Remove collapsible white-space from HTML.
+
+        HTML in docstrings often contains extraneous white-space around tags,
+        for readability. Browsers would collapse this white-space before
+        rendering. If not removed before conversion to RST where white-space is
+        part of the syntax, for example for indentation, it can result in
+        incorrect output.
+        """
+        self.lstrip()
+        self.rstrip()
+        for child in self.children:
+            child.collapse_whitespace()
+
 
 class TagNode(StemNode):
     """
@@ -159,6 +191,32 @@ class TagNode(StemNode):
         self._write_children(doc)
         self._write_end(doc, next_child)
 
+    def collapse_whitespace(self):
+        """Remove collapsible white-space.
+
+        All tags collapse internal whitespace. Block-display HTML tags also
+        strip all leading and trailing whitespace.
+
+        Approximately follows the specification used in browsers:
+        https://www.w3.org/TR/css-text-3/#white-space-rules
+        https://developer.mozilla.org/en-US/docs/Web/API/Document_Object_Model/Whitespace
+        """
+        if self.tag in HTML_BLOCK_DISPLAY_TAGS:
+            self.lstrip()
+            self.rstrip()
+        # Collapse whitespace in situations like ``</b> <i> foo</i>`` into
+        # ``</b><i> foo</i>``.
+        for prev, cur in zip(self.children[:-1], self.children[1:]):
+            if prev.endswith_whitespace() and cur.startswith_whitespace():
+                cur.lstrip()
+        # Same logic, but for situations like ``<b>bar </b> <i>``:
+        for cur, nxt in zip(self.children[:-1], self.children[1:]):
+            if cur.endswith_whitespace() and nxt.startswith_whitespace():
+                cur.rstrip()
+        # Recurse into children
+        for child in self.children:
+            child.collapse_whitespace()
+
     def _write_start(self, doc):
         handler_name = 'start_%s' % self.tag
         if hasattr(doc.style, handler_name):
@@ -174,33 +232,6 @@ class TagNode(StemNode):
                 getattr(doc.style, handler_name)()
 
 
-class LineItemNode(TagNode):
-    def __init__(self, attrs=None, parent=None):
-        super().__init__('li', attrs, parent)
-
-    def write(self, doc, next_child=None):
-        self._lstrip(self)
-        super().write(doc, next_child)
-
-    def _lstrip(self, node):
-        """
-        Traverses the tree, stripping out whitespace until text data is found
-        :param node: The node to strip
-        :return: True if non-whitespace data was found, False otherwise
-        """
-        for child in node.children:
-            if isinstance(child, DataNode):
-                child.lstrip()
-                if child.data:
-                    return True
-            else:
-                found = self._lstrip(child)
-                if found:
-                    return True
-
-        return False
-
-
 class DataNode(Node):
     """
     A Node that contains only string data.
@@ -210,28 +241,72 @@ class DataNode(Node):
         super().__init__(parent)
         if not isinstance(data, str):
             raise ValueError("Expecting string type, %s given." % type(data))
-        self.data = data
+        self._leading_whitespace = ''
+        self._trailing_whitespace = ''
+        self._stripped_data = ''
+        if data == '':
+            return
+        if data.isspace():
+            self._trailing_whitespace = data
+            return
+        first_non_space = next(
+            idx for idx, ch in enumerate(data) if not ch.isspace()
+        )
+        last_non_space = len(data) - next(
+            idx for idx, ch in enumerate(reversed(data)) if not ch.isspace()
+        )
+        self._leading_whitespace = data[:first_non_space]
+        self._trailing_whitespace = data[last_non_space:]
+        self._stripped_data = data[first_non_space:last_non_space]
+
+    @property
+    def data(self):
+        return (
+            f'{self._leading_whitespace}{self._stripped_data}'
+            f'{self._trailing_whitespace}'
+        )
+
+    def is_whitespace(self):
+        return self._stripped_data == '' and (
+            self._leading_whitespace != '' or self._trailing_whitespace != ''
+        )
+
+    def startswith_whitespace(self):
+        return self._leading_whitespace != '' or (
+            self._stripped_data == '' and self._trailing_whitespace != ''
+        )
+
+    def endswith_whitespace(self):
+        return self._trailing_whitespace != '' or (
+            self._stripped_data == '' and self._leading_whitespace != ''
+        )
 
     def lstrip(self):
-        self.data = self.data.lstrip()
+        if self._leading_whitespace != '':
+            self._leading_whitespace = ''
+        elif self._stripped_data == '':
+            self.rstrip()
+
+    def rstrip(self):
+        # If there is no content, consolidate all white-space on the right.
+        if self._stripped_data == '' and self._leading_whitespace != '':
+            self._trailing_whitespace = (
+                f"{self._leading_whitespace}{self._trailing_whitespace}"
+            )
+            self._leading_whitespace = ''
+        # Up to one trailing space is always preserved
+        if self._trailing_whitespace != '':
+            self._trailing_whitespace = ' '
+
+    def collapse_whitespace(self):
+        """Noop, ``DataNode.write`` always collapses whitespace"""
+        return
 
     def write(self, doc):
-        if not self.data:
-            return
-
-        if self.data.isspace():
-            str_data = ' '
-            if isinstance(self.parent, TagNode) and self.parent.tag == 'code':
-                # Inline markup content may not start or end with whitespace.
-                # When provided <code> Test </code>, we want to
-                # generate ``Test`` instead of `` Test ``.
-                str_data = ''
-        else:
-            end_space = self.data[-1].isspace()
-            words = self.data.split()
-            words = doc.translate_words(words)
-            str_data = ' '.join(words)
-            if end_space:
-                str_data += ' '
-
-        doc.handle_data(str_data)
+        words = doc.translate_words(self._stripped_data.split())
+        str_data = (
+            f'{self._leading_whitespace}{" ".join(words)}'
+            f'{self._trailing_whitespace}'
+        )
+        if str_data != '':
+            doc.handle_data(str_data)
diff --git a/botocore/docs/bcdoc/style.py b/botocore/docs/bcdoc/style.py
index 568808df7..14813df5e 100644
--- a/botocore/docs/bcdoc/style.py
+++ b/botocore/docs/bcdoc/style.py
@@ -89,26 +89,27 @@ class ReSTStyle(BaseStyle):
         self.doc.write('\n%s' % self.spaces())
 
     def _start_inline(self, markup):
+        # Insert space between any directly adjacent bold and italic inlines to
+        # avoid situations like ``**abc***def*``.
+        try:
+            last_write = self.doc.peek_write()
+        except IndexError:
+            pass
+        else:
+            if last_write in ('*', '**') and markup in ('*', '**'):
+                self.doc.write(' ')
         self.doc.write(markup)
 
     def _end_inline(self, markup):
-        # Sometimes the HTML markup has whitespace between the end
-        # of the text inside the inline markup and the closing element
-        # (e.g. <b>foobar </b>).  This trailing space will cause
-        # problems in the ReST inline markup so we remove it here
-        # by popping the last item written off the stack, striping
-        # the whitespace and then pushing it back on the stack.
-        last_write = self.doc.pop_write().rstrip(' ')
-
-        # Sometimes, for whatever reason, a tag like <b/> is present. This
-        # is problematic because if we simply translate that directly then
-        # we end up with something like ****, which rst will assume is a
-        # heading instead of an empty bold.
+        # Remove empty and self-closing tags like ``<b></b>`` and ``<b/>``.
+        # If we simply translate that directly then we end up with something
+        # like ****, which rst will assume is a heading instead of an empty
+        # bold.
+        last_write = self.doc.pop_write()
         if last_write == markup:
             return
-
         self.doc.push_write(last_write)
-        self.doc.write(markup + ' ')
+        self.doc.write(markup)
 
     def start_bold(self, attrs=None):
         self._start_inline('**')
@@ -251,10 +252,7 @@ class ReSTStyle(BaseStyle):
         last_write = doc.pop_write()
         while not last_write.startswith('`'):
             last_write = doc.pop_write() + last_write
-        doc.push_write('`')
 
-        # Remove whitespace from the start of link text.
-        last_write = last_write[1:].lstrip(' ')
         if last_write != '':
             doc.push_write(last_write)
 
@@ -282,15 +280,6 @@ class ReSTStyle(BaseStyle):
                 self.doc.write('`__')
             self.a_href = None
 
-        if (
-            next_child is None
-            or next_child.data[0] not in PUNCTUATION_CHARACTERS
-        ):
-            # We only want to add a trailing space if the link is
-            # not followed by a period, comma, or other gramatically
-            # correct special character.
-            self.doc.write(' ')
-
     def start_i(self, attrs=None):
         self.doc.do_translation = True
         self.start_italics()
diff --git a/botocore/docs/params.py b/botocore/docs/params.py
index 335d6d502..cddaf12fc 100644
--- a/botocore/docs/params.py
+++ b/botocore/docs/params.py
@@ -136,13 +136,14 @@ class BaseParamsDocumenter(ShapeDocumenter):
         special_py_type = self._get_special_py_type_name(shape)
         py_type = py_type_name(shape.type_name)
 
-        type_format = '(%s) -- '
+        type_format = '(%s) --'
         if special_py_type is not None:
             # Special type can reference a linked class.
             # Italicizing it blows away the link.
             type_section.write(type_format % special_py_type)
         else:
             type_section.style.italics(type_format % py_type)
+        type_section.write(' ')
 
     def _start_nested_param(self, section):
         section.style.indent()
@@ -162,7 +163,8 @@ class ResponseParamsDocumenter(BaseParamsDocumenter):
         name_section = section.add_new_section('param-name')
         name_section.write('- ')
         if name is not None:
-            name_section.style.bold('%s ' % name)
+            name_section.style.bold('%s' % name)
+            name_section.write(' ')
         type_section = section.add_new_section('param-type')
         self._document_non_top_level_param_type(type_section, shape)
 
@@ -259,14 +261,16 @@ class RequestParamsDocumenter(BaseParamsDocumenter):
             name_section = section.add_new_section('param-name')
             name_section.write('- ')
             if name is not None:
-                name_section.style.bold('%s ' % name)
+                name_section.style.bold('%s' % name)
+                name_section.write(' ')
             type_section = section.add_new_section('param-type')
             self._document_non_top_level_param_type(type_section, shape)
 
         if is_required:
             is_required_section = section.add_new_section('is-required')
             is_required_section.style.indent()
-            is_required_section.style.bold('[REQUIRED] ')
+            is_required_section.style.bold('[REQUIRED]')
+            is_required_section.write(' ')
         if shape.documentation:
             documentation_section = section.add_new_section(
                 'param-documentation'
diff --git a/docs/Makefile b/docs/Makefile
index c7f72ef94..4e31a4c39 100644
--- a/docs/Makefile
+++ b/docs/Makefile
@@ -6,6 +6,7 @@ SPHINXOPTS    = -j auto
 SPHINXBUILD   = sphinx-build
 PAPER         =
 BUILDDIR      = build
+SERVICESDIR   = source/reference/services
 
 # Internal variables.
 PAPEROPT_a4     = -D latex_paper_size=a4
@@ -40,6 +41,7 @@ help:
 
 clean:
 	-rm -rf $(BUILDDIR)/*
+	-rm -rf $(SERVICESDIR)/*
 
 html:
 	$(SPHINXBUILD) -b html $(ALLSPHINXOPTS) $(BUILDDIR)/html


Test patch:
diff --git a/tests/unit/docs/bcdoc/test_docstringparser.py b/tests/unit/docs/bcdoc/test_docstringparser.py
index 3e816cdcd..d29a40c38 100644
--- a/tests/unit/docs/bcdoc/test_docstringparser.py
+++ b/tests/unit/docs/bcdoc/test_docstringparser.py
@@ -18,6 +18,8 @@
 # WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
 # OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS
 # IN THE SOFTWARE.
+import pytest
+
 import botocore.docs.bcdoc.docstringparser as parser
 from botocore.docs.bcdoc.restdoc import ReSTDocument
 from tests import mock, unittest
@@ -40,6 +42,11 @@ class TestDocStringParser(unittest.TestCase):
             beginning = contents.index(line)
             contents = contents[beginning:]
 
+    def test_tag_with_collapsible_spaces(self):
+        html = "<p>  a       bcd efg </p>"
+        result = self.parse(html)
+        self.assert_contains_exact_lines_in_order(result, [b'a bcd efg '])
+
     def test_nested_lists(self):
         html = "<ul><li>Wello</li><ul><li>Horld</li></ul></ul>"
         result = self.parse(html)
@@ -58,7 +65,7 @@ class TestDocStringParser(unittest.TestCase):
         html = "<p>This is a test <a href='https://testing.com'>Link</a></p>"
         result = self.parse(html)
         self.assert_contains_exact_lines_in_order(
-            result, [b'This is a test `Link <https://testing.com>`__ ']
+            result, [b'This is a test `Link <https://testing.com>`__']
         )
 
     def test_link_with_period(self):
@@ -69,39 +76,69 @@ class TestDocStringParser(unittest.TestCase):
         )
 
     def test_code_with_empty_link(self):
-        html = "<code> <a>Link</a> </code>"
+        html = "<p>Foo <code> <a>Link</a> </code></p>"
         result = self.parse(html)
-        self.assert_contains_exact_lines_in_order(result, [b'``Link`` '])
+        self.assert_contains_exact_lines_in_order(result, [b'Foo ``Link``'])
 
     def test_code_with_link_spaces(self):
-        html = "<code> <a href=\"https://testing.com\">Link</a> </code>"
+        html = "<p>Foo <code> <a href=\"https://aws.dev\">Link</a> </code></p>"
         result = self.parse(html)
-        self.assert_contains_exact_lines_in_order(result, [b'``Link`` '])
+        self.assert_contains_exact_lines_in_order(result, [b'Foo ``Link``'])
 
     def test_code_with_link_no_spaces(self):
-        html = "<code><a href=\"https://testing.com\">Link</a></code>"
+        html = "<p>Foo <code><a href=\"https://aws.dev\">Link</a></code></p>"
         result = self.parse(html)
-        self.assert_contains_exact_lines_in_order(result, [b'``Link`` '])
+        self.assert_contains_exact_lines_in_order(result, [b'Foo ``Link``'])
 
     def test_href_with_spaces(self):
         html = "<p><a href=\" https://testing.com\">Link</a></p>"
         result = self.parse(html)
         self.assert_contains_exact_lines_in_order(
-            result, [b' `Link <https://testing.com>`__ ']
+            result, [b' `Link <https://testing.com>`__']
         )
 
     def test_bold_with_nested_formatting(self):
         html = "<b><code>Test</code>test<a href=\" https://testing.com\">Link</a></b>"
         result = self.parse(html)
         self.assert_contains_exact_lines_in_order(
-            result, [b'``Test`` test `Link <https://testing.com>`__ ']
+            result, [b'``Test``test `Link <https://testing.com>`__']
         )
 
     def test_link_with_nested_formatting(self):
         html = "<a href=\"https://testing.com\"><code>Test</code></a>"
         result = self.parse(html)
         self.assert_contains_exact_lines_in_order(
-            result, [b'`Test <https://testing.com>`__ ']
+            result, [b'`Test <https://testing.com>`__']
+        )
+
+    def test_indentation_with_spaces_between_tags(self):
+        # Leading spaces inserted as padding between HTML tags can lead to
+        # unexpected indentation in RST. For example, consider the space between
+        # ``<p>`` and ``<b>`` in the third line of this example:
+        html = (
+            "<p>First paragraph </p> "
+            "<note> <p> Second paragraph in note </p> </note> "
+            "<p> <b>Bold statement:</b> Third paragraph</p>"
+            "<p>Last paragraph</p> "
+        )
+        # If kept, it will appear as indentation in RST and have the effect of
+        # pulling the third paragraph into the note:
+        #
+        # ```
+        # First paragraph
+        #
+        # ..note::
+        #   Second paragraph in note <-- intentionally indented
+        #
+        #  **Bold statement:** Third paragraph <-- unintentionally indented
+        #
+        # Last paragraph <-- first non-indented paragraph after note
+        # ```
+        result = self.parse(html)
+        self.assert_contains_exact_lines_in_order(
+            result,
+            #  ↓ no whitespace here
+            [b'**Bold statement:** Third paragraph'],
         )
 
 
@@ -181,6 +218,7 @@ class TestDataNode(unittest.TestCase):
         self.style = mock.Mock()
         self.doc = mock.Mock()
         self.doc.style = self.style
+        self.doc.translate_words.return_value = []
 
     def test_string_data(self):
         node = parser.DataNode('foo')
@@ -201,58 +239,103 @@ class TestDataNode(unittest.TestCase):
         node.write(self.doc)
         self.doc.handle_data.assert_called_once_with('foo bar baz')
 
-    def test_write_space(self):
-        node = parser.DataNode(' ')
-        node.write(self.doc)
-        self.doc.handle_data.assert_called_once_with(' ')
-        self.doc.handle_data.reset_mock()
-
-        node = parser.DataNode('              ')
-        node.write(self.doc)
-        self.doc.handle_data.assert_called_once_with(' ')
-
     def test_write_empty_string(self):
         node = parser.DataNode('')
         node.write(self.doc)
         self.assertFalse(self.doc.handle_data.called)
 
 
-class TestLineItemNode(unittest.TestCase):
-    def setUp(self):
-        self.style = mock.Mock()
-        self.doc = mock.Mock()
-        self.doc.style = self.style
-        self.doc.translate_words.return_value = ['foo']
-        self.node = parser.LineItemNode()
-
-    def test_write_strips_white_space(self):
-        self.node.add_child(parser.DataNode('  foo'))
-        self.node.write(self.doc)
-        self.doc.handle_data.assert_called_once_with('foo')
-
-    def test_write_strips_nested_white_space(self):
-        self.node.add_child(parser.DataNode('  '))
-        tag_child = parser.TagNode('foo')
-        tag_child.add_child(parser.DataNode('  '))
-        tag_child_2 = parser.TagNode('foo')
-        tag_child_2.add_child(parser.DataNode(' foo'))
-        tag_child.add_child(tag_child_2)
-        self.node.add_child(tag_child)
-
-        self.node.write(self.doc)
-        self.doc.handle_data.assert_called_once_with('foo')
-
-    def test_write_only_strips_until_text_is_found(self):
-        self.node.add_child(parser.DataNode('  '))
-        tag_child = parser.TagNode('foo')
-        tag_child.add_child(parser.DataNode('  '))
-        tag_child_2 = parser.TagNode('foo')
-        tag_child_2.add_child(parser.DataNode(' foo'))
-        tag_child_2.add_child(parser.DataNode(' '))
-        tag_child.add_child(tag_child_2)
-        self.node.add_child(tag_child)
-
-        self.node.write(self.doc)
-
-        calls = [mock.call('foo'), mock.call(' ')]
-        self.doc.handle_data.assert_has_calls(calls)
+@pytest.mark.parametrize(
+    'data, lstrip, rstrip, both',
+    [
+        # Note for cases with trailing white-space: If any white-space exists
+        # at the end of the string, stripping will leave behind a single space.
+        ('foo', 'foo', 'foo', 'foo'),
+        (' foo', 'foo', ' foo', 'foo'),
+        ('   foo', 'foo', '   foo', 'foo'),
+        ('\tfoo', 'foo', '\tfoo', 'foo'),
+        ('\t \t foo', 'foo', '\t \t foo', 'foo'),
+        ('foo ', 'foo ', 'foo ', 'foo '),
+        ('foo  ', 'foo  ', 'foo ', 'foo '),
+        ('foo\t\t', 'foo\t\t', 'foo ', 'foo '),
+        (' ', ' ', ' ', ' '),
+        ('  ', ' ', ' ', ' '),
+        ('\t', ' ', ' ', ' '),
+    ],
+)
+def test_datanode_stripping(data, lstrip, rstrip, both):
+    doc = mock.Mock()
+    doc.style = mock.Mock()
+    doc.translate_words.side_effect = lambda words: words
+
+    node = parser.DataNode(data)
+    node.lstrip()
+    node.write(doc)
+    doc.handle_data.assert_called_once_with(lstrip)
+    doc.handle_data.reset_mock()
+
+    node = parser.DataNode(data)
+    node.rstrip()
+    node.write(doc)
+    doc.handle_data.assert_called_once_with(rstrip)
+    doc.handle_data.reset_mock()
+
+    node = parser.DataNode(data)
+    node.lstrip()
+    node.rstrip()
+    node.write(doc)
+    doc.handle_data.assert_called_once_with(both)
+
+
+@pytest.mark.parametrize(
+    'html, expected_lines',
+    [
+        ('<p>  foo</p>', [b'foo']),
+        ('<p>  <span>  </span> <span> <span> foo</span></span></p>', [b'foo']),
+        # if there are trailing white-spaces, right-stripping text always
+        # leaves one space character behind
+        ('<p>foo  </p>', [b'foo ']),
+        ('<p>foo\t</p>', [b'foo ']),
+        ('<p>  foo  </p>', [b'foo ']),
+        ('<p>  <span>foo  </span>  </p>', [b'foo ']),
+        # ... but right-stripping tag-nodes does not
+        ('<p>  <span>foo</span>  </p>', [b'foo']),
+        ('<p>  <span>  foo</span>  </p>', [b'foo']),
+        ('<p>  <span>  foo  </span>  </p>', [b'foo ']),
+        # various nested markup examples
+        ('<p><i>italic</i></p>', [b'*italic*']),
+        ('<p><i>italic</i> </p>', [b'*italic*']),
+        ('<p><i>italic </i></p>', [b'*italic *']),
+        ('<p>  <span> foo <i> bar</i> </span>  </p>', [b'foo *bar*']),
+        ('<p>  <span> foo<i> bar</i> </span>  </p>', [b'foo* bar*']),
+        ('<p>  <span> foo <i>bar</i> </span>  </p>', [b'foo *bar*']),
+        # list items
+        ('<li>  foo</li>', [b'* foo']),
+        ('<li>  <foo>  </foo><foo> foo</foo></li>', [b'* foo']),
+        ('<li>  <foo> foo</foo><foo>  </foo></li>', [b'* foo']),
+        ('<li><foo>  </foo><foo> foo</foo> <foo>bar</li>', [b'* foo bar']),
+        ('<li><foo>  </foo><foo> foo</foo> <foo> bar</li>', [b'* foo bar']),
+        ('<li><foo>  </foo><foo> foo</foo><foo> bar</li>', [b'* foo bar']),
+        # multiple block tags in sequence are each left and right stripped
+        ('<p>  foo</p><p>  bar\t</p>', [b'foo', b'bar ']),
+        ('<p>  foo</p><li>  bar  </li>', [b'foo', b'* bar ']),
+        # nested block tags also work
+        (
+            '<p> <p> foo </p> <p> <span> bar </span> </p> </p>',
+            [b'foo ', b'bar '],
+        ),
+    ],
+)
+def test_whitespace_collapsing_foo(html, expected_lines):
+    docstring_parser = parser.DocStringParser(ReSTDocument())
+    docstring_parser.feed(html)
+    docstring_parser.close()
+    actual = docstring_parser.doc.getvalue()
+
+    # Get each line and filter out empty lines
+    contents = actual.split(b'\n')
+    contents = [line for line in contents if line and not line.isspace()]
+    for line in expected_lines:
+        assert line in contents
+        beginning = contents.index(line)
+        contents = contents[beginning:]
diff --git a/tests/unit/docs/bcdoc/test_document.py b/tests/unit/docs/bcdoc/test_document.py
index 1895de91a..10a8d065d 100644
--- a/tests/unit/docs/bcdoc/test_document.py
+++ b/tests/unit/docs/bcdoc/test_document.py
@@ -38,7 +38,7 @@ class TestReSTDocument(unittest.TestCase):
     def test_include_doc_string(self):
         doc = ReSTDocument()
         doc.include_doc_string('<p>this is a <code>test</code></p>')
-        self.assertEqual(doc.getvalue(), b'\n\nthis is a ``test`` \n\n')
+        self.assertEqual(doc.getvalue(), b'\n\nthis is a ``test``\n\n')
 
     def test_remove_doc_string(self):
         doc = ReSTDocument()
diff --git a/tests/unit/docs/bcdoc/test_style.py b/tests/unit/docs/bcdoc/test_style.py
index 3b474c26e..05bf179f5 100644
--- a/tests/unit/docs/bcdoc/test_style.py
+++ b/tests/unit/docs/bcdoc/test_style.py
@@ -43,7 +43,7 @@ class TestStyle(unittest.TestCase):
     def test_bold(self):
         style = ReSTStyle(ReSTDocument())
         style.bold('foobar')
-        self.assertEqual(style.doc.getvalue(), b'**foobar** ')
+        self.assertEqual(style.doc.getvalue(), b'**foobar**')
 
     def test_empty_bold(self):
         style = ReSTStyle(ReSTDocument())
@@ -54,7 +54,7 @@ class TestStyle(unittest.TestCase):
     def test_italics(self):
         style = ReSTStyle(ReSTDocument())
         style.italics('foobar')
-        self.assertEqual(style.doc.getvalue(), b'*foobar* ')
+        self.assertEqual(style.doc.getvalue(), b'*foobar*')
 
     def test_empty_italics(self):
         style = ReSTStyle(ReSTDocument())
@@ -72,7 +72,7 @@ class TestStyle(unittest.TestCase):
     def test_code(self):
         style = ReSTStyle(ReSTDocument())
         style.code('foobar')
-        self.assertEqual(style.doc.getvalue(), b'``foobar`` ')
+        self.assertEqual(style.doc.getvalue(), b'``foobar``')
 
     def test_empty_code(self):
         style = ReSTStyle(ReSTDocument())
@@ -192,7 +192,7 @@ class TestStyle(unittest.TestCase):
         style.doc.write('example')
         style.end_a()
         self.assertEqual(
-            style.doc.getvalue(), b'`example <http://example.org>`__ '
+            style.doc.getvalue(), b'`example <http://example.org>`__'
         )
 
     def test_escape_href_link(self):
@@ -202,14 +202,14 @@ class TestStyle(unittest.TestCase):
         style.end_a()
         self.assertEqual(
             style.doc.getvalue(),
-            b'`foo\\: the next bar <http://example.org>`__ ',
+            b'`foo\\: the next bar <http://example.org>`__',
         )
 
     def test_handle_no_text_hrefs(self):
         style = ReSTStyle(ReSTDocument())
         style.start_a(attrs=[('href', 'http://example.org')])
         style.end_a()
-        self.assertEqual(style.doc.getvalue(), b'`<http://example.org>`__ ')
+        self.assertEqual(style.doc.getvalue(), b'`<http://example.org>`__')
 
     def test_sphinx_reference_label_html(self):
         style = ReSTStyle(ReSTDocument())
diff --git a/tests/unit/docs/test_client.py b/tests/unit/docs/test_client.py
index 800912de7..c80cebc6e 100644
--- a/tests/unit/docs/test_client.py
+++ b/tests/unit/docs/test_client.py
@@ -144,7 +144,7 @@ class TestClientExceptionsDocumenter(BaseDocsTest):
         self.assert_contains_lines_in_order(
             [
                 '.. py:class:: MyService.Client.exceptions.SomeException',
-                '**Example** ::',
+                '**Example**::',
                 'except client.exceptions.SomeException as e:',
                 '.. py:attribute:: response',
                 '**Syntax**',
diff --git a/tests/unit/docs/test_service.py b/tests/unit/docs/test_service.py
index 37d1d2d5c..ebdb1d435 100644
--- a/tests/unit/docs/test_service.py
+++ b/tests/unit/docs/test_service.py
@@ -78,7 +78,7 @@ class TestServiceDocumenter(BaseDocsTest):
         self.assert_contains_lines_in_order(
             [
                 '.. py:method:: MyService.Client.sample_operation(**kwargs)',
-                '  **Examples** ',
+                '  **Examples**',
                 '  Sample Description.',
                 '  ::',
                 '    response = client.sample_operation(',
diff --git a/tests/unit/test_client.py b/tests/unit/test_client.py
index c42920668..eb7a30787 100644
--- a/tests/unit/test_client.py
+++ b/tests/unit/test_client.py
@@ -1213,7 +1213,7 @@ class TestAutoGeneratedClient(unittest.TestCase):
                 '    Creates an iterator that will paginate through responses '
                 'from :py:meth:`MyService.Client.test_operation`.'
             ),
-            '    **Request Syntax** ',
+            '    **Request Syntax**',
             '    ::',
             '      response_iterator = paginator.paginate(',
             "          Foo='string',",
@@ -1234,16 +1234,16 @@ class TestAutoGeneratedClient(unittest.TestCase):
                 '      A dictionary that provides parameters to control '
                 'pagination.'
             ),
-            '      - **MaxItems** *(integer) --* ',
+            '      - **MaxItems** *(integer) --*',
             (
                 '        The total number of items to return. If the total '
                 'number of items available is more than the value specified '
                 'in max-items then a ``NextToken`` will be provided in the '
                 'output that you can use to resume pagination.'
             ),
-            '      - **PageSize** *(integer) --* ',
+            '      - **PageSize** *(integer) --*',
             '        The size of each page.',
-            '      - **StartingToken** *(string) --* ',
+            '      - **StartingToken** *(string) --*',
             (
                 '        A token to specify where to start paginating. This is '
                 'the ``NextToken`` from a previous response.'
diff --git a/tests/unit/test_waiters.py b/tests/unit/test_waiters.py
index 13e4c2480..62544e180 100644
--- a/tests/unit/test_waiters.py
+++ b/tests/unit/test_waiters.py
@@ -771,7 +771,7 @@ class TestCreateWaiter(unittest.TestCase):
                 'seconds until a successful state is reached. An error '
                 'is returned after 1 failed checks.'
             ),
-            '    **Request Syntax** ',
+            '    **Request Syntax**',
             '    ::',
             '      waiter.wait(',
             "          bar='string'",