Repository: sphinx-doc/sphinx

Issue:
Text writer breaks on hyphens
The text writer uses a custom `TextWrapper` class that breaks hyphenated words. The [parent class][1] from Python standard library has a [`break_on_hyphens`][2] option that allows to disable that, however Sphinx’ `TextWrapper` has a custom `wordsep_re` regular expression so it is not easy to override that:
https://github.com/sphinx-doc/sphinx/blob/8c7faed6fcbc6b7d40f497698cb80fc10aee1ab3/sphinx/writers/text.py#L259-L263

This leads to two issues.

1. Some technical terms, program names, email addresses, etc., are being cut at their hypen at the end of line. This makes copy&paste more difficult, and it reads confusingly.

2. The second issue is related to footnotes. When processing a footnote reference, the `add_text` method is called with `first` argument being `'[N]'`, where N is the footnote number.

   When `first` is not None, the text is wrapped:
   https://github.com/sphinx-doc/sphinx/blob/8c7faed6fcbc6b7d40f497698cb80fc10aee1ab3/sphinx/writers/text.py#L451

   then joined by space and wrapped again:
   https://github.com/sphinx-doc/sphinx/blob/8c7faed6fcbc6b7d40f497698cb80fc10aee1ab3/sphinx/writers/text.py#L456-L457

   Because of this, the words that had a hyphen get an extra space after the hyphen.

[1]: https://docs.python.org/3/library/textwrap.html#textwrap.TextWrapper
[2]: https://docs.python.org/3/library/textwrap.html#textwrap.TextWrapper.break_on_hyphens

**To Reproduce**
Steps to reproduce the behavior:
```bash
$ cat >index.rst <<EOF
Please see the footnote. [#]_

.. [#]
   Another common way to do this is for ``build`` to depend on
   ``build-stamp`` and to do nothing else, and for the ``build-stamp``
   target to do the building and to ``touch build-stamp`` on completion.
EOF
$ touch conf.py
$ sphinx-build -b text . _build/text
```

The result is:
```
$ cat _build/text/index.txt 
Please see the footnote. [1]

[1] Another common way to do this is for "build" to depend on
    "build- stamp" and to do nothing else, and for the "build-stamp"
    target to do the building and to "touch build-stamp" on
    completion.
```

**Expected behavior**
In the second line of footnote there should be no space between `build-` and `stamp`.

I see two possible solutions to achieve that:
- Do not break words on hyphens, or make that configurable.
- Refactor the `end_state` method to avoid calling `do_format()` twice.

**Environment info**
- OS: Debian GNU/Linux sid
- Python version: 3.7.5
- Sphinx version: latest master

This issue is based on two bugs in Debian: [#944330] and [#944331].

[#944330]: https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=944330
[#944331]: https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=944331

Discussion:

Patch:
diff --git a/CHANGES b/CHANGES
index 0887943e5..bed017d55 100644
--- a/CHANGES
+++ b/CHANGES
@@ -11,6 +11,7 @@ Incompatible changes
   match the first line of the code block.
 * #1331: Change default User-Agent header to ``"Sphinx/X.Y.Z requests/X.Y.Z
   python/X.Y.Z"``.  It can be changed via :confval:`user_agent`.
+* #6867: text: content of admonitions starts after a blank line
 
 Deprecated
 ----------
@@ -78,6 +79,7 @@ Bugs fixed
   fully matched in a text paragraph on the same page, the search does not
   include this match.
 * #6848: config.py shouldn't pop extensions from overrides
+* #6867: text: extra spaces are inserted to hyphenated words on folding lines
 
 Testing
 --------
diff --git a/sphinx/writers/text.py b/sphinx/writers/text.py
index 879ce5ada..a71aefa80 100644
--- a/sphinx/writers/text.py
+++ b/sphinx/writers/text.py
@@ -440,15 +440,14 @@ def do_format() -> None:
                 toformat = []
         do_format()
         if first is not None and result:
-            itemindent, item = result[0]
-            result_rest, result = result[1:], []
-            if item:
-                toformat = [first + ' '.join(item)]
-                do_format()  # re-create `result` from `toformat`
-                _dummy, new_item = result[0]
-                result.insert(0, (itemindent - indent, [new_item[0]]))
-                result[1] = (itemindent, new_item[1:])
-                result.extend(result_rest)
+            # insert prefix into first line (ex. *, [1], See also, etc.)
+            newindent = result[0][0] - indent
+            if result[0][1] == ['']:
+                result.insert(0, (newindent, [first]))
+            else:
+                text = first + result[0][1].pop(0)
+                result.insert(0, (newindent, [text]))
+
         self.states[-1].extend(result)
 
     def visit_document(self, node: Element) -> None:
@@ -916,12 +915,20 @@ def depart_admonition(self, node: Element) -> None:
     def _visit_admonition(self, node: Element) -> None:
         self.new_state(2)
 
-        if isinstance(node.children[0], nodes.Sequential):
-            self.add_text(self.nl)
-
     def _depart_admonition(self, node: Element) -> None:
         label = admonitionlabels[node.tagname]
-        self.end_state(first=label + ': ')
+        indent = sum(self.stateindent) + len(label)
+        print(self.states[-1])
+        if (len(self.states[-1]) == 1 and
+                self.states[-1][0][0] == 0 and
+                MAXWIDTH - indent >= sum(len(s) for s in self.states[-1][0][1])):
+            # short text: append text after admonition label
+            self.stateindent[-1] += len(label)
+            self.end_state(first=label + ': ')
+        else:
+            # long text: append label before the block
+            self.states[-1].insert(0, (0, [self.nl]))
+            self.end_state(first=label + ':')
 
     visit_attention = _visit_admonition
     depart_attention = _depart_admonition


Test patch:
diff --git a/tests/test_build_text.py b/tests/test_build_text.py
index 5b19f43ef..efdc0f5eb 100644
--- a/tests/test_build_text.py
+++ b/tests/test_build_text.py
@@ -31,16 +31,18 @@ def test_maxwitdh_with_prefix(app, status, warning):
     lines = result.splitlines()
     line_widths = [column_width(line) for line in lines]
     assert max(line_widths) < MAXWIDTH
-    assert lines[0].startswith('See also: ham')
-    assert lines[1].startswith('  ham')
-    assert lines[2] == ''
-    assert lines[3].startswith('* ham')
-    assert lines[4].startswith('  ham')
-    assert lines[5] == ''
-    assert lines[6].startswith('* ham')
-    assert lines[7].startswith('  ham')
-    assert lines[8] == ''
-    assert lines[9].startswith('spam egg')
+    assert lines[0].startswith('See also:')
+    assert lines[1].startswith('')
+    assert lines[2].startswith('  ham')
+    assert lines[3].startswith('  ham')
+    assert lines[4] == ''
+    assert lines[5].startswith('* ham')
+    assert lines[6].startswith('  ham')
+    assert lines[7] == ''
+    assert lines[8].startswith('* ham')
+    assert lines[9].startswith('  ham')
+    assert lines[10] == ''
+    assert lines[11].startswith('spam egg')
 
 
 @with_text_app()
diff --git a/tests/test_intl.py b/tests/test_intl.py
index a6bd17512..080ee9221 100644
--- a/tests/test_intl.py
+++ b/tests/test_intl.py
@@ -315,7 +315,8 @@ def test_text_seealso(app):
               "\n*********************\n"
               "\nSee also: SHORT TEXT 1\n"
               "\nSee also: LONG TEXT 1\n"
-              "\nSee also: SHORT TEXT 2\n"
+              "\nSee also:\n"
+              "\n  SHORT TEXT 2\n"
               "\n  LONG TEXT 2\n")
     assert result == expect
 
@@ -356,7 +357,9 @@ def test_text_figure_captions(app):
               "14.4. IMAGE UNDER NOTE\n"
               "======================\n"
               "\n"
-              "Note: [image: i18n under note][image]\n"
+              "Note:\n"
+              "\n"
+              "  [image: i18n under note][image]\n"
               "\n"
               "     [image: img under note][image]\n")
     assert result == expect