Repository: mgeisler/textwrap (language: rust)

Issue:
Support not squeezing whitespace
We currently split the input string by whitespace and process the words one by one. This implies that we squeeze runs of whitespace to a single space.

We could instead have a mode where whitespace is significant and kept in the output. This would make it possible to wrap text like this line by line:
```
foobar:    Have lots of foos and bars in the output.
baz:       Show a single bar in the output, but only if it is a full moon.
```
Lines that wrap would still line up somewhat:
```
foobar:    Have lots of foos and bars in
the output.
baz:       Show a single bar in the output,
but only if it is a full moon.
```

In [Python's textwrap][1] this option is called `replace_whitespace`, but I think I'll call it `squeeze_whitespace` instead. The option should come in handy to support the output wanted in https://github.com/kbknapp/clap-rs/issues/617.

[1]: https://docs.python.org/3/library/textwrap.html


Discussion / hints:
Hi,
I needed to wrap text  in rust and thus tried this crate. I quickly ran into limitations, like squeezing the whitespace, breaking on non-break spaces, before punctuation etc... so I set off trying to remedy that, which was a bit more hassle than I anticipated, but currently I have something that does the job and that has quite some margin for improvement, extension, so I thought it might interest you. The code is at: https://github.com/najamelan/unicode-wrap and it is very much draft. I'm a total beginner to rust, and not a unicode expert either. 

I kind of stole your example and benchmarks. I hope that's ok, if not let me know. I can also add attribution if you would like. Haven't done that yet. Having the same benchmarks allows comparing performance and thus, unicode-wrap is 10 times slower than textwrap on my system. 

ps: currently there is no documentation...

greets,
Naja
Hey @najamelan -- feel free to use the code if you can, just make sure to keep the license and the copyright notices intact. For example, you should probably not put it under public domain since it is MIT licensed right now.

I took a brief look at your crate and it seems you have built an algorithm that is much more fancy than what I did :-) So I can imagine it will be slower, but with the filters you can get overall better results.

The points about respecting non-breaking spaces is great -- I should double check how they're handled by textwrap and fix any problems.

Fix patch:
diff --git a/src/lib.rs b/src/lib.rs
index 6db75bf2..8bc190d6 100644
--- a/src/lib.rs
+++ b/src/lib.rs
@@ -207,6 +207,15 @@ pub struct Wrapper<'a> {
     /// When set to `false`, some lines be being longer than
     /// `self.width`.
     pub break_words: bool,
+    /// This crate treats every whitespace character (space, newline,
+    /// TAB, ...) as a space character. When this option is set to
+    /// `true`, the whitespace between words will be squeezed into a
+    /// single space character. Otherwise, the whitespace is
+    /// significant and will be kept in the output. Whitespace
+    /// characters such as TAB will still be turned into a single
+    /// space -- consider expanding TABs manually if this is a
+    /// concern.
+    pub squeeze_whitespace: bool,
     /// The method for splitting words. If the `hyphenation` feature
     /// is enabled, you can use a `hyphenation::language::Corpus` here
     /// to get language-aware hyphenation.
@@ -227,6 +236,7 @@ impl<'a> Wrapper<'a> {
             initial_indent: "",
             subsequent_indent: "",
             break_words: true,
+            squeeze_whitespace: false,
             splitter: Box::new(HyphenSplitter {}),
         }
     }
@@ -301,7 +311,7 @@ impl<'a> Wrapper<'a> {
 
         for mut word in s.split(|c: char| c.is_whitespace() && c != NBSP) {
             // Skip over adjacent whitespace characters.
-            if word.is_empty() {
+            if self.squeeze_whitespace && word.is_empty() {
                 continue;
             }
 
@@ -569,9 +579,25 @@ mod tests {
         assert_eq!(wrap("foo bar", 0), vec!["foo", "bar"]);
     }
 
+    #[test]
+    fn whitespace_is_significant() {
+        assert_eq!(wrap("foo:   bar baz", 10), vec!["foo:   bar", "baz"]);
+    }
+
+    #[test]
+    fn extra_whitespace_start_of_line() {
+        // Whitespace is only significant inside a line. After a line
+        // gets too long and is broken, the first word starts in
+        // column zero and is not indented. The line before might end
+        // up with trailing whitespace.
+        assert_eq!(wrap("foo               bar", 5), vec!["foo  ", "bar"]);
+    }
+
     #[test]
     fn whitespace_is_squeezed() {
-        assert_eq!(wrap(" foo \t  bar  ", 10), vec!["foo bar"]);
+        let mut wrapper = Wrapper::new(10);
+        wrapper.squeeze_whitespace = true;
+        assert_eq!(wrapper.wrap(" foo \t  bar  "), vec!["foo bar"]);
     }
 
     #[test]


Test patch: