D-Programming-Language/phobos — pull request #934 by monarchdodra (2012-11-07 18:46:44+00:00)
Title: Splitter improvements/repairs

Covers:

5977: http://d.puremagic.com/issues/show_bug.cgi?id=5977
6730: http://d.puremagic.com/issues/show_bug.cgi?id=6730
8013: http://d.puremagic.com/issues/show_bug.cgi?id=8013
8166: http://d.puremagic.com/issues/show_bug.cgi?id=8166
8866: http://d.puremagic.com/issues/show_bug.cgi?id=8866
8867: http://d.puremagic.com/issues/show_bug.cgi?id=8867

Features:
1. Adds support for ranges as basic as "ForwardRange" (for both input and separator);
Current implementation required both to be RA + hasLength.
2. Adds (undocumented) biddir support for all flavors of splitter that take RA as input.
3. If no separator is provided, and the input range's element type is char, then a convenience "string splitter" is taken. This has the same behavior has array's split: "no empty tokens, runs of white are run together"
3.1. `(D std.algorithm.splitter(R)(R SomeCharacterRange)` now only works with non-strings, as that is already covered in array (so it avoids ambiguity).
4. Array's "splitter(C)(C[]) now splits the same way as `split(S)(S)` (so no add end of splitter empty tokens).
5.1. Array's `split(R)` will now turn down infinite ranges.

Regarding the implementation:
I used an aggressive implementation, that extracts the first token on construction. This makes the implementation simpler. It initially slows down iteration because it evaluates back. I may change this in a future edit, but for now, I think "something that works, and is easy to maintain" trumps "efficient".

Because of massive amounts of code duplication, I defined most duplicate code in mixin templates. Not pretty, but efficient.

Standing issues:
1. IMO, splitter should not offer bidirectional unless explicitly requested by the user, just the same way we have `filterBidir`. Suggest doing `splitterBidir`, and dropping backwards support for standard `splitter`
2. Outright deprecated support for bidirectionality of spliter(R1, R2), because the result is dependent on the splitting direction:
```D
string base = "aaabaaadaaa";
string sep = "aa";
splitter(base); => ["", "ab", "ad", "a"]
splitter(base, sep).array.retro; => ["a", "ad", "ab", ""]
splitter(base, sep).retro; => ["", "da", "ba", "a"]
```
As you can see, the range's retro is not the retro of the range's array... Since the behavior is undocumented, I though deprecation is fine.

Because of the complete rewrite, there are two commits: One to remove the old one, and one to insert the new one. This should make it easier to read the diff.
There are a few changes in the unittests, that should help see what has changed where.

Don't hesitate to ask if there are any doubts, this is a long commit.

I'll post some use cases here to illustrate the changes in the morning.