D-Programming-Language/phobos — pull request #1502 by monarchdodra (2013-08-22 15:06:50+00:00)
Title: Fix splitter!pred and splitter(string)

This fixes:
* Issue 6730 - std.algorithm.splitter conflicts with std.array.splitter
* Issue 6791 - std.algorithm.splitter random indexes utf strings
* Issue 8013 - splitter() and split() give different results

(links:)
* http://d.puremagic.com/issues/show_bug.cgi?id=6730
* http://d.puremagic.com/issues/show_bug.cgi?id=6791
* http://d.puremagic.com/issues/show_bug.cgi?id=8013

as well as "unlocks" `splitter!pred` for forward ranges. Also fixes the splitting behavior of `splitter!pred`, which was just wrong, which, in turn, also fixes `std.array.splitter`.

--------

FWI, this is a toned down failed pull I had oppened 9 months ago: #934.

This fix can be review in 3 parts
1: A complet `splitter!pred` rewrite

I did this because `splitter!pred` was *completely* inconsistent with the rest of the splitter functions:
* leading delimiters where kept, but leading delimiters where omitted
* empty tokens where kept, but *multiple* runs of empty tokens where merged into a single token.

The new implementation has the exact same behavior as the other splitters. Also, the new implementation works on forward ranges. The new implementation is arguably simpler, even though it handles more types.

2: This leads to a `std.array.splitter(string)` rewrite:

`std.array.split` has a string specific overload that takes no teminator: This special case takes a string, and splits it on unicode white, and returns *no* empty tokens. Unfortunatly, `std.array.splitter(string)` was simply `return std.algorithm.splitter!(std.uni.isWhite)`. This means that `split` and `splitter` did not have the same output. Un-acceptable. Now it does.

3: Deprecate `std.algorithm.splitter(string)`

This *string* function specific function has no reason to be in algorithm. The only thing it is doing, is creating ambiguity for users who have both algorithm and string included.

(4: Also included: `std.array.split!pred`, which was missing for some stringe reason)