D-Programming-Language/phobos — pull request #906 by monarchdodra (2012-10-28 10:52:37+00:00)
Title: Split(string) unicode support

Added unicde support, plus unittest, for split.

The implementation was completely changed, so not much point looking at it in "diff" mode.

I'm not 100% sure about `appender`'s usage, but the documentation says to use `appender` instead of `~=` when making several appends in a row, so I did.

Addresses #7689 issues 7689.
http://d.puremagic.com/issues/show_bug.cgi?id=7689
(what syntax am I supposed to use for automatic tagging?)

```D
S[] split(S)(S s)
if (isSomeString!S)
{
alias ElementEncodingType!S C;

S[] result;
auto app = appender!(S[])();

s = s.strip(); //Means there is no trailing ws: Usefull so we only check empty once.
if(!s.empty)
{
restart:
foreach (size_t i, dchar c; s)
{
if(std.uni.isWhite(c))
{
app.put(s[0 .. i]);
//increment i manually, because we already know the first index is white.
i += c.codeLength!C();
//Then remove the rest of the whites
s = s[i .. $].stripLeft();
goto restart; //And look for the next one
}
}
//When we reach here, we know that s has no whites, but is not empty, so it HAS
//to contain a word.
app.put(s);
}

return app.data;
}
```