D-Programming-Language/phobos — pull request #299 by dawgfoto (2011-10-22 03:32:56+00:00)
Title: UTF decoding optimizations

- Use fast path tests for non-complex unicode sequences that can be
inlined. These rely on the built-in array bounds check.

- Factor out complex cases into separate functions that do exception
based validity checks. The char[] and wchar[] versions use
pointers to avoid redundant array bounds checks, thus they can only
be trusted.

- Complete rewrite of decode for char[] to use less branching and
unrolled loops. This requires less registers AND less instructions.
The overlong check is done much cheaper on the code point.

- The decode functions were made templates to short circuit the very
restricted function inlining possibilities.

As a rough number, I get a 2x-4x speedup in streamed string decoding,
even for unicode heavy input.

We should think of moving these functions to druntime
to avoid the duplication.