D-Programming-Language/phobos — pull request #1685 by blackwhale (2013-11-10 20:11:37+00:00)
Title: Introduce UTF matchers into std.uni.

It's a step zero to get decode-less std.regex.

UTF matchers are efficient functors around a set of
specific tries. Processing Unicode characters
without decoding at speeds on par with decoding itself.

Along the way make staticIota at 'package' protected and reuse it.
Fixes a couple of shameful typos.

Some ballpark numbers with LDC.
m8-Alpha is new UTF matcher
new-trie - is decode + std.uni Trie
trie - tried and true std.regex Trie
decode-only - counts codepoints by running decode in a loop
noop - counts numbers of bytes > 0x20

|Name | Found | Time (s) | Mb/s|
|m8-Alpha [arwiki]| 273256 hits| 1.35852| 459.92 Mb/s|
|trie-Alpha [arwiki]| 273256 hits| 2.35152| 265.70 Mb/s|
|new-trie-alpha [arwiki]| 273256 hits| 2.27336| 274.84 Mb/s|
|decode-only [arwiki]| 342832 hits| 1.63872| 381.28 Mb/s|
|noop [arwiki]| 599328 hits| 0.30504| 2048.28 Mb/s|

|Name | Found | Time (s) | Mb/s|
|m8-Alpha [dewiki]| 1435456 hits| 3.84244| 442.65 Mb/s|
|trie-Alpha [dewiki]| 1435456 hits| 7.92868| 214.52 Mb/s|
|new-trie-alpha [dewiki]| 1435456 hits| 6.7128| 253.38 Mb/s|
|decode-only [dewiki]| 1680050 hits| 3.4514| 492.81 Mb/s|
|noop [dewiki]| 1607454 hits| 0.88392| 1924.23 Mb/s|

|Name | Found | Time (s) | Mb/s|
|m8-Alpha [enwiki]| 1152212 hits| 3.90008| 459.83 Mb/s|
|trie-Alpha [enwiki]| 1152212 hits| 7.90484| 226.87 Mb/s|
|new-trie-alpha [enwiki]| 1152212 hits| 6.95048| 258.02 Mb/s|
|decode-only [enwiki]| 1786517 hits| 3.53968| 506.65 Mb/s|
|noop [enwiki]| 1699856 hits| 0.97012| 1848.61 Mb/s|

|Name | Found | Time (s) | Mb/s|
|m8-Alpha [ruwiki]| 414460 hits| 2.0152| 452.02 Mb/s|
|trie-Alpha [ruwiki]| 414460 hits| 3.34324| 272.46 Mb/s|
|new-trie-alpha [ruwiki]| 414460 hits| 3.29996| 276.03 Mb/s|
|decode-only [ruwiki]| 496887 hits| 2.33088| 390.80 Mb/s|
|noop [ruwiki]| 884374 hits| 0.45652| 1995.31 Mb/s|