Voxer/sse4_crc32 — pull request #11 by uskudnik (2014-02-06 16:26:59+00:00)
Title: Switch to Mark Adler's hardware and software implementation

After semi-extensive benchmarking I switched to [Mark Adler's implementation](http://stackoverflow.com/a/17646775/2696894) for low level operations. While hardware version differences between your and his code are so small that they could be attributed to any number of other causes, the main benefit is software fallback.

JS string	138.41425 +- 1.51806890077	Speed up (current String): 1.0	Speed up (Array): 0.145467320019
JS array	20.13475 +- 0.478236539277	Speed up (current String): 6.87439625523	Speed up (Array): 1.0

C call only, make debug (build -d)
CRC C software	1.74 +- 0.0158113883008	Speed up (current String): 79.5484195402	Speed up (Array): 11.5716954023
Node - voxer	0.7675 +- 0.0178535710714	Speed up (current String): 180.344299674	Speed up (Array): 26.2342019544
CRC - Mark Adler	0.735 +- 0.0111803398875	Speed up (current String): 188.318707483	Speed up (Array): 27.3942176871

(The results here for C versions were benchmarked in the crc32c.c file itself, so absolutely no overhead due to bindings or anything. Input string was constant. Not really a fair comparison, though only compiled with `make debug`.)

Our initial version was based on [Javascript string-based operations](http://stackoverflow.com/a/18639975/2696894), a switch to [array based version](http://stackoverflow.com/a/18639999/2696894) resulted in ~6x of a speed up. Going to C, software implementation is 11 times faster, while both hardware versions are 26-27 times faster than array-based implementation (and 180-188 times faster than string based version :))

Now, for the more interesting part - full call (NodeJS -> C++ bindings -> C), compiled `make build`

Iterations = 50000000; Sample size = 11
Full call (NodeJS -> C++ bindings -> C); make build
String: "text" + Math.random() * 10000;
JS Array	65.4263333333 +- 0.90362283184
Node - Voxer	33.014 +- 0.61320632743
C - Mark Adler	35.889 +- 1.54114654289

C software-based implementations, "text" + Math.random() * 10000:
Generated array	34.8908 +- 0.211416555643
Static array	35.1686666667 +- 1.41606222399

Hardware accelerated, static string, "Kevin van Zonneveld"
Vox	11.2927272727 +- 0.798183174569
Mark	11.0386363636 +- 0.528981916296

Software implementation, static string, "Kevin van Zonneveld"
Static table	11.8988181818 +- 1.01931124858
Generated table	12.0049090909 +- 0.87978566135
Javascript array 19.1868181818 +- 0.2252847968

(Static table means that the crc32c table was hard-coded.)

While I would prefer to test with a number generator, it turned out to be responsible for a large portion of the time taken for iterations (so large that I would say that those measurements are useless), therefore I had to turn it off and moved the string assignment out of the `for` loop as it also had a noticeable impact on the performance (although I forgot to take the numbers).

Mark's implementation still leads you when using static string, although only by a very small margin (~2%) - the use of software fallback is the main reason why I propose his version.

Static / generated table result should be almost identical as the table is computed only on the first iteration, but there is a small difference, so I kept it.