andrew/split — pull request #221 by caser (2013-11-18 21:36:50+00:00)
Title: Zscore fix

Hey Andrew,

I saw [this post](https://github.com/andrew/split/issues/199) in the open issues, and upon closer inspection, realized that the z_score algorithm wasn't calculating z-scores correctly.

I re-implemented the z_score algorithm with the correct math, and then refactored it out into its own module. This effectively decouples the z_score calculation from the alternative class. The alternative class now implements its own z_score method which simply passes data along to the calculate method in the new module.

I also tweaked the z_score specs to work with the new implementation, and changed the UI output in the case that the experiment doesn't yet have sufficient data.

Finally, I updated the documentation with a section about statistical validity. It may make sense in future versions to implement a minimum sample size calculator in Split that hides significance data until the experiment has run its course, which would minimize false positives (and is best practices in A/B testing). Until then, it seems prudent to have some sort of warning or advice to users so that they are running statistically valid tests.

Happy to chat or tweak things if need be.

Cheers,
Casey