stripe/brushfire — pull request #18 by avibryant (2014-12-20 20:47:21+00:00)
Title: replace Evaluator with Error

This address #17 by removing Evaluator and replacing it with a minimization of the training error. The default is now to minimize brier score rather than maximizing chi^2.

This required adapting `BrierScore` to work when the `actual` distribution was more than just an indicator. @danielhfrank, you might want to review that part?

@snoble you should be able to play min-weight games by having the error go to infinity if the `actual` distribution's weight is too low, so I don't think we lose any expressivity that we care about.

There's no longer an obvious hook for CHAID-style split-merging, but we can cross that bridge when we come to it (having that be part of Evaluator was always suspicious anyway).

Also cc @brahn who this might be directly relevant to right now.