twitter/scalding — pull request #996 by bholt (2014-08-04 18:03:22+00:00)
Title: hRaven Reducer Estimator and other minor estimator fixes

Adds new "scalding-hraven" module with an estimator that looks up job history in hRaven to determine the actual amount of bytes reaching reducers. The `HRavenHistory` trait can be added to a `ReducerEstimator` to add functionality for looking up job history, and the `RatioBasedReducerEstimator` does this, scaling the estimate provided by `InputSizeReducerEstimator` based on the past ratio between mapper/reducer input sizes.

This PR also includes a couple other minor changes:
- Using a case class as argument to `estimateReducers` to hold the common reducer estimator info (`flow`,`step`,`predecessorSteps`)
- Add optional `fallbackEstimator` which lets estimators be chained

So far I don't have a story for unit testing hRaven. Any ideas are welcome.