twitter/tormenta — pull request #24 by sritchie (2013-06-07 03:30:05+00:00)
Title: Improve upon ScalaSpout

This pull requests gives us a number of improvements to the ScalaSpout API. (The code here is based off of my branch with the TwitterSpout and build modularization... we can either drop that, or merge it in first then rebase this off of develop.)

* `ScalaSpout` no longer requires a `ScalaScheme`. I made two subclasses, one that implements `IRichSpout` directly and one that uses our current scheme transformation.
* Spouts can now be filtered, flatMapped and mapped directly (vs having to go through the scheme)
* Added `ScalaScheme.fromFn`, `ScalaScheme. fromTraversable`, and implicit bijections for ScalaScheme and ScalaSpout
* Recoded `TwitterSpout` with the new API
* Made a pull req into chill with the current `StormKryoFactory`. If we get this in and decide we want to scrap the bijections, then we can lose the bijection and chill dependencies entirely.

This new code will make it MUCH easier to create a `tormenta-testing` module. (We can make an `ArbitrarySpout` with a function that both gets a new `T` from the generator and adds it to a mutable accumulator. Then, at the end of the topology's local run for testing, we'll know exactly what came out of the spout.