mesos/spark — pull request #369 by pwendell (2013-01-14 04:04:38+00:00)
Title: Java API For Spark Streaming

This patch adds a Java API for Spark Streaming. It is modeled after, and fully integrated with, the existing Spark Java API. This merge includes full API functionality for both normal streams and paired streams. It also includes an example in Java and a comprehensive test suite. The test suite improves overall streaming test coverage while also validating the Java <=> Scala type conversions which constitute most of the logic in the API code.

There are two features I plan to add before this is fully "merge ready":
- Documentation for remaining JavaPairDStream classes (those docs don't exist yet in the streaming code)
- Test coverage for a few additional methods.

This currently does not support creating a custom DStream or InputDStream in Java. I'm planning to include that in a subsequent merge that is independently reviewed.

It would be great to get some feedback on this in the next few days as I add the additional tests.