mesos/spark — pull request #519 by Reinvigorate (2013-03-11 23:43:52+00:00)
Title: Streaming/Kafka improvements

This pull request makes available all config params of the high level kafka consumer (see http://kafka.apache.org/configuration.html). This allows for more fine-grain control over how events are consumed from kafka.

Example of consuming from very start of topic:
```scala
val kafkaParams = Map(
"zk.connect" -> "zoo01,zoo02,zoo03",
"groupid" -> "my-consumer-group",
"zk.connectiontimeout.ms" -> "10000",
"autooffset.reset" -> "smallest"
)

val inputStream = ssc.kafkaStream(kafkaParams, topicMap, Map[KafkaPartitionKey, Long](), StorageLevel.MEMORY_AND_DISK)
```

The existing/simpler kafkaStream interface was maintained so that it's non-breaking. This new functionality is just an override for when you need more of the kafka config bells and whistles.