SPARK-1729. Make Flume pull data from source, rather than the current push model

Update to the previous patch fixing some error cases and also excluding Netty dependencies. Also updated the unit tests.