start of a refac to streaming_hadoop_step to enable better hooks for testing