cleanup:
- dataframes made optional
- streams closed when job is deleted
- additional checks on hdfs apis
- optimized single byte read from hdfs stream