some refactoring, added HDFS, Spark