Added more thorough check for hdfs. Currently testing this change