do not perform cleanup to check temporary files
fix terrier's ways of traversing hdfs dirs into new hadoop api