The local dir should not be in HDFS... unless YARN can somehow ask jobs to put their scratch space on HDFS. It should be just a directory on the machine, for fast access. We don't assume that it's reliable. Usually in YARN I think you can just use the working directory of the process.