Fix HDFS flush behavior

org.apache.hadoop.fs.FSDataOutputStream only supports sync(), but not flush(). This means that flushing
higher level streams doesn't actually do anything if the Hadoop FileSystem is used. This is now fixed.

Further, this clarifies why logging local files is handled as a special case by referencing a known,
unresolved bug in HDFS (HADOOP-7844).
Fix bug: storage status was not reflected on UI in the local case

This is because the executor ID from task events and that from the storage status
list are different. The former is "localhost", but the latter is "<driver>." To be
consistent, we will only use "<driver>."
Use a more mnemonic path for logging

Also get rid of a couple of unused vals.