[SPARK-3875] Add TEMP DIRECTORY configuration

Related to SPARK-3875.

PS this can all be one line if you set the default value to `... .get("spark.tmp.dir", System.getProperty("java.io.tmpdir"))`