prevent Spark from overwriting directory silently and leaving dirty directory