create directory dependencies in spark setup script