Make sure config directory exists before trying to write to it