Make sure parent directories for admin and bootstrap keys exist