Make sure .ssh gets right permissions