Set correct group to home dir and .ssh if provided