Usually you need to set tight permissions on .ssh and authorized_keys for it to work, maybe we should assert that?