Fix SSH 'Too many authentication failures' error.

When the system provides an SSH agent, which has many loaded keys, the
SSH client may attempt to use these for authentication before trying
the identity specified by the SSH config or invocation.

This can result in the guest system aborting the connection (due to
too many failed attempts) before the correct key is tried.