Get the fingerprint only when necessary.

This speeds up connection initialization in cases where getting the fingerprint is expensive, like for containers/jails on a remote host.