Add sshd_config and restart ssh after config file change.
Get a little crazy and try to remove the passphrase from the key.