Upgrade OpenSSH to 6.4.
Disable the use of DNS to speed up the time it takes to establish a connection.
Reduce the login noise by disabling the login banner.