When the user tries to log in, the client tells the server the public key that the user wishes to use for authentication. The server then checks if this public key is admissible. If so, it generates a 256 bit random number, encrypts it with the public key, and sends the value to the client. The client then decrypts the number with its private key, computes a 128 bit MD5 checksum from the resulting data, and sends the checksum back to the server. The server computes a checksum from the correct data, and compares the checksums.
Authentication is accepted if the checksums match.
The RSA private key can and should be protected with a passphrase . With passphrase, authorization requires access to the key file and the passphrase. It does not rely on the network, routers, domain name servers, or the client machine. The only thing that matters is access to the private key. The program also asks for a passphrase.