In SSH, the public key is stored on the remote server to verify your identity, while the private key is kept on your local machine for authentication. When you connect to a remote server, your local device provides the private key, which the server matches with the associated public key to establish the connection.