Bomb out from addKeyToAgent() if we do not have the private key.