Electronically keys can be made to be fairly secure. The private key can and should be stored encrypted itself by some other scheme. Ascertaining a key by trial and error is realistically computationally infeasible. Trial and error may be possible if the key was generated by a computer. Someone, knowing how it was generated, might narrow the range of possible keys and systematically go through them.
Short messages are most vulnerable.
First, if the adversary can find a plain text - cyphertext pair where the cyphertext matches the cyphertext in question then the adversary knows the plain text may also be paired with the cyphertext in question. Second, if the adversary can obtain enough samples of the cyphertext to find patterns that match patterns found in natural language, they may be able to gain some meaning. These threats can be easily minimized by padding short messages and periodically changing keys. Besides cracking the encryption messages themselves, the scheme may be compromised through fraud, particularly impersonation. An adversary may publish their own public key and pass it off to be that of someone else.