Let ensure it is deleted before writing a new id_rsa private key.