Removes old public key.