You can easily remove a Legacy Contact by going back into your Legacy Contact settings and tapping to remove them. They won’t be directly notified if you remove them, but you will be removed from their Legacy Contact List and their access key will no longer work.