Notify the admin when a key is replaced