Fix bug where the clear whitelist option would not always appear