Clear the CAP handlers properly when reloading the config
Fixes #83