HouseKeeper: remove from the list of listeners in the destructor

This fixes a segfault when exiting the FE caused by the HouseKeeper receiving
messages after it had been deleted.