Keep track of all active listeners so that they can be shut down properly