worker: Slight refactor in collect_new_clients
accept: After reconfiguring close clients which are connected to
listeners removed in the reconfiguration, and clients which no
longer match the ACLs on each listnere