Delay cleanup of listeners

until all requests using them are done.

Also do stats just before we free the request, instead of when
we remove it from the request hash.  This is because some requests
aren't in the hash...