server: report metric loss after all secondaries finished

If we report our qeueusize when we stop trying to send them, we may
report more than are actually in the queue when we really terminate.
Our secondaries can (and should) process our queue, which means nothing
get lost.