Add missing files and only pull more data from amqp if there's space in the metric queue.