Keep track of slow workers in a better way

...using a mutex, a condition variable, and a counter that counts
un-acked notifications.