If there's only a single worker, do not block after accept treshold has been
reached