Handle case where slow workers cause assertionerrors

Signed-off-by: Rick Copeland <rick@synapp.io>