Check infrequently to avoid blocking the actual worker

Signed-off-by: Zachary Auerbach <zauerbach@pivotal.io>