Decline offer when we reject running jobs.

@florianleibert @andykram