Fix exceptions causing builds to queue up and not finish

Reviewers: ar, cramer

Reviewed By: cramer

Subscribers: cramer

Maniphest Tasks: T19834

Differential Revision: https://tails.corp.dropbox.com/D70129