Add cleanup for running builds to prevent the issue of builds just finishing causing an inconsistent state.