Clean up first. This allows users to get logs from failed runs before they're wiped out. Ref: #354