Cleanup: Provide option to not fail tests immediately