Correctly handle failure during handler change