ENG-5769 Fix test for not automatically retrying.