Adjust test to bring it inline with new retry behavior