Always add a shouldRetry option to tests

Always add a shouldRetry option to force Unit tests down our code path