Test that gem doesn't auto-retry.