Add simple retry logic into contrib/test