tests: Split test runs by release, and retry as needed.