Deflakifying integration tests

**Warning** this is still a draft, please don't publish this yet! But comments, questions, concerns are all welcome.

~Max