Add some extra checks to (hopefully) prevent flakes.
Merge pull request #3114 from brendandburns/e2e

Add some extra checks to (hopefully) prevent flakes.