Add the ability to multiple test iterations without rebuilding.

Address comments.
Merge pull request #1055 from brendandburns/flake

Add the ability to multiple test iterations without rebuilding.