I was about to add this to the doc just now - but as I did I realised that it's not quite true.

If you have a remote application set up to point to a known mock address then even with IP filtering you won't be able to set up that mock any differently as all of the requests to the mock server will come from the same place - your remote application.

At the moment, if you are pre-configuring remote applications to point to a known address, we will always have a dependency on there only being one set of tests running against that node at any time (unless it only wants to return the same thing always). As soon as you want a test for "what happens if we get a 500 back" this only-one-test-suite rule kicks in.

I can't really see how we can get away from that. Unless you inject configuration into your remote running application, but even then how can you do that reliably so that only your test is using that current configuration.

I think the rule is there is a one to one mapping between test-suite and group of remote apps. If you want concurrent tests then you'll need to deploy a new stack I guess.