Adding more fail-safes for stopping tests

After a preview run, there were still some tests that
claimed that their required ports were not available.
This change moves `stop` from the framework test to
the benchmarker (since we don't need the test to stop
anymore). Additionally, it adds another stop call to
any place that checks whether the port is bound (read:
before a test is started and after a test is stopped).