TechEmpower/FrameworkBenchmarks — pull request #1257 by msmith-techempower (2014-12-16 21:54:47+00:00)
Title: Meganuke

This changes the suite to run every test in a "sandbox" via a executor user in order to fix #1225.

A test will be run by `testrunner` and then instead of a `stop` function being called to ask the framework to shut down nicely, we exit the shell started for `testrunner` and `kill -9` everything that that user started.

Preface: none of the numbers posted here are official in any capacity; they are here only as a proof of concept that tests are able to start, be stopped, and bind to ports. These numbers should be considered suspect since this is a new hardware environment and no tuning has been done towards tuning to a preview or final run. These numbers are simply here to prove that tests can be reliably stopped now.

[This](https://gist.github.com/msmith-techempower/4c81cfb49dcc12977169) is a preview-style run done on the Peak hardware using a build of the `meganuke` branch from about a week ago. One can search for "port" and will see a few, but not many, errors trying to bind to a port. I added more checks before tests are run to ensure that the ports are available before failing.

[This](https://gist.github.com/msmith-techempower/1e4b7595bb10bcc9ab00) is an preview-style run done on the Peak hardware illustrating that every test that started successfully bound to the port to which it was trying to bind.