seedifferently/the-great-web-framework-shootout — Issue #26: Write a Fabric script to automate testing
Author: seedifferently
Created: 2012-02-22 04:21:16+00:00
State: open

Now that these tests have grown to more than just a few frameworks, it would be nice to have a script which automates things a bit more.

Being primarily a Python dev myself, I'd like to see a [Fabric](http://docs.fabfile.org/en/latest/index.html) script written which utilizes [boto's](http://boto.cloudhackers.com/en/latest/index.html) EC2 API.

The script would need to:

1. Boot the AMI.
2. Set up the test environment depending on the test. For example, a Python test would need:
1. apt-get install libapache2-mod-wsgi
2. Copy over the Pyramid test.
3. Run pip/easy_install to install dependencies.
4. Set up any necessary Apache configs (i.e. specify WSGIScriptAlias).
3. Run apachebench several times and log the output of reqs/second (and also make sure there were no errors).

We would want to be able to run all tests (e.g. `fab test_all`) or just specific tests (e.g. `fab test_pyramid`).