gsathya/torperf2 — pull request #2 by Ryman (2013-09-30 20:48:55+00:00)
Title: Core Http proxy implementation

I think this is in a state where we should be able to build at least the static experiments on top of it relatively easy.

This runs a local http proxy for each version of Tor that's running. (Haven't actually run multple versions yet) The Http proxy takes care of the majority of timing information that would be required for simple experiments (static file downloads, canonical median webpage, hidden service performance) while also providing a simple interface with which our selenium experiment can use. (The selenium experiment will likely need a lot more timings, but it will not have to worry about Tor management or using a socks proxy.)

Note that my alexa experiment is just for testing and doesn't really represent the intended experiment. (Easier to run that test than static download test as I have no public ip for testing without tunnelling).

Also, there's a bug in txtorcon so you'll need to install the version in requirements.txt (This mattered more during the dev when I was killing tor instances regularly, it doesn't matter as much currently due to port cycling but it's a problem if the tor initialisation times out)

Things todo to get static_file_downloads completed:
- [ ] Resetting identity on the Tor instance for each experiment
- [ ] Take public host in torperf config (as you have in recent commit)
- [ ] New experiment class that generates random files on each run (into it's local public folder) and then does the a simple http get for each file with an expected bytes header. The proxy should react to this header and record data/percent timings for the request.
- [ ] Grab timing information from the file server for the request (The http proxy should probably add the uniqueid header to the request so the fileserver can use that as an identifier)

Things required for the selenium experiment:
- [ ] Support https proxying
- [ ] Handling the HAR format (probably)
- This will require the postprocessing part to replace timings for all requests in the HAR format (including redirects)
- [ ] The actual selenium experiment code!

Circuit information for each experiment is also missing (as it was labelled optional in the data spec) so I haven't looked into that at all yet, but it shouldn't be that wild to get working.

If you want me to rebase let me know, I haven't rebased on your file server changes yet as I don't think relying on the timestamp alone is a good idea. I plan to reply to karsten tonight to clarify on the sequential vs parallel experiments to know if this is actually going to be a realistic problem or not. I'm open to discussing the timestamp issue.

So yeah, review, rip it apart, and hopefully we can have static_downloads done soon!