int3/doppio — pull request #50 by jvilk (2012-10-31 19:09:28+00:00)
Title: Benchmark mode, Makefile fixes, and other goodies!

Changes:

* Makefile updated to reflect source file changes.
* Added a mock console, which is a drop-in replacement for the JQuery console. It receives a 'shell script' (a JSON array) for the Doppio browser shell to execute and pipes output back to the web server.
* Added a benchmarking mode to the web server, which allows you to:
* Specify a list of Doppio 'shell scripts' to run (which, again, are JSON arrays of valid Doppio browser shell commands).
* Specify a list of browsers to test with.
* The server will automatically launch / kill browsers, have them run each of your scripts, and record the output to separate log files.
* Added a benchmark target to the Makefile, which is equivalent to release but with the mock console.
* Doppio console in the browser has a ``-classpath`` option for the ``java`` command.
* You can now 'mount' directories into the webserver, e.g. if you have a bunch of class files in ``/Users/jvilk/benchmarks``, you can do ``./tools/webrick.rb -m /Users/jvilk/benchmarks`` and it will mount it in ``/benchmarks``. Then you can just tell Doppio to use classes in the folder with the ``-classpath`` option in the browser...
* Fixed a bug with the web server where ``third_party`` would not be mounted properly for release mode (``third_party`` is not in the ``build`` folder).
* Server now has ``--help`` output, which should be invaluable for figuring out all of those switches I added. :)
* Added logic to the Makefile to install any needed node dependencies into a local ``node_modules`` folder and to check for other required dependencies that may need to be installed with ``setup.sh``. This resolves issues with having the wrong version of coffee-script installed on the system.
* Removed some things from ``setup.sh`` that are now handled by the Makefile, and added a check for ``pygmentize`` which is required to build docs.

Current limitations:
* Due to time constraints, the server's benchmark mode is limited to Macs at the moment.
* Opera will pop up with a dialog box that the user *must* answer if you open it after it gets killed. As a result, running multiple scripts with Opera automatically requires a user to answer the dialog box on runs past the first one.
* Firefox likes to complain about long running JavaScript threads, which I believe pauses the script. You must change the value ``dom.max_script_run_time`` in ``about:config`` to the largest value you can (or you could alter Doppio to yield the JS thread more frequently...).
* Benchmark mode only supports firefox, safari, chrome, and opera.
* I want to eventually support scripts that are like bash scripts -- one Doppio command per line; then, webrick.rb can convert it into an appropriate JSON array for easy parsing on the browser side of things. Shouldn't be too hard if you have a library that can do string escaping...

Dependencies:
* You need Ruby 1.9 to use ``webrick.rb``'s benchmark mode, as I use the ``spawn`` method (you can get it with ``brew install ruby``). 1.8 works as usual for dev or release mode.

I'm sure there are others; I'm still in the process of using / testing it / improving it.

Here's a guide for how to run a simple benchmark:

``test.json``: ``["ls","java","ls"]``

```
make benchmark
./tools/webrick.rb -b test.json -e simpletest -w "opera:Opera,safari:Safari,firefox:Firefox,chrome:Google Chrome"
```

Log files from the test will be in ``./logs/simpletest``.

Before merging, *please* make sure the Makefile and benchmark script work for you. Also, if you have suggestions for how to improve the ``webrick.rb`` code, let me know. I had to 'learn' Ruby for this, so it may be a bit rough.