yui/yeti — pull request #29 by reid (2013-03-10 02:00:20+00:00)
Title: Fix test timeout feature; rapid disconnects for crashed browsers

TL;DR:

* Replace timeout feature with a test-specific timeout and a browser-response timeout.
* Browser has to keep responding or else it gets killed within 3-5 seconds.
* Each test must complete within the test-specific timeout.
* Introduce some robustness enhancements to the client-side script.
* If a browser crash happens during a test run, attempt to run that test again.

Recent refactoring broke the timeout feature. Additionally, the old implementation of test timeouts was based on a very fragile setTimeout in `AllAgents` that had to be rescheduled every time a new batch was created with a different timeout value. Timeouts were supposed to abort very slow tests, but they were also occurring in previous 0.2.x releases if a browser crashed without letting Yeti know.

This pull request addresses both problems.

Separate internal timeouts are needed for determining if a test is taking too long vs. if a browser has stopped responding. These two timeouts are needed per browser. Instead of creating a lot of setTimeouts, we introduce a `PeriodicRegistry` to run all of our health checks with one setTimeout. This replaces the complex logic inside of `AllAgents`.

We also introduce a browser ping. The ping is sent if no response (e.g., results or beat events) come from the browser for a few seconds. After 3 pings without a response, we consider it dead. This should allow users of the Yeti API to take action quickly if a browser is stalled (e.g. by restarting it). This applies to tests and the capture page.

This also updates the UI for old browsers that cannot use streaming protocols to communicate with Yeti: instead of getting updates in the UI after every test, we now see movement of the spinner every 3-5 seconds or so.

Having two internal timeouts lets us increate the default test timeout value to 5 minutes from 45 seconds, since dead browsers would be caught by the unresponsive pings.

We also introduce some logic in the client-side `tempest.js` to:

* Resend test results if the first send failed.
* Wait longer for IE 6, 7, and Android browsers before navigating to the next test.
* Respond to server ping requests.

Finally, we also no longer use a separate pending test queue in `Tests`. We instead check for tests without results that are not already being currently ran. This significant change lets us give a test that one browser failed to run to another identical browser after we consider the first instance dead.

These changes help make using old browsers more reliable and catch crashed browsers much faster and accurately.