web-animations/web-animations-next — pull request #125 by mithro (2014-06-30 06:33:29+00:00)
Title: Adding "grunt sauce:xxxx" tasks to allow checking of unit tests on many browsers

This pull request allows us to run the unit tests on SauceLab platform. There is a lot more that could be done here to make it work better, but I think this is probably a good start.

Things are a little bit complicated because we have a stack which is; `grunt -> karma -> (chia|mocha)` and Karma, Grunt and Mocha, each of which support running on SauceLabs.

I ended up choosing to use Karma as the integration point as,
* it was the best documented,
* it seems to be commonly used,
* I actually got it to work.

Karma still continues to be launched via Grunt. Every grunt `test` target now has an equivalent grunt `sauce` target. For example, `sauce:minifill` is the equivalent of `test:minifill`. For these sauce targets to run, you need to set up the `SAUCE_ACCESS_KEY` and `SAUCE_USERNAME` environment variables, these can be retrieved via your SauceLabs account.

This **doesn't** use the Javascript Unit testing stuff that SauceLabs have released because it needs some more work on the structuring of the Grunt/Karma/Mocha set up (but I will give that a look too). This pull request **does not** make update Travis to use these targets. That is something Alan and myself should discuss and figure.

We should also figure out what targets we actually want to run on SauceLabs. I assume it should be similar to what Polymer supports?

--------------------

Example output; (it's slightly easier to read this on the console as karma colorizes the output nicely.)
```
tansell@tansell-z620-l2:/fast/web-animations-next$ grunt sauce:minifill
Running "sauce:minifill" (sauce) task
[2014-06-27 19:55:22.544] [DEBUG] config - Loading config /fast/web-animations-next/test/karma-config-ci.js
INFO [karma]: Karma v0.12.16 server started at http://localhost:9876/
INFO [launcher]: Starting browser chrome (Windows 7) on SauceLabs
INFO [launcher]: Starting browser firefox 27 on SauceLabs
INFO [launcher]: Starting browser iphone 7.1 (OS X 10.9) on SauceLabs
INFO [launcher]: Starting browser internet explorer 11 (Windows 8.1) on SauceLabs
INFO [launcher.sauce]: chrome (Windows 7) session at https://saucelabs.com/tests/5ba8fe38797a4dd888a80c7b09b4a28a
INFO [launcher.sauce]: internet explorer 11 (Windows 8.1) session at https://saucelabs.com/tests/27d009bd00a54dc49c71499ec67368f5
INFO [Chrome 27.0.1453 (Windows 7)]: Connected on socket _N20qImo-dOcjDLpwDXS with id 95007235
INFO [IE 11.0.0 (Windows)]: Connected on socket gWDgmFqT8C1O4qnPwDXT with id 19853181
............................................................................
Chrome 27.0.1453 (Windows 7) effect-convertEffectInput Convert effect input for a simple effect with one property. FAILED
AssertionError: expected NaN to be close to 0 +/- 0.001
at Function.assert.closeTo (/fast/web-animations-next/node_modules/chai/chai.js:3315:35)
at Context. (/fast/web-animations-next/test/js/effect.js:358:12)
at callFn (/fast/web-animations-next/node_modules/mocha/mocha.js:4338:21)
at Test.Runnable.run (/fast/web-animations-next/node_modules/mocha/mocha.js:4331:7)
at Runner.runTest (/fast/web-animations-next/node_modules/mocha/mocha.js:4728:10)
at /fast/web-animations-next/node_modules/mocha/mocha.js:4806:12
at next (/fast/web-animations-next/node_modules/mocha/mocha.js:4653:14)
at /fast/web-animations-next/node_modules/mocha/mocha.js:4663:7
at next (/fast/web-animations-next/node_modules/mocha/mocha.js:4601:23)
at /fast/web-animations-next/node_modules/mocha/mocha.js:4625:7
.....
Chrome 27.0.1453 (Windows 7): Executed 168 of 84 (6 FAILED) (0.832 secs / 0.116 secs)
Chrome 27.0.1453 (Windows 7): Executed 168 of 84 (6 FAILED) (0.866 secs / 0.116 secs)
................................................................................
................................................................................
........
IE 11.0.0 (Windows): Executed 168 of 84 SUCCESS (0.632 secs / 0.106 secs)
IE 11.0.0 (Windows): Executed 168 of 84 SUCCESS (0.642 secs / 0.106 secs)
INFO [launcher.sauce]: iphone 7.1 (OS X 10.9) session at https://saucelabs.com/tests/4de6b46ce1934f8e80998ed957e48f5c
INFO [Mobile Safari 7.0.0 (iOS 7.1)]: Connected on socket eX8F9GPWR0zOT_F_wDXU with id 94945318
INFO [launcher.sauce]: firefox 27 session at https://saucelabs.com/tests/029afe01439c4f53bef33ab82ec17048
WARN [Mobile Safari 7.0.0 (iOS 7.1)]: Disconnected (1 times), because no message in 10000 ms.
Mobile Safari 7.0.0 (iOS 7.1): Executed 0 of 0 DISCONNECTED (10.005 secs / 0 secs)
INFO [Firefox 27.0.0 (Linux)]: Connected on socket 6MXHWUYScLqaQnP1wDXV with id 89516128
................................................................................
................................................................................
........
Firefox 27.0.0 (Linux): Executed 168 of 84 SUCCESS (0.598 secs / 0.145 secs)
TOTAL: 6 FAILED, 498 SUCCESS
TOTAL: 6 FAILED, 498 SUCCESS
INFO [launcher.sauce]: Shutting down Sauce Connect
Firefox 27.0.0 (Linux): Executed 168 of 84 SUCCESS (0.61 secs / 0.145 secs)
TOTAL: 6 FAILED, 498 SUCCESS
TOTAL: 6 FAILED, 498 SUCCESS
WARN [Mobile Safari 7.0.0 (iOS 7.1)]: Disconnected (2 times), because no message in 10000 ms.
Mobile Safari 7.0.0 (iOS 7.1): Executed 0 of 84 DISCONNECTED (10.002 secs / 0 secs)
Warning: Task "sauce:minifill" failed. Use --force to continue.

Aborted due to warnings.
```

https://github.com/web-animations/web-animations-next/issues/64