mobify/stencil — pull request #31 by ry5n (2014-05-31 23:58:56+00:00)
Title: Add automated visual regression testing

Adds an automated way to prevent regressions in previously-tested components. This only adds the testing infrastructure. As full device testing is done on components and they are battle-hardened, we can use the functionality introduced here to commit visual test fixtures to the repo (screenshots representing known-good-state components). Future commits against those components should run the tests set up here to verify things haven’t been broken.

Status: **Opened for visibility**

Reviewers: @jeffkamo @kpeatt @scalvert
Ticket: none
Linked PRs: none

## Changes
- Add PhantomCSS for visual regression testing
- Add test suite to test all components based on the project directory structure. New components will be included automatically without having to update the suite.
- Adds grunt tasks for running all tests (`grunt test`) or a single component’s tests (e.g. `grunt test:components/arrange`).
- Does not include test fixtures (screenshots) yet. These should only be committed once components have been manually tested and green-lighted for production use
- All comparison and failure screenshots are ignored and will not be committed to the repo.

## How to test-drive this PR
- **Important**: Install CasperJS and its dependency PhantomJS. These are not package-managed and must be installed separately. This is especially true since the test suite requires the latest dev version of CasperJS to work correctly. Use **only** the “install with Git” instructions on this page: http://docs.casperjs.org/en/latest/installation.html
- Verify that Casper is installed by typing `casperjs` on the command line. You should see a message with the Casper and Phantom version numbers
- In the Stencil directory, run `grunt test:components/arrange`. You can also run `grunt test` but running all tests will take some time. This time delay is the reason verbose terminal output is on by default. You can supress this by appending :terse to the end of your grunt command, e.g. `grunt test:components/arrange:terse`.
- Check that the correct fixture screenshots have been created in tests/fixtures/screenshots
- Run the same test command again. PhantomCSS will compare the existing screenshots with the new set and all tests should pass.

## Research resources
- https://github.com/Huddle/PhantomCSS
- http://casperjs.org/
- The reason a dev version of Casper is required for the time being: https://github.com/n1k0/casperjs/pull/918