travis-ci/travis-cookbooks — pull request #186 by gildegoma (2013-05-22 09:48:29+00:00)
Title: Consider to add Test Kitchen

This is a first experiment to demonstrate how we could easily run integration tests on travis-cookbooks. It uses Test-Kitchen *without Berskhelf*. Some patches on Test-Kitchen and Vagrant driver were needed for following reasons:
* vagrant must connect as `travis` (not `vagrant`) on the travis base boxes
* test-kitchen must provision within `bash -l` on the travis boxes because preinstalled `chef-solo` is not in default SSH PATH.
* cookbooks to upload are stored in `ci_environment` directory, not `cookbooks`.

Depending on where and how we want to go further, we may have to adapt on our sides (e.g. create symbolic link `cookbooks -> ci_environment`) or get some PRs accepted on Test-Kitchen side (to obtain required flexibility). But none of these points are showstoppers.

### Goals / Advantages

By providing "ready to use" Test-Kitchen suites in travis-cookbooks, I think we can improve in following aspects:
* Contributors can smoothly validate their changes with an environment close to Travis production environment. Test-Kitchen setup is straightforward (and becoming standard).
* Pro-active validation of new operating systems (new ubuntu release, other Unix/Linux flavors, ...)
* Pro-active validation of new Chef versions (e.g. upgrade to Chef 11)
* Be more confident/agile in integrating external community cookbooks (and thus reduce maintenance cost? pros/cons to be debated...)
* Potential for "heavy" continuous integration (at the moment, we only check some foodcritic rules, no more).

### TODO list

* Which vagrant base boxes should be used? (from travis, from opscode, from canonical)
* Automatic way to synchronize worker setup definitions (from external repositories like travis-boxes, travis-images) into `.kitchen.yml`
* Experiment bluebox integration tests (+ continuous integration via http://travis-ci.org)
* Experiment macosx (and other OS) provisioning

**Attention:** This pull request is still in "open discussion" and "work in progress" status. Even if "merge" step is not expected yet, I preferred to open the issue as a pull request to better illustrate the proposal by appending the prototype code. I hope it's okay for everybody so...