travis-ci/travis-cookbooks — pull request #288 by maxlinc (2014-03-28 03:12:04+00:00)
Title: Dev tools (prep for multi-platform work)

_Note: This PR doesn't make any changes that would effect production. It's just some ideas for dev/test tools to hookup. I'm looking for feedback - no need to merge right away unless others want to start using these tools._

This pull request adds some tools for testing cookbooks in isolation (via strainer) or as part of a larger run_list (via test-kitchen). I was trying this out in preparation for [travis-ci#2110 -
Multi-platform cookbooks](https://github.com/travis-ci/travis-ci/issues/2110).

Adds:
- [x] [strainer](https://github.com/customink/strainer) so you can test individual cookbooks with a variety of tools (knife test, foodcritic, rubocop, more later...) by running `bundle exec strainer test ` or `bundle exec rake strainer[]`
- [x] [test-kitchen](https://github.com/test-kitchen/test-kitchen) to test run lists that correspond to [travis-images templates](https://github.com/travis-ci/travis-images/tree/master/templates).
- [x] [serverspec](http://serverspec.org/) for testing the resulting nodes created by test-kitchen.

Sample usage:
- `bundle exec rake strainer[travis_worker]` (I realize the linting tools are too strict right now)
- `bundle exec kitchen verify worker-python`

Notes:
- I've only added the test and run list for the python worker. The rest will be similar so I wanted to open the PR for feedback.
- I used chef roles for the run list. There are arguments [for](http://www.getchef.com/blog/2013/11/19/chef-roles-arent-evil/) and [against](http://dougireton.com/blog/2013/02/16/chef-cookbook-anti-patterns/) roles, so I don't know if they should replace the [travis-images templates](https://github.com/travis-ci/travis-images/tree/master/templates), but I thought it was better to put the run lists and attributes in roles, which have the potential to be used by travis-images, then to hardcode them in .kitchen.yml.
- Strainer and test-kitchen expect a normal [chef-repo](http://docs.opscode.com/essentials_repository.html) structure. This project is different, mostly because the cookbooks are spread across `ci_environment` and `worker_host` rather than a single `cookbooks` directory. There is a rake task, `bundle exec rake fool_strainer`, which merges the cookbooks together into one `cookbooks` directory so strainer and test-kitchen can be used.
- I choose serverspec because it has [windows support](https://github.com/serverspec/serverspec/blob/master/WINDOWS_SUPPORT.md), though it is currently hooked into test-kitchen rather than running on its own. Test-kitchen does not have windows support yet (https://github.com/test-kitchen/kitchen-vagrant/issues/77).

Todo:
- [ ] Other roles/run_lists
- [ ] The python worker run list doesn't explicitly depend on whatever installs travis and git. I did it manually, but should track down which cookbook does that and add to the role.
- [ ] Adjust foodcritic/rubocop rules to match the project's style.
- [ ] Better serverspec tests - test/integration/worker_python/serverspec/python_worker/python_spec.rb is just a quick sample.