rackspace/rumm — pull request #67 by maxlinc (2013-09-18 16:33:20+00:00)
Title: Example "journey" tests, VCR filters, and other ideas

This is more of a "discussion" PR than a fix. I wanted to show a couple of ideas and I thought looking at actual code would help. I'm not sure there's anything that's going to be ready to merge, without discussing, deciding, and possibly splitting into separate PRs. Sorry for including so much in a single PR, but it seemed like the most effective way to share some ideas.

Background: In the process of exploring alternatives to testing with VCR (which we'd like to continue exploring via "branch-by-abstraction"), we noticed a few issues with the current VCR-based tests:
- Some tests require test preparation that is not documented or automated by the test suite (e.g. creating a load-balancer named laughing-barrel.
- A pending test is skipped, but may cause the following tests to fail (e.g. show server fails because create server is pending).
- We are a bit confused by mixing Aruba with RSpec. I think it looks good, except I don't know how to use some Aruba features, like "@announce" to get debug info.
- Currently, recording server creation records the polling of the server in a "building" state, so playback takes just as long as recording. Ideally, these should be filtered out.

Our main proposal is to use the concept of [User Journey Tests](http://martinfowler.com/bliki/UserJourneyTest.html) instead of just integration tests for individual features. This would mean grouping features similar features together if there are common use-cases that cover both. Highlighting the users goals instead of Rumm's features could also help drive innovation.

We just created a simple "manage servers" journey to show the basic idea. A more complete example might be trying to do something like a journey to [create a static site with Rackspace Cloud Files](http://docs.rackspace.com/files/api/v1/cf-devguide/content/Create_Static_Website-dle4000.html). That journey involves some things Rumm currently doesn't handle, so the test would help highlight some areas for improvement. Right now the journey would look something like this:

```bash
#!/bin/bash

bundle exec rumm create container --name my-site
for f in `find fixtures/my-site -type f | sed 's/fixtures\/my-site\///g'`; do
bundle exec rumm create file "$f" in container my-site --file "fixtures/my-site/$f"
done
bundle exec rumm show container my-site

echo "Now run a custom script to:"
echo " CDN enable the container"
echo " Set X-Container-Meta-Web-Index to index.html"
echo " Set X-Container-Meta-Web-Error to error.html"
echo " Optionally set a cache TTL or other options"
```

Items to discuss:
* Does journey testing make sense, and if so what journeys should we include?
* Can we keep mixing Aruba/RSpec or will that create confusion that deters contributions?
* We created a FogFilters project to share reusable VCR "filters" (or hooks) that make sense for rumm, knife-rackspace, and any other project that wants to use Fog+VCR.

If this doesn't look good, then we should discuss other ways to make re-recording cassettes easy and deterministic.