alphagov/whitehall — pull request #186 by h-lame (2013-02-05 19:29:56+00:00)
Title: Don't render views in functional tests

Thought I might experiment with this to see if it has any appreciable effect on our test run time.

Results are from my machine running via the VM.

On master:

Finished tests in 902.666978s, 2.8216 tests/s, 6.4852 assertions/s.

2547 tests, 5854 assertions, 0 failures, 0 errors, 0 skips
bundle exec rake test:functionals 551.25s user 218.51s system 82% cpu 15:31.18 total

On this branch:

Finished tests in 874.869522s, 2.9113 tests/s, 6.6913 assertions/s.

2547 tests, 5854 assertions, 0 failures, 0 errors, 0 skips
bundle exec rake test:functionals 539.33s user 212.89s system 83% cpu 15:04.95 total

So... not great. 30 seconds on a 15minute test run. Not ideal at all.

However!

On master, our slowest functional test (admin/publications_controller_test.rb):

Finished tests in 72.644295s, 2.4641 tests/s, 5.3549 assertions/s.

179 tests, 389 assertions, 0 failures, 0 errors, 0 skips
bundle exec ruby -I test test/functional/admin/publications_controller_test.r 51.88s user 18.17s system 77% cpu 1:30.32 total

On this branch:

Finished tests in 65.748172s, 2.7225 tests/s, 5.9165 assertions/s.

179 tests, 389 assertions, 0 failures, 0 errors, 0 skips
bundle exec ruby -I test test/functional/admin/publications_controller_test.r 50.46s user 17.73s system 82% cpu 1:22.40 total

Which is more interesting, 8 seconds on a single test. Might be worth investigating why this controller is so dependent on stuff happening in the view.

Part of doing this was to change all the tests that require the view to be rendered to use ``view_test`` instead of ``test``. Turns out we have a lot of tests that do something simple like ``refute_select('.errors')`` which could be replaced by ``refute assigns().errors.empty?`` - e.g. test the errors on the object, rather than the rendered view.

The one major caveat is that if we do go down this road (and I'm not sure it's worth it yet) we remove any confidence that our views are actually renderable - unless we're sure that we're rendering all the paths in our views via cucumber.