Running Rails 5 System Tests on Travis CI with Chromedriver
Tags: rails, selenium, chromedriver, testing
Published: 2018-07-29 22:35:59

---
title: Running Rails 5 System Tests on Travis CI with Chromedriver
published: true
description: How to setup Rails 5 system tests to run on Travis CI with Google Chrome and Chromedriver
tags: rails, selenium, chromedriver, testing
---

Rails 5.1 introduced the concept of [system tests](https://weblog.rubyonrails.org/2017/4/27/Rails-5-1-final/#system-tests) using Capybara and Selenium. System tests takes the practice of "feature" tests that many Rails apps were already doing and brings it into an official part of the Rails testing strategy. Since system tests utilize a browser, they can execute Javascript, something that Rails integration tests could not do.

System tests work great out of the box on local machines, but getting them set up to run on a remote server like Travis CI is a little harder. There's not a lot of good documentation out there for what needs to be done, and a lot of solutions have to be dug up through Stack Overflow and GitHub comment threads.

In this post, I'll show how I set up system tests to run on Travis CI using headless Chrome and Chromedriver.

## Rails 5.1.6

Although system tests were announced with 5.1, you really should make sure you are using Rails 5.1.6 to get the best experience. This is because after the initial release there were a few other small quality of life improvements made to the framework that are great to have.

Among these are a change to ActiveRecord that allows the Capybara thread and the test server thread to use the same database connection (see [rails/rails#28083](https://github.com/rails/rails/pull/28083)). This is important when using transactions because otherwise the two threads would not be able to see DB changes made by the other thread.

In the past, you would have had to work around this by turning off transactions and adding Database Cleaner or some similar gem to clean up data created during test runs. But this would have meant that either Rails would need to pull Database Cleaner in as a dependency, or anyone wanting to use system tests would need to add another gem to their Gemfile manually. Neither of those were good for the future adoption of system tests, so instead the Rails team updated ActiveRecord to allow the connection to be shared.

## RSpec 3.7

I prefer using RSpec as my test framework. RSpec 3.7 added [support for system tests](http://rspec.info/blog/2017/10/rspec-3-7-has-been-released/#rails-actiondispatchsystemtest-integration-system-specs) into the framework, replacing the previous feature specs.

If you are using RSpec, you will need to make sure you are using version 3.7 in order to use system tests.

## Installing Chromedriver

[Chromedriver](http://chromedriver.chromium.org/) allows Chrome to be controlled by Selenium and is necessary in order for us to use Chrome as the browser in system tests.

You can download Chromedriver from the project's website, or it can be installed with Homebrew Cask:

```
$ brew cask install chromedriver
```

### Removing chromedriver-helper

If you have worked with Chromedriver in the past, you may have installed the Ruby gem [chromedriver-helper](https://rubygems.org/gems/chromedriver-helper). If you want to use Chromedriver directly, you will need to remove chromedriver-helper and it's executables from your system.

If you're using rbenv, you can use the commands I used:

```
$ gem uninstall chromedriver-helper
$ ls ~/.rbenv/versions/*/bin/* | grep chromedriver | xargs rm
$ rbenv rehash
```

> _**Note**: Removing chromedriver-helper may or may not be necessary for you, depending on your setup. I found it was causing problems for me, so I decided to remove it._

## Selecting a driver

System tests abstract Capybara's drivers using the `driven_by` method. With regular system tests, you would include `driven_by` in your `ApplicationSystemTestCase`. RSpec, however, does not use `ApplicationSystemTestCase`, so you need to configure the driver in a `before` block.

The easiest way to do this is by adding a global `before(:each)` to `rails_helper.rb`:

```ruby
RSpec.configure do |config|
  config.before(:each, type: :system) do
    driven_by :selenium, using: :chrome
  end
end
```

This will set all system tests to be driven by Selenium using Chrome.

`type: :system` indicates that this `before` block will only apply to tests marked as system tests. RSpec will infer that a test is a system test if it is located in `spec/system`, or you can tag specs manually.

Now running `bundle exec rspec` should open a Chrome window and run your system specs. Cool!

## Going headless

Opening a browser window works fine on your local machine, but it won't work on a remote server because there's no monitor to render the window. Instead, we have to use Chrome in headless mode.

To use headless Chrome, we can pass some extra arguments to `driven_by`:

```diff
 RSpec.configure do |config|
   config.before(:each, type: :system) do
-    driven_by :selenium, using: :chrome
+    driven_by :selenium, using: :chrome, options: { args: ["headless"] }
   end
 end
```

There are other ways to use headless mode, and the downside of this method is that it causes Selenium to throw a deprecation warning. For reasons we'll see later, we have to use this method of passing arguments.

Now running `bundle exec rspec` should **not** open a browser window. Neat!

## Running on Travis

Getting the tests to run on Travis is the trickiest part. For my build, I wanted to use Travis's Docker infrastructure with `sudo: false` in my `.travis.yml` file. This meant that I had to work around several limitations.

### Installing Chrome and Chromedriver

To install packages on Travis, you need to use the `addons` section in your config file. Although Travis has an [addon specifically for Chrome](https://docs.travis-ci.com/user/chrome), I ended up having some problems using this. Instead, I used the `apt` addon to install both Chrome and Chromedriver:

```yml
addons:
  apt:
    - google-chrome-stable
    - chromium-chromedriver
```

Selenium expects to find Chromedriver on the PATH, but the default installation on Travis does not put the binary on the PATH. To fix this, I created a symlink in the `before_script` step:

```yml
before_script:
  - ln -s /usr/lib/chromium-browser/chromedriver ~/bin/chromedriver
```

Now the Travis environment is set up correctly.

### Chrome crashes on launch

Even with the environment set up correctly, my tests were still failing because Chrome would immediately crash on launch. After a lot of digging, I found [this troubleshooting page](https://developers.google.com/web/tools/puppeteer/troubleshooting#running_puppeteer_on_travis_ci).

There are two takeaways:

1. We need to [run Chrome with `--no-sandbox`](https://developers.google.com/web/tools/puppeteer/troubleshooting#running_puppeteer_on_travis_ci) in order for it to work properly on Travis.
2. We need to [disable usage of `/dev/shm`](https://developers.google.com/web/tools/puppeteer/troubleshooting#tips) because Docker provisions too little space in that device and Chrome crashes if there's not enough space.

In order to do this, we need to be able to pass custom arguments to Selenium, which is why we are using the `driven_by` options that we have.

```diff
 RSpec.configure do |config|
   config.before(:each, type: :system) do
-    driven_by :selenium, using: :chrome
+    driven_by :selenium, using: :chrome, options: { args: ["headless", "disable-gpu", "no-sandbox", "disable-dev-shm-usage"] }
   end
 end
```

The `disable-gpu` argument is there on recommendation from [this page](https://developers.google.com/web/updates/2017/04/headless-chrome#cli).

Now your system tests should successfully run on Travis! Awesome!

## Extra credit: Filter system tests with metadata

As we have it set up now, RSpec will run all system tests whenever we run the full test suite. But system tests are slow and we probably don't want to run them all the time. Luckily RSpec has a way to filter out certain specs from the run based on the metadata on the spec.

Let's tag all specs in `spec/system` with the `browser` tag. To do this, we can use `define_derived_metadata`:

```ruby
RSpec.configure do |config|
  config.define_derived_metadata(file_path: /spec\/system/) do |metadata|
    metadata[:browser] = true
  end
end
```

Then we can use `filter_run_excluding` to filter `browser` specs by default:

```ruby
RSpec.configure do |config|
  config.filter_run_excluding browser: true
end
```

Now when running `bundle exec rspec`, system tests will be filtered out. To run the system tests, we can use the `-t` option to **include** `browser` specs.

```
$ bundle exec rspec -t browser
```

Remember to change your `.travis.yml` to use `-t` to run the system tests. Having system tests split off from the rest of the test suite allowed me to parallelize running the slow system tests with running the rest of the suite.

## Conclusion

System tests are a really easy way to get started with feature testing. The default configuration shipped with Rails should work for most people, and extending it to use a different browser or pass different options is straightforward. What was harder was trying to figure out how to use system tests on Travis, with not a lot of documentation or examples to go off of.

With this post, I have gathered together everything I learned in getting system tests running on Travis CI. I hope it's useful!