Adding Code Quality Checks To Your Ruby Test Suite
Tags: ruby, rails, testing
Published: 2018-08-15 06:19:55

---
title: Adding Code Quality Checks To Your Ruby Test Suite
published: true
tags: Ruby, Rails, Testing
canonical_url: https://www.briankephart.com/adding-code-quality-checks-to-your-ruby-test-suite
---

Over time I've integrated a few static analysis tools into my workflow. These have been extremely helpful in maintaining high quality code, but running them manually is tedious. For many projects, this would be automated in some kind of CI pipeline. For my current project, however, there is no automated CI pipeline. This app is deployed to Heroku straight from the command line. So, I decided to see if I could integrate these tools into my test suite, reducing manual quality checks to a single step.

## Tool #1: Brakeman

Since I started learning Rails development, I have been using [Brakeman](https://brakemanscanner.org) to scan my code for security vulnerabilities. This tool uses static analysis to check for vulnerabilities to attacks like SQL injection and cross-site scripting. Brakeman was designed to be run from the command line, but a little digging into the docs revealed that [it can be run as a Ruby library](https://brakemanscanner.org/docs/brakeman_as_a_library/) within an app. Following these instructions, I opened up a Pry console and ran:

```ruby
require 'brakeman'
=> true
Brakeman.run Rails.root.to_s
=> #<Brakeman::Tracker:0x00007fde2d546da8 ... tons of output in this object
```

That object had way too much output to read. Next, I ran the same command, but stored the result as a variable...

```ruby
result = Brakeman.run Rails.root.to_s
=> # same result as before
```

... so I could query its methods and attributes:

```ruby
result.methods.sort
=> [:warnings] # There were tons of other methods in this array
result.warnings
=> []

result.instance_variables.sort
=> [:@errors] # Lots more here too
result.errors
=> [{:error=>"/Users/briankephart/Sites/some_app/app/jobs/some_job.rb:14 :: parse error on value \"**\" (error)",
   :backtrace=>["Could not parse /Users/briankephart/Sites/some_app/app/jobs/some_job.rb"]}]
```

As you can see, I had no security warnings, but one error due to Brakeman failing to parse a `**` operator. This was helpful in showing me that the errors are readable in this form. Note that both the errors and warnings are returned as Arrays.

Using what I learned in my Pry console, I wrote the following test:

```ruby
class BrakemanTest < ActiveSupport::TestCase
  require 'brakeman'

  test 'no brakeman errors or warnings' do
    result = Brakeman.run Rails.root.to_s
    assert_equal [], result.errors
    assert_equal [], result.warnings
  end
end
```

This worked perfectly. I could see the Brakeman output during my test run, and the test responded correctly to the error seen above.

## Tool #2: RuboCop

I've recently been using [RuboCop](http://batsov.com/rubocop/) in a lot more of my work to check for syntax errors, good styling, and general readability. Like Brakeman, Rubocop is designed as a command line tool. Unlike Brakeman, the docs were not helpful to me in using it any other way. Instead, I had to dive into the source code. The most important line I found was [here](https://github.com/rubocop-hq/rubocop/blob/9b7e480d99723b7ab17039bb3e280637cc3a7fb6/exe/rubocop#L13), in the file that defines the command line executable. The important points:

```ruby
require 'rubocop'
cli = RuboCop::CLI.new
result = cli.run
```

What I needed to do was instantiate a `RuboCop::CLI` object and call its `#run` method. Looking at those files, I found that [the `#initialize` method takes no arguments](https://github.com/rubocop-hq/rubocop/blob/9b7e480d99723b7ab17039bb3e280637cc3a7fb6/lib/rubocop/cli.rb#L21), while [the `#run` method takes all arguments from the command line](https://github.com/rubocop-hq/rubocop/blob/9b7e480d99723b7ab17039bb3e280637cc3a7fb6/lib/rubocop/cli.rb#L36) as an array (I usually run `rubocop` on the command line without arguments, though). Back to the Pry console:

```ruby
require 'rubocop'
=> true

RuboCop::CLI.new.run
# Inspecting 397 files
# ...
#
# 397 files inspected, no offenses detected
=> 0
```

Notice that the return value is `0`. Knowing this, I wrote the following test:

```ruby
# Do not use this code
class RubocopTest < ActiveSupport::TestCase
  require 'rubocop'

  test 'no rubocop offenses' do
    assert_equal 0, RuboCop::CLI.new.run
  end
end
```

I noticed something strange, though. When I ran my full test suite, RuboCop checked all my files, but when I ran the test above on its own, RuboCop only checked that test file. The context in which I called the test became the context for RuboCop. The fix for this was to add a path argument to the run method, just as RuboCop allows a path argument on the command line.

```ruby
# Use this instead of the code above
class RubocopTest < ActiveSupport::TestCase
  require 'rubocop'

  test 'no rubocop offenses' do
    cop = RuboCop::CLI.new
    args = [Rails.root.to_s]
    assert_equal 0, cop.run(args)
  end
end
```

##Tool #3: Bundler Audit

Bundler Audit checks your Gemfile.lock for known-vulnerable versions of gems. This tool is pretty wedded to the command line, to the point where it proved easiest to just run system commands.

```ruby
class RubocopTest < ActiveSupport::TestCase
  require 'bundler/audit/cli'

  test 'no vulns in bundle' do
    `bundle audit update -q` # Update vulnerability database
    result = `bundle audit`  # Run the audit
    code = `echo $?`.squish  # Returns '0' if successful, otherwise '1'

    # Print the scan result as the error message if it fails.
    assert_equal '0', code, result

    # If successful, output the success message
    puts "\nMessage from bundler-audit: #{result}"
  end
end
```

That's all! With these three small tests, I can check my code style and security whenever I run my test suite, with no extra steps.