Profiling and Speeding Up Our Rails Test Suite with TestProf
Tags: ruby, rails, testing
Published: 2020-03-22 13:49:13

_Originally published on [https://ali-ilman.com/blog/profiling-and-speeding-up-our-rails-test-suite-with-test-prof](https://ali-ilman.com/blog/profiling-and-speeding-up-our-rails-test-suite-with-test-prof)_.

RSpec best practices can only go so far in speeding up our test suite. Are you looking to speed up your Rails test suite? You've come to the right place! In this article, we'll take a look at [TestProf](https://github.com/palkan/test-prof), a Ruby gem that provides profilers for analysing our test suite, and helpers to improve the test suite's performance.

## How did I end up using TestProf?
A month ago, I gave a go at speeding up the test suite of a project I'm working on. For a bit of background, the project's codebase is almost 6 years old, developers have come and go, new technologies and practices have appeared after all these years. We had a sprint where we can make our lives easier, in other words, enhance the codebase! And we decided speeding up the test suite was one of them. 🕵️‍♂️

The test suite took around 45 minutes (!) to run on my Mac. That's horrendous! 😭 Initially, I applied practices (which you can read about [here](https://dev.to/thealiilman/3-practices-to-apply-to-speed-up-our-rails-test-suite-12p0)) that can speed up the test suite, but the improvement was little. It was clear that something else is making the test suite take longer than usual. And that's when I decided to add TestProf to the project.

## What are the common causes of a lengthy test suite?
There are a few common causes, and the most common cause is arguably the **_factory usage_**. And what is a factory?

>A factory generates test data that are used in tests. These data can be an Active Record object, a plain Ruby object and so on.

Factory usage can negatively affect the performance of the test suite because of these two reasons.

### 1. Creation of records for each test
_Wait a minute. Don't we sometimes need to seed our database before running our tests?_  
Yup, that's correct. But wouldn't it be great if we can create one single record for the entire test file? Let's take a look at the example below. 🤓
```ruby
# factories/books.rb
FactoryBot.define do
  factory :book do
    title { Faker::Book.title }
  end
end

# requests/books_controller_spec.rb
RSpec.describe BooksController, type: :request do
  let(:book) { create(:book) }

  describe 'GET #show' do
    it do
      get book_path(book)
      expect(response).to have_http_status :ok
    end
  end

  describe 'DELETE #destroy' do
    it do
      delete book_path(book)
      expect(response).to have_http_status :ok
    end
  end
end
```
As we can see in the code above, the `book` variable is invoked for each test (`it`), which means we're telling the `book` factory to create a `book` for **each test**! This is unnecessary for the tests above.  
Can't we initialize a `@book` variable inside of a `before(:all)` to create a record just once? 🤔 We can, but `before(:all)` doesn't wrap the tests within a transaction, so nothing is rolled back after they have been ran.
We'll see later on in the article how a TestProf helper allows us to create a record only once **and** without going through the headache of cleaning our test database.
### 2. Nested factory invocations
Nested factory invocations can cause _factory cascades_.  
> A factory cascade is a term given to the process of creating new associated objects inside of each factory through nested factory invocations. 🧠

Here's an example of factory cascades, copied mostly from [test-prof's documentation](https://test-prof.evilmartians.io/#/factory_default).
```ruby
factory :account do
end

factory :user do
  account
end

factory :project do
  account
  user
end

factory :task do
  account
  project
  user
end

create(:task)
```
What is happening when we tell FactoryBot to create a task?
1. The task's `account` is created
2. The task's `project` is created, followed by the project's `account` and `user`, followed by the user's `account`
3. The task's `user` is created, followed by the user's `account`.

In overall, four accounts are created even though all we need is one account.

Here's an example of simple nested factory invocations.
```ruby
# factories/publishers.rb
FactoryBot.define do
  factory :publisher do
    name { Faker::Company.name }
  end
end

# factories/authors.rb
FactoryBot.define do
  factory :author do
    name { Faker::Book.author }
    publisher
  end
end

# factories/books.rb
FactoryBot.define do
  factory :book do
    title { Faker::Book.title }
    author
  end
end
```
If we were to use the `book` factory to create a book, it'll invoke the `author` factory, which in turn will invoke the `publisher` factory! Do we really need a publisher and an author for all tests? Not really, because some tests can be written with record(s) from only one single model, which in our case is the `Book` model.

One of the reasons why a factory is used is to help generate data for associations, but at the same time, this can come at the cost of unnecessary factory usage.

## How can TestProf help us improve our test suite?
TestProf comes with various of profilers and helpers, but in this article, I'm only going to share the ones that can make a significant improvement to the test suite and the ones that I've utilised. Let's start with the profilers!

### 1. FactoryDoctor
_FactoryDoctor_ will go through our test suite for any tests that perform unnecessary database queries.
Here's an example.  

```
FDOC=1 rspec spec/services/books/update_availability_spec.rb


Total (potentially) bad examples: 1
Total wasted time: 00:00.018

Book (./spec/models/book_spec.rb:3) (2 records created, 00:00.018)
  is expected to validate that :title cannot be empty/falsy (./spec/models/book_spec.rb:10) – 2 records created, 00:00.018

Finished in 0.12646 seconds (files took 1.18 seconds to load)
2 examples, 0 failures
```

We can also make FactoryDoctor ignore a test, by adding `:fd_ignore` to the test.
```ruby
RSpec.describe Book, type: :model do
  describe '.published' do
    subject { described_class.published }

    it(:fd_ignore) { should eq([]) }
  end
end
```
There's one thing to be wary of when using FactoryDoctor, though. There's a chance that it'll report false positives and false negatives as it's still learning, so don't always trust it. Dig deeper into certain tests if you're looking to eliminate unnecessary database queries.

### 2. FactoryProf
_FactoryProf_ will analyse our factory usage, and after the tests have finished running, it'll display the following report.
```
FPROF=1 rspec spec/services/books/update_availability_spec.rb

Finished in 21.08 seconds (files took 1.15 seconds to load)
3 examples, 0 failures

[TEST PROF INFO] Factories usage

 Total: 6
 Total top-level: 6
 Total time: 0.0515s
 Total uniq factories: 2

   total   top-level     total time      time per call      top-level time               name

       3           3        0.0304s            0.0101s             0.0304s             author
       3           3        0.0211s            0.0070s             0.0211s               book
```
What does the report show?
1. The number of times factories were invoked
2. The number of times factories were invoked (excluding nested factory invocations)
3. The time spent generating records
4. The number of factories, in our case is 2 for `author` and `book`.

You will love FactoryProf. I was utterly astounded to see the report for the project I work on, I found out that we spent 36 minutes just for generating records! 🤯

Let's move on to the helpers.

### 1. FactoryDefault
_FactoryDefault_ helps with eliminating factory cascades by reusing associated records.
Going back to the factory cascades example, we can use FactoryDefault's `create_default` to reuse associated records.
```ruby
factory :account do
end

factory :user do
  account
end

factory :project do
  account
  user
end

factory :task do
  account
  project
  user
end

create_default(:account)
create(:task)
```
What is happening here?
1. An `account` is created and the same object is reused for `user`, `project` and `task`
2. The task's `account` will refer to the one created with `create_default`
3. The task's `project` is created, followed by the reusing of the `account`, and the creation of `user`, followed again by the reusing of the `account`
4. The task's `user` is created, followed by the reusing of the `account`

In overall, only **one** account is created.

Do be careful with using FactoryDefault, as it can reduce the readability of your tests. 👁

### 2. before_all
`before_all` is similar to RSpec’s `before(:all)`. The only difference is that when we use `before_all`, it wraps the tests within a transaction, thus destroying the records afterwards. For RSpec’s `before(:all)`, we have to destroy the records ourselves through `after(:all)`.
#### Before
```ruby
RSpec.describe Book, type: :model do
  before(:all) { create_list(:book, 5) }

  # test
  # test
  # test

  after(:all) do
    Book.destroy_all
    Author.destroy_all
    Publisher.destroy_all
  end
end
```
#### After
```ruby
RSpec.describe Book, type: :model do
  before_all { create_list(:book, 5) }

  # test
  # test
  # test
end
```
This keeps our tests tidy. 🤓

### 3. let_it_be
`let_it_be` works similarly to `before_all`, with the addition of allowing us to use variables.
#### Before
```ruby
# requests/books_controller_spec.rb
RSpec.describe BooksController, type: :request do
  let(:book) { create(:book) }

  describe 'GET #show' do
    it do
      get book_path(book)
      expect(response).to have_http_status :ok
    end
  end

  describe 'DELETE #destroy' do
    it do
      delete book_path(book)
      expect(response).to have_http_status :ok
    end
  end
end
```
`book` is invoked twice, thus creating two book records.
#### After
```ruby
# requests/books_controller_spec.rb
RSpec.describe BooksController, type: :request do
  let_it_be(:book) { create(:book) }

  describe 'GET #show' do
    it do
      get book_path(book)
      expect(response).to have_http_status :ok
    end
  end

  describe 'DELETE #destroy' do
    it do
      delete book_path(book)
      expect(response).to have_http_status :ok
    end
  end
end
```
`book` is invoked twice but only **one** book record is created.

There is one **caveat** with using `let_it_be`.
If we were to a modify record generated within a `let_it_be` outside of `let_it_be`, we need to _reload_ or _re-find_ the record. The cost of reloading or querying for a record is still less than generating one.
#### Before
```ruby
RSpec.describe Book, type: :model do
  let_it_be(:book) { create(:book) }

  describe '#update' do
    context 'when title is valid' do
      let(:new_title) { 'And Then There Were None' }
      before do
        book.update(title: new_title)
      end

      it do
        expect(book.title).to eq(new_title) # => false
      end
    end
  end
end
```
#### After
```ruby
RSpec.describe Book, type: :model do
  # let_it_be(:book, refind: true) { create(:book) }
  let_it_be(:book, reload: true) { create(:book) }

  describe '#update' do
    context 'when title is valid' do
      let(:new_title) { 'And Then There Were None' }
      before { book.update(title: new_title) }

      it do
        expect(book.title).to eq(new_title) # => true
      end
    end
  end
end
```

### Conclusion
Using TestProf can help speed up your test suite massively. Benchmark your test suite by utilising the profilers that come with the gem, though you might find FactoryProf and FactoryDoctor become your go-to profilers. Then, identify the slowest tests, and try to determine the tests that can gain significant improvement in speed when helpers provided by TestProf are used. Performant tests are a nice-to-have, but don't forget, keep the tests readable.

Cheers for reading and I hope you find this article helpful! 🤓