Improve Rspec Pipeline Performance
Tags: testing, rails, rspec, tdd
Published: 2019-08-22 09:57:52

At [Mekari](mekari.com), we highly encourage that developers should write tests on every projects. Testing prevents bugs in production, forces better code, and encourages developer confidence when making changes to existing code bases. We use rspec, selenium with capybara as our standard unit tests and integration tests tools.

### Rspec best practice
If you're new to ruby on rails and never used rspec before, here are some links which will give little guidance to start.
- [RSpec (Rails) Best Practices](https://medium.com/@somethvictory/rspec-rails-best-practices-c457679e4c04)
- [RSpec best practices by thoughtbot](https://thoughtbot.com/upcase/videos/rspec-best-practices)
- [Our best practice](https://dev.to/bambangsinaga/rspec-best-practice-26k5)


### Working with Large Code Bases
In a large code base, testing becomes even more important. While some argue that testing slows development down, we argue that it increases development velocity over time, especially when refactoring existing code and eliminating technical debt.

[Test Driven Development(TDD)](https://en.wikipedia.org/wiki/Test-driven_development) encourages running your tests frequently and often. What you might find in a large code base when running the full test suite are it may takes 25+ minutes to run 2500+ tests. That will make you unproductive when you push changes and wait test suite to finish.

In this post, I will share how we accelerate our pipeline test. For my project, we use dedicated [CI (Continuous Integration)](https://en.wikipedia.org/wiki/Continuous_integration) server bitbucket for running the full test suite. Every minutes, developers make changes on a branch, commit, and push changes to bitbucket. Then, create PR (Pull Request) regarding ticket and bitbucket will pull down these changes and run the entire test suite.

#### Increase Hardware capability
When we want to achieve performance without doing much, our first option come to improve hardware specifications. You can easily see double the non parallel performance on your test suite by upgrading your CI server from an old or low spec machine into faster one.

#### Parallel Tests
Running specs across multiple CPU cores can further decrease the test time for your suite. We use [parallel_tests](https://github.com/grosser/parallel_tests) gem for this. It uses a database per thread and can run on CI servers like bitbucket. Running test suite in parallel also make hardware upgrades even more effective, scaling out to machines with more cores. Single core performance is no longer your bottleneck.

When parallelizing your suite, beware of shared state between tests that lives outside of the database such as [Elasticsearch](https://www.elastic.co/). These interdependencies need to be addressed too in parallel.

This solution solve your issue about slow running your test suite, but they come at a price.

#### Profiling The Specs
When you have implemented the above solutions, but you still feel slow with your test suite. Then, what you should try to consider is your database strategy setup and test suite.

##### Database Cleaner configuration
If you write ruby or rails code that interacts with a database and write unit test, chances are you have heard of or used the [database_cleaner](https://github.com/bmabey/database_cleaner) gem. It's a terrific gem that abstracts away the various ORM APIs for getting the DB into a "blank slate" state. Some setup needed to use it in optimal mode. Here are some article that give you idea to setup your own:
  * [Configuring database_cleaner with Rails, Rspec, Capybara, and Selenium](http://www.virtuouscode.com/2012/08/31/configuring-database_cleaner-with-rails-rspec-capybara-and-selenium/)
  * [Testing with Rspec, FactoryGirl, Faker and Database cleaner](https://medium.com/brief-stops/testing-with-rspec-factorygirl-faker-and-database-cleaner-651c71ca0688)

##### Factory cascade
Let's say you have `User` model and each user has one `Contact` and one `Location` record. Your factory should be looking similar like this:
```ruby
factory :user do
  contact
  location
end
```
Now, try to guess how many records are created in the database once you call `create(:user)`?

Each time you call `create(:user)`, then 2 additional records are created. __Yes, even with build__. Now imagine that you are using this base factory across all tests. There is 2 ways to eliminate this issue.
* **Explicit associations**
The first thing that comes to mind is to remove all (or almost all) associations from our factories:
```ruby
factory :user do
  # do not declare associations
  # contact
  # location
end
```
With this approach, you have to explicitly specify all required associations when using a factory
```ruby
create(:user, contact: contact, location: location)

# But!!
create(:user) # => raises ActiveRecord::InvalidRecord
```
* **Traits**
Your base factory should always have only these attributes that are needed to pass validation. If you sometimes need associations then consider using [traits](https://thoughtbot.com/blog/remove-duplication-with-factorygirls-traits)
```ruby
factory :user do
  name { "Zuned #{Faker::Name.unique.name}" }
  username { "#{ name.parameterize.underscore}_#{ SecureRandom.hex(3) }" }
  email { "#{ username + Random.rand(1000).to_s }@example.com"}

  trait :with_location do
    location
  end
end
```
and you can call your factory using
```ruby
create :user, :with_location
```
##### let_it_be and Anyfixture
Our specs always have relations or required company and 2 or more employees (with association to user, job title, location, and job level). Imagine if we call `create :company` and so on for every specs. `1 spec` need `1 company + 2 employee + 2 locations + 2 job_title + 2 job_level`, it means 9 records only for initial data. Can we generate those 9 records only once for every specs?

Yes, here comes anyfixture. Built in `fixture` are different with `anyfixture`. Both represent different approach. `Fixture` declare a static state of the data that is loaded into a test database right away and usually persists between test runs. But with `anyfixture`, we able to hard reload record to get fresh copy of record via refind method.
Here are our implementation with (Anyfixture)[https://test-prof.evilmartians.io/#/any_fixture].
```ruby
require 'test_prof/any_fixture/dsl'
require 'test_prof/ext/active_record_refind'

using TestProf::AnyFixture::DSL
using TestProf::Ext::ActiveRecordRefind

# Fixtures are kept to be outside db transaction such that it can be leveraged
# for specs. https://github.com/palkan/test-prof/blob/master/docs/any_fixture.md
RSpec.shared_context 'company with staffs' do
  before(:all) do
    # register to fixture
    fixture(:company) { FactoryBot.create(:company) }
    fixture(:location) { FactoryBot.create(:location, company: fixture(:company)) }
    fixture(:job_title_ceo) {
      FactoryBot.create(:job_title, :ceo, company: fixture(:company))
    }

    fixture(:job_title_hr) {
      FactoryBot.create(
        :job_title,
        title: 'HR Manager', parent: fixture(:job_title_ceo), company: fixture(:company)
      )
    }

    fixture(:job_title_staff) {
      FactoryBot.create(
        :job_title,
        title: 'Staff', parent: fixture(:job_title_ceo), company: fixture(:company)
      )
    }

    fixture(:org_general) {
      FactoryBot.create(
        :organisation_node,
        name: 'General', company_id: fixture(:company).id
      )
    }

    fixture(:org_hr) {
      FactoryBot.create(
        :organisation_node,
        name: 'HR', company_id: fixture(:company).id
      )
    }

    fixture(:job_level_ceo) {
      FactoryBot.create(:job_level, :CEO, company: fixture(:company))
    }

    fixture(:job_level_hr) {
      FactoryBot.create(:job_level, name: 'HR', company: fixture(:company))
    }

    fixture(:job_level_staff) {
      FactoryBot.create(:job_level, name: 'Staff', company: fixture(:company))
    }

    fixture(:owner) {
      FactoryBot.create(
        :employee, :owner,
        company: fixture(:company),
        location: fixture(:location),
        job_level: fixture(:job_level_ceo),
        organisation_node: fixture(:org_general),
        job_title: fixture(:job_title_ceo)
      )
    }

    fixture(:hr_admin) {
      FactoryBot.create(
        :employee, :hr_admin,
        company: fixture(:company),
        location: fixture(:location),
        job_level: fixture(:job_level_hr),
        organisation_node: fixture(:org_hr),
        job_title: fixture(:job_title_hr)
      )
    }

    fixture(:staff) {
      FactoryBot.create(
        :employee, :staff,
        company: fixture(:company),
        location: fixture(:location),
        job_level: fixture(:job_level_staff),
        organisation_node: fixture(:org_general),
        job_title: fixture(:job_title_staff)
      )
    }
  end

  # Ensure that every example has a fresh copy of a record
  %i[
    company location job_title_ceo job_title_hr org_general org_hr
    job_level_ceo job_level_hr owner hr_admin staff
  ].each do |object|
    let(object) { fixture(object).refind }
  end
end
```
Imagine this example too.
```ruby
describe User do
  let!(:user) { FactoryGirl.create :user }

  it 'does something' do
    # test with user
  end

  it 'does something' do
    # test with user
  end

  it 'does something' do
    # test with user
  end
end
```
How many User records are created in the database? One record for each example so the answer is 3 records. To avoid it you can use (let_it_be)[https://test-prof.evilmartians.io/#/let_it_be]. This helper uses Rails transactional tests feature which means that the record is created only once at the beginning of the test and removed when the test is finished.
```ruby
describe User do
  let_it_be(:user) { FactoryGirl.create :user }

  it 'does something' do
    # test with user
  end

  it 'does something' do
    # test with user
  end

  it 'does something' do
    # test with user
  end
end
```
### Additional suggestion and resource
There are some condition where we doesn't need data to persists first into db like checking custom validation on model and unit test on view or mailer specs. Then if this happer, we can utilize `build` and `build_stubbed` create strategy.
```ruby
let(:company) { FactoryBot.build_stubbed(:company) }
```
some usefull links:
* (Test Profiling)[https://test-prof.evilmartians.io]
* (Factory therapy for your ruby tests)[https://evilmartians.com/chronicles/testprof-2-factory-therapy-for-your-ruby-tests-rspec-minitest#bonus-anyfixture]
* (build and build_stubbed)[https://thoughtbot.com/blog/use-factory-girls-build-stubbed-for-a-faster-test]