How to optimize factory creation.
Tags: rspec, factorybot, profiling, ruby
Published: 2022-12-21 20:24:45

At **[Potloc](https://www.potloc.com/)** we have a test stack which is pretty standard in the Rails ecosystem. We run tests with **RSpec**, we use **FactoryBot** for setting up our test data, **Capybara** for user interactions, **Github Actions** as a CI etc..

These great tools allow us to code at a fast pace with good test coverage. But this pace comes at a cost. The more the team grows, the bigger the codebase gets and the more tests get written.

### :face_with_head_bandage: The issue

As developer we usually take care of optimizing our own code and queries, we are used to test new implementations with all the edge cases. But tests optimization is likely a topic that we put **at the bottom of the list** and that’s if we ever even think about it.
Up until the moment where quietly but surely you will end up with a CI that take ages to run and a whole test suite to speed up.

Let’s see how we took on this challenge at **Potloc**.

For the purpose of demonstration we will rely on this simple test file:

```ruby
RSpec.describe Purging::QuestionnaireWorker, type: :worker do
  let(:questionnaire) { create(:questionnaire) }

  describe "#perform" do
    it "destroys a survey form" do
      expect { subject.perform(questionnaire.id) }.to change(Questionnaire, :count).by(-1)
    end

    context "given associations" do
      it "destroys a questionnaire and its associations" do
        create(:question, :postal_code, questionnaire: questionnaire)

        expect { subject.perform(questionnaire.id) }.to change(SurveyQuestion, :count).by(-1)
      end
    end
  end
end
```


### :firefighter: The solutions

The [factory-bot](https://github.com/thoughtbot/factory_bot) gem is used in almost in all of our spec files and it make our set up much more easier than when we use fixtures. 
Here is the tradeoff, the easier the gem is to use, the  more likely you’ll end up with some pain to control its usage. And when the times come to tackle slow tests, **the best bet you can take is to start digging into you factories** because it’s likely they are the primary reason why your test suite is slowing down

- **Avoiding the factory cascades**

To quote [Evil Martian](https://evilmartians.com/chronicles/testprof-2-factory-therapy-for-your-ruby-tests-rspec-minitest) a factory cascade is an

> uncontrollable process of generating excess data through nested factory invocations.

In our test we use two factories a `questionnaire` and `question` who could be represented like this as a tree:

```
questionnaire
|
|---- survey

question
|
|---- survey
```

In this simple example each factory calls a nested factory. This means that every time we create a `questionnaire` factory, we also create a `survey` factory.

To have a better vision of what objects are created in our spec file we can use [test-prof](https://test-prof.evilmartians.io/), a powerful gem that provides a collection of different tools to analyse your test suite performance. One of this tool is really useful to **identify a factory cascade**, let’s introduce [factory profiler](https://test-prof.evilmartians.io/#/profilers/factory_prof).

If we run `FPROF=1 bundle exec rspec` the factory profiler, it will generate the following report:

```
[TEST PROF INFO] Factories usage

 Total: 6
 Total top-level: 3
 Total time: 00:01.005 (out of 00:26.087)
 Total uniq factories: 3

   total   top-level     total time      time per call      top-level time               name

       3           0        0.6911s            0.2304s             0.0000s             survey
       2           2        0.6397s            0.3199s             0.6397s             questionnaire
       1           1        0.3658s            0.3658s             0.3658s             question
```

The most interesting insight is the **difference** between the `total` and the `top-level`. The more this difference is important, the more you end up with factory cascade, meaning that **you are creating useless factories**.

Let’s take the `survey` , we don’t instantiate any surveys in our test suite but during the execution we create 4.

The easiest workaround it is to instantiate a `survey` at the top-level and to associate the factories to it.

```ruby
RSpec.describe Purging::QuestionnaireWorker, type: :worker do
  let(:survey) { create(:survey) }
  let(:questionnaire) { create(:questionnaire, survey: survey) }
  ....

  it "destroys a questionnaire and its associations" do
    create(:question, :postal_code, questionnaire: questionnaire, survey: survey)
    ....
```
If we run the factory profiler we now have a different report:

```
[TEST PROF INFO] Factories usage

 Total: 5
 Total top-level: 5
 Total time: 00:00.868 (out of 01:09.067)
 Total uniq factories: 3

   total   top-level     total time      time per call      top-level time               name

       2           2        0.6986s            0.3493s             0.6986s             survey
       2           2        0.0973s            0.0487s             0.0973s             questionnaire
       1           1        0.0729s            0.0729s             0.0729s             question
```

Nice! No more factory cascades. The `total` and the `top-level` columns are the same. **We now create 5 factories instead of 8**. We have decreased the time spent creating factories **by 30%**.

The caveat of this method it that it could be **a heavy process to maintain**. Thankfully, `test-prof` as a recipe called `FactoryDefault`. Removing factory cascades manually could be good enough most of the time but if you want to go further you can follow the [documentation](https://test-prof.evilmartians.io/#/recipes/factory_default?id=instructions).

That being said, `test-prof` has even more to offer, it’s time to introduce an awesome helper named `let_it_be`.

- **Reuse the factory you need**

> Let's bring a little bit of magic and introduce a new way to set up a shared test data.

`let_it_be` is a [helper](https://test-prof.evilmartians.io/#/recipes/let_it_be) that allows you to reuse the same factory for all your spec file. In our example we don’t need to create 2 `survey` and 2 `questionnaire` we could re-use the same ones for all our file.

```ruby
RSpec.describe Purging::QuestionnaireWorker, type: :worker do
  let_it_be(:survey) { create(:survey) }
  let_it_be(:questionnaire) { create(:questionnaire, survey: survey) }
  ...
end
```

If we run the factory profiler we now have a different report:

```
[TEST PROF INFO] Factories usage

 Total: 3
 Total top-level: 3
 Total time: 00:00.272 (out of 00:24.264)
 Total uniq factories: 3

   total   top-level     total time      time per call      top-level time               name

       1           1        0.2024s            0.2024s             0.2024s             survey
       1           1        0.0323s            0.0323s             0.0323s             questionnaire
       1           1        0.0375s            0.0375s             0.0375s             question
```

So now we only create the factories we need, by reusing the same ones throughout our file.

Be aware that `let_it_be` come with a [caveat](https://test-prof.evilmartians.io/#/recipes/let_it_be?id=caveats) section. I strongly encourage you to read the documentation and use this powerful helper in accordance with your needs.

### :rocket:  Conclusion

Let’s take a step back and relish our improvements:

|               | Initial | Without cascades  | With let_it_be  |
| ------------- | -------------| -----| -----|
| Factories creation time | 00:01.00 | 00:00.868 | 00:00.272 

Numbers look nice for this simple example. But what is the impact in real life at **Potloc**? 

So far we just applied this recipe for a specific folder of our codebase. Below the result by profiling locally that folder:

Before we spent **3.50 min** in factories creation, now **2 min. (~ -50%)**
Before we created **6824** factories, now **4378. (~ -35%)**

 `test-prof` is the swiss army knife we needed to speed up our test suite. It’s still a long journey but by embracing this topic we have already taken an important step!


Want to go further? Watch this [99 problems of slow test](https://www.youtube.com/watch?v=eDMZS_fkRtk&ab_channel=parisrb) talk by [Vladimir Dementyev](https://github.com/palkan)


*Interested in what we do at [Potloc](https://www.potloc.com/)? Come join us! [We are hiring](https://jobs.lever.co/Potloc) 🚀*