System testing for Rails application without Capybara, using Puppeteer
Tags: puppeteer, rails, capybara, rspec
Published: 2021-09-03 05:58:03

---
title: System testing for Rails application without Capybara, using Puppeteer
published: true
description: Capybara is easy to configure, but sometimes brings unstable/flaky tests in system testing for Rails. This article shows how to use the system testing without Capybara DSL. 
tags: Puppeteer,Rails,Capybara,RSpec
cover_image: https://raw.githubusercontent.com/YusukeIwaki/puppeteer-ruby/main/puppeteer-ruby.png
---

> Refer [this article](https://zenn.dev/yusukeiwaki/articles/449a860a750561) instead if you can read Japanese sentences.
> 日本語を読める方は[こちらの記事](https://zenn.dev/yusukeiwaki/articles/449a860a750561)を見てください

## Background

Rails introduced "system testing", that makes it easy to configure Capybara: acceptance test framework for Ruby. We may often use it not for "acceptance test" but for just UI testing of the Rails application with React/Vue or other rich JavaScript-powered features.

Capybara DSL is actually a little **inaccurate** for working with pages with rich JavaScript features.

Capybara has internally polling with interval 10msec for waiting for Elements available and actionable.
https://github.com/teamcapybara/capybara/blob/0468de5a810aae75ab9de20447e246c5c35473f0/lib/capybara/node/base.rb#L91
I don't know this is really the main cause, but we may often face flaky/unstable testcases in using Capybara.

On the other hand, [Puppeteer](https://pptr.dev/) or [Playwright](https://playwright.dev/) have a relyable solution for waiting for Elements: `Page#waitForSelector` and `Page#waitForNavigation`. (Playwright has more advanced feature of [auto-waiting](https://playwright.dev/docs/actionability) )

I actually ported the libraries for using them in Rails application.

* Playwright for Ruby: https://playwright-ruby-client.vercel.app/
* Puppeteer for Ruby: https://github.com/YusukeIwaki/puppeteer-ruby

However **the relyable features cannot be used from Capybara DSL**. So let's try to configure system tests without Capybara, and using Puppeteer.

## What happens if Capybara is absent??

Since Capybara is a huge framework, it would be difficult for some users to imagine what happens without Capybara.

Actually Rails application without Capybara loses the functionalities

* Launching test HTTP server on starting tests
* Capybara::DSL
  * Using `visit` or other methods raises `NoMethodError` instead of NotImplementedError
  * System spec (on RSpec) or SystemTestCase (on MiniTest), which depends on Capybara::DSL

However rspec-rails still provides features like

* Attaching `type: :feature` `type: :system` to example.metadata
* Defining `feature` `background` `scenario` methods for feature spec

So actually all we have to prepare is

* **Launching test HTTP server**
* Performing automation with browsers
  * This will be satisfied by Puppeteer

## Launching test HTTP server without Capybara

Capybara is really kind that

* finds available port
* allows to configure host
* allows to launch Webrick and Puma

However let's forget them at this moment, and assume that **we only have to launch Puma with the baseURL `http://127.0.0.1:3000`**

With referring the logics of Capybara [preparing server](https://github.com/teamcapybara/capybara/blob/master/lib/capybara/server.rb) and [launching Puma server](https://github.com/teamcapybara/capybara/blob/master/lib/capybara/registrations/servers.rb), then we can easily find the minimum configuration for launching test server like below:

```ruby
RSpec.configure do |config|
  config.before(:suite) do
    # launching Rails server for system testing
    require 'rack/builder'
    testapp = Rack::Builder.app(Rails.application) do
      map '/__ping' do # debugging endpoint for heartbeat
        run ->(env) { [200, { 'Content-Type' => 'text/plain' }, ['OK']] }
      end
    end

    require 'rack/handler/puma'
    server_thread = Thread.new do
      Rack::Handler::Puma.run(testapp,
        Host: '127.0.0.1',
        Port: 3000,
        Threads: '0:4',
        workers: 0,
        daemonize: false,
      )
    end

    # Waiting for Rails server is ready, using Net::HTTP
    require 'net/http'
    require 'timeout'
    Timeout.timeout(3) do
      loop do
        puts Net::HTTP.get(URI("http://127.0.0.1:3000/__ping"))
        break
      rescue Errno::EADDRNOTAVAIL
        sleep 1
      rescue Errno::ECONNREFUSED
        sleep 0.1
      end
    end
  end

  # Configure puppeteer-ruby for automation
  config.around(type: :feature) do |example|
    Puppeteer.launch(channel: :chrome, headless: false) do |browser|
      @server_base_url = 'http://127.0.0.1:3000'
      @puppeteer_page = browser.new_page
      example.run
    end
  end
end
```

By putting this configuration into spec/support/integration_test_helper.rb or another place you prefer, now we can enjoy system testing with Puppeteer like below!!

As is already described, **we cannot use system spec. Use feature spec (provided by rspec-rails) instead**.

```ruby
require 'rails_helper'

describe 'example' do
  let(:base_url) { @server_base_url }
  let(:page) { @puppeteer_page }

  let(:user) { FactoryBot.create(:user) }

  it 'can browse' do
    page.goto("#{base_url}/tests/#{user.id}")
    page.wait_for_selector('input', visible: true)
    page.type_text('input', 'hoge')
    page.keyboard.press('Enter')

    text = page.eval_on_selector('#content', 'el => el.textContent')
    expect(text).to include('hoge')
    expect(text).to include(user.name)
  end
end
```

![integ_rails](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/nfgmuh1orp6d6rvk9hv2.gif)


### A little refactoring for production use

The logic of launching test server, introduced in the previous section, is really straightforward. However most users would hesitate to copy/paste it into your own products because it's too dirty :(

No worry, here is a better code with more relyable `Rack::Server.start` to launch HTTP server, which is actually used in [the implementation of `rackup` command](https://github.com/rack/rack/blob/2.2.3/bin/rackup#L5)

```rb
class RackTestServer
  def initialize(app:, **options)
    @options = options

    @options[:Host] ||= '127.0.0.1'
    @options[:Port] ||= 3000

    require 'rack/builder'
    @options[:app] = Rack::Builder.app(app) do
      map '/__ping' do
        run ->(env) { [200, { 'Content-Type' => 'text/plain' }, ['OK']] }
      end
    end
  end

  def base_url
    "http://#{@options[:Host]}:#{@options[:Port]}"
  end

  def start
    require 'rack/server'
    Rack::Server.start(**@options)
  end

  def ready?
    require 'net/http'
    begin
      Net::HTTP.get(URI("#{base_url}/__ping"))
      true
    rescue Errno::EADDRNOTAVAIL
      false
    rescue Errno::ECONNREFUSED
      false
    end
  end

  def wait_for_ready(timeout: 3)
    require 'timeout'
    Timeout.timeout(3) do
      sleep 0.1 until ready?
    end
  end
end
```

With this test server, the RSpec configuration file can be much shorter and simplified :)

```rb
RSpec.configure do |config|
  config.before(:suite) do
    # Launch Rails application
    test_server = RackTestServer.new(
      # options for Rack::Server
      # https://github.com/rack/rack/blob/2.2.3/lib/rack/server.rb#L173
      app: Rails.application,
      server: :puma,
      Host: '127.0.0.1',
      Port: 3000,
      daemonize: false,

      # options for Rack::Handler::Puma
      # https://github.com/puma/puma/blob/v5.4.0/lib/rack/handler/puma.rb#L84
      Threads: '0:4',
      workers: 0,
    )

    Thread.new { test_server.start }
    test_server.wait_for_ready
  end

  # Configure puppeteer-ruby for automation
  config.around(type: :feature) do |example|
    Puppeteer.launch(channel: :chrome, headless: false) do |browser|
      @server_base_url = 'http://127.0.0.1:3000'
      @puppeteer_page = browser.new_page
      example.run
    end
  end
end
```

Now, you can now get rid of Capybara from your app if you just want to launch HTTP server for testing :)

You can also use [rack-test_server](https://rubygems.org/gems/rack-test_server) Gem instead of defining your own RackTestServer.