Testing external APIs with Rspec and WebMock
Tags: ruby, rails, rspec, testing
Published: 2021-04-16 16:05:40

###Testing code you don't own can be a bit trickier than writing tests for code you own. If you're interacting with 3rd party APIs, you don't actually want to make real calls every time you run a test.

Reasons why you should not make real requests to external APIs from your tests: 

- it can make your tests slow
- you're potentially creating, editing, or deleting real data with your tests
- you might exhaust the API limit
- if you're paying for requests, you're just wasting money
- ...

Luckily we can use mocks to simulate the HTTP requests and their responses, instead of using real data.

I've recently written a simple script that makes a `POST` request to my external newsletter service provider every time a visitor subscribes to my newsletter. Here's the code:

```ruby
module Services
  class NewSubscriber
    BASE_URI = "https://api.buttondown.email/v1/subscribers".freeze

    def initialize(email:, referrer_url:, notes: '', tags: [], metadata: {})
      @email = email
      @referrer_url = referrer_url
      @notes = notes
      @tags = tags
      @metadata = metadata
    end

    def register!
      uri = URI.parse(BASE_URI)
      request = Net::HTTP::Post.new(uri.request_uri, headers)

      http = Net::HTTP.new(uri.host, uri.port)
      http.use_ssl = true
      http.request(request, payload.to_json)
    end

    private

    def payload
      {
        "email" => @email,
        "metadata" => @metadata,
        "notes" => @notes,
        "referrer_url" => @referrer_url,
        "tags" => @tags
      }
    end

    def headers
      {
        'Content-Type' => 'application/json',
        'Authorization' => "Token #{Rails.application.credentials.dig(:buttondown, :api_key)}"
      }
    end
  end
end
```

The first approach could be to mock the `::Net::HTTP` ruby library that I'm using to make the request:

```ruby
describe Services::NewSubscriber do
  let(:email) { 'user@example.com' }
  let(:referrer_url) { 'www.blog.com' }
  let(:tags) { ['blog'] }
  let(:options) { { tags: tags, notes: '', metadata: {} } }

  let(:payload) do
    {
      email: email,
      metadata: {},
      notes: "",
      referrer_url: referrer_url,
      tags: tags
    }
  end

  describe '#register' do
    it 'sends a post request to the buttondown API' do
      response = Net::HTTPSuccess.new(1.0, '201', 'OK')

      expect_any_instance_of(Net::HTTP)
        .to receive(:request)
        .with(an_instance_of(Net::HTTP::Post), payload.to_json)
        .and_return(response)

      described_class.new(email: email, referrer_url: referrer_url, **options).register!

      expect(response.code).to eq("201")
    end
  end
```

This test passes but there are some caveats to this approach:

- I'm too tied to the implementation. If one day I decide to use [`Faraday`](https://github.com/lostisland/faraday) or [`HTTParty`](https://github.com/jnunemaker/httparty) as my HTTP clients instead of `Net::HTTP`, this test will fail. 
- It's easy to break the code without making this test fail. For instance, this test is indifferent to the arguments that I'm sending to the `Net::HTTP::Post` and `Net::HTTP` instances. 

### Testing behavior with `WebMock`

`WebMock` is a library that helps you stub and set expectations on HTTP requests.

You can find the setup instructions and examples on how to use WebMock, in their [github documentation](https://github.com/bblimke/webmock). WebMock will prevent any external HTTP requests from your application so make sure you add this gem under the `test` group of your `Gemfile`.
 
With Webmock I can stub requests based on method, uri, body, and headers. I can also customize the returned response to help me set some expectations base on it. 

```ruby
require 'webmock/rspec'

describe Services::NewSubscriber do
  let(:email) { 'user@example.com' }
  let(:referrer_url) { 'www.blog.com' }
  let(:tags) { ['blog'] }
  let(:options) { { tags: tags, notes: '', metadata: {} } }

  let(:payload) do
    {
      email: email,
      metadata: {},
      notes: '',
      referrer_url: referrer_url,
      tags: tags
    }
  end

  let(:base_uri) { "https://api.buttondown.email/v1/subscribers" }

  let(:headers) do
    {
      'Content-Type' => 'application/json',
      'Accept'=>'*/*',
      'Accept-Encoding'=>'gzip;q=1.0,deflate;q=0.6,identity;q=0.3',
      'User-Agent'=>'Ruby'
    }
  end

  let(:response_body) { File.open('./spec/fixtures/buttondown_response_body.json') }

  describe '#register!' do
     it 'sends a post request to the buttondown API' do
        stub_request(:post, base_uri)
          .with(body: payload.to_json, headers: headers)
          .to_return( body: response_body, status: 201, headers: headers)

       response = described_class.new(email: email, referrer_url: referrer_url, **options).register!

       expect(response.code).to eq('201')
    end
  end
end
```

Now I can pick another library to implement this script and this test should still pass. But if the script makes a different call from the one registered by the stub, it will fail. This is what I want to test - behavior, not implementation. So if, for instance, I run the script passing a different subscriber email from the one passed to the stub I'll get this failure message:

```
1) Services::NewSubscriber#register! sends a post request to the buttondown API
     Failure/Error: http.request(request, payload.to_json)                                                                                                                        
                                            
     WebMock::NetConnectNotAllowedError:
       Real HTTP connections are disabled. Unregistered request: POST https://api.buttondown.email/v1/subscribers with body '{"email":"ana@test.com","metadata":{},"notes":"","ref
errer_url":"www.blog.com","tags":["blog"]}' with headers {'Accept'=>'*/*', 'Accept-Encoding'=>'gzip;q=1.0,deflate;q=0.6,identity;q=0.3', 'Authorization'=>'Token 26ac0f19-24f3-4ac
c-b993-8b3d0286e6a0', 'Content-Type'=>'application/json', 'User-Agent'=>'Ruby'}
                                            
       You can stub this request with the following snippet:
                                            
       stub_request(:post, "https://api.buttondown.email/v1/subscribers").
         with(                              
           body: "{\"email\":\"ana@test.com\",\"metadata\":{},\"notes\":\"\",\"referrer_url\":\"www.blog.com\",\"tags\":[\"blog\"]}",
           headers: {                                                                    
          'Accept'=>'*/*',                                                                                                                                                        
          'Accept-Encoding'=>'gzip;q=1.0,deflate;q=0.6,identity;q=0.3',
          'Authorization'=>'Token 26ac0f19-24f3-4acc-b993-8b3d0286e6a0',            
          'Content-Type'=>'application/json',                                                                                                                                     
          'User-Agent'=>'Ruby'
           }).                                                                           
         to_return(status: 200, body: "", headers: {})
                                            
       registered request stubs:            
                                            
       stub_request(:post, "https://api.buttondown.email/v1/subscribers").                                                                                                        
         with(                              
           body: "{\"email\":\"user@example.com\",\"metadata\":{},\"notes\":\"\",\"referrer_url\":\"www.blog.com\",\"tags\":[\"blog\"]}",                                                    headers: {                       
          'Accept'=>'*/*',
          'Accept-Encoding'=>'gzip;q=1.0,deflate;q=0.6,identity;q=0.3',
          'Content-Type'=>'application/json',
          'User-Agent'=>'Ruby'
           })                               
                                            
       Body diff:                           
        [["~", "email", "ana@test.com", "user@example.com"]]
```

There's not much more to it than this. [VCR](https://github.com/vcr/vcr) is another tool that also stubs API calls but works a little differently by actually making a first real request that will be saved in a file for future use. For simple API calls, WebMock does the trick for me!