Fast Type-Checked Serializers for Ruby Web APIs
Tags: ruby, json, api, web
Published: 2018-02-11 07:52:15

---
title: "Fast Type-Checked Serializers for Ruby Web APIs"
published: true
description: "Primalize — Fast Type-Checked Serializers for Ruby Web APIs"
tags: ruby,json,api,web
cover_image: https://i.imgur.com/Gok5Gc6.png
---

When you need two different processes to be able to talk to each other, they need to be able to connect and speak the same language. If one of them is a web app, you're using HTTP (and everything it's built on top of), so that covers most of it. If you're sending requests and receiving responses serialized as JSON, you're even closer. But we still need to figure out that last mile — what JSON objects are we going to send and receive?

If our web server is running on Ruby, we just need to get our models converted into hashes and collections converted into arrays. Conversion to JSON is a simple `to_json` call at that point. How we convert them into those hashes and arrays is the choice to be made.

## Conversion Options

There are plenty of gems available to convert arbitrary Ruby objects into our JSON-serializable format. The most well-known of these is [`active_model_serializers`](https://github.com/rails-api/active_model_serializers), but a couple weeks ago Netflix announced [their own](https://medium.com/netflix-techblog/fast-json-api-serialization-with-ruby-on-rails-7c06578ad17f) serialization gem called `fast_jsonapi`.

With both gems, you declare your serializers like this:

```ruby
class OrderSerializer < ActiveModel::Serializer
  attributes(
    :id,
    :customer_name,
    :customer_id,
    :line_item_ids,
    :delivery_address,
    :delivery_instructions,
    :total_price,
  )
end
```

## The Downside of Those Options

Both of those gems are great. You can tell by looking at this serializer exactly what fields it'll be sending. The thing they're missing is validation of the types of the data they are emitting. Is `delivery_address` a string or is it broken out into a hash with each address part? Which of these fields can be `nil`? Is `total_price` sent as a string to be used presentationally, a floating-point number for calculation, or an integer number of `cents` to avoid floating-point error? These are impossible to tell by looking at the attribute declaration in the serializer.

You might be thinking "why do I want type checking in a dynamic language?!" While I can see the benefits of static typing, I really do enjoy the freedom that dynamic typing gives us to build applications. However, when you have two different applications that must both agree on how they speak to each other, you need to be 100% sure about all of the details, just like getting contract details ironed out in a consulting agreement. Adding unit tests to verify that a serializer emits the right data types is a lot of work. What if the serializers did that for us and would even let us know if our data was incorrect in production?

Well, I wouldn't be writing this if I didn't already have a solution for that. Enter [`primalize`](https://github.com/jgaskins/primalize), a serialization gem that comes with type checking out of the box. Primalize is so named because it converts more advanced objects into their primitive counterparts, but it does it in an intelligent way.

## How does it work?

Let's start by declaring a serializer:

```ruby
class OrderSerializer < Primalize::Single
  attributes(
    id: string, # UUIDs for primary keys are great
    customer_name: string,
    customer_id: string,
    line_item_ids: array(string),
    delivery_address: primalize(AddressSerializer),
    delivery_instructions: optional(string),
    total_price: integer,
  )
end
```

Notice the attributes we declare all have types. We say the `id` is a `string`, the `line_item_ids` is an `array` containing `string` values. The `total_price` is an `integer`. These are pretty easy to understand and they tell us exactly what we're getting.

Notice the `delivery_instructions` attribute is marked as `optional`. This means it can be `nil`. If an attribute isn't marked as `optional`, it can't be `nil`.

Here's a list of the various types that `Primalize` supports for model serializers:

- `integer`: whole numbers
- `float`: floating-point numbers
- `number`: any numeric value
- `string`: text
- `boolean`: explicitly `true` or `false` (not "truthy" or "falsy" values)
- `array(*types)`: an array containing values of the specified types
  - Example: `array(string, integer)`
- `optional(*types)`: any of the specified types or `nil`
  - Example: `optional(string)`, both `"foo"` and `nil` are acceptable values
- `enum(*values)`: must be one of the specified values
  - Example: `enum('requested', 'shipped', 'delivered')`
- `timestamp`: a `Date`, `Time`, or `DateTime` value
- `any(*types)`: any value of the given types
  - Example: `any(string, integer)` will only match on strings and integers
  - If no types are specified, any value will match
- `primalize(YourPrimalizerClass)`: primalizes the specified attribute with the given `Primalize::Single` subclass
  - Example: `primalize(OrderSerializer)`
- `object(**types)`: a hash of the specified structure
  - Example: `object(id: integer, name: string)`
  - Only the required keys need to be specified. The rest of the hash will pass.
  - If no keys are specified, all of them are optional and it will match any hash.
  - Ruby objects already define a method called `hash` that's used for resolving hash keys and determining `Set` inclusion, so we had to use the more language-agnostic name `object`. If we'd used `hash`, it would be effectively impossible to use a serializer class as a hash key or store it in a `Set`.

These type declarations are composable, so we can set up some really complex type declarations if our API calls for it:

```ruby
attributes(
  user: object(
    name: optional(string),
    email: string,
    nicknames: array(string),
    role: optional(enum('user', 'agent', 'manager', 'admin')),
  ),
)
```

If you find yourself writing a lot of nested objects, though, it might be worth extracting that to another serializer and using `primalize(ThatSerializer)`.

## What happens if the type check fails?

In Ruby, we can't enforce that our models don't have the wrong types of attributes because any variable can hold any value, but we can run a type check at the time of serialization.

Serializers have a default type-mismatch handler, which is a `call`able object (as in, responds to `call`) that receives `serializer_class, attribute, type, value`. By default, it raises an exception, which is great for a development environment. In production, though, it might be preferable to let it pass through while still sending alerts.

You can customize what you do when a type mismatch occurs on your individual serializer class by setting `MySerializerClass.type_mismatch_handler` to your preferred method of handling it. To set it for _all_ model serializers, use `Primalize::Single.type_mismatch_handler`. For example:

```ruby
MySerializerClass.type_mismatch_handler = proc do |serializer, attr, type, value|
  message = "#{serializer}##{attr} is specified as #{type.inspect}, but is #{value.inspect}"

  Slack.notify '#bugs', message
  BugTracker.notify message

  # the return value of the block is the value to be used for that attribute
  value
end
```

## Attribute conversion

If an attribute isn't already the type you're expecting, you can provide a block to its type declaration to specify how to coerce it to that type. For example, if an `Address#city` returns a `City` object instead of just the name, we could serialize it like this:

```ruby
class AddressSerializer < Primalize::Single
  attributes(
    city: string { |city| city.name },
    # ...
  )
end
```

This indicates that we would call `name` on the city that's in the address's `city` attribute. The type check will still occur here. If the result of that block isn't a string, we'll trigger a type mismatch.

## Virtual attributes

Sometimes you want to provide attributes that don't actually exist on the model being serialized. Just like how our server-side domain models don't need to match the database schema, a client doesn't need to know that the server-side model doesn't have a particular attribute. For those "virtual" attributes, you can define a method that will compute the attribute from what the model _does_ have:

```ruby
class AddressSerializer < Primalize::Single
  attributes(
    latitude: float,
    longitude: float,
    # ...
  )

  def latitude
    object.coordinates.latitude
  end

  def longitude
    object.coordinates.longitude
  end
end
```

## Composite Serializers

That's just individual model serializers. There's also first-class support for returning associated objects in a single response without nesting them with `primalize(AnotherSerializer)`:

```ruby
class OrderResponse < Primalize::Many
  attributes(
    order: OrderSerializer,
    line_items: enumerable(LineItemSerializer),
    address: AddressSerializer,
    customer: CustomerSerializer,

    # Only required for corporate accounts
    purchase_order: optional(PurchaseOrderSerializer),
  )
end
```

I typically refer to these as "response serializers" and, while it's possible to serialize just the domain model in an API response, I almost always wrap them inside a serializer like this in case I need to return associated models in the future. If you have any API consumers you don't control, once you start returning the model as the top-level object, you're stuck with it most of the time.

To use this serializer, you instantiate it with the keys you gave it:

```ruby
OrderResponse.new(
  order: order,
  line_items: order.line_items,
  address: order.delivery_address,
  customer: order.customer,
  purchase_order: order.purchase_order,
)
```

`Primalize::Many` doesn't traverse the object graph for you, and this might feel inconvenient, but it's intended functionality. It ensures that you can customize what you're sending. For example, if your `order` is an ActiveRecord model, you may not want to send _all_ of its `line_items` together. You might split them between `taxable_line_items` and `non_taxable_line_items` for some business reason. If the serializer traversed the association for you, you might not be able to specify that without adding a method on the `Order` model. With Primalize, though, you can set up your serializer like this:

```ruby
class OrderResponse < Primalize::Many
  attributes(
    # ...
    taxable_line_items: enumerable(LineItemSerializer),
    non_taxable_line_items: enumerable(LineItemSerializer),
  )
end

OrderResponse.new(
  # ...
  taxable_line_items: order.line_items.taxable,
  non_taxable_line_items: order.line_items.non_taxable,
)
```

The type checking comes into play with response serializers, as well. For example, if we passed a single line item in where we specified `enumerable(LineItemSerializer)`, we would get an error. If we passed `nil` for a field without specifying it as `optional`, we would also get an error.

All this helps make more robust API endpoints.

## JSONAPI

Some client applications might be written in a way that is more suited to the JSONAPI response structure. For such applications, there is a [JSONAPI wrapper](https://github.com/jgaskins/primalize-jsonapi) for Primalize.

## Performance

I mentioned in the title that these serializers are fast, but I haven't touched on that part yet. When I [benchmarked Primalize::JSONAPI against AMS and Netflix's `fast_jsonapi` gem](https://github.com/Netflix/fast_jsonapi/pull/42#issuecomment-364693269), it was over 1200% as fast as AMS and about half as fast as the Netflix gem. When you consider that the selling point of `fast_jsonapi` is that … well … it's fast, and the difference between it and Primalize to serialize over 1000 models is 12ms (less than 12µs each), it's still close enough. On typical payloads (dozens of models, maybe 100), you'll have garbage-collection pauses longer than the difference between them.

Also, that benchmark was using the JSONAPI wrapper, which is doing considerably more work (including its own naive traversal of associated models). If you don't require that particular format, you can stick with `Primalize::Many` to cut off about 25% of that time.

I don't know about you, but I'm certainly willing to trade 12ms (much less on typical payloads) for peace of mind that my response payloads match what the client expects.

## Future Development

Primalize is a gem based on a pattern I implemented at a previous employer. It has been used in production for around two years and has stabilized the communication between their API and its consumers significantly while also improving its performance. With that said, there are still ways to improve.

One idea that a coworker at that employer requested was the ability to generate RAML or Swagger documentation. For example, if an API endpoint returns an `OrderResponse`, you should be able to generate the exact structure a client could expect for that endpoint. RAML docs can be [imported into Postman](http://blog.getpostman.com/2015/11/04/supporting-raml-folders-in-postman/) for easier testing and consumption of REST APIs.

Performance could probably be improved even more, both in the baseline `Primalize` classes and the `JSONAPI` wrapper.

If you'd like to contribute, I'm always open to conversation ([gitter](https://gitter.im/jgaskins/primalize) / [twitter](https://twitter.com/jamie_gaskins) / comments on this post), [suggestions](https://github.com/jgaskins/primalize/issues), and [pull requests](https://github.com/jgaskins/primalize/pulls). Thanks, everyone! ❤️