Payload and parameters validation in Rails
Tags: ruby, rails, dry, programming
Published: 2024-03-09 19:08:10

## Overview

Here we go again! A lot of text and code in this post :smile: We are going to continue where we stopped [last time](https://dev.to/kimrgrey/tuning-rails-application-structure-5f74) with focus on input parameters validation. In my opinion, this is the weakest and almost missing part of the standard Rails setup which can be easily improved, polished and extended. We are going to show how with a few tricks and tips one can solve validations once and for all.

In order to read and understand this article you don't necessary need to go through the previous one, but if you have not read first part you may notice some strange details which are not standard for Rails and may look awkward. If you find yourself a bit confused try to go read first part and come back. It explains a lot of basics we use here.

Consider subscribing to my Dev.to blog to to stay in touch! 😉 And feel free to ask questions in the comments section below.

You can also find me here:

- Telegram: https://t.me/kgcodes
- Twitter: https://twitter.com/kimrgrey

See you!

## What do we have?

My professional life developed in such a way that most of the time I was working on different kinds of mobile applications mostly in transportation area: taxi, delivery, bike and car sharing, public transport. When you are building a common API based mobile app a lot of your every day work can be boiled down to the following sequence:

1. Take input from mobile
2. Validate this input
3. Store something in database
4. Make it possible to fetch records you created

Of course, you have some business logic and algorithms around that which creates actual value and essence of your application. This logic is what makes it worth money in the end of the day. It is often complicated and requires a lot of thinking to cook and deliver. That is why it is so important to offload most of the routine to your framework. And Rails really shines in such an environment. 

Convention over configuration principle saves you from living in the world of making stack traces out of XML files, as we were joking back in the Java days :smile: 

Framework takes care of a database connection for you. It provides you with very convenient way of creating, validating and accessing your records via `ActiveRecord`. 

You can render and send an e-mail in a few lines of code thanks to `ActionMailer`. Which I actually rarely use for my projects, but that is a topic for another day :wink: 

And, of course, asynchronous jobs are taken care of by `ActiveJobs`. 

And so on and so far. 

In many aspects Rails decides and does a lot of heavy lifting so you can think about important things. It saves a lot of arguing time as well. I remember good old days of battles about the best way to layout your data objects or structure your configuration files. Such a waste of time! 

It is really good to live in a world where boilerplate is solved and be efficient, isn't it?

## What do we want?

With time though I realized that there are a few caveats which spoil the party. One of the things almost neglected by Rails is input parameters validation. 

Let's say we are developing delivery application for a dark store or dark kitchen. Shortly speaking, we have an app which allows user to pick products, put them into a cart, estimate a price, order instant or scheduled delivery and pay for it. 

If you want to create such an app it is necessary to take care of many questions. How do we represent and calculate the price? Should we provide some promo codes? How do we allow service fee or handle surges? How user can sign up and login into the system? How do we manage and keep track of stocks? How picking and packing processes are organized? What should we do if something ordered is actually missing? All of that creates value. All of that is what matters the most. All of that pays your bills.

In addition to hard questions we also should provide our procurement team with ability to manage assortment. Our admins should be able to create products, categorize them, hide and show them in store, fill in descriptions, specify prices, tags and so on. Doesn't matter how big and cool your application is anyway you have to solve simple things too. It all grows up from the back office.

Let's start from creating very simplistic table and model for our products:

```ruby
# db/migrate/20240309154032_create_products.rb

class CreateProducts < ActiveRecord::Migration[7.1]
  def change
    create_table :products do |t|
      t.string :name, null: false
      t.string :description, null: true
      t.timestamps
    end
  end
end
```

```ruby
# app/models/product.rb

class Product < ApplicationRecord
  validates :name, presence: true
end
```

Now we should make it possible to create products through API requests. Again, obviously oversimplified code which does the job may look like this:

```ruby
# app/controllers/products_controller.rb

class ProductsController < ApplicationController
  def create
    product = Product.create!(create_params)

    render json: product.as_json(only: [:name, :description])
  end

  private def create_params
    params.permit(:name, :description)
  end
end
```

Should we try to use it?

```shell
curl --location '127.0.0.1:3000/products' \
--header 'Content-Type: application/json' \
--data '{
    "name": "Lindt Excellence Dark 70% Cacao Bar",
    "description": "Extra fine dark chocolate Lindt EXCELLENCE Dark 70% Bar 100g. A finely balanced taste sensation, finished with a hint of fine vanilla."
}'
```

Response is looking good:

```json
{
    "name": "Lindt Excellence Dark 70% Cacao Bar",
    "description": "Extra fine dark chocolate Lindt EXCELLENCE Dark 70% Bar 100g. A finely balanced taste sensation, finished with a hint of fine vanilla."
}
```

We should also have at least ability to see the list with pagination, update existing products and delete them. Let's keep it very simple as well:

```ruby
# app/controllers/products_controller.rb

class ProductsController < ApplicationController
  DEFAULT_LIMIT = 25
  DEFAULT_OFFSET = 0

  def index
    products = Product.order(id: :desc)
      .limit(params[:limit] || DEFAULT_LIMIT)
      .offset(params[:offset] || DEFAULT_OFFSET)

    render json: {
      products: products.as_json(only: [:id, :name, :description])
    }
  end

  def create
    product = Product.create!(create_params)

    render json: product.as_json(only: [:id, :name, :description])
  end

  def show
    product = Product.find(params[:product_id])

    render json: product.as_json(only: [:id, :name, :description])
  end

  def update
    product = Product.find(params[:product_id])
    product.update!(update_params)

    render json: product.as_json(only: [:id, :name, :description])
  end

  def destroy
    product = Product.find(params[:product_id])
    product.destroy!

    render json: {}
  end

  private def create_params
    params.permit(:name, :description)
  end

  private def update_params
    params.permit(:name, :description)
  end
end
```

We have entire set of standard CRUD operations for our products. But even from this little example it is easy to see capabilities which Rails lacks out of the box. 

First, it mixes all actions together. Which is not such big of a problem, but we have to do some naming exercise to differentiate `create_params` from `update_params`, for example. Plus constants `DEFAULT_LIMIT` and `DEFAULT_OFFSET` which are only relevant for `index` are present for every action. Core models, such as product, tend to have a lot of methods: update this, update that, toggle, activate, hide, show, etc. More actions you have in one controller harder it is going to find relevant stuff. And more tempting it will be to re-use some parts and bits. Which is not always a great idea.

Second, we have URL parameters, such as `product_id` for some methods. For others we have payload parameters, such as `name` or `description`. And sometimes we have both. Also `name` is required while `description` is completely optional. 

Unless you can read and understand Ruby in general and Rails models particularly there is no way to know all of that. Code is going to grow and we will have more parameters and rules. Mobile and back office teams would like to know what endpoints do we have, what they should send and in which form. We can't ask them to use source code to find this out, because it is not obvious from the first glance at all. So, in order to provide some insights on our API we gonna need to spend time on documenting it. 

Can we improve it a bit and save some time on documenting internal API-s without going too far?

## Let's move things around, shall we?

What if for the beginning we isolate every action into separated class. That is not classic Rails approach, but taking into account our module friendly project structure from the previous article it should be possible to do it quite easily and the result should look quite nicely. 

We call such classes "handlers". Their role is to take some set of parameters, payload, handle the request and respond with some data. It is important to remember, that our goal here is to improve developer's experience, coding speed and readability without going too far away from vanilla Rails common sense, so that our code won't surprise new people too much.

Here we define a base class for our handlers to share common logic:

```ruby
# app/application_handler.rb

class ApplicationHandler
  attr_reader :controller

  def initialize(controller)
    @controller = controller
  end

  protected def render(**args)
    controller.render(**args)
  end

  protected def params
    controller.params
  end

  protected def headers
    controller.headers
  end
end
```

After that we can move our first action out from controller to a handler:

```ruby
# app/products/handlers/create.rb

module Products
  module Handlers
    class Create < ApplicationHandler
      def handle
        product = Product.create!(payload)

        render json: product.as_json(only: [:id, :name, :description])
      end

      private def payload
        params.permit(:name, :description)
      end
    end
  end
end
```

As you can see, this file is very narrow focused. It covers one specific flow - product creation. It shares zero space with other flows. All methods, names and parameters are scoped. In the same moment, it contains all information required about this specific endpoint. In order to work with product creation flow you need this file and this file only.

I think it is quite obvious how other handlers will look like, so I'll share one more example here just to make a point clear:

```ruby
# app/products/handlers/index.rb

module Products
  module Handlers
    class Index < ApplicationHandler
      DEFAULT_LIMIT = 25
      DEFAULT_OFFSET = 0

      def handle
        products = Product.order(id: :desc)
          .limit(params[:limit] || DEFAULT_LIMIT)
          .offset(params[:offset] || DEFAULT_OFFSET)

        render json: {
          products: products.as_json(only: [:id, :name, :description])
        }
      end
    end
  end
end
```

Once we moved all of our actions to handlers we should change the controller which, frankly speaking, doesn't make a lot of work now:

```ruby
# app/controllers/products_controller.rb

class ProductsController < ApplicationController
  def index
    handler = Products::Handlers::Index.new(self)
    handler.handle
  end

  def create
    handler = Products::Handlers::Create.new(self)
    handler.handle
  end

  def show
    handler = Products::Handlers::Show.new(self)
    handler.handle
  end

  def update
    handler = Products::Handlers::Update.new(self)
    handler.handle
  end

  def destroy
    handler = Products::Handlers::Destroy.new(self)
    handler.handle
  end
end
```

We could make one extra step here and introduce some DSL or convention, so that calling of handlers is done for us "automagically". But based on my experience it is much better idea to keep a little boilerplate, because it makes whole thing understandable for any new developer joining the team. What we are looking at are just plain simple Ruby classes and methods. Nothing too fancy, too scary or too controversial.

Good job, but this move achieves only one thing. It gives us some namespace for programming and isolates one handler from another. But there is very little improvement in other areas. 

Can we make things better? And what about validations?

## Finally, validations!

As it was mentioned before, Rails does very good job in solving almost every common web development problem. But somehow it is not true for the most frequent one - input validation. 

The best tool we have in Rails for validating parameters is `params.require(:product).permit(:name, :description)`. Which is at very least limited, if not to say "primitive". Without going out of the box it is not easy to make any next step.

Luckily, there is a large pool of community wisdom around and outside of Rails which may help us a lot here. Instead of inventing our own wheel for now we will use one invented before us by others. Pretty much sure you have seen this magic used outside of Hogwarts before: https://dry-rb.org/gems/dry-validation.

As we agreed, each handler represents one endpoint with some set of parameters and it's payload. But while logic is right there, in the `handle` method, other stuff stays implicit. We can't immediately tell from code what is expected as an input.  The idea is to make our contract with client explicit and very much official by using `dry-schema` right in our handlers.

First, let's add `dry-validation` into our `Gemfile`:

```ruby
gem "dry-validation", "~> 1.10"
```

Now I'll add a few more methods into `AppplicationHandler`:

```ruby
# app/application_handler.rb

def self.payload(&block)
  if block_given?
    klass = Class.new(ApplicationContract) do
      json(&block)
    end

    @payload_contract = klass.new
  end

  @payload_contract
end

def self.params(&block)
  if block_given?
    klass = Class.new(ApplicationContract) do
      params(&block)
    end

    @params_contract = klass.new
  end

  @params_contract
end
```

Method `payload` helps us to define a contract for the payload we expect to come in request. It uses helper method `json` which takes care of type coercion for us. And method `params` does the same for URL parameters. It uses another helper conveniently called `params` which coerces data coming from URL query. You can read more about it [here](https://dry-rb.org/gems/dry-validation/1.10/schemas/#defining-a-schema-with-params-coercion) and [here](https://dry-rb.org/gems/dry-validation/1.10/schemas/#defining-a-schema-with-json-coercion).

Now we can go to one of the handlers and define what parameters and payload it expects:

```ruby
# app/products/handlers/update.rb

module Products
  module Handlers
    class Update < ApplicationHandler
      params do
        required(:product_id).filled(:integer, gt?: 0)
      end

      payload do
        required(:name).filled(:string)
        optional(:description).maybe(:string)
      end

      def handle
        product = Product.find(params[:product_id])
        product.update!(payload)

        render json: product.as_json(only: [:id, :name, :description])
      end

      private def payload
        params.permit(:name, :description)
      end
    end
  end
end
```

That helps with documentation issues, but sadly contracts are not applied and enforced, so it is easy to break what you agreed on. Let's add some more code into `ApplicationHandler` to actually use our definitions:

```ruby
# app/application_handler.rb

attr_reader :controller
attr_reader :params
attr_reader :payload
attr_reader :errors

def initialize(controller)
  @errors = []

  if self.class.params.present?
    result = self.class.params.call(controller.params.to_h)

    if result.failure?
      @errors << result.errors(full: true).to_h
    end

    @params = result.to_h
  end

  if self.class.payload.present?
    result = self.class.payload.call(controller.params.to_h)

    if result.failure?
      @errors << result.errors(full: true).to_h
    end

    @payload = result.to_h
  end

  @controller = controller
end

protected def render(**args)
  controller.render(**args)
end

protected def headers
  controller.headers
end
```

In the constructor we check if our class has contracts for `params` and `payload` defined. Both contracts if present are applied and the result is stored in an appropriate instance variable available as an attribute. In case of validation errors we store relevant messages in another instance variable which we can use later on. In addition we had to remove method `params` from `ApplicationHandler`. It is used to call `controller.params` but now it was replaced by the attribute.

There is one more problems to solve. By default Rails doesn't allow you to call `to_h` on `params` because it expects you to explicitly permit specific keys. But in our case contracts are taking care of validations and they are way-way more powerful than build-in tools. So, let's just get rid of that check:

```ruby
# app/application_controller.rb

class ApplicationController < ActionController::API
  before_action { params.permit! }
end
```

Handler can now be changed to utilize all of what has been done:

```ruby
# app/products/handlers/update.rb

module Products
  module Handlers
    class Update < ApplicationHandler
      params do
        required(:product_id).filled(:integer, gt?: 0)
      end

      payload do
        required(:name).filled(:string)
        optional(:description).maybe(:string)
      end

      def handle
        if errors.present?
          render json: { errors: errors }, status: 422
          return
        end

        product = Product.find(params[:product_id])
        product.update!(payload)

        render json: product.as_json(only: [:id, :name, :description])
      end
    end
  end
end
```

Let's hit our endpoint with some corrupted payload just to test it and see what happens:

```shell
curl --location --request PATCH '127.0.0.1:3000/products/7' \
--header 'Content-Type: application/json' \
--data '{
    "name": 100
}'
```

Here is what response with status `422 Unprocessable Entity` looks like:

```json
{
    "errors": [
        {
            "name": [
                "name must be a string"
            ]
        }
    ]
}
```

That's great! Contracts we defined in handler are not just there, they are actually being used and enforced. Checking errors in every handler is a bit noisy though, so let's move this part out to `ApplicationHandler` as well:

```ruby
# app/application_handler.rb

def handle!
  if errors.present?
    render json: { errors: errors }, status: 422
    return
  end

  handle
end
```

Now instead of calling directly `handle` method from specific handler controller will call `handle!` defined in `ApplicationHandler`. First it checks validation errors and if everything is fine it call down to the handler logic itself.

```ruby
# app/controllers/products_controller.rb

def update
  handler = Products::Handlers::Update.new(self)
  handler.handle!
end
```

The result is pretty much the same, but we don't need to remember about checking validation errors every time now. We just have to define a contract, everything else will work by itself:

```ruby
# app/products/handlers/update.rb

module Products
  module Handlers
    class Update < ApplicationHandler
      params do
        required(:product_id).filled(:integer, gt?: 0)
      end

      payload do
        required(:name).filled(:string)
        optional(:description).maybe(:string)
      end

      def handle
        product = Product.find(params[:product_id])
        product.update!(payload)

        render json: product.as_json(only: [:id, :name, :description])
      end
    end
  end
end

```

Here is another handler after adapting it to the latest changes:

```ruby
# app/products/handlers/index.rb

module Products
  module Handlers
    class Index < ApplicationHandler
      DEFAULT_LIMIT = 25
      DEFAULT_OFFSET = 0

      params do
        optional(:limit).filled(:integer, gt?: 0)
        optional(:offset).filled(:integer, gteq?: 0)
      end

      def handle
        products = Product.order(id: :desc)
          .limit(limit)
          .offset(offset)

        render json: {
          products: products.as_json(only: [:id, :name, :description])
        }
      end

      private def limit
        params.fetch(:limit, DEFAULT_LIMIT)
      end

      private def offset
        params.fetch(:offset, DEFAULT_OFFSET)
      end
    end
  end
end
```

## Conclusion

With just one gem added and a few lines of code written on top of it we managed to solve one of the most annoying problems in Rails applications. Important part here is that we have not moved too far away from default Rails path, so everything looks and feels familiar and can be easily traced through. No magic tricks, just simple code which one can read and understand in ~ 15 minutes.

Next topic we are going to talk about is authentication and authorization. In the next article I'm going to show how we can authenticate users and admins and check their permissions in handlers using quite simple but very powerful approach. Please, subscribe to stay tuned!