More Custom Validation Work: Manipulating a Keyword List To and From a Map
Tags: elixir, phoenix, learning, thisweekilearned
Published: 2020-06-30 15:37:21

Without getting too deep into the weeds, when storing and inserting these large CSV files, we are kind of meta programming the casting and validation aspect of the process, ultimately storing an individual row's errors in a `source_file_errors` table. Currently we were only storing the `message` from changeset. Up to this point this was working fine, because the default Ecto errors we were displaying were things like `"is not valid"`, and almost all of the more complex validations were custom, and therefore provided custom error messages. 

But now we are starting to utilize more of the available Ecto validations, like `validate_number`. When using `validate_number` and something like `less_than_or_equal_to`, Ecto produces a default message of `"must be less than or equal to %{number}"`. In this message `%{number}` is intended to be interpolated to the actual number found in the field. 

Normally, when dealing with Ecto out of the box, error messages are run through an `error_helpers.ex` module before being displayed in views. The helper takes in the `message` and `opts` from the changeset, determines if any interpolation needs to be performed, and outputs a message.

{% gist https://gist.github.com/noelworden/a754e438b0b719105cffbc5b66a89fb1 %}

But, because we are meta programming the error handling for this particular schema, a bit more manual work is needed before we can take advantage of the helper module. We need to store the `opts` field of the changeset in the `source_file_errors` table so that we can pass it, along with the stored `message`, to `error_helpers.ex`.

Sounded easy enough, spin up a migration to add an `opts` field to the already existing `source_file_errors` table. But the catch was what type to cast the new field as? The `changeset.opt` field is built out like this:

{% gist https://gist.github.com/noelworden/25f8b27cc2e50ed1e2d9bc563339c415 %}

It is a keyword list. Taking a look at the Ecto Primitive Types [documentation](https://hexdocs.pm/ecto/Ecto.Schema.html#module-primitive-types), there is an option for storing a casting a field as a list there's no explicit option for storing a keyword list. But, a keyword list _can_ be converted to a map with `Map.new/0`.

So, what needs to happen, is to take the keyword list from `changeset.opts`:

{% gist https://gist.github.com/noelworden/25f8b27cc2e50ed1e2d9bc563339c415 %}

Convert it to a map, with `Map.new/0` and store it in the database:

{% gist https://gist.github.com/noelworden/59c3dc569e3d62d8c6268a4d915e0078 %}

Once that gets stored in the database, the syntax shifts a bit, it's shaped like this:

{% gist https://gist.github.com/noelworden/cabcd19316dfedb3ceb5711ea9da7d12 %}

And when it gets called from the database, the syntax shifts yet again, and is shaped like this: 

{% gist https://gist.github.com/noelworden/c3ca7e42aecb9d00d4b09171f8e404c7 %}

Now, the `opts` field is being stored, but I found out it's not clear sailing after that. The `error_helpers.ex` has a `translate_error` function, which takes in `message` and `opts` as arguments, and uses `Gettext.dgettext` to output either a simple or interpolated message, whichever is necessary. What I realized is that the function is expecting the `opts` field to be in a specific shape, you guessed it, a keyword list with keys as atoms. One might assume that a keyword list that has been converted to a map can just be converted back to a keyword list in the same shape it originally was. Well, this is what happens when using the `Map.to_list/1`:

{% gist https://gist.github.com/noelworden/902184b37104960e02f9fa2a733f4f48 %}

And, if you try to run a keyword list of that shape through the `translate_error` function, it doesn't recognize the `number` key because it's not an atom, and therefore it outputs the uninterpolated message:

{% gist https://gist.github.com/noelworden/fb35a076e7c3235477e40d5aa513f88e %}

So, that means the map needs to be converted in a way not covered by the built-in functions. I did a little digging, and StackOverflow offered up this little gem:

{% gist https://gist.github.com/noelworden/aa7de97532f73452f96b0f2fc83c2496 %}

That converts the map to a keyword list, with the keys as atoms:

{% gist https://gist.github.com/noelworden/c56ea8591f6243cbc0335a81d8d4fc8d %}

And, now that `opts[:number]` is an atom, it will be recognized by `Gettext.dgettext/4`, and the message will be successfully interpolated:

{% gist https://gist.github.com/noelworden/6c07f5e2ce7c1e1f605d1e163ebeed0d %}

Now that all the pieces are there, a custom function can be added to a `error_helpers.ex` module to gather the data, convert the list, and output the error message utilizing the existing functions in that module. 

{% gist https://gist.github.com/noelworden/455f481f500117b5ee1218507f5d46cb %}

Although it can be seen as less than ideal, I always find it interesting when forced to step outside of a language's scaffold. One can become a bit complacent when slapping that `<%= error_tag =>` helper around forms and magically seeing the appropriate error message appear. Although it seemed a bit daunting at first, I'm happy I was able to take this deep dive into the guts of the error message handling. 

------

This post is part of an ongoing *This Week I Learned* [series](https://dev.to/noelworden/beginning-of-a-blog-series-5aj3). I welcome any critique, feedback, or suggestions in the comments.