Cache the data using Redis in Elixir/Phoenix
Tags: elixir, redis, github
Published: 2020-03-01 12:25:10

Recently I stumble upon a problem where I had to cache the data which I was getting from the database. But for this, you need to have a little bit of understanding of genservers and supervisors. 

Let's start with Redis and why we need it? 

The distributed cache is a widely-adopted design pattern for building robust and scalable applications. By serving data from the cache, and instead of fetching it from slower devices or re-computing it, the application’s overall performance is improved. The cache also relieves
some of the load on external data sources as well as improves the availability of the application during outages.

Redis is the most popular distributed caching engine today. It is a production-proven and provides a host of capabilities that make it the ideal distributed caching layer for applications. Redis Cloud and RLEC provide additional capabilities that extend the abilities of open source
Redis to make it even more suitable for caching purposes.

Redix

Redix provides the main API to interface with Redis. We will use this library to interact with redis server. But we'll get to this later. 

    def start_link(url) do
       GenServer.start_link(__MODULE__, {url})
    end

start_link here starts a process that connects to Redis. Each Elixir process started with this function maps to a client TCP connection to the specified Redis server.

  
    def init({url}) do
      Logger.info("connect to url #{url}");
      case Redix.start_link(url) do
      {:ok, conn} -> {:ok, conn}
      {:error, err} -> {:error, err}
      end
    end

So here start_link function will invoke the init function. Now just run this 

    iex -S mix phx.server
    Database.start_link("redis://127.0.0.1:6379")

    iex(2)> Database.start_link("redis://127.0.0.1:6379")
    [info] connect to url redis://127.0.0.1:6379
    {:ok, #PID<0.448.0>}
    iex(3)> {:ok, pid} =  Database.start_link("redis://127.0.0.1:6379") 
    [info] connect to url redis://127.0.0.1:6379
    {:ok, #PID<0.452.0>}

Now here you're doing a pattern matching. On the left side, it is a tuple that has the first argument as atom :ok and second has a variable. If you check the variable it would return a process id. Now you're connected to Redis server and you have a process running with that pid. 

Now to set a key-value command in Redix. You have to use some binaries if you're not using one of those binaries it will suggest to you use those binaries. Now you're confused here about the binaries let me tell you the Elixir Compiler expects certain parameters to pass in case if you miss something it would tell you to use those binaries.

Okay let's get back to code so it would suggest you to something like this 

     Redix.command!(conn, ["SET", "mykey", "foo"])
     "OK"

Now I wanted to set a key and value. But I wanted the value to be the data that I was fetching from the database. But here the problem is the data which I was getting database was a map. So I figured if I could somehow  set a normal key and value then I will resolve the other problem. 

So here what I did

     def set(conn, key, value) do
       GenServer.call(conn, {:set, key, value})
     end

     def handle_call({:set, key, value}, _from, state ) do
       reply = Redix.command(state, ["SET", key, value])
       {:reply, {:ok, reply}, state}
     end

What I did here is I have a set function that is expecting a conn, key, and value which you want to set. The set function will invoke a handle_call function which is where the main logic happens and you will able to set a key and value. That's the beauty of genserver because it let you maintain the state. If you run this it would give you something like this

    iex(6)> Database.set(pid, "onetwo", "three")
    # {:ok, {:ok, "OK"}}


But now what I want is to fetch a database from the database for which I want to create a function which will take a pid as first argument and id which I wanted to fetch. 

    def userone(pid, id) do
      GenServer.call(pid, {:userone, id})
    end

So I created a function that expects that argument and will invoke the handle_call function.

    def handle_call({:userone, id}, _from, state) do
      user = Accounts.get_user!(id)
      {:reply, user, state}
    end

Now this will get the single user from the database. Just call this function and it would return a map. But our set function expects an only string, not a map. So we need to figure out the way to convert a map to a string. 

There are many libraries that can help you to convert the map to string but I used the Poison module which you have to add under your dependencies. 

    def handle_call({:set, key, value}, _from, state) when is_map(value) do
     value = Poison.encode!(value)
     reply = Redix.command(state, ["SET", key, value])
     {:reply, {:ok, reply}, state}
    end

So I did exactly that so I used an Elixir guard function and if the value is map go-ahead and converts that map to a string. Now if you see it will convert that map to string and will save it to Redis.

Now if you want to see how can you get the data which you've just set. Don't worry there is a function for that. But first, let's see how it works in redix. 

     Redix.command(conn, ["GET", "mykey"])

Using this command I created another function that will just get the key which you've just set.

     # get the key function
     def get(pid, key) do
       GenServer.call(pid, {:get, key})
     end

     # handle call for get function
     def handle_call({:get, key}, _from, state) do
       reply = Redix.command(state, ["GET", key])
       {:reply, {:ok, reply}, state}
     end

This is how it will look like in the end

     # iex(6)> Database.set(pid, "onetwo", "three")
     # {:ok, {:ok, "OK"}}
     # iex(7)> Database.get(pid, "onetwo")
     # {:ok, {:ok, "three"}}
     # iex(8)>



Some useful information about redix connection:- 

Redix tries to be as resilient as possible: it tries to recover automatically from most network errors.

If there's a network error when sending data to Redis or if the connection to Redis drops, Redix tries to reconnect. The first reconnection attempt will happen after a fixed time interval; if this attempt fails, reconnections are attempted until successful, and the time interval between reconnections is increased exponentially.


Here how it will look like in the end. 

    defmodule Gorm.Database do
      use GenServer
      require Logger
      alias Gorm.Accounts

      # "redis://localhost:6379/3"

      #connect the redis server
      def start_link(url) do
        GenServer.start_link(__MODULE__, {url})
      end

     def init({url}) do
        Logger.info("connect to url #{url}");
        case Redix.start_link(url) do
        {:ok, conn} -> {:ok, conn}
        {:error, err} -> {:error, err}
        end
     end

     # checking the connection if it's connected to redis or not
    def check(pid) do
       GenServer.call(pid, :check)
    end

     # handle call for check function
    def handle_call(:check, _from, state) do
      checking = Redix.command!(state, ["PING"])
      {:reply, checking, state}
    end

    # get the key function
    def get(pid, key) do
      GenServer.call(pid, {:get, key})
    end

    # handle call for get function
    def handle_call({:get, key}, _from, state) do
      reply = Redix.command(state, ["GET", key])
      {:reply, {:ok, reply}, state}
    end

    # set the key function
    def set(conn, key, value) do
      GenServer.call(conn, {:set, key, value})
    end

    #handle call for set function
    def handle_call({:set, key, value}, _from, state ) do
      state = Exq.enqueue(Exq, "q1", SetWorker, [key, value])
      {:reply, state, state}
    end

    def handle_call({:set, key, value}, _from, state) when is_map(value) do
      value = Poison.encode!(value)
      reply = Redix.command(state, ["SET", key, value])
      {:reply, {:ok, reply}, state}
    end

     # Get the single user from the database
     def userone(pid, id) do
       GenServer.call(pid, {:userone, id})
     end

     # Get the list of user from the database
     def list(pid) do
       GenServer.call(pid, :list)
     end

     # handle call for the list function
     def handle_call(:list, _from, state) do
       my_models = Accounts.list_users()

      {:reply, my_models, state}
     end

     # handle call for the single user function
      def handle_call({:userone, id}, _from, state) do
        user = Accounts.get_user!(id)
        {:reply, user, state}
      end
    end

If you want to see a full code and how I've implemented it please go to my github. 

   github.com/siddhant3030/Micro/gorm