URL shortener with Elixir and Phoenix
Tags: elixir, phoenix, postgres, webdev
Published: 2020-09-10 20:55:57

One evening I was thinking a topic for a small Elixir + Phoenix project and came up with an idea of implementing URL shorten-er with Elixir and Phoenix. The implementation is quite easy and quick, and overall the application is going to be very small.

The technology stack consists following technologies, `Elixir`, `Phoenix framework` and `PostgreSQL`. This stack is run in docker containers.

If you want to follow through this article I have public [GitLab repository](https://gitlab.com/hlappa/url-shortener) which contains the application.

## Features

I started defining key user paths:

* User is able to create new link
* User is able to navigate to created link and redirected to original URL
* User is able to review statistics of the created link

These 3 paths are pretty trivial to implement, so lets dive in to it!

## Implementation

I'm not going to go through the initial project set-up in this article. You can do so [here](https://dev.to/hlappa/development-environment-for-elixir-phoenix-with-docker-and-docker-compose-2g17)!

### Resource

The only resource we are going to need is Links. We can create one running the following command inside our docker container. If you didn't follow the earlier link for setting up the local docker + docker-compose environment ignore the docker-compose related commands.

```
$ docker-compose run web mix phx.gen.html Links Link links
```

This command will create `Links` and `Link` modules, Link-controller, migrations and test files.

A lot of files are generated when creating new resource, but take your time to go through the files. You can read more information of the `phx.gen.html` [here](https://hexdocs.pm/phoenix/Mix.Tasks.Phx.Gen.Html.html#content).

### Migration and schema

In previous step we generated resource for our application. The `mix phx.gen.html` accepts fields as parameter for the database table etc. You can put them there or define the fields manually in migration and schema file.

#### Migration

Our migration file looks pretty neat. The only notable change is that we are saying "Don't create primary key" for the `links`-table, since we are defining the `id`-field to be the primary key of the `links` table and setting its length to 8-digits.

```elixir
defmodule Shorturl.Repo.Migrations.CreateLinks do
  use Ecto.Migration

  def change do
    create table(:links, primary_key: false) do
      add :id, :string, size: 8, primary_key: true
      add :url, :text, null: false
      add :visits, :integer, default: 0

      timestamps()
    end
  end
end
```

#### Schema and changeset

Here we define the schema for our model/struct. Pretty basic stuff, but we need to define that we are not using the default primary key.

```elixir
defmodule Shorturl.Links.Link do
  use Ecto.Schema
  import Ecto.Changeset

  @primary_key {:id, :string, []}
  schema "links" do
    field :url, :string
    field :visits, :integer

    timestamps()
  end

...

end
```
Below the schema block we have our changeset function with a custom validation for the user defined original URL.

```elixir
defmodule Shorturl.Links.Link do
  use Ecto.Schema
  import Ecto.Changeset

...

  @doc false
  def changeset(link, attrs) do
    link
    |> cast(attrs, [:id, :url, :visits])
    |> validate_required([:id, :url])
    |> validate_url(:url)
  end

  def validate_url(changeset, field, options \\ %{}) do
    validate_change(changeset, field, fn :url, url ->
      uri = URI.parse(url)

      if uri.scheme == nil do
        [{field, options[:message] || "Please enter valid url!"}]
      else
        []
      end
    end)
  end
end

```

The `validate_url/3` checks if the original URL is an URL with scheme (http, https). If not, error message is attached to the `:url` field of the changeset.

### Routes

If you are not familiar with the Phoenix routing, I suggest you to read [this](https://hexdocs.pm/phoenix/routing.html).

When creating a new Phoenix project, there is already some routes defined in `routes.ex`. The LiveView routes for metrics dashboard are enabled only in `dev` and `test`  environments, we can leave it there. The initial route for PageController can be removed, since we are only using one controller in this application, `LinkController`.

At this point we need following routes

* Path to show the form of the new shortened URL
* Path where to post the form
* Path which redirects the user to original URL
* Path where the stats of the shortened URL are shown

```elixir
defmodule ShorturlWeb.Router do
  use ShorturlWeb, :router

  pipeline :browser do
    plug :accepts, ["html"]
    plug :fetch_session
    plug :fetch_flash
    plug :protect_from_forgery
    plug :put_secure_browser_headers
  end

  scope "/", ShorturlWeb do
    pipe_through :browser

    get "/", LinkController, :new
    post "/links", LinkController, :create
    get "/:id", LinkController, :redirect_to
    get "/:id/stats", LinkController, :show
  end

  ...
end
```

Now we have defined our routes for our application, wonderful! Each `atom` after `LinkController` is the action where the request is passed to.

### LinkController

#### New

This is the action which is responsible for rendering the front page of the application. The form helper in the view needs `Link` changeset to be available. If you look the corresponding template file (`new.html.eex`), you can see that we also pass `@action` variable for the form-template. This is used to tell the form helper where to send all the form data.

```elixir
defmodule ShorturlWeb.LinkController do
  use ShorturlWeb, :controller

  alias Shorturl.Links
  alias Shorturl.Links.Link

  def new(conn, _params) do
    changeset = Links.change_link(%Link{})
    render(conn, "new.html", changeset: changeset)
  end

...

end

```

#### Create

All the magic happens here what comes to the creation of the link. First we need to create random 8-digit identifier for the link, this is the earlier mentioned primary key for the database entry. We put the identifier to our parameter map and try to create the link. If the link is successfully created and persisted in database, we redirect the user to `:show`-action, otherwise we just render the front page (`:new`-action) with the Link changeset which holds the possible errors and display them in the UI.

```elixir
defmodule ShorturlWeb.LinkController do
  use ShorturlWeb, :controller

  alias Shorturl.Links
  alias Shorturl.Links.Link

...

  def create(conn, %{"link" => link_params}) do
    case create_link(link_params) do
      {:ok, link} ->
        conn
        |> put_flash(:info, "Link created successfully.")
        |> redirect(to: Routes.link_path(conn, :show, link))

      {:error, changeset} ->
        render(conn, "new.html", changeset: changeset)
    end
  end


...

defp create_link(link_params) do
    key = random_string(8)
    params = Map.put(link_params, "id", key)

    try do
      case Links.create_link(params) do
        {:ok, link} ->
          {:ok, link}

        {:error, %Ecto.Changeset{} = changeset} ->
          {:error, changeset}
      end
    rescue
      Ecto.ConstraintError ->
        create_link(params)
    end
  end

...

  defp random_string(string_length) do
    :crypto.strong_rand_bytes(string_length)
    |> Base.url_encode64()
    |> binary_part(0, string_length)
  end
end
```

Edit:
I totally forgot to handle situation where the randomly generated ID is already taken by some other URL. I updated the above code to handle also this case with a nice recursive function! I abstracted the actual business logic of this case to it's own function, so the `create` controller function stays clean and readable. Thank you [Roberto Amorim](https://dev.to/rhamorim)!

#### Show

This is purely just displaying the information of the shortened link. Original link, shortened URL and visits are shown in the view. In the controller we fetch the Link with the `id` from our path parameters and pass the link for the view. If the query results in `Ecto.NoResultsError` the user is redirected to front page with error flash message. [Flash message documentation](https://hexdocs.pm/phoenix/controllers.html#flash-messages). We also pass domain to the view, this is used for prefixing the shortened URL.

```elixir
defmodule ShorturlWeb.LinkController do
  use ShorturlWeb, :controller

  alias Shorturl.Links
  alias Shorturl.Links.Link

...

  def show(conn, %{"id" => id}) do
    try do
      link = Links.get_link!(id)
      domain = System.get_env("APP_BASE_URL") || nil
      render(conn, "show.html", link: link, domain: domain)
    rescue
      Ecto.NoResultsError ->
        conn
        |> put_flash(:error, "Invalid link")
        |> redirect(to: Routes.link_path(conn, :new))
    end
  end

...

end
```

### Redirect_to

This is the action which is responsible for redirecting the users to original URL when they click a shortened URL/link somewhere. The structure is pretty much the same as in `:show`, but we need to update the visits count for the associated link. The `Task.start/1` is used for side-effects in Elixir applications and we don't care what the task returns. Also the function call in the task is run on its own process, so it doesn't block the execution of the following code.

```elixir
defmodule ShorturlWeb.LinkController do
  use ShorturlWeb, :controller

  alias Shorturl.Links
  alias Shorturl.Links.Link

...

  def redirect_to(conn, %{"id" => id}) do
    try do
      link = Links.get_link!(id)
      # Start task for side-effect
      Task.start(fn -> update_visits_for_link(link) end)
      redirect(conn, external: link.url)
    rescue
      Ecto.NoResultsError ->
        conn
        |> put_flash(:error, "Invalid link")
        |> redirect(to: Routes.link_path(conn, :new))
    end
  end

...

  defp update_visits_for_link(link) do
    Links.update_link(link, %{visits: link.visits + 1})
  end

...

end

```

### Templates and styling

I'm not going through styling and templates in this article, but you can look for them in the GitLab repository which were linked earlier in this article.

### Testing

There is small portion of tests done in the repository, go look it up! For the large part they are generated by the `mix`-task and modified to meet the functionalities of our application.

### Are we done? No.

Lets say that the application gets huge amount of links daily and you persist all of them in your database. At some point your DB will blow up. We need to clean old URLs away which are not used any more, sounds fair?

To begin with the implementation, I took [Quantum](https://github.com/quantum-elixir/quantum-core) library and configured it according the documentation. This will be responsible for scheduling jobs and running them.

We want to delete all the links which are not visited during the last 7 days and we want to run this cleaning job daily.

```yaml
config :shorturl, Shorturl.Scheduler,
  jobs: [
    {"@daily", {Shorturl.Clean, :delete_old_links, []}}
  ]
```
```elixir
defmodule Shorturl.Clean do
  alias Shorturl.Links
  require Logger

  def delete_old_links do
    Logger.info("Running job for old links")
    {count, _} = Links.delete_all_old()
    Logger.info("Removed #{count} old links")
  end
end
```
In `links.ex`
```elixir
  def delete_all_old() do
    from(l in Link, where: l.updated_at < ago(7, "day")) |> Repo.delete_all()
  end
```
Just like that, with a few lines of code the database is cleaned daily. The Ecto query API is just wonderful and we don't have to delete the links one by one when using `delete_all/1`.

## Conclusion

This was a nice tiny project which was fun and quick to implement, no biggie. I tried to make this article beginner friendly as possible!

The next step is to deploy this application to AWS with some GitLab CI/CD and Terraform magic!