Background Jobs with Lucky
Tags: lucky, crystal
Published: 2019-02-14 23:19:28

---
title: Background Jobs with Lucky
published: true
description: Running some background task jobs with Lucky.
tags: lucky, crystal
---

As with a lot of web applications and APIs, there comes a point where you need to process some tasks in the background. To do this, you'll usually use a background job processor. This helps to make some things asynchronous and speed along the users of your app. 

An example of this might be logging when your users login to your site. Maybe you store the IP address they used, when they logged in, how many attempts it took to sign in, and the device they used. If you did all this in-line, then the process would be to track all that info, and the user has to wait while you create this new record in your database before they can actually login. Or, you push that info to a background job, and log the user in. Then you can update that record at your own pace since there's nothing relying on the data.

If you're new to [Crystal Lang](https://crystal-lang.org/) and/or you're new to [Lucky](https://luckyframework.org/), but you're coming from another language like Rails, you may be familiar with Sidekiq. Now, [Sidekiq](https://github.com/mperham/sidekiq.cr/) does exist in the Crystal world which is amazing. It even exists by the same guy that created the Ruby version! But at the time of this writing, I spent nearly an hour trying to figure out how to get it integrated with Lucky, and I was running in to quite a few issues. Thankfully, I found an alternative that I had installed and running locally in maybe 2 minutes.

## Mosquito

Take a look at [Mosquito](https://github.com/robacarp/mosquito). The API for it really isn't too far off from what Sidekiq uses, so for me, the concept was quick and easy. This also uses Redis, and since I was going to use Redis with Sidekiq, there's no additional things I needed to install.

Assuming you have your Lucky app all setup and ready to go, you can start by adding Mosquito in:

```yml
dependencies:
  mosquito:
    github: robacarp/mosquito
```

Run your `shards install`, and you'll be good to go.

Now we just need to setup our mosquito runner. Create a new file `./src/mosquito.cr`.


```crystal
require "./dependencies"
require "./workers/*"

Mosquito::Runner.start
```

## Setting up workers

Next we need to setup the workers. Create a new folder `./src/workers/`, and let's make a `login_worker.cr` file in there.


```crystal
class LoginWorker < Mosquito::QueuedJob
  params user_id : Int64
  params ip : String

  def perform
    user = UserQuery.find(user_id)
    if user
      # create new Login record from the user
    end
  end
end
```

## Using the worker

At some point we have to kick off the `LoginWorker` we made, so in one of the actions, we can add this:

```crystal
post "/login" do
  form = SessionForm.new(params)
  form.submit do |f, user|
    if user
      LoginWorker.new(user_id: user.id, ip: "127.0.0.1").enqueue
      # redirect logged in
    else
      # show your errors
    end
  end
end
```

## Booting the whole thing

Now that you have mosquito installed, and setup, you'll just need to boot the whole app. Since Mosquito runs in it's own process, you have to run it in a separate process.

In your `Procfile.dev` add in the line `worker: crystal src/mosquito.cr`.

```
web: lucky watch --reload-browser
assets: yarn watch
worker: crystal src/mosquito.cr
```

Now when you run `lucky dev`, it'll boot mosquito in it's on separate process! 


### ProTip:

1.  If your app is like mine, and just an api, your `Procfile.dev` might look more like

```
web: crystal src/server.cr
worker: crystal src/mosquito.cr
```

2. Mosquito needs to know about your stack since it runs in a separate process. If you're trying to slim this down, just keep in mind that whatever you put in your worker file, you'll need to require that separately from your app stack.

3. By default, Mosquito looks at your `REDIS_URL` ENV, or pulls just a normal localhost value. Be sure to update that for production, as well as your `Procfile` for your production.

4. If you would rather build the targets (maybe for docker), and run those separately, you can update your `shard.yml` with:

```
targets:
  server:
    main: src/server.cr
  mosquito:
    main: src/mosquito.cr
``` 

Then `shards build --production --static --release --no-debug` to get your `server` and `mosquito` binary files. Just run those however you need, and you should be good.