A Deep Dive Into Laravel Queues
Tags: php, laravel
Published: 2022-10-26 07:00:00

*This article was originally written by [Farhan Hasin Chowdhury](https://www.honeybadger.io/blog/authors/farhanhasinchowdhury/) on the [Honeybadger Developer Blog](https://www.honeybadger.io/blog/laravel-queues-deep-dive/).*

Web applications and software are generally satisfactory when they are fast. However, some tasks take time, such as sending an automated email or creating a detailed report by analyzing thousands of data points. Has a website ever made you wait on a loading screen while it sent a verification email to you? Probably not, because most web applications, at least the good ones, perform long-running tasks asynchronously in the background, which is what [Laravel Queues](https://laravel.com/docs/8.x/queues) allows you to do.

In this article, you'll learn how to process jobs asynchronously using queues and workers in Laravel. This article assumes that you have PHP, Composer, MySQL and Redis installed on your system, and you know how to bootstrap and start a new Laravel project.

## Table of Contents

- [A Complete Guide to Queues and Workers in Laravel](#a-complete-guide-to-queues-and-workers-in-laravel)
  - [Table of Contents](#table-of-contents)
  - [Introduction to Jobs, Queues, and Workers](#introduction-to-jobs-queues-and-workers)
    - [Queue Back-ends](#queue-back-ends)
    - [Queue Workers](#queue-workers)
  - [Job Classes and Dispatch Modes](#job-classes-and-dispatch-modes)
    - [Dispatch Modes](#dispatch-modes)
  - [Connections and Queues](#connections-and-queues)
    - [Working with Multiple Connections](#working-with-multiple-connections)
    - [Working with Multiple Queues](#working-with-multiple-queues)
    - [More Common Configuration Options](#more-common-configuration-options)
  - [Job Failures and Retries](#job-failures-and-retries)
    - [Automatic Retries](#automatic-retries)
    - [Automatic Retry Interval](#automatic-retry-interval)
    - [Catching Job Failures](#catching-job-failures)
  - [Chains and Batches](#chains-and-batches)
    - [Chains](#chains)
    - [Batches](#batches)
  - [Laravel Horizon](#laravel-horizon)
  - [Running Workers in Production](#running-workers-in-production)
    - [Running Laravel Horizon in Production](#running-laravel-horizon-in-production)
  - [Conclusion](#conclusion)

## Introduction to Jobs, Queues, and Workers

Before diving into code, I would like to present you with a real-life analogy of doing things asynchronously. Imagine that you're the owner of a local grocery shop that takes orders by phone and then delivers the goods to the customer's doorstep.

Now, imagine that you get a phone call from a customer and take a new order. You then go to the storeroom, look for the individual products, put them in a box, and send them out for delivery. When using this approach, you cannot receive any new orders until you send out the current one. This is synchronous, and it blocks your I/O until you finish processing a long-running task.

Instead of taking a synchronous approach, you could hire an additional __worker__ for the storeroom. This way, you can take a new order, note it down on a piece of paper (let's call it a __job__), and put the order in a __queue__.

One of the storeroom workers will pick one of the queued jobs, prepare the order, and send it out for delivery. The worker will then return to the queue, pick the next job, and start processing it. This is an asynchronous approach. Although hiring additional workers will require more resources, you'll be able to process more orders without blocking the I/O, resulting in better performance and customer satisfaction.

Now, let's see how you could implement the idea of a job, a worker, and a queue in Laravel. I hope you have already created a new Laravel project somewhere on your computer. Open the project in your favorite IDE and update the code for the `routes/web.php` file as follows:

```php
Route::get('/', function () {

    dispatch(function() {
        sleep(5);
        logger('job done!');
    });

    return view('welcome');
});
```

The `dispatch()` helper function in Laravel sends a new job to a queue. Here, a job is regular PHP code in the form of a closure or class. You'll use a closure for now, but I'll introduce you to job classes very soon.

To simulate a long-running task, I've put an explicit delay of five seconds using the sleep function in the job closure. Then, I've used the `logger()` helper function to print the line "job done!" in the project's log file.

Start the project by executing the `php artisan serve` command or any other method you like and visit the `/` route. You'll see the browser hanging in a loading state for five seconds, and then the welcome page will show up. If you view the `storage/logs/laravel.log` file at this point, you'll see the line "job done!" printed there.

```log
[2021-08-23 14:37:50] local.DEBUG: job done!
```

This means that the job ran successfully after the five-second delay, but there was a problem. The job was supposed to run asynchronously without blocking the I/O, but as you just observed, the browser was in a loading state for five seconds.

To understand why this happened, open up the project's `.env` file and look for the `QUEUE_CONNECTION` variable. This variable indicates the connection to the backend queue service in use. By default, Laravel sets the value of this variable to `sync`, which means that the framework will process all the jobs synchronously.

### Queue Back-ends

To make asynchronous job-processing possible, you'll have to use a different backend. You can get a list of the pre-configured back-ends inside the `config/queue.php` file:

```php
    'connections' => [

        'sync' => [
            'driver' => 'sync',
        ],

        'database' => [
            'driver' => 'database',
            'table' => 'jobs',
            'queue' => 'default',
            'retry_after' => 90,
            'after_commit' => false,
        ],

        'beanstalkd' => [
            'driver' => 'beanstalkd',
            'host' => 'localhost',
            'queue' => 'default',
            'retry_after' => 90,
            'block_for' => 0,
            'after_commit' => false,
        ],

        'sqs' => [
            'driver' => 'sqs',
            'key' => env('AWS_ACCESS_KEY_ID'),
            'secret' => env('AWS_SECRET_ACCESS_KEY'),
            'prefix' => env('SQS_PREFIX', 'https://sqs.us-east-1.amazonaws.com/your-account-id'),
            'queue' => env('SQS_QUEUE', 'default'),
            'suffix' => env('SQS_SUFFIX'),
            'region' => env('AWS_DEFAULT_REGION', 'us-east-1'),
            'after_commit' => false,
        ],

        'redis' => [
            'driver' => 'redis',
            'connection' => 'default',
            'queue' => env('REDIS_QUEUE', 'default'),
            'retry_after' => 90,
            'block_for' => null,
            'after_commit' => false,
        ],

    ],
```

As you can see, all Laravel projects come with five predefined back-end configurations. I'll introduce you to all of them, but for simplicity, let's use the `database` connection first. To do so, open the project's `.env` file and change the value of `QUEUE_CONNECTION` to `database` from `sync` and save the file.

```dotenv
QUEUE_CONNECTION=database
```

Next, execute the `php artisan queue:table` command to generate the migration script for creating the `jobs` table and migrate the database by running the `php artisan migrate` command. This table will hold all the queued jobs until a worker processes them. Now, clear the `storage/logs/laravel.log` file and view the `/` route.

This time, you'll see no delay in code execution, and the page will render almost immediately, indicating that the job is running in the background. After waiting for five seconds, if you check the `storage/logs/laravel.log` file, you'll find it empty.

### Queue Workers

At this point, you may think that there must be something wrong, while the reality is that everything's working as expected. If you use a database client tool and look at the jobs table in your database, you'll see that the framework has indeed pushed a new job to the queue, but there are no workers to process it.

![Unprocessed Job In Database](https://www.honeybadger.io/images/blog/posts/laravel-queues-deep-dive/unprocessed-job-in-database.jpg)

A queue worker is a regular process that runs in the background and polls the queue backend for unprocessed jobs. To start a new worker, execute the `php artisan queue:work` command inside your project directory. The worker will start and begin processing the unprocessed job immediately.

```bash
[2021-08-23 15:30:34][1] Processing: Closure (web.php:18)
[2021-08-23 15:30:39][1] Processed:  Closure (web.php:18)
```

Look at the `storage/logs/laravel.log` file, and you'll see the "job done!" line printed again. If you use a job class instead of a closure, the worker will print the job name instead of the word closure in the output. To process jobs automatically in the future, you'll have to keep the worker running. I'll show you how to keep the process running in the background in a later section.

## Job Classes and Dispatch Modes

Now that you're familiar with fundamental concepts like jobs, queues, and workers, it's time to create your first job class. You can do this by executing the `php artisan make:job <name>` command. Create a new job and name it something like `SendVerificationEmail` for now.

```bash
php artisan make:job SendVerificationEmail

# Job created successfully.
```

The job classes in Laravel reside inside the `app/Jobs` directory. Open the `app/Jobs/SendVerificationEmail.php` file in your IDE.

```php
<?php

namespace App\Jobs;

use Illuminate\Bus\Queueable;
use Illuminate\Contracts\Queue\ShouldBeUnique;
use Illuminate\Contracts\Queue\ShouldQueue;
use Illuminate\Foundation\Bus\Dispatchable;
use Illuminate\Queue\InteractsWithQueue;
use Illuminate\Queue\SerializesModels;

class SendVerificationEmail implements ShouldQueue
{
    use Dispatchable, InteractsWithQueue, Queueable, SerializesModels;

    /**
     * Create a new job instance.
     *
     * @return void
     */
    public function __construct()
    {
        //
    }

    /**
     * Execute the job.
     *
     * @return void
     */
    public function handle()
    {
        //
    }
}
```

Every job class implements the `ShouldQueue` interface and comes with `constructor` and `handle()` methods. The class also comes with four traits by default: `Dispatchable`, `InteractsWithQueue`, `Queueable`, and `SerializesModels`. You'll learn about these later on.

As you may have already guessed, the `handle()` method is where the magic happens. So, go ahead and update the `handle()` method as follows:

```php
    /**
     * Execute the job.
     *
     * @return void
     */
    public function handle()
    {
        sleep(5);
        logger('email sent!');
    }
```

Now that you've moved the job code from the `routes/web.php` file, you'll have to update the route closure to use the job class. Open the `routes/web.php` file and update its code as follows:

```php
<?php

Route::get('/', function () {
    dispatch(new \App\Jobs\SendVerificationEmail());

    return view('welcome');
});
```

The change is simple. Instead of passing a closure to the `dispatch()` helper method, you now pass an instance of the `SendVerificationEmail` class. Alternatively, you can also dispatch a job using the following syntax:

```php
Route::get('/', function () {
    \App\Jobs\SendVerificationEmail::dispatch();

    return view('welcome');
});
```

The `dispatch()` method on the class itself comes from the `Dispatchable` trait that I mentioned earlier. Apart from the regular `dispatch()` method, the trait provides a few other methods allowing you to process a job in different ways:

- `dispatchIf()` — Dispatch the job with the given arguments if the given truth test passes.
- `dispatchUnless()` — Dispatch the job with the given arguments unless the given truth test passes.
- `dispatchSync()` — Dispatch a command to its appropriate handler in the current process.
- `dispatchAfterResponse()` — Dispatch a command to its appropriate handler after the current process.
- `withChain()` — Set the jobs that should run if this job is successful.

I've taken these short descriptions directly from the trait's source code. Let's see a few of these methods in action.

### Dispatch Modes

The `dispatchIf()` method, for example, lets you dispatch a job based on a conditional. Update the code of the `routes/web.php` file as follows:

```php
Route::get('/', function () {
    $userSignedUp = false;

    \App\Jobs\SendVerificationEmail::dispatchIf($userSignedUp);

    return view('welcome');
});
```

One thing to remember is that __you must restart the queue worker whenever you make some changes to your job code__; otherwise, the queue will not pick up the changes and will run the previous code. Now, if you revisit the `/` route, you'll see no new jobs running. However, if you change the value of the `$userSignedUp` variable to `true` and revisit the `/` route, the job will run immediately. Just like the `dispatchIf()` method, the `dispatchUnless()` method lets you run a job based on a conditional. The difference is that the `dispatchUnless()` method only dispatches a job if the given conditional evaluates to `false`, such as sending a verification email unless a user signs up (which is a weird condition). To test this method, keep the value of the `$userSignedUp` variable set as `true` and replace the `SendVerificationEmail::dispatchIf($userSignedUp);` line with `SendVerificationEmail::dispatchUnless($userSignedUp);`. Save the file, revisit the `/` route, and you will not see any new jobs running.

The `dispatchSync()` and `dispatchAfterResponse()` methods let you run a job synchronously. The difference between the two methods is that the `dispatchSync()` method runs the job immediately, whereas the `dispatchAfterResponse()` method first returns the response to the user and then runs the job before closing the connection. Neither of these methods require a worker to be running in the background, and you can use them to run short-running jobs.

The `withChain()` method lets you run a chain of additional jobs if the current job succeeds. I'll discuss the concept of chains and batches in a later section. You may find it daunting to memorize all these methods, and the truth is you don't have to. You can look at the source code of the `Illuminate\Foundation\Bus\Dispatchable` trait or visit the Laravel API [reference](https://laravel.com/api/8.x/Illuminate/Foundation/Bus/Dispatchable.html) whenever you want.

## Connections and Queues

You've already seen in a previous section that Laravel comes with several pre-configured back-end connections, and you've been using the `database` connection so far. In this section, you'll learn a bit more about the other ones, as well as go deeper into queues. To begin, let's go back to the `config/queue.php` file again and have a look at the available connection configurations.

```php
    'connections' => [

        'sync' => [
            'driver' => 'sync',
        ],

        'database' => [
            'driver' => 'database',
            'table' => 'jobs',
            'queue' => 'default',
            'retry_after' => 90,
            'after_commit' => false,
        ],

        'beanstalkd' => [
            'driver' => 'beanstalkd',
            'host' => 'localhost',
            'queue' => 'default',
            'retry_after' => 90,
            'block_for' => 0,
            'after_commit' => false,
        ],

        'sqs' => [
            'driver' => 'sqs',
            'key' => env('AWS_ACCESS_KEY_ID'),
            'secret' => env('AWS_SECRET_ACCESS_KEY'),
            'prefix' => env('SQS_PREFIX', 'https://sqs.us-east-1.amazonaws.com/your-account-id'),
            'queue' => env('SQS_QUEUE', 'default'),
            'suffix' => env('SQS_SUFFIX'),
            'region' => env('AWS_DEFAULT_REGION', 'us-east-1'),
            'after_commit' => false,
        ],

        'redis' => [
            'driver' => 'redis',
            'connection' => 'default',
            'queue' => env('REDIS_QUEUE', 'default'),
            'retry_after' => 90,
            'block_for' => null,
            'after_commit' => false,
        ],

    ],
```

As you can see, the `connections` array contains individual configurations for the different queue back-ends. These configurations have distinct options, but the `driver`, `queue`, and `after_commit` options are the most commonly used.

People often get confused about the difference between the `driver` and the connection name. The connection name can be anything. Just because a queue connection uses a database as its back-end, you don't have to name it `database`. You can name it anything you want. The driver, however, must match one of the names the framework ships.

Now, does this mean you're limited to just these five driver options? What if you want to use RabbitMQ as the queue back-end? In such cases, Laravel allows you to use third-party queue drivers. Take the very popular [vyuldashev/laravel-queue-rabbitmq](https://github.com/vyuldashev/laravel-queue-rabbitmq), for example. It lets you use RabbitMQ as a queue back-end like the built-in ones.

### Working with Multiple Connections

I've already explained how to set the default queue connection on the project's environment `.env` file using the `QUEUE_CONNECTION` variable, but what if you want to have multiple connections? You can do that as well. By default, when you dispatch a job, the framework will push it into the default queue connection. However, if you want to push a job into another queue connection, such as redis, you can use the `onConnection()` method.

```php
Route::get('/', function () {
    \App\Jobs\SendVerificationEmail::dispatch()
    ->onConnection('redis');

    return view('welcome');
});
```

### Working with Multiple Queues

Apart from multiple connections, there can also be different queues. The `queue` option indicates the default queue for a connection. Thus, if you dispatch a job without specifying a queue, the framework will push the job into the default queue. Imagine a scenario where you have jobs that must be processed before any of the others, regardless of which one comes first. To do so, update the route closure as follows:

```php
Route::get('/', function () {
    \App\Jobs\SendVerificationEmail::dispatch();

    \App\Jobs\SendVerificationEmail::dispatch()
    ->onQueue('high-priority');

    return view('welcome');
});
```

You are dispatching the same job twice: on the default queue and then on the high priority queue. The name `high-priority` is made-up, and you can name a queue whatever you want. First, start the application and visit the `/` route without starting any workers. Then, look at the `jobs` database table.

![Jobs On Multiple Queues](https://www.honeybadger.io/images/blog/posts/laravel-queues-deep-dive/multiple-queues.jpg)

As you can see, the first job is in the default queue, and the second one is in the high-priority queue. Dispatching a job in the high priority queue will not tell the framework which job to process first. You'll have to change the `'queue' => 'default'` line for the database connection to `'queue' => 'high-priority,default'` inside the `config/queue.php` file. Now, start a queue worker by executing the `php artisan queue:work` command, and the worker will prioritize the queue that comes first in the configuration value.

```bash
[2021-08-28 15:25:26][2] Processing: App\Jobs\SendVerificationEmail
[2021-08-28 15:25:31][2] Processed:  App\Jobs\SendVerificationEmail
[2021-08-28 15:25:31][1] Processing: App\Jobs\SendVerificationEmail
[2021-08-28 15:25:36][1] Processed:  App\Jobs\SendVerificationEmail
```

Indeed, the worker processes job 2 first and then processes job 1 because it has a low priority. Instead of setting the queue priority in the `config/queue.php` file, you can set it in the `queue:work` command by writing it as `php artisan queue:work --queue="high-priority,default"`, which will have the same effect.

### More Common Configuration Options

The third most common option among all the configurations is the `after_commit` option. If you dispatch a job within a database transaction, it's possible that while the job is running, the framework has not yet committed changes to the database. In such situations, especially if the job depends on values from the database, things can get out of hand. Setting the `after_commit` option to `true` will ensure that the framework dispatches the job only after committing database changes. You can set this option globally or for each job, which I'll teach you in a later section.

The `retry_after` configuration option is present in all of the back-end configurations except one. Amazon SQS uses [SQS Visibility Settings](https://docs.aws.amazon.com/AWSSimpleQueueService/latest/SQSDeveloperGuide/sqs-visibility-timeout.html) instead. By setting the value of this option to 90 seconds, you're telling the framework that if a job doesn't succeed or fails within 90 seconds of being processed, the framework should consider it failed and try again. Typically, you should set this value to the maximum number of seconds your jobs should reasonably take to complete processing.

There are also worker timeouts that you can set by executing the `php artisan queue:work --timeout=60` command. Setting the timeout to 60 means that if a job doesn't succeed or fails within 60 seconds, the worker will throw an error and exit. The official docs recommend setting the timeout to less than the retry interval. Otherwise, the framework may end up running your job twice before the worker timeout kicks in.

The effects of `retry_after` and `timeout` can be confusing, but they'll become clearer once you've dealt with them a few times. Apart from these four configuration options, the others in the `sqs` and `beanstalkd` back-end are mostly related to the back-end connection, such as the hostname and secret keys. However, I would like to discuss two options from the `redis` connection in a bit more detail.

The first one is the `block_for` option. When you set this option to 5, the worker will first open a connection with Redis and check for unprocessed jobs. If it doesn't find any, the worker will wait for 5 seconds before checking again. This option can save you some valuable CPU resources if you tune it properly.

The second one is the `connection` option. Seeing `default` as the value often makes people confused about whether it's multiple queue connections or something else. To understand this option, first, open the `config/database.php` file in your project, scroll to the bottom, and find code similar to the following:

```php
    'redis' => [

        'client' => env('REDIS_CLIENT', 'phpredis'),

        'options' => [
            'cluster' => env('REDIS_CLUSTER', 'redis'),
            'prefix' => env('REDIS_PREFIX', Str::slug(env('APP_NAME', 'laravel'), '_').'_database_'),
        ],

        'default' => [
            'url' => env('REDIS_URL'),
            'host' => env('REDIS_HOST', '127.0.0.1'),
            'password' => env('REDIS_PASSWORD', null),
            'port' => env('REDIS_PORT', '6379'),
            'database' => env('REDIS_DB', '0'),
        ],

        'cache' => [
            'url' => env('REDIS_URL'),
            'host' => env('REDIS_HOST', '127.0.0.1'),
            'password' => env('REDIS_PASSWORD', null),
            'port' => env('REDIS_PORT', '6379'),
            'database' => env('REDIS_CACHE_DB', '1'),
        ],

    ],
```

As you can see, there are two separate Redis connection configurations here. One is the `default` connection, and the other is the `cache` connection. By default, the framework uses the `default` connection as the queue back-end, but you can define new ones here if you want. Thus, the `connection` option under `redis` inside the `config/queue.php` file refers to the default database connection for Redis.

## Job Failures and Retries

So far, you've worked with jobs that have succeeded, but this will not always be the case. Like other code, jobs may fail, and based on your requirements, you can either retry a failed job or ignore it. A job can fail because of a timeout or an exception. Let's have a look at timeouts first.

To demonstrate this, go back to the `app/Jobs/SendVerificationEmail.php` file and update its code as follows:

```php
<?php

namespace App\Jobs;

use Illuminate\Bus\Queueable;
use Illuminate\Contracts\Queue\ShouldQueue;
use Illuminate\Foundation\Bus\Dispatchable;
use Illuminate\Queue\InteractsWithQueue;
use Illuminate\Queue\SerializesModels;

class SendVerificationEmail implements ShouldQueue
{
    use Dispatchable, InteractsWithQueue, Queueable, SerializesModels;

    public $timeout = 1;

    // ...

    /**
     * Execute the job.
     *
     * @return void
     */
    public function handle()
    {
        sleep(5);
        logger('email sent!');
    }
}
```

The timeout public property indicates how long the queue worker should allow a job to keep running. Setting the timeout of a job to 1 indicates to the worker that this job should not run more than 1 second. Inside the `handle()` method, I've left the `sleep()` statement intact. So, theoretically, the worker will terminate the job after 1 second and mark it as a failed job. Restart the queue worker and revisit the `/` route. Look at the output from the worker, and you should see something similar to the following:

```bash
[2021-09-13 06:42:22][2] Processing: App\Jobs\SendVerificationEmail
[2021-09-13 06:42:22][2] Failed:     App\Jobs\SendVerificationEmail
fish: Job 1, 'php artisan queue:work' terminated by signal SIGKILL (Forced quit)
```

The last line in the output indicates that the queue worker has exited. I hope you understand by now that setting the `$timeout` to 1 inside a job is similar to setting it via the `queue:work --timeout=1` command. In a production environment, a process manager, like a supervisor, will be in charge of restarting the queue worker.

The other way a job can fail is by simply throwing an exception. Open the `app/Jobs/SendVerificationEmail.php` file and update its content as follows:

```php
<?php

namespace App\Jobs;

use Exception;
use Illuminate\Bus\Queueable;
use Illuminate\Contracts\Queue\ShouldQueue;
use Illuminate\Foundation\Bus\Dispatchable;
use Illuminate\Queue\InteractsWithQueue;
use Illuminate\Queue\SerializesModels;

class SendVerificationEmail implements ShouldQueue
{
    use Dispatchable, InteractsWithQueue, Queueable, SerializesModels;

    // ...

    /**
     * Execute the job.
     *
     * @return void
     */
    public function handle()
    {
        throw new Exception();

        sleep(5);
        logger('email sent!');
    }
}
```

The `throw` statement will simulate a typical code execution failure. I've also removed the timeout. If you restart the queue worker and revisit the `/` route, you'll see the following output from the worker:

```bash
[2021-09-13 06:32:26][1] Processing: App\Jobs\SendVerificationEmail
[2021-09-13 06:32:26][1] Failed:     App\Jobs\SendVerificationEmail
```

The worker attempted the job and marked it as failed as soon as the exception was thrown. You might wonder where these failed jobs go; they are stored the database. If you take a look at your database, you'll see there is a table called `failed_jobs`, which is responsible for holding all the jobs that didn't succeed for one reason or another.

![Failed Jobs In Database](https://www.honeybadger.io/images/blog/posts/laravel-queues-deep-dive/failed-jobs-in-database.jpg)

The table stores important information, such as the connection, queue, payload, exception, and time of the failure. Instead of exploring the database directly, you can also use the `php artisan queue:failed` command to get a list of all the failed jobs.

```bash
+--------------------------------------+------------+---------+--------------------------------+---------------------+
| ID                                   | Connection | Queue   | Class                          | Failed At           |
+--------------------------------------+------------+---------+--------------------------------+---------------------+
| d10fbacb-7cce-444f-ab56-c78bf7612f32 | database   | default | App\Jobs\SendVerificationEmail | 2021-09-14 15:53:54 |
| 978ce868-90b5-49c4-a202-9038b20d745b | database   | default | App\Jobs\SendVerificationEmail | 2021-09-14 15:53:22 |
+--------------------------------------+------------+---------+--------------------------------+---------------------+
```

You can retry jobs stored in this table either manually or automatically. To manually retry a job, use the `php artisan queue:retry` command. The command takes a job ID as an argument.

```bash
php artisan queue:retry d10fbacb-7cce-444f-ab56-c78bf7612f32

# The failed job [d10fbacb-7cce-444f-ab56-c78bf7612f32] has been pushed back onto the queue!
```

The framework has released the job back to the queue. Start a queue worker, and it will process the job immediately. The job will fail again because you've programmed the job to always fail. Even if you remove the `$timeout` value, the job will remember the old configuration and repeat it.

The `queue:retry` command has some alternate syntaxes as well. It can take multiple job IDs or even a range of IDs. You can even retry all jobs at once by executing the `php artisan queue:retry --all` command. Alternatively, if you want to retry failed jobs from a certain queue, you can execute the `php artisan queue:retry --queue=high-priority` command. You can even remove failed jobs from the database by executing the `php artisan queue:forget` command. Just like the `queue:retry` command, this one also takes the job ID as an argument.

### Automatic Retries

I know that retrying jobs using an artisan command in a production environment seems awkward, and I personally have never done it. You'll either use a GUI (which I'll talk about in a later section) or configure the jobs to retry automatically. To retry a job automatically, set the `$tries` public property on the job class. Update the `app/Jobs/SendVerificationEmail.php` file's content as follows:

```php
<?php

namespace App\Jobs;

use Exception;
use Illuminate\Bus\Queueable;
use Illuminate\Contracts\Queue\ShouldQueue;
use Illuminate\Foundation\Bus\Dispatchable;
use Illuminate\Queue\InteractsWithQueue;
use Illuminate\Queue\SerializesModels;

class SendVerificationEmail implements ShouldQueue
{
    use Dispatchable, InteractsWithQueue, Queueable, SerializesModels;

    public $tries = 5;

    // ...

    /**
     * Execute the job.
     *
     * @return void
     */
    public function handle()
    {
        throw new Exception();

        sleep(5);
        logger('email sent!');
    }
}
```

Now, restart a queue worker and visit the `/` route. You'll see that instead of considering the job failed on the first exception, the worker automatically retried it five times.

```bash
[2021-09-26 14:42:13][1] Processing: App\Jobs\SendVerificationEmail
[2021-09-26 14:42:13][2] Processing: App\Jobs\SendVerificationEmail
[2021-09-26 14:42:13][3] Processing: App\Jobs\SendVerificationEmail
[2021-09-26 14:42:13][4] Processing: App\Jobs\SendVerificationEmail
[2021-09-26 14:42:13][5] Processing: App\Jobs\SendVerificationEmail
[2021-09-26 14:42:13][5] Failed:     App\Jobs\SendVerificationEmail
```

As you can see in the output, the worker marked the job as failed only after the fifth retry. If you set the value of the `$tries` to `-1`, the worker will retry the job an unlimited number of times. Apart from setting the number of retries, you can also configure the job so that the worker retries a specific number of times. Update the `app/Jobs/SendVerificationEmail.php` file's content as follows:

```php
<?php

namespace App\Jobs;

use Exception;
use Illuminate\Bus\Queueable;
use Illuminate\Contracts\Queue\ShouldQueue;
use Illuminate\Foundation\Bus\Dispatchable;
use Illuminate\Queue\InteractsWithQueue;
use Illuminate\Queue\SerializesModels;

class SendVerificationEmail implements ShouldQueue
{
    use Dispatchable, InteractsWithQueue, Queueable, SerializesModels;

    public $tries = -1;

    // ...

    /**
     * Execute the job.
     *
     * @return void
     */
    public function handle()
    {
        throw new Exception();

        sleep(5);
        logger('email sent!');
    }

    public function retryUntil()
    {
        return now()->addSeconds(2);
    }
}
```

The `retryUntil()` method returns the number of times the job should be retried. Restart the queue worker and and revisit the `/` route. This time, you'll see that the worker retries the job for only 2 seconds and then declares it failed.

```bash
# ...
[2021-09-26 15:02:32][160] Processing: App\Jobs\SendVerificationEmail
[2021-09-26 15:02:32][161] Processing: App\Jobs\SendVerificationEmail
[2021-09-26 15:02:32][162] Processing: App\Jobs\SendVerificationEmail
[2021-09-26 15:02:32][163] Processing: App\Jobs\SendVerificationEmail
[2021-09-26 15:02:32][164] Processing: App\Jobs\SendVerificationEmail
[2021-09-26 15:02:32][165] Processing: App\Jobs\SendVerificationEmail
[2021-09-26 15:02:32][166] Processing: App\Jobs\SendVerificationEmail
[2021-09-26 15:02:32][167] Processing: App\Jobs\SendVerificationEmail
[2021-09-26 15:02:33][168] Processing: App\Jobs\SendVerificationEmail
[2021-09-26 15:02:33][168] Failed:     App\Jobs\SendVerificationEmail
```

### Automatic Retry Interval

You may have thought that 2 seconds is a very brief length of time, but within this timespan, the worker has retried the job 168 times on my local computer. You may not want this to happen. You may want the worker to wait for 1 second before retrying the job. The `$backoff` public property does this. To see it in action, update the `app/Jobs/SendVerificationEmail.php` file as follows:

```php
<?php

namespace App\Jobs;

use Exception;
use Illuminate\Bus\Queueable;
use Illuminate\Contracts\Queue\ShouldQueue;
use Illuminate\Foundation\Bus\Dispatchable;
use Illuminate\Queue\InteractsWithQueue;
use Illuminate\Queue\SerializesModels;

class SendVerificationEmail implements ShouldQueue
{
    use Dispatchable, InteractsWithQueue, Queueable, SerializesModels;

    public $tries = -1;
    public $backoff = 1;

    // ...

    /**
     * Execute the job.
     *
     * @return void
     */
    public function handle()
    {
        throw new Exception();

        sleep(5);
        logger('email sent!');
    }

    public function retryUntil()
    {
        return now()->addSeconds(30);
    }
}
```

Restart the queue worker and revisit the `/` route. This time, the worker will wait 1 second before processing the job. Within 30 seconds, the worker will only retry the job 10 times:

```bash
[2021-09-26 15:16:25][1] Processing: App\Jobs\SendVerificationEmail
[2021-09-26 15:16:28][2] Processing: App\Jobs\SendVerificationEmail
[2021-09-26 15:16:32][3] Processing: App\Jobs\SendVerificationEmail
[2021-09-26 15:16:35][4] Processing: App\Jobs\SendVerificationEmail
[2021-09-26 15:16:38][5] Processing: App\Jobs\SendVerificationEmail
[2021-09-26 15:16:41][6] Processing: App\Jobs\SendVerificationEmail
[2021-09-26 15:16:44][7] Processing: App\Jobs\SendVerificationEmail
[2021-09-26 15:16:47][8] Processing: App\Jobs\SendVerificationEmail
[2021-09-26 15:16:50][9] Processing: App\Jobs\SendVerificationEmail
[2021-09-26 15:16:53][10] Processing: App\Jobs\SendVerificationEmail
[2021-09-26 15:16:53][10] Failed:     App\Jobs\SendVerificationEmail
```

The `$backoff` property is one of the most interesting ones. Assume that you want the job to be tried 5 times and want the worker to wait 1 second before making the first attempt, 2 seconds before making the second attempt, 3 seconds before making the third attempt, 4 seconds before making the fourth attempt, and 5 seconds before making the fifth attempt. To do so, update the `app/Jobs/SendVerificationEmail.php` file as follows:

```php
<?php

namespace App\Jobs;

use Exception;
use Illuminate\Bus\Queueable;
use Illuminate\Contracts\Queue\ShouldQueue;
use Illuminate\Foundation\Bus\Dispatchable;
use Illuminate\Queue\InteractsWithQueue;
use Illuminate\Queue\SerializesModels;

class SendVerificationEmail implements ShouldQueue
{
    use Dispatchable, InteractsWithQueue, Queueable, SerializesModels;

    public $tries = 5;
    public $backoff = [1, 2, 3, 4, 5];

    // ...

    /**
     * Execute the job.
     *
     * @return void
     */
    public function handle()
    {
        throw new Exception();

        sleep(5);
        logger('email sent!');
    }
}
```

I guess you can understand what the `$tries` and `$backoff` values indicate. Restart the queue worker and revisit the `/` route. This time, you'll see the backoff time increasing with each attempt. Now, if you set the `$tries` to 10 and leave the `$backoff` as-is, the last duration will persist for the rest of the retries. Thus, after the fifth attempt, the rest of the attempts will have a 5-second backoff.

### Catching Job Failures

There are two more concepts that I would like to discuss regarding job failures and retries. The first one is the `failed()` public method. You can use this function to execute a piece a code after a job has failed. It's like a catch statement for jobs. The second one is the maximum number of allowed exceptions. You can achieve some quite interesting results by combining these methods and properties.

Consider a scenario where you want to release a failed job back to the queue based on a certain condition, but you also want to limit the number of allowed exceptions. To see this in action, update the `app/Jobs/SendVerificationEmail.php` file's content as follows:

```php
<?php

namespace App\Jobs;

use Exception;
use Illuminate\Bus\Queueable;
use Illuminate\Contracts\Queue\ShouldQueue;
use Illuminate\Foundation\Bus\Dispatchable;
use Illuminate\Queue\InteractsWithQueue;
use Illuminate\Queue\SerializesModels;
use Illuminate\Support\Facades\Log;

class SendVerificationEmail implements ShouldQueue
{
    use Dispatchable, InteractsWithQueue, Queueable, SerializesModels;

    public $maxExceptions = 5;

    // ...

    /**
     * Execute the job.
     *
     * @return void
     */
    public function handle()
    {
        throw new Exception('Failed!', 999);

        sleep(5);
        logger('email sent!');
    }

    public function failed(Throwable $th)
    {
        if ($th->getCode() == 999) {
            $this->release();
        } else {
            logger($th->getMessage());
        }
    }
}
```

Inside the `failed()` method, I'm releasing the job back to the queue if it has encountered the 999 error code. The `release()` method comes from the `InteractsWithQueue` contract, and the 999 error code is made up. I've also set the maximum number of allowed exceptions to 5, so if the the job throws 5 consecutive exceptions, the worker will consider it failed.

I could've used the `$tries` property here, but in this case, the worker would've retried the job regardless of the exception code. If you use `$tries` and `$maxExceptions` in the same job, the `$maxExceptions` will override the `$tries` property. Restart the queue worker and revisit the `/` route. You'll see the job being retried 5 times and declared failed.

## Chains and Batches

So far in this article, you've worked with stand-alone jobs, but jobs can also be arranged in chains or batches. Both of these are multiple jobs grouped together. The main difference between a batch and chain is that jobs in a batch are processed simultaneously, whereas jobs in a chain are processed sequentially.

### Chains

In a chain, a job is dependent on the previous one. Thus, if you build a job chain with three jobs, the worker will process them sequentially; if any of the jobs fail, the worker will not process any subsequent jobs. To see this in action, create three new jobs inside the project. To do so, execute the following commands inside your project directory:

```bash
php artisan make:job EncodeVideoClip
php artisan make:job GenerateSubtitles
php artisan make:job PublishVideoClip
```

Now, open the `routes/web.php` file and update the code for the `/` route handler:

```php
use App\Jobs\EncodeVideoClip;
use App\Jobs\GenerateSubtitles;
use App\Jobs\PublishVideoClip;

Route::get('/', function () {

    \Illuminate\Support\Facades\Bus::chain([
        new EncodeVideoClip,
        new GenerateSubtitles,
        new PublishVideoClip,
    ])->dispatch();

    return view('welcome');
});
```

This imaginary snippet of code could be a part of some video sharing platform, and it describes the steps necessary for publishing a new video clip on the platform. There are three steps in the publishing process:

1. Encode the video clip to an optimized format.
2. Generate subtitles by analyzing the audio stream.
3. Publish the video on the platform.

These three steps are handled within three separate jobs in the back-end, and if any of them fails, the remaining jobs will not be processed. The `chain()` method inside the `Illuminate\Support\Facades\Bus` facade returns an instance of the `Illuminate\Foundation\Bus\PendingChain` class, which is like an array of multiple jobs and encapsulates methods, such as the `dispatch()` method. To see this in action, start a new queue worker and revisit the `/` route:

```bash
[2021-09-28 15:33:05][1] Processing: App\Jobs\EncodeVideoClip
[2021-09-28 15:33:05][1] Processed:  App\Jobs\EncodeVideoClip
[2021-09-28 15:33:05][2] Processing: App\Jobs\GenerateSubtitles
[2021-09-28 15:33:05][2] Processed:  App\Jobs\GenerateSubtitles
[2021-09-28 15:33:05][3] Processing: App\Jobs\PublishVideoClip
[2021-09-28 15:33:05][3] Processed:  App\Jobs\PublishVideoClip
```

As you can see from the output, the worker indeed processed the three jobs sequentially. Now, open the `app/Jobs/GenerateSubtitles.php` file and update its code as follows:

```php
<?php

namespace App\Jobs;

use Exception;
use Illuminate\Bus\Queueable;
use Illuminate\Contracts\Queue\ShouldBeUnique;
use Illuminate\Contracts\Queue\ShouldQueue;
use Illuminate\Foundation\Bus\Dispatchable;
use Illuminate\Queue\InteractsWithQueue;
use Illuminate\Queue\SerializesModels;

class GenerateSubtitles implements ShouldQueue
{
    use Dispatchable, InteractsWithQueue, Queueable, SerializesModels;

    // ...

    /**
     * Execute the job.
     *
     * @return void
     */
    public function handle()
    {
        throw new Exception();
    }
}
```

Throwing an exception in the second job should theoretically stop the worker from processing the later jobs in the queue. Restart the queue worker and revisit the `/` route.

```php
[2021-09-28 15:36:08][4] Processing: App\Jobs\EncodeVideoClip
[2021-09-28 15:36:08][4] Processed:  App\Jobs\EncodeVideoClip
[2021-09-28 15:36:08][5] Processing: App\Jobs\GenerateSubtitles
[2021-09-28 15:36:08][5] Failed:     App\Jobs\GenerateSubtitles
```

As you can see, the worker stopped as soon as it encountered the exception in the second job. The `Illuminate\Foundation\Bus\PendingChain` also encapsulates a `chain()` method for handling job failure in chains. Open the `routes/web.php` file again and update the code for the `/` route handler:

```php
use App\Jobs\EncodeVideoClip;
use App\Jobs\GenerateSubtitles;
use App\Jobs\PublishVideoClip;

Route::get('/', function () {

    \Illuminate\Support\Facades\Bus::chain([
        new EncodeVideoClip,
        new GenerateSubtitles,
        new PublishVideoClip,
    ])->catch(function (Throwable $th) {
        Log::error('The job failed with code: ' . $th->getCode());
    })
    ->dispatch();

    return view('welcome');
});
```

Now, look inside the `storage/logs/laravel.log` file and you should see something like the following code:

```log
[2021-09-28 15:50:23] local.ERROR: The job failed with code: 0  
[2021-09-28 15:50:23] local.ERROR:  {"exception":"[object] (Exception(code: 0):  at D:\
epos\\laravel\\queue-worker-lessons\\app\\Jobs\\GenerateSubtitles.php:33)
[stacktrace]
# ...
```

Of course, in a real-life scenario, you'll have a much more complex error-handling mechanism, but for now, I hope that you understand the idea. You can also override the default queue or connection by chaining calls to `onQueue()` or `onConnection()` methods.

```php
use App\Jobs\EncodeVideoClip;
use App\Jobs\GenerateSubtitles;
use App\Jobs\PublishVideoClip;

Route::get('/', function () {

    \Illuminate\Support\Facades\Bus::chain([
        new EncodeVideoClip,
        new GenerateSubtitles,
        new PublishVideoClip,
    ])->onConnection('database')
    ->onQueue('video-processing')
    ->catch(function (Throwable $th) {
        Log::error('The job failed with code: ' . $th->getCode());
    })
    ->dispatch();

    return view('welcome');
});
```

### Batches

Apart from job chains, there are also batches. Batches are similar to chains, but they are collection of jobs independent from each other, and the worker processes a batch of jobs simultaneously. To see this in action, open the `routes/web.php` file and update the code for the `/` route handler:

```php
use App\Jobs\EncodeVideoClip;

Route::get('/', function () {

    \Illuminate\Support\Facades\Bus::batch([
        new EncodeVideoClip,
        new EncodeVideoClip,
        new EncodeVideoClip,
    ])->dispatch();

    return view('welcome');
});
```

Like the `chain()` method, the `batch()` method returns an instance of the `Illuminate\Bus\PendingBatch` class, which encapsulates a bunch of helpful methods for dealing with batches. Before you can see this in action, you'll have to create the `job_batches` table inside your database. To do so, execute the following commands:

```php
php artisan queue:batches-table
php artisan migrate
```

As the name suggests, this table stores information about the job batches. Apart from creating this table, you'll also have to add the `Illuminate\Bus\Batchable` trait to each job inside the batch. So, open the `app/Jobs/EncodeVideoClip.php` file and update its code as follows:

```php
<?php

namespace App\Jobs;

use Illuminate\Bus\Batchable;
use Illuminate\Bus\Queueable;
use Illuminate\Contracts\Queue\ShouldQueue;
use Illuminate\Foundation\Bus\Dispatchable;
use Illuminate\Queue\InteractsWithQueue;
use Illuminate\Queue\SerializesModels;

class EncodeVideoClip implements ShouldQueue
{
    use Batchable, Dispatchable, InteractsWithQueue, Queueable, SerializesModels;

    // ...
}
```

Now, restart the queue worker. Make sure the window running the worker is visible and revisit the `/` route. You'll see all three jobs appear on the terminal simultaneously. If you look at the `job_batches` table using the database tool, you'll see information about the job batch appear in the table.

![Unprocessed Job Batches In Database](https://www.honeybadger.io/images/blog/posts/laravel-queues-deep-dive/job-batches-in-database.jpg)

As you can see, it holds information like the total number of jobs, number of pending jobs, number of failed jobs, and more. I've already said that jobs in a batch are not dependent on each other. However, if any of the jobs in a batch fail, the worker will declare the entire batch as canceled. You can override this default behavior by calling the `allowFailures()` method before dispatching the job.

```php
use \App\Jobs\EncodeVideoClip;

Route::get('/', function () {

    \Illuminate\Support\Facades\Bus::batch([
        new EncodeVideoClip,
        new EncodeVideoClip,
        new EncodeVideoClip,
    ])->allowFailures()
    ->dispatch();

    return view('welcome');
});
```

Apart from this method, the `Illuminate\Bus\PendingBatch` class also encapsulates regular methods, such as the `then()`, `catch()`,  and `finally()` methods.

```php
use App\Jobs\EncodeVideoClip;

Route::get('/', function () {

    \Illuminate\Support\Facades\Bus::batch([
        new EncodeVideoClip,
        new EncodeVideoClip,
        new EncodeVideoClip,
    ])->allowFailures()
    ->onConnection('database')
    ->onQueue('video-encoding')
    ->then(function(\Illuminate\Bus\Batch $batch) {
        logger('Batch ' . $batch->id . ' finished successfully!');
    })
    ->then(function(\Illuminate\Bus\Batch $batch) {
        logger('Batch ' . $batch->id . ' did not finish successfully!');
    })
    ->finally(function(\Illuminate\Bus\Batch $batch) {
        logger('Cleaning leftovers from batch ' . $batch->id);
    })
    ->dispatch();

    return view('welcome');
});
```

The `then()` method only runs if the batch has been processed successfully. The `catch()` method runs if a failure occurs, and the `finally()` method runs regardless of a failure. You can use a combination of these three methods to whip up some pretty complex workflows. I've also sneaked in the `onConnection()` and `onQueue()` methods to let you know that they also exist in the case of batches.

There is another interesting thing that you can do with chains and batches: dispatch batches of chains. I know it sounds complicated, but bear with me. Open the `routes/web.php` file and update the code for the `/` route handler:

```php
use App\Jobs\EncodeVideoClip;
use App\Jobs\GenerateSubtitles;
use App\Jobs\PublishVideoClip;

Route::get('/', function () {

    \Illuminate\Support\Facades\Bus::batch([
        [
            new EncodeVideoClip,
            new GenerateSubtitles,
            new PublishVideoClip,
        ],
        [
            new EncodeVideoClip,
            new GenerateSubtitles,
            new PublishVideoClip,
        ],
    ])->dispatch();

    return view('welcome');
});
```

In this case, you'll have to put the chains within the batch array and pass them to the `batch()` method as a parameter. Remove the exception from the `app/Jobs/GenerateSubtitles.php` file, make sure to add the `Illuminate\Bus\Batchable` trait to all three jobs, restart the queue worker, and revisit the `/` route.

```bash
[2021-09-28 16:46:19][11] Processing: App\Jobs\EncodeVideoClip
[2021-09-28 16:46:19][11] Processed:  App\Jobs\EncodeVideoClip
[2021-09-28 16:46:19][12] Processing: App\Jobs\EncodeVideoClip
[2021-09-28 16:46:19][12] Processed:  App\Jobs\EncodeVideoClip
[2021-09-28 16:46:19][13] Processing: App\Jobs\GenerateSubtitles
[2021-09-28 16:46:19][13] Processed:  App\Jobs\GenerateSubtitles
[2021-09-28 16:46:19][14] Processing: App\Jobs\GenerateSubtitles
[2021-09-28 16:46:19][14] Processed:  App\Jobs\GenerateSubtitles
[2021-09-28 16:46:19][15] Processing: App\Jobs\PublishVideoClip
[2021-09-28 16:46:19][15] Processed:  App\Jobs\PublishVideoClip
[2021-09-28 16:46:19][16] Processing: App\Jobs\PublishVideoClip
[2021-09-28 16:46:19][16] Processed:  App\Jobs\PublishVideoClip
```

As you can see, the worker has processed the jobs as chains within a batch. You can also do the opposite (i.e., dispatch batches within a chain), but the processes is not that straightforward. Open the `routes/web.php` file and update the code for the `/` route handler:

```php
use App\Jobs\EncodeVideoClip;
use App\Jobs\GenerateSubtitles;
use App\Jobs\PublishVideoClip;

Route::get('/', function () {

    \Illuminate\Support\Facades\Bus::chain([
        new EncodeVideoClip,
        function() {
            \Illuminate\Support\Facades\Bus::batch([
                new GenerateSubtitles,
                new GenerateSubtitles,
                new GenerateSubtitles,
            ])->dispatch();
        },
        new PublishVideoClip,
    ])->dispatch();

    return view('welcome');
});
```

Inside the chain array, you can define a closure dispatch and a batch within the closure. If you run this code, the worker will process the encode job, process the subtitle generation jobs all at once, and then process the publish job.

## Laravel Horizon

So far in this tutorial, you've worked with queues and jobs using the terminal window, but Laravel has a gorgeous graphical user interface. Laravel Horizon is an official Laravel packages, and it's a elegant admin panel for managing queues. To use this in your application, you'll have to install the package and switch to the `redis` queue driver.

I am assuming that you have Redis installed on your system or are using a managed service, such as the official [offering](https://redis.com/try-free/). Open the `.env` file of your project and change the value of `QUEUE_CONNECTION` to `redis` from `database`.

```dotenv
QUEUE_CONNECTION=redis
```

Now, install the Laravel Horizon package by executing the following commands:

```bash
composer require laravel/horizon
php artisan horizon:install
```

The second command will publish the `app/Providers/HorizonServiceProvider.php` file, the `config/horizon.php` file, and a bunch of new assets. I'll dive into the configuration options later, but let’s have a look at Horizon first. To start Laravel Horizon, execute the following command:

```bash
php artisan horizon
```

Then, visit the `/horizon` route on your browser, and you should see the Horizon Dashboard:

![Laravel Horizon](https://www.honeybadger.io/images/blog/posts/laravel-queues-deep-dive/horizon.jpg)

There isn't a lot to discuss about the UI. It's elegant and pretty self explanatory. You may try firing a number of jobs and see how they're presented within the UI. What I'm more interested in talking about is the configuration options. Open the `config/horizon.php` file inside your project, and you'll see an array containing all the different configuration options.

Most of these are very well documented both in the file and in the official [docs](https://laravel.com/docs/8.x/horizon). One thing that you may have noticed already is when you use Laravel Horizon, you don't have to start the worker queues manually. Laravel Horizon remains in charge of starting the queue worker. The queue worker configuration can be found at the bottom of the configuration file.

```php
    'defaults' => [
        'supervisor-1' => [
            'connection' => 'redis',
            'queue' => ['default'],
            'balance' => 'auto',
            'maxProcesses' => 1,
            'memory' => 128,
            'tries' => 1,
            'nice' => 0,
        ],
    ],

    'environments' => [
        'production' => [
            'supervisor-1' => [
                'maxProcesses' => 10,
                'balanceMaxShift' => 1,
                'balanceCooldown' => 3,
            ],
        ],

        'local' => [
            'supervisor-1' => [
                'maxProcesses' => 3,
            ],
        ],
    ],
```

The `defaults` array contains the default configuration for all environments. Inside the `environments` array, you can customize the options for `production` or `local` environments. Laravel Horizon will match these environment names against the value of `APP_ENV` inside the `.env` file. You can add more environments in this array.

## Running Workers in Production

So far in this article, you’ve started the queue workers manually. In a production server, that's not an option. Instead, you'll have to use a program called [supervisor](http://supervisord.org/). It's a popular process control system for UNIX-like operating systems. If you're deploying an application on a Ubuntu server, supervisor will come pre-installed. However, if you're on a platform that doesn't come with the program pre-installed, follow the official installation [instructions](http://supervisord.org/installing.html).

Supervisor, as the name suggests, supervises programs. If you instruct supervisor to start a queue worker, supervisor will make sure to keep it running. To do so, create a file `/etc/supervisor/conf.d/laravel-worker.conf` file and put following content in it:

```conf
[program:laravel-worker]
process_name=%(program_name)s_%(process_num)02d
command=php /srv/awesome-laravel-application/artisan queue:work
autostart=true
autorestart=true
stopasgroup=true
killasgroup=true
user=root
numprocs=8
redirect_stderr=true
stdout_logfile=/srv/awesome-laravel-application/storage/logs/worker.log
stopwaitsecs=3600
```

It's a typical program section in a supervisor configuration file. The most important configuration directives to understand here are `process_name`, `command`, and `numprocs`.

A typical program section in a supervisor configuration file usually represents a homogeneous process group containing multiple copies of a process. Thus, if you set the `process_name` of a program section to something like `laravel-worker`, then it'll have a single process named `laravel-worker` in it. However, if you set the name to the `%(program_name)s_%(process_num)02d` expression and `numprocs` (number of processes) to 8, then there will be 8 processes in this group, named `laravel-worker_00`, `laravel-worker_01`, `laravel-worker_02`, `laravel-worker_03`, `laravel-worker_04`, `laravel-worker_05`, `laravel-worker_06`, and `laravel-worker_07`.

The `command` directive represents the actual command supervisor will run. Assuming that you've deployed the application inside the `/srv/awesome-laravel-application` directory on your server, the `php /srv/awesome-laravel-application/artisan queue:work` command will start 8 queue workers. Another important directives to understand is the `user` directive. Depending on how you've configured your server, this could change. The `stdout_logfile` points to the log file where supervisor will record important events about this program. I usually keep the logs alongside the application log files, but you're free to keep them anywhere you like.

Once the configuration is in place, execute the following commands to start the processes:

```bash
sudo supervisorctl reread

sudo supervisorctl update

sudo supervisorctl start laravel-worker:*
```

The first command will reread the configuration file. The update command will update the processes that have changed. Finally, the start command will start the program. I hope you remember that queues need to be restarted on every code update. On a production server, you'll have to restart all the processes. To do so, you may execute the `sudo supervisorctl restart laravel-worker:*` command. However, if you've added a new configuration to the server, you'll have to first run the reread and update commands. Otherwise, supervisor will not know about the new configuration. The restart command should be a part of your CI/CD pipeline.

### Running Laravel Horizon in Production

Running Laravel Horizon is similar to running queue workers, but the configuration is slightly different.

```conf
[program:horizon]
process_name=%(program_name)s
command=php /srv/awesome-laravel-application/artisan horizon
autostart=true
autorestart=true
user=root
redirect_stderr=true
stdout_logfile=/srv/awesome-laravel-application/storage/logs/horizon.log
stopwaitsecs=3600
```

The `numprocs` directive is irrelevant, as the number of workers is handled by Horizon itself. The remainder of the configuration is unchanged. The number of workers you should run really depends on the type of server and your requirements. There are no rules-of-thumb here. You'll understand once you've worked with them for some time.

## Conclusion

That's a wrap on this article. I've covered almost all the important concepts regarding Laravel Queues, but this not a definitive guide. There are tons of other aspects of running queues that you should understand, such as job uniqueness, dealing with race conditions, and throttling. I could've included all of them in a single article, but that would've made it overwhelming. It's best if you start using queues in your applications and learn by doing. Also, make sure to read the official [docs](https://laravel.com/docs/8.x/queues) on queues thoroughly.

Thank you for being patient throughout the entire article. I hope that you've gained a solid understanding of the Laravel queue system. If you have any confusion regarding anything shown in this article, feel free to connect with me on [Twitter](https://twitter.com/frhnhsin), [LinkedIn](https://linkedin.com/in/farhanhasin), or [GitHub](https://github.com/fhsinchy). Till the next one, stay safe and keep on learning.