Features and Changes Coming to Laravel 8's Queue System
Tags: laravel
Published: 2020-09-04 07:02:24

Laravel 8 ships with cool new features as well as some changes to the queue system. In this post we'll look into these features and changes.

## Changes

### Backoff

The `retryAfter()` method and the `retryAfter` property of queued jobs, mailers, notifications, and listeners has been renamed to `backoff`.

The `--delay` option of the `php artisan queue:work` command has been renamed to `--backoff` as well.

You should also know that you can now pass an array to the `backoff` property to instruct the worker to implement an exponential backoff:

```php
public $backoff = [30, 60];
```

Or return an array from the `backoff()` method:

```php
public function backoff ()
{
    return [30, 60]; 
}
```

Using an exponential backoff here, we instructed the worker to delay retrying the job 30 seconds after the first failure, then 60 seconds for each failure after that.

You can also use an exponential backoff on the `queue:work` command:

```
php artisan queue:Work --backoff=30,60
```

### Job Expiration

The `timeoutAt` property of queued jobs, notifications, and listeners has been renamed to `retryUntil`.

Using `$retryUntil` instructs the worker to keep retrying the job until a certain time in the future.

You can add `retryUntil` as a public property on the job class or a `retryUntil` method:

```php
public function retryUntil()
{
    return now()->addDay();
}
```

## New Features

### Queued Closures

You can now chain a `catch()` method while dispatching a queued closure:

```php
dispatch(function () {
    // Job logic...
})->catch(function (Throwable $e) {
    // Handle Failure...
});
```

The closure provided to the `catch()` method will be invoked if the job fails.

### Database Driver Reliability

When releasing a job back to the queue while using the database queue driver, Laravel will perform the operation inside a transaction now. That means the job will not be deleted from the queue unless the new—released—instance is added. This decreases the chance of losing jobs significantly.

### Redis Driver Efficiency

When dispatching a group of jobs in bulk while using the redis queue driver, Laravel will perform the operation by sending a single command to redis. Previously, Laravel used to send multiple `rpush` command to redis; one for each job.

### Worker Graceful Termination

Starting Laravel 8, workers will exit gracefully and call any terminating callbacks registered by `App::terminating()`.

### Worker Self Termination

In order to avoid memory leaks, it's a common practice to terminate your workers every now and then and let your process monitoring tool start new ones. This was usually done by adding a CRON job that runs the `queue:restart` artisan command.

In Laravel 8, you can instruct workers to exit after processing a certain number of jobs or after running for a specific number of seconds:

```php
php artisan queue:work --max-jobs=1000 --max-time=3600
```

### Naming Workers

You can now add a `--name` option to the `queue:work` command:

```php
php artisan queue:work --name=notifications
```

This feature was mainly added to allow people to customize how the worker picks which queue to process jobs from at runtime:

```php
Worker::popUsing('notifications', function ($pop) {
    $queues = time()->atNight() 
        ? ['mail', 'webhooks'] 
        : ['push-notifications', 'sms', 'mail', 'webhooks'];

    foreach ($queues as $queue) {
        if (! is_null($job = $pop($queue))) {
            return $job;
        }
    }
});
```

### Job Batching

Laravel's job batching allows you to dispatch a number of jobs to be processed in parallel by your workers. You can perform an action after all jobs in the batch are processed or if any of the batch jobs fail:

```php
Bus::batch([
    new ProcessFile(1),
    new ProcessFile(2),
    new ProcessFile(3),
])->dispatch();
```

You can find more information on Job Batching in the [official documentation](https://laravel.com/docs/master/queues#job-batching).

### Job Chaining

You can now dispatch a chain of jobs using the `Bus` facade:

```php
Bus::chain([
    new ExtractReports,
    new GenerateReport,
    new SendResults,
])->dispatch();
```

You can also add a `catch()` callback that'll be invoked if any of the jobs in the chain fails:

```php
Bus::chain([
    new ExtractReports,
    new GenerateReport,
    new SendResults,
])->catch(function(){
    // Handle the chain failure.
})
->dispatch();
```

### Horizon Balancing Rate

Two new configuration options were added to Horizon; `balanceMaxShift` and `balanceCooldown`.

```php
 'environments' => [
    'environment' => [
        'supervisor-1' => [
            'balanceMaxShift' => 5,
        ],
    ],
],
```

`balanceMaxShift` sets the maximum number of worker processes to add or remove each time Horizon scales the workers pool. In previous versions on Horizon, only a single worker was added or removed, now you can control that number.

As for `balanceCooldown`, it sets the number of seconds to wait between each scaling action. This was hard-coded to 3 seconds in previous versions of Horizon.

```php
 'environments' => [
    'environment' => [
        'supervisor-1' => [
            'balanceCooldown' => 1,
        ],
    ],
],
```