Laravel Worker with super-visor
Tags: laravel, supervisor, worker, job
Published: 2022-10-10 07:01:02

Many of us struggled to setup **supervisor** with **Laravel Worker** in our docker container or web-server. If you don't then you are lucky not as me. 😉 

If you are struggling to do or you want to know about what is job, how supervisor maintain the jobs inside a docker-container or web-server, how to setup all these stuffs then you are in right post i guess.
 
Follow my journey that let you know something informative.

## Pre-requisites

- Setup laravel queue:
```
php artisan queue:table
php artisan migrate
```
After migrating, You should find 2 tables named **failed_jobs** & **jobs**. If you don't then follow the procedure again or check [laravel queue](https://laravel.com/docs/9.x/queues) for better understanding.

- Setup job batches to keep track of jobs:
```
php artisan queue:batches-table
php artisan migrate
```
You should see table named **job_batches**. To learn more about batch jobs, Check the beautiful documentation of [batch jobs laravel](https://laravel.com/docs/9.x/queues#defining-batchable-jobs)

- Setup some test methods for batch jobs:
```
    public function testJob() {
        $path = storage_path('test');
        $files = glob("$path/user.csv"); // Parse the stored csv
        $header = [];
        foreach ($files as $key => $file) {
            $data = array_map('str_getcsv', file($file));
            if($key === 0) {
                $header = $data[0];
                unset($data[0]);
            }
        }
        $batch = Bus::batch([new TestCsvForSupervisor($header, 
        $data)->dispatch();
        return $batch->id;
    }

```

<u>What we do here?</u>
Ok, we've imported a csv with 100 thousand users information ex. email, username, address etc. **TestCsvForSupervisor** is the job class for handling some checks or operation like insert, update or comparing datas with any table ex. users. Simple as writing artisan command. 😉

**In TestCsvForSupervisor Job Class:**
```
    public function handle()
    {
        try{
            foreach ($this->data as $row) {
                $result = array();
                $result = User::whereName($row[0])->first() ? 'Found' : 'Not Found';
                Log::info('User checked successfully with email: '.$row[1]);
            }
        } catch (Exception $err) {
            Log::info('Error occured with message: '.$err);
        }
    }
```

Tough one, I have 20 thousand users information in my users table and what I've done here? Compare csv data with users table data and store the checking result in log (You might find it clueless but it's just a test case actually not good test case😒). Then dispatch the job to store the pending jobs to be done in table.

## Setup supervisor
Now it's time for setup supervisor into docker container where you can follow 2 steps:
  
  1. You can access docker bash and install supervisor by command scripts (It is for local testing not for global deployment or team collaboration)
  2. You can integrate the command scripts in Dockerfile.

**Before digging into both please let me inform you some details about supervisor**
> _Supervisor is basically linux based daemon that runs in the background. You have to configure your laravel app so that supervisor know about background jobs. It monitors the executing or executable jobs and if for any interference the jobs accidentally stop ex. server down after the restarting the supervisor automatically starts the job from where it stops and monitor the failed, pending and success jobs through this time and can deliver us jobs report._

Install from docker bash:
```
sudo apt-get install supervisor
```

**Install from Dockerfile using Daemon**

In your app/docker-conf/php/Dockerfile, Write the following script:
```
RUN apt-get install -y supervisor
```

Then, build and start the docker:
```
docker compose build
docker compose up -d // -d for running the docker in background
```

I will suggest you to follow the docker daemon-based (step-2) supervisor because it can be used globally. So, we will be continuing with that.

In your root directory make file named **laravel-worker.conf** and write and save the file:
```
[program:laravel-worker]
process_name=%(program_name)s_%(process_num)02d
command=php /app/artisan queue:work // you may use extra attributes like --sleep=3
autostart=true
autorestart=true
# stopasgroup=true
# killasgroup=true
;user=
numprocs=8
redirect_stderr=true
stdout_logfile=/app/storage/logs/worker.log​
stopwaitsecs = 100
```

In the docker bash, you will find file named **supervisord.conf** in /etc/supervisor/ directory. Overwrite the last line of the **supervisord.conf** file by following script and save it:

`[include]
files = /etc/supervisor/conf.d/*.conf  /app/laravel-worker.conf`

After that, Execute the following commands to initiate supervisor:

```
# Start the service
  service supervisor start

# Read the new config from supervisord file
  supervisorctl reread

# Activate the configuration
  supervisorctl update

# Start queue command for running jobs
  supervisorctl start laravel-worker:* or laravel-worker

# Check the status of new config operation
  supervisorctl status​
```

---

Congrats 🥳, You've just did a great job not just laravel-job but the real automated jobs using supervisor.

You may find the job list with status and unique id from **job_batches** table after starting supervisor like below shot:

![Batch jobs](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/blfkpyn8psty83igwzf0.png)

**There is one interesting fact about supervisor. Do you know what language used to develop supervisor?**

It's **Python** 🐍😲

             Thanks for reading with so much patience. ❤️