Config Laravel Horizon, queue, balance, processes and priority in redis

configure horizon for multiple installation on the same server

I assume you are using redis and Laravel Horizon, hence before starting to config the horizon, just clean horizon dashboard from all processed queued jobs with the following command I described in the a tutorial published in this series This Link:

php artisan flush:redis

So now everything is clear and we are ready to learn all the following concepts from A to I.

I assume you have already installed Horizon according to its official document. To understand how to config Laravel queue professionally you should be familiar with these simple concepts:

Concept A:

In config/queue.php file, you can find the configuration for redis connection driver.

'queue' => env('REDIS_QUEUE', 'default'),

It means if you do not specify for a job “the kind of queue status”, it will get the default status which is now ‘default’.

Then run:

php artisan horizon

Then trigger a job ( I assume you know how to trigger a job using onQueue() method. If you don’t know, check this link). You will see in horizon dashboard:

Attention: Obviously, the retry_after value must be greater than the time it takes a job to run. Taylor set it 90 sec.

Before you continue with the rest of this tutorial, I would like to introduce two packages for Laravel that I have recently developed: Laravel Pay Pocket, a modern multi-wallet package, and Laravel Failed Jobs, a UI for the Laravel Failed Jobs Table. I hope they may be of help to you.

https://github.com/HPWebdeveloper/laravel-pay-pocket
https://github.com/HPWebdeveloper/laravel-failed-jobs

Concept B:

You may set (let’s say change) the default queue status for the redis connection to any name you want!

First change the ‘default’ to ‘foo’ in config/queue.php for redis connection.

'queue' => env('REDIS_QUEUE', 'foo'),

Then clear the config cache:

php artisan config:cache

So after triggering a job ( I assume you don’t specify for a job the “kind of queue status” and leave onQueue() method blank ), and then do:

php artisan horizon

in horizon dashboard you will see that queued job will get the foo queue status. Pay attention to the “Status”.

As you see, the queued job is not processed! To process it, follow the next concept.

GitHub - HPWebdeveloper/laravel-pay-pocket: A multi-wallet Laravel package with comprehensive…

Concept C:

Let’s take a look at the config/horizon.php configuration file. You will see at the end of the file there are two array keys. One for Production and one for local. Choose what you want according to the environment you already set in the .env file.

In both production and local, you can see the queue status is set as ‘default’.

It means if you run horizon with the artisan command :

php artisan horizon

by default, it will process only those jobs that have the ‘default’ queue status (let’s say priority). While we have already changed the default queue status in the redis connection in Concept B from ‘default’ to ‘foo’ and every triggered job will get the ‘foo’ as its queue status.

What can we do? We can add ‘foo’ to the queue status values as below:

In this way, we introduce what we set in config/queue.php to config/horizon.php.

Then:

php artisan config:cache

and then do:

php artisan horizon

and trigger that job again and you will see:

Note:

Until now we didn’t specify for a job “the status of queue” and each job gets the default value which is preset for redis driver.

But we can set a value as a queue status for each job at the moment of triggering that job, something rather than the default value. If you want this, let’s follow the next concepts.

Concept D:

Priority or queue status is just a *name* and nothing more, i.e. when you set ‘high’ as the queue status for a job, it does not have a higher priority compare to the other names and it does not have any meaning in the Laravel until you define your exact meaning. After you define ( I will show later), Laravel will push a job with a ‘high’ queue status to a higher priority queue.

Concept E:

How does Laravel understand the priority of each queue status (queue value)?

Simply! Just by its *order* that comes in this array

'queue' => ['default','foo'],

I will show you an example.

The order we put the “queue status values” is: “how we prioritize them”.

Example:

First stop the horizon. Then trigger two jobs in this order, first Job1 (queue=foo) and then Job2 (queue=default).

In concept G, I will say how to set a name.

As you see the jobs are not processed. So run the horizon with artisan command:

php artisan horizon

You will see in terminal:

It means Job2 has a priority over Job1, i.e. ‘default ’ value has a priority over ‘foo’, just because of the order they are defined in the array. As you see, first ‘default’ comes and then‘ foo’ as you can see below:

'environments' => [
        'production' => [
            'supervisor-1' => [
                'connection' => 'redis',
                'queue' => ['default','foo'],
                'balance' => 'simple',
                'processes' => 10,
                'tries' => 3,
            ],
        ],
       'local' => [
            'supervisor-1' => [
                'connection' => 'redis',
                'queue' => ['default','foo'],
                'balance' => 'simple',
                'processes' => 3,
                'tries' => 3,
            ],
        ],
    ],

Note:

If you don’t use horizon, to start a worker (that verifies all of the ‘default’ queue jobs being processed before continuing to any jobs on the ‘foo’ queue), pass a comma-delimited list of queue names to the artisan work command:

php artisan queue:work --queue=default,foo

Refer to Laravel document here.

Concept F:

You may want to assign different priorities (queue status) to different queued jobs. In other words, you may want prioritize how your queues are processed.

According to the following styles, you have two options in naming the queue status :

Style I:

You may follow the style which implies the kind of priority such as:

‘low’, ‘high’, ‘default’

Style II:

You may give a name to the queued jobs based on how you categorize them such as:

‘email’, ‘render’, ‘resize’, etc.

The question is where should we use each kind of styles? In the following, you will find the answer. So follow Concept H and Concept I.

Concept G:

Dispatching to a particular queue:

dispatch((new TheJob)->onQueue('foo'));

or using dishpatchNow method like:

TheJob::dispatchNow()->onQueue('foo'));

and also both of the below commands work.

PostPaymentJobTest::dispatch();
dispatch(new PostPaymentJobTest());

I also will publish a new article or update this tutorial for setting a queue status for Laravel notification. Stay tuned!

Concept H:

Let’s talk about ‘balance’ in Laravel Horizon configuration file. In config/horizon.php you see:

'environments' => [
        'production' => [
            'supervisor-1' => [
                'connection' => 'redis',
                'queue' => ['default'],
                'balance' => 'simple',
                'processes' => 10,
                'tries' => 3,
            ],
        ],
        'local' => [
            'supervisor-1' => [
                'connection' => 'redis',
                'queue' => ['default'],
                'balance' => 'simple',
                'processes' => 3,
                'tries' => 3,
            ],
        ],
    ],

If you want to follow Style I, described in Concept F, you should turn off balance and set it ‘false’ and not ‘simple’.

'queue' => ['high','default','low'],
'balance' => 'false',

When the ‘balance’ option is set to ‘false’ , the default Laravel behavior will be used, which processes queues in the *order* they are listed in the queue array.

'queue' => [‘high’,‘default’,‘low’],

First high then default and finally low

Concept I:

As you see in horizon configuration, there is a ‘processes’ key.

'processes' => 3,

When the ‘balance’ value is something other than ‘false’, the ‘processes’ will be meaningful and you should follow the Style II, described in Concept F.

In a new tutorial or an update, I will say more about “how Laravel dedicates the resources to each process”. Stay tuned!

Important final notes:

Note A:

• Whenever you edit your code, you have to terminate Horizon and restart it.

Note B:

• When you are running multiple installations of Horizon on the same server, you should choose different prefixes for each installation. Otherwise you will face some nerve-wracking issues.

According to what is set in config/horizon.php, we can set a custom value in .env file:

HORIZON_PREFIX=ApplicationName-horizon:

or even without setting in .env file we can edit the config/horizon.php to something like :

'prefix' => env('APP_NAME').env('HORIZON_PREFIX', '-horizon:'),

Note C:

• Horizon would monitor all the queues across all servers, while processing only the queue that is in config file for each server. So for a better experience, it is highly recommended to use different list of status names for each installation (or website):

As you see in Concept B: we can even set foo ( an arbitrary name). So we can use the .env file to set the specific queue names for each server that should run and we run “php artisan horizon” on every server.

As you see I set this array in config/horizon.php config file for production or local :

'queue' => ['high','default','low'],

Now we can use custom names for each installation in a way that in .env file we can set:

QUEUE_HIGH=ApplicationName-high
QUEUE_DEFAULT=ApplicationName-default
QUEUE_LOW=ApplicationName-low

and put in config/horizon.php something like this:

'queue' => [
                    env('QUEUE_HIGH'),
                    env('QUEUE_DEFAULT'),
                    env('QUEUE_LOW'),
                ],

and don’t forget to introduce the default one in the config/queue.php for redis connection.

Note D:

• This is simple but it is very important to remember it firmly:

Production setting works when you set in .env file this:

APP_ENV=production

and local setting works only when you set in .env file this:

APP_ENV=local

If you set something in production, you should clone it in local too! Otherwise you will be crazy when you see your apps do not listen to you and there are many conflicts between your site in your local or server.

This is also the list of available Laravel horizon commands:

Some tips:

• If you face an error when using Laravel job and queue always check the queue
• One of the difference between dispatch_now() and dispatch() is that in the first one you can use in handle method of Job class
  Auth::user() while at the second one it is meaningless
• When passing $request to a Job be careful that you cannot serialize request. Only eloquent model can be serialized and unserialized. See here: https://laravel.com/docs/5.2/queues#writing-job-classes You should use $request->all() instead of $request. Since Request is treated as closure.

At the end, if you enjoyed this article:

Clap it ! Share it! Follow Me in Medium!

Panjeh — Medium

Laravel Auth::routes() Email verification Reset password Authentication Registration routes

Install Laravel on Ubuntu 18.04 with Apache, MySQL, PHP7 (LAMP) stack

Laravel prepare migration file for Polymorphic Relationships

touch Laravel model touch relationships in depth explanations

Laravel Auth::user() alternatives