Config Laravel Horizon, queue, balance, processes and priority in redis

configure horizon for multiple installation on the same server

Image for post
Image for post

I assume you are using redis and , hence before starting to config the horizon, just clean horizon dashboard from all processed queued jobs with the following command I described in the a tutorial published in this series :

php artisan flush:redis

So now everything is clear and we are ready to learn all the following concepts from A to I.

I assume you have already installed Horizon according to . To understand how to config Laravel queue professionally you should be familiar with these simple concepts:

Concept A:

In config/queue.php file, you can find the configuration for redis connection driver.

'queue' => env('REDIS_QUEUE', 'default'),
Image for post
Image for post

It means if you do not specify for a job “the kind of queue status”, it will get the default status which is now ‘default’.

Then run:

php artisan horizon

Then trigger a job ( I assume you know how to trigger a job using onQueue() method. If you don’t know, check ). You will see in horizon dashboard:

Image for post
Image for post

Attention: Obviously, the retry_after value must be greater than the time it takes a job to run. Taylor set it 90 sec.

Image for post
Image for post

Concept B:

You may set (let’s say change) the default queue status for the redis connection to any name you want!

First change the ‘default’ to ‘foo’ in config/queue.php for redis connection.

'queue' => env('REDIS_QUEUE', 'foo'),

Then clear the config cache:

php artisan config:cache

So after triggering a job ( I assume you don’t specify for a job the “kind of queue status” and leave onQueue() method blank ), and then do:

php artisan horizon

in horizon dashboard you will see that queued job will get the foo queue status. Pay attention to the “Status”.

Image for post
Image for post

As you see, the queued job is not processed! To process it, follow the next concept.

Concept C:

Let’s take a look at the config/horizon.php configuration file. You will see at the end of the file there are two array keys. One for Production and one for local. Choose what you want according to the environment you already set in the .env file.

Image for post
Image for post

In both production and local, you can see the queue status is set as ‘default’.

It means if you run horizon with the artisan command :

php artisan horizon

by default, it will process only those jobs that have the ‘default’ queue status (let’s say priority). While we have already changed the default queue status in the redis connection in Concept B from default to foo’ and every triggered job will get the ‘foo’ as its queue status.

What can we do? We can add ‘foo’ to the queue status values as below:

In this way, we introduce what we set in config/queue.php to config/horizon.php.

Image for post
Image for post

Then:

php artisan config:cache

and then do:

php artisan horizon

and trigger that job again and you will see:

Image for post
Image for post

Note:

Until now we didn’t specify for a job “the status of queue” and each job gets the default value which is preset for redis driver.

But we can set a value as a queue status for each job at the moment of triggering that job, something rather than the default value. If you want this, let’s follow the next concepts.

Concept D:

Priority or queue status is just a *name* and nothing more, i.e. when you set ‘high’ as the queue status for a job, it does not have a higher priority compare to the other names and it does not have any meaning in the Laravel until you define your exact meaning. After you define ( I will show later), Laravel will push a job with a ‘high’ queue status to a higher priority queue.

Concept E:

How does Laravel understand the priority of each queue status (queue value)?

Simply! Just by its *order* that comes in this array

'queue' => ['default','foo'],

I will show you an example.

The order we put the “queue status values” is: “how we prioritize them”.

Example:

First stop the horizon. Then trigger two jobs in this order, first Job1 (queue=foo) and then Job2 (queue=default).

In concept G, I will say how to set a name.

Image for post
Image for post

As you see the jobs are not processed. So run the horizon with artisan command:

php artisan horizon

You will see in terminal:

Image for post
Image for post

It means Job2 has a priority over Job1, i.e. ‘default ’ value has a priority over ‘foo’, just because of the order they are defined in the array. As you see, first ‘default’ comes and then‘ foo’ as you can see below:

'environments' => [
'production' => [
'supervisor-1' => [
'connection' => 'redis',
'queue' => ['default','foo'],
'balance' => 'simple',
'processes' => 10,
'tries' => 3,
],
],
'local' => [
'supervisor-1' => [
'connection' => 'redis',
'queue' => ['default','foo'],
'balance' => 'simple',
'processes' => 3,
'tries' => 3,
],
],
],

Note:

If you don’t use horizon, to start a worker (that verifies all of the ‘default queue jobs being processed before continuing to any jobs on the ‘foo queue), pass a comma-delimited list of queue names to the artisan work command:

php artisan queue:work --queue=default,foo

Refer to Laravel document .

Concept F:

You may want to assign different priorities (queue status) to different queued jobs. In other words, you may want prioritize how your queues are processed.

According to the following styles, you have two options in naming the queue status :

Style I:

You may follow the style which implies the kind of priority such as:

‘low’, ‘high’, ‘default’

Style II:

You may give a name to the queued jobs based on how you categorize them such as:

‘email’, ‘render’, ‘resize, etc.

The question is where should we use each kind of styles? In the following, you will find the answer. So follow Concept H and Concept I.

Concept G:

Dispatching to a particular queue:

dispatch((new TheJob)->onQueue('foo'));

or using dishpatchNow method like:

TheJob::dispatchNow()->onQueue('foo'));

and also both of the below commands work.

PostPaymentJobTest::dispatch();
dispatch(new PostPaymentJobTest());

I also will publish a new article or update this tutorial for setting a queue status for Laravel notification. Stay tuned!

Concept H:

Let’s talk about ‘balance’ in Laravel Horizon configuration file. In config/horizon.php you see:

'environments' => [
'production' => [
'supervisor-1' => [
'connection' => 'redis',
'queue' => ['default'],
'balance' => 'simple',
'processes' => 10,
'tries' => 3,
],
],
'local' => [
'supervisor-1' => [
'connection' => 'redis',
'queue' => ['default'],
'balance' => 'simple',
'processes' => 3,
'tries' => 3,
],
],
],

If you want to follow Style I, described in Concept F, you should turn off balance and set it ‘false’ and not ‘simple’.

'queue' => ['high','default','low'],'balance' => 'false',

When the ‘balance’ option is set to ‘false , the default Laravel behavior will be used, which processes queues in the *order* they are listed in the queue array.

'queue' => [‘high’,‘default’,‘low’],

First high then default and finally low

Concept I:

As you see in horizon configuration, there is a ‘processes’ key.

'processes' => 3,

When the ‘balance value is something other than ‘false’, the ‘processes’ will be meaningful and you should follow the Style II, described in Concept F.

In a new tutorial or an update, I will say more about “how Laravel dedicates the resources to each process”. Stay tuned!

Important final notes:

Note A:

  • Whenever you edit your code, you have to terminate Horizon and restart it.

Note B:

  • When you are running multiple installations of Horizon on the same server, you should choose different prefixes for each installation. Otherwise you will face some nerve-wracking issues.

According to what is set in config/horizon.php, we can set a custom value in .env file:

HORIZON_PREFIX=ApplicationName-horizon:

Image for post
Image for post

or even without setting in .env file we can edit the config/horizon.php to something like :

'prefix' => env('APP_NAME').env('HORIZON_PREFIX', '-horizon:'),

Note C:

  • Horizon would monitor all the queues across all servers, while processing only the queue that is in config file for each server. So for a better experience, it is highly recommended to use different list of status names for each installation (or website):

As you see in Concept B: we can even set foo ( an arbitrary name). So we can use the .env file to set the specific queue names for each server that should run and we run “php artisan horizon” on every server.

As you see I set this array in config/horizon.php config file for production or local :

'queue' => ['high','default','low'],

Now we can use custom names for each installation in a way that in .env file we can set:

QUEUE_HIGH=ApplicationName-high
QUEUE_DEFAULT=ApplicationName-default
QUEUE_LOW=ApplicationName-low

and put in config/horizon.php something like this:

'queue' => [
env('QUEUE_HIGH'),
env('QUEUE_DEFAULT'),
env('QUEUE_LOW'),
],

and don’t forget to introduce the default one in the config/queue.php for redis connection.

Note D:

  • This is simple but it is very important to remember it firmly:

Production setting works when you set in .env file this:

APP_ENV=production

and local setting works only when you set in .env file this:

APP_ENV=local

Image for post
Image for post

If you set something in production, you should clone it in local too! Otherwise you will be crazy when you see your apps do not listen to you and there are many conflicts between your site in your local or server.

This is also the list of available Laravel horizon commands:

Image for post
Image for post

Some tips:

  • If you face an error when using Laravel job and queue always check the queue
  • One of the difference between dispatch_now() and dispatch() is that in the first one you can use in handle method of Job class
    Auth::user() while at the second one it is meaningless
  • When passing $request to a Job be careful that you cannot serialize request. Only eloquent model can be serialized and unserialized. See here: You should use $request->all() instead of $request. Since Request is treated as closure.

At the end, if you enjoyed this article:

Clap it ! Share it! Follow Me in Medium!

Written by

Web geek, Self-taught full-stack web developer, Learning Python, Laravel, Vuejs, UX/UI design, Nuclear Physicist PhD

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store