Build a Livewire CSV Importer
This episode is for members only

Sign up to access "Build a Livewire CSV Importer" right now.

Get started
Already a member? Sign in to continue
Playing
16. Setting up for queues

Episodes

0%
Your progress
  • Total: 2h 20m
  • Played: 0m
  • Remaining: 2h 20m
Join or sign in to track your progress

Transcript

00:00
So we are very nearly at the point where we can start pushing these jobs to our queue but we need to a little bit of setup before we do anything else. The first thing that we're gonna do is go ahead and install Redis. The reason being
00:10
is we're gonna be using Laravel Horizon here which works with Redis. I'd always recommend Redis as a queue driver when you're working with Laravel. So the first thing that you're gonna need to do is make sure you have that installed on your local environment. Of course every environment is different so go ahead and
00:25
look up how you'll get this installed if you don't already. But once you have got that we can start to work with Laravel Horizon. So Laravel Horizon is, if we just head over to the introduction here, a dashboard and code-driven configuration for your Laravel powered Redis queues. So this allows you to manage things a lot
00:43
easier. You just run a simple command phpArts and Horizon. All of your queues come up. You can configure this however you like and particularly with something like this where we want to potentially scale it, Horizon is always a good choice. So we're gonna go ahead and get this installed. Let's just
00:58
head straight over to the installation section and go ahead and pull this in. You might have already used it. So we're gonna do a composer require on Laravel Horizon and just wait for that to finish. And once we have done that we want to go ahead and install it. So let's head over, go ahead and run this command and there
01:14
we go. So we've got the Horizon service provider, the assets and the configuration. Now the reason we have assets and stuff like that is because we have a dashboard. So we can just go ahead and run this and view the dashboard. Now we're not going to talk too much about configuration. We might come back to this
01:27
a little bit later. The configuration we're just going to leave as the default for now. But you'll want to tweak that based on how many people you expect to be importing CSVs in your app. So the other thing that we're going to do is of course change over the queue driver or the queue connection. So for this we're
01:46
going to use Redis of course. So now that we've done that we're pretty much set up and ready to go. What we can do is run phpArts and Horizon. That's going to start up all of our queue workers and it's also going to start up the Horizon dashboard which we can use over here. So let's go ahead and just duplicate this
02:03
over and let's head over to slash Horizon within our app and we are pretty much done. So we've got this overview, jobs per minute, total processes. We just have a single process at the moment which is the default and that's just going to have one process in it. So it's just going to add a job, wait for that to
02:21
finish, go on to the next job and so on and so forth. And we've got some other things in here as well. We can check failed jobs, we can check completed jobs, there's a lot that we can do inside of here. But let's also talk about job batching. Now when we looked at our generator we had lots of different
02:38
chunks here. Ideally what we want to do is add these to an entire batch. We don't want to fire these off individually. The reason being is that when we batch these we can execute a batch of jobs and then perform some action when the batch of jobs has completed executing. So in order to get job batching working we need to
02:58
create a batches table which is going to keep an eye on all of these. If we head over to our database here at the moment we don't have anything in here like that but when we go ahead and create this batches table that's going to add to that batches table and it's going to have a batch of all of them chunks that
03:15
we've already generated. So let's go ahead and just cancel this off for a second because we're running out of room here and let's go ahead and create that batches table and let's go ahead and run phprs and migrate. So that's going to go ahead and create this job batches table for you. So it's the name of the batch,
03:31
the total amount of jobs, pending failed jobs and the IDs as well and so on and so forth. So lots of information here that we can read from. So now that we've got all of that set up we can go ahead and rerun phprs and horizon. We've got that up and running, it's working, it's got the default queue in here that we've
03:48
already seen and what we can now do is when we import this grab all of them chunks that we've already looked at. So if we head back over to our CSV importer and for each of these chunks that we have create a new job and push it. So let's do that in the next episode.

Episode summary

In this episode, we get everything set up to start working with queues in our Laravel app! First things first, we install Redis since it's the recommended queue driver, especially when using Laravel Horizon. If you don’t have Redis yet, make sure to check out how to install it based on your system.

Next, we pull in the Laravel Horizon package through Composer. Horizon gives us a really nice dashboard and makes managing queue workers super easy. Once installed, you just have to start it up with a simple command, and you'll be able to view all your jobs and processes in the dashboard right in your browser.

We also talk through why we're changing our queue connection to Redis, and get Horizon running. You'll see the dashboard, how it displays jobs per minute and lets you dig into completed and failed jobs.

Then, we switch gears and cover job batching. Rather than pushing each chunk as a separate job, batching lets us run a group of jobs and then take additional steps once the whole batch is finished. To support this, we add the necessary batches table with a database migration – making it easy to track batches and see their progress.

By the end, everything's properly set up: Horizon is running, batching is ready, and our system is prepped for handling CSV imports in batches. Next time, we’ll dive into creating those jobs and pushing them onto the queue!

Episode discussion

No comments, yet. Be the first!