Build An Uptime Monitor with Inertia
This episode is for members only

Sign up to access "Build An Uptime Monitor with Inertia" right now.

Get started
Already a member? Sign in to continue
Playing
32. Queuing the endpoint check

Episodes

0%
Your progress
  • Total: 4h 59m
  • Played: 0m
  • Remaining: 4h 59m
Join or sign in to track your progress
01. Introduction
01. Introduction and demo
6m 33s
0%
02. A fresh Inertia project
6m 7s
0%
02. Sites
03. Site and dashboard setup
8m 59s
0%
04. Building the site selector dropdown
12m 22s
0%
05. Making use of API resources
4m 14s
0%
06. Setting the default site
10m 37s
0%
07. Setting up Vue modals
4m 34s
0%
08. Creating a site
18m 18s
0%
03. Endpoints
09. Setting up endpoints
4m 23s
0%
10. Enum endpoint frequencies
3m 50s
0%
11. Creating the endpoint form
12m 58s
0%
12. Storing an endpoint
7m 21s
0%
13. Validating and authorizing endpoint stores
7m 6s
0%
14. Normalizing the endpoint location
6m 4s
0%
15. Displaying endpoints
6m 22s
0%
16. Deleting endpoints
6m 21s
0%
17. Toggling inline editing for endpoints
5m 12s
0%
18. Applying a watcher to the endpoint form
5m 31s
0%
19. Storing endpoint edits
8m 56s
0%
04. Checking endpoints
20. Determining endpoints to check
2m 57s
0%
21. Creating a job to perform a check
3m 21s
0%
22. Making an HTTP request
3m 39s
0%
23. Scheduling checks every second
4m 27s
0%
24. Setting up the check relationship
3m 14s
0%
25. Updating the job
2m 10s
0%
26. Showing the last status
12m 50s
0%
27. Expanded list of checks
14m 10s
0%
28. The response body modal
2m 49s
0%
29. Authorizing the endpoint details
2m 27s
0%
30. Calculating uptime for endpoints
8m 29s
0%
05. Queuing
31. Setting up queues
3m 21s
0%
32. Queuing the endpoint check
7m 26s
0%
06. Email notifications
33. Setting up email in Laravel
5m 8s
0%
34. Site model email notification storage
3m 5s
0%
35. Adding email addresses via the UI
17m 24s
0%
36. Listing and deleting email addresses
7m 14s
0%
07. Downtime notifications
37. Observing when to notify
10m 51s
0%
38. Creating an endpoint down notification
10m 46s
0%
39. Creating an endpoint recovery notification
8m 9s
0%
40. Making the emails a bit friendlier
3m 50s
0%
08. Extras
41. Deleting sites
5m 9s
0%
42. Eager loading
4m 44s
0%
43. Recovery notification bug
2m 31s
0%
44. No sites fix and placeholder
3m 25s
0%

Transcript

00:00
Now that we have queuing set up, we're actually going to go ahead and queue the job that performs the endpoint check. So the first thing that we're going to do is just head over to the database and make sure we clear out all checks here. We're going to see an issue with the way things are
00:12
going to be set up at the moment and we're going to go ahead and fix that in this video. So we want to make sure we have no checks just so we can monitor everything nicely. Okay, so let's hop over to the endpoint check job. So that's the one that makes the HTTP request to that endpoint URL. Now what we're going to do and something that we removed earlier
00:34
is go ahead and implement the should queue interface. Now just by doing this, what's now going to happen is when we dispatch this job, that is now going to be put onto the queue. And if you remember over in our perform checks command here, this is where we're dispatching this. So now that we have configured our queue, when our scheduler dispatches this,
00:55
this will now be put onto the queue. Now we can test this out by going over and running the PHP artisan short schedule run command. And let's just run that through and just see what happens to our jobs table here. This is where all of the jobs are being put onto the queue. These haven't actually been run yet. And what you can see at the moment is these just being consistently and
01:18
constantly put onto our queue. Now I'm going to go ahead and just cancel this command off. The reason that these are being put onto the queue one after another, these are pretty much the same checks happening over and over again, is because let's think about how our endpoint works. We have this next check date. And our console command is going ahead and grabbing all of the
01:39
endpoints where the next check is less than or equal to now. Now the job itself actually updates the next check. So if the next check is never updated, it's always going to be checked every single second. Once the job finishes, the next check date will be incremented. And then it won't be put back onto the queue until the next check date comes around. So really, until we actually
02:04
handle this job on the queue by running queue work, it will just constantly put this onto the queue. So let's go ahead and run both of these commands at the same time to get this working. Well, almost. We're going to see an issue with this. So let's go over to our jobs table and just clear everything out of here. And we're going to go over and first of all start our queue worker. So php artisan
02:25
queue and work. And that's going to run through and listen for jobs that are in the queue. There are none at the moment. Now if we go ahead and run our short scheduler, this is now going to put them jobs onto the queue. And they're going to be processed one after another. So let's go over and just run the short schedule command. You can see them jobs are being put in and slowly processed.
02:45
The checks are coming in. Now already we can see an issue with this. We've got a check here for the endpoint here 8, a check here for endpoint 9, but we've got a duplicate check here for endpoint 9 as well. So even though these two endpoints were only due to be checked once each, we have a duplicate in here. And we're going to see that happen over and over again as our scheduler runs
03:09
and as our jobs are processed in the queue. Now what we're going to do is think about why this might be happening. And to do that, we're going to go straight back over to our console kernel here. So we run the checks perform console command every single second. We do that because we want the date and time here to be very, very precise. We want these to be run exactly when the next check needs
03:34
to be run because we're working with minutes here, potentially say every 30 seconds if you were to set it up to do that. So this is fine and this is what we need because we need this to be checked every second. But what's happening here is over in perform checks, what we're doing is we're grabbing where the next check is less than or equal to now. Now that's absolutely fine. It's going to go ahead
03:52
and find, say, two records. But remember, over in the perform endpoint check job, this is where we are updating the next check. Now if this request takes any longer than one second, which in this case it has, we're not going to see the next check updated until this request is finished. So if this request takes longer than a second, the next check will not have been updated. And what
04:18
that means is the next time this runs, so second two, so the first second it's going to put two onto here, the second second is going to put another one on there because the next check hasn't had time to update. So basically what we want to do here is make sure that we only ever have an endpoint being checked in our queue at once. So we do not want any duplicate checks to happen because this
04:43
is an inevitable part of going ahead and pushing these onto our queue. So how are we going to do this? Well, let's just head back over to the database and give this a refresh. You can see that we've got a few more duplicates here. This is not good. We don't want to send too many requests here. It's going to create duplicate checks and it's going to mess our data up. So I'm going to
05:03
come over and just cancel off our queue, cancel off our scheduler. Let's just clear up the checks table by getting rid of everything in here. And you'll also want to delete any jobs that have hung around in here as well. Okay, so we're going to go back over to our perform endpoint check job. And what we're actually going to do is assign a unique ID to this. So we want this to be unique
05:25
in the database. So to do this, we go ahead and say should be unique for another interface on here and make sure you pull in the namespace for that if it's not already in there. And then we want to assign a unique ID to this using the unique ID method. So unique ID. And let's go ahead and return a unique ID from here. So what's unique about an endpoint that we already
05:47
have passed into this job? Well, just the ID from the database that will do. So let's go ahead and say this endpoint ID. And that's pretty much it. Now what's going to happen is if this job is attempted to be put onto the queue again, and it already has a job in there with that endpoint ID, it's going to fail. Now we need to be really careful here because if you start to queue other
06:07
things within your application that are not endpoints, this ID could be the same as something else. Let's say you are queuing something completely different. And we had an endpoint with an ID of one and a completely different model with an ID of one. That thing would then not be put onto the queue. So what you could do is go ahead and add in maybe endpoint underscore to this to make it
06:29
a little bit unique, more unique to what you have. So let's leave it at that just so we have future proof this. Okay, so now that we've added that unique ID, and we've implemented this should be unique interface, let's go ahead and see how this behaves differently. So I'm going to run the queue worker, going to go ahead and run the schedule. And we're going to head straight over to the
06:48
database and see both of them checks roll in. Already you can see only two of these have been processed, which is exactly what we want. One for this endpoint, and one for this endpoint. I'm going to leave this for a couple of minutes just to see some more checks roll in. And we'll see the difference. Okay, so I've left this run for a few minutes. And as you can see, our checks are
07:07
performing nicely. If you look at the times here as well, they're pretty much going on time. We have a couple of seconds in between, but that's kind of normal because obviously the request takes a little bit of time to actually process and send the HTTP request. But everything is lining up nicely. And our checks are being made as they should.

Episode summary

In this episode, we're taking the next step with our queuing setup by actually queuing the job that checks our endpoints. First up, we do a bit of housekeeping by clearing out previous checks from the database so we can see our changes more clearly.

Next, we jump into the endpoint check job and add the ShouldQueue interface. This means when the job is dispatched, it gets put on our queue instead of running immediately. We run a quick test using php artisan short-schedule:run to dispatch jobs and watch them pop up in the jobs table. However, there's a problem: our jobs are piling up really quickly because nothing is stopping them from being queued repeatedly every second (since the next check date isn't being updated until after the job runs).

We explore why this happens (the core of it: the job's update to the next check date doesn't happen until the job actually processes, so the scheduler keeps queuing it if the previous one hasn't finished), and see that we end up with duplicate jobs for the same endpoint.

To fix this, we introduce job uniqueness by using Laravel's ShouldBeUnique interface. By assigning a unique key to each job (based on the endpoint's ID), we make sure only one job per endpoint can be in the queue at a time. As a further safeguard, we make the unique ID even more specific (like 'endpoint_1'), in case we have other jobs with overlapping IDs.

After making these adjustments, we run both our scheduler and queue worker together and watch the database to confirm: now we're only processing one check per endpoint at a time, just as we intended! The times might lag by a second or two (which is normal), but everything is running smoothly without duplicate checks.

Episode discussion

No comments, yet. Be the first!