Build An Uptime Monitor with Inertia
This episode is for members only

Sign up to access "Build An Uptime Monitor with Inertia" right now.

Get started
Already a member? Sign in to continue
Playing
37. Observing when to notify

Episodes

0%
Your progress
  • Total: 4h 59m
  • Played: 0m
  • Remaining: 4h 59m
Join or sign in to track your progress
01. Introduction
01. Introduction and demo
6m 33s
0%
02. A fresh Inertia project
6m 7s
0%
02. Sites
03. Site and dashboard setup
8m 59s
0%
04. Building the site selector dropdown
12m 22s
0%
05. Making use of API resources
4m 14s
0%
06. Setting the default site
10m 37s
0%
07. Setting up Vue modals
4m 34s
0%
08. Creating a site
18m 18s
0%
03. Endpoints
09. Setting up endpoints
4m 23s
0%
10. Enum endpoint frequencies
3m 50s
0%
11. Creating the endpoint form
12m 58s
0%
12. Storing an endpoint
7m 21s
0%
13. Validating and authorizing endpoint stores
7m 6s
0%
14. Normalizing the endpoint location
6m 4s
0%
15. Displaying endpoints
6m 22s
0%
16. Deleting endpoints
6m 21s
0%
17. Toggling inline editing for endpoints
5m 12s
0%
18. Applying a watcher to the endpoint form
5m 31s
0%
19. Storing endpoint edits
8m 56s
0%
04. Checking endpoints
20. Determining endpoints to check
2m 57s
0%
21. Creating a job to perform a check
3m 21s
0%
22. Making an HTTP request
3m 39s
0%
23. Scheduling checks every second
4m 27s
0%
24. Setting up the check relationship
3m 14s
0%
25. Updating the job
2m 10s
0%
26. Showing the last status
12m 50s
0%
27. Expanded list of checks
14m 10s
0%
28. The response body modal
2m 49s
0%
29. Authorizing the endpoint details
2m 27s
0%
30. Calculating uptime for endpoints
8m 29s
0%
05. Queuing
31. Setting up queues
3m 21s
0%
32. Queuing the endpoint check
7m 26s
0%
06. Email notifications
33. Setting up email in Laravel
5m 8s
0%
34. Site model email notification storage
3m 5s
0%
35. Adding email addresses via the UI
17m 24s
0%
36. Listing and deleting email addresses
7m 14s
0%
07. Downtime notifications
37. Observing when to notify
10m 51s
0%
38. Creating an endpoint down notification
10m 46s
0%
39. Creating an endpoint recovery notification
8m 9s
0%
40. Making the emails a bit friendlier
3m 50s
0%
08. Extras
41. Deleting sites
5m 9s
0%
42. Eager loading
4m 44s
0%
43. Recovery notification bug
2m 31s
0%
44. No sites fix and placeholder
3m 25s
0%

Transcript

00:00
By the end of this episode, we will be able to determine whether to notify the user if an endpoint goes down.
00:07
Now, you might be thinking, well, surely if a check is just unsuccessful, we should notify the user. Well, if we think about it, if we're checking every one minute this endpoint, what's going to happen is we don't want to send an email every minute. And if we just check if an endpoint is unsuccessful, we're going to be emailing the user every single minute.
00:29
And that might not be great. If you want it to work like that, that's great. It is very simple to do. But we want to be a little bit more nuanced. We want to say, well, we want to notify the user when an endpoint goes down. But then we want to wait and not send another email until it's recovered. So in this part, we can look at the logic around how to properly notify the user.
00:51
We're not actually going to send any emails just yet, but we're just going to log out when we should. When we get to the point when we email, we can implement that. So before we do anything, we're just going to go over and delete all of the endpoints that we currently have, just so we can sort of start fresh. That will get rid of the checks for us.
01:08
Let's just make sure there are no jobs in here and we can pretty much start from scratch. What I'm also going to do just for the purpose of this is come over to the endpoint frequency and I'm going to go ahead and set in a 10 second frequency. The reason being is just while we're testing this, we want this to be pretty fast.
01:30
So let's go ahead and add in a 10 second case in here, which is, of course, just going to be 10 seconds. So we can choose that now when we create our new endpoint. And let's go ahead and just say forward slash, because that does work on CodeCourse.com. And we want to monitor for this every 10 seconds.
01:49
So, of course, we want to go ahead and start up our queue and our scheduler. So let's go ahead and say PHP artisan queue work and let's say PHP artisan short schedule and run. So now that is going to be checking this and we should see a status of 200. And that is now checking every 10 seconds.
02:09
So if I just give this a refresh, we're now getting these in every 10 seconds. And that's going to give us a really opportunity to get some feedback about what we're doing. And if it works. So the question is, how are we going to detect when we get an unsuccessful status?
02:24
Well, if we go over to our event service provider, we've already created some observers for our site and our endpoint. And we could actually create an observer for our check model. So when a check is created in the database, that's a really good opportunity to then check if that check is successful. So we can just go ahead and create out an observer for this.
02:47
So let's go ahead and say PHP artisan make observer. And we're going to call that check observer. Great. And I'm just going to pull these over a little bit. So we've got a little bit more room to breathe.
03:00
OK, so check observer. Let's go ahead and hook this up first of all. So check and check observer. And we want to figure out what to do when a check is created.
03:14
So let's go ahead and build in that method. Pull in the check that we get. And we can go ahead and do something with this. Now, while we're doing this, we're going to be using the log facade to log something out to our Laravel log.
03:28
So we're going to go ahead and hang in here over in our Laravel log. And once these checks are performed and once these are created, we should see them roll in here. Now, if we make any changes while our queue is running, we're not going to see these reflected. So I'm going to go ahead and just cancel off our queue in here.
03:47
And I'm just going to restart the queue. And now if we come over to our Laravel log, we should see these now logged out. So when you make any changes to this, you're going to need to just restart your queue. So any changes that you've made are going to start working again.
04:03
So what we should see now as we come back to our Laravel log are more things being logged out. So we're just going to keep opening and closing that to see them changes. Okay, so we want to log out something in here if we should notify the user. That's pretty much what we want to do.
04:20
Now, I've made some notes on this just to make this process a little bit easier. So if the check wasn't successful, we don't want to just go ahead and log straight away. We only want to log this if the previous check was successful itself or if this is the first check that we're performing. So let's just imagine we create an endpoint out in here to monitor and it fails.
04:45
Of course, we want to go ahead and notify the user straight away that this endpoint is down. If we had a previously successful endpoint check, so if this is successful and the next one fails, we want to notify the user. But if the previous check failed, so for example, if this was a 404, we don't want to send another one. So that is where we get into the point where we're not sending repeated endpoint down notifications.
05:11
So with these notes, let's go ahead and start to implement what we've got. Okay, so we know that we already have over on our check model. Let's just open that up really quickly before we start an isSuccessful method. So we can use that to start to build this up.
05:30
So if the check was not successful, we could implement another method for that. Then we want to go ahead and just log out here. Notify user and let's go ahead and add in the check. Let's leave that out, actually.
05:47
So do we want to notify the user at this point if the check wasn't successful? Well, we do, but we want an additional layer of checking on here as well. So we want to say and the previous endpoint check was not successful or was successful. So if the previous one was successful and this one, the next one, was not successful, then we want to notify the user.
06:10
So from this, how do we get the previous check on a check itself? So on this model itself, how do we go back in time and look at the previous one? There are lots of different ways to do that, but we're going to go ahead and implement a really handy method over in the check model that gives us the previous record. So let's go ahead and create our method called previous in here.
06:36
And let's go ahead and say this endpoint checks. So we're going upper level onto the endpoint itself, going ahead and grabbing the checks in here. We're going to go ahead and order these by the ID in descending order. So really crucially here, we are relying on the fact that the ID is sequential, which it pretty much always is.
06:59
If you're using UUIDs, for example, this wouldn't work. And then we want to go ahead and say where the ID is less than this ID and grab the first record. So basically, we're ordering all of the checks by the ID in descending order, checking this where the ID is less than or equal to the ID that we currently have. That's going to be the one at the very top of the list, so the previous one, and then just grabbing the first one.
07:29
So that will give us the previous check. So now what we can do is say and check previous. Now we may not have a previous check in here, so we want to be really careful and use a null safe operator here and say is successful. So this will return null if the previous item didn't exist.
07:53
So what we have got so far is was the check unsuccessful and the previous check was successful. So we're only going to notify if the previous check was successful. That's only going to notify once. But we also need to take into account that this might only have one check.
08:13
So we're actually going to wrap this in parentheses, and we're also going to say or. So or check endpoint and checks count is one. So that pretty much wraps up the logic that we've got here. And I'm actually going to pull this down just so it's a little bit clearer like that.
08:37
So just one more time, we're only going to notify the user if the check is unsuccessful and the previous check was successful or there's only one check. Because remember, if we create an endpoint and it fails straight away, we then always want to notify the user. So let's just try this out. So we're going to go over to our Laravel log.
08:57
I'm going to get rid of everything that's being logged out in here. I'm going to go ahead and close off and just delete that. Close off the QWorker that we have currently running. And we're going to go ahead and start the QWorker.
09:10
So obviously, at the moment, we have an endpoint which is successfully running. This is running every 10 seconds. We should not see anything logged out because we're getting successful requests to this endpoint. Now, we shouldn't really be able to do this.
09:24
But if we edit the location, of course, that's then going to fail. So I'm going to go ahead and change this over to something like ABC. And I'm going to hit Done. Now, eventually, this is going to fail.
09:35
So we should see this logged out in here. So let's just wait for this to be run through. And sure enough, we get notify user. Now, in another 10 seconds, this request is still going to be unsuccessful.
09:47
But because we've set this up because the previous check was successful only when the previous check was successful, it's not going to work. So if we come over to our Laravel log, we should not see this logged out again despite the fact that we are creating more than one check. So that's the second time that check has run. And we should get a third one here as well.
10:08
But we are only notifying the user once. Really, really important. So, of course, this was quite long. And we introduced quite a lot here.
10:17
If you wanted to email the user every single time the endpoint was still down, of course, what you could do is just get rid of all of this and just have a single unsuccessful check in here to then notify the user. But really, if we think about it, it only makes sense to notify the user when the endpoint goes down and until it recovers. And we're going to get onto the recovery part of this a little bit later. Okay, so now that we've got the logic down for this, we actually want to move on to the next part and really notify the user that the endpoint went down.

Episode summary

In this episode, we're focusing on when to notify a user if one of their monitored endpoints goes down. It might sound simple—just notify every time a check fails, right? But that would be super annoying, especially if you're checking every minute (or, in our testing case, every 10 seconds!). No one wants a flood of emails, so we need to be a little smarter.

We start with a bit of setup: clearing out old endpoints and checks, and adjusting the check frequency so we can see results quickly. After that, we build the logic to detect when a notification should be sent. Instead of notifying on every failure, we set things up to only send a notification the first time an endpoint goes down, then stay quiet until the endpoint comes back up and fails again in the future. This involves creating a new observer for our check model in Laravel, and writing code to log (for now, not actually email!) when a notification would go out.

The main checks are: if the current check fails and the last one succeeded (or if it's the first check ever and it's a failure), that's when we notify. We also cover how to grab the previous check for an endpoint and how to make this logic efficient and clear.

To wrap up, we test our logic by changing an endpoint to deliberately fail and watch our logs to make sure notifications are handled just right. If you really want to be notified every single time, you could, but that's not the best user experience! Next time, we'll move on to actually sending out those email notifications when endpoints go down.

Episode discussion

No comments, yet. Be the first!