In this episode, we tackle a sneaky bug related to our recovery notifications. If you've been following along, you might've noticed that when you add a fresh endpoint (one that's working), you weirdly get a recovery email—when nothing was actually down to begin with! So, to demonstrate, we walk through adding a new endpoint while watching the logs and the email client. Sure enough, that "recovered" email pops up immediately, which isn't what we want.
Digging into the code, we check the logic around when to send out recovery notifications—specifically, in our check observer. The bug comes from a check that doesn't properly handle freshly added endpoints that have never failed before. There's a logic assumption that there would always be a previous check, but for brand new endpoints there isn't, so things break.
To fix this, we change the condition from checking if the count of checks is greater than one, to verifying that the checks count does not equal one. That way, we steer clear of sending recovery emails right when someone adds a fresh and healthy endpoint for the first time.
After a quick code tweak and restarting processes, we test the workflow: add an endpoint, confirm no "recovered" email is sent at first, then simulate a fail and recovery to ensure the notifications work correctly. Everything works as expected. In the end, a simple logic fix saves us from accidentally spamming users with unnecessary recovery notifications!