Failing tests in CI

Howdy! 🤠 Let's dive in.

We have three specific tests that constantly fail in CI. We are using GitHub Actions.

LoginTest

LogoutTest

RegisterTest

  FAILED  Tests\Feature\LoginTest > a user can login   NotFoundHttpException   
  POST http://localhost/api/v1/login

The tests pass locally just fine. I can get them to fail locally though. Running php artisan optimize:clear followed by pest (alias) will reproduce the same errors and behaviours. If I then run php artisan optimize then the tests pass again. I don't want to clear the cache in CI because it yields other unexpected behaviours and quirks. e.g.

  SQLSTATE[08006] [7] connection to server at "127.0.0.1", port 5432 failed: FATAL:  role "root" does not exist (Connection: pgsql, SQL: drop database if exists "laravel_test_1")

root is not even referenced anywhere. I have no idea where it pulls that from, or why. Skipping the failed tests and not attempting to clear the cache, allows all other tests to pass. Including ones that interact with the database.

These three tests never used to fail before switching our authentication logic to be cookie-based. The only thing that has changed in these tests is one use statement.

use Illuminate\Auth\Events\Login;

We weren't previously using Fortify, so we used our Login event.

Here are the failing tests:

test('a user can login', function () {
    Event::fake();

    $user = User::factory()->create();

    postJson('/api/v1/login', [
        'email'    => $user->email,
        'password' => 'password',
    ])
        ->assertSuccessful();

    Event::assertListening(Login::class, TrackUserLogin::class);
    Event::assertDispatched(Login::class);
})
    ->skip();

test('a user can logout', function () {
    $user = User::factory()->create();

    actingAs($user);

    postJson('/api/v1/logout')
        ->assertSuccessful();

    assertGuest();
})
    ->skip();

test('a user can register', function () {
    Event::fake();

    postJson('/api/v1/register', [
        'name'                  => 'Ernest Thornhill',
        'company'               => 'Thornhill Corporation',
        'email'                 => 'ernest.thornhill@example.com',
        'email_confirmation'    => 'ernest.thornhill@example.com',
        'password'              => 'password',
        'password_confirmation' => 'password',
    ])
        ->assertSuccessful();

    Event::assertDispatched(Registered::class);
})
    ->skip();

The routes are correct. The requests made in our tests work just fine in Postman.