The PR Looked Fine Until OAuth Sent Users Somewhere Else

A pull request goes green. Unit tests pass. Integration tests pass. Linting is clean. Preview deploy looks fine at a glance.

Then the first real user clicks Continue with Google and lands on the wrong callback URL, gets stuck in a popup loop, loses session state between domains, or returns authenticated but stranded on a blank onboarding page.

This is one of the most common ways modern teams ship broken software while believing their process is working.

The problem is not that engineers do not care about testing. The problem is that most testing stacks were built to validate code behavior inside a controlled boundary, while the failure happened in a user workflow that crossed several boundaries at once: app domain, auth provider, cookies, redirects, local state, backend session exchange, and a browser environment that behaves differently in CI than in localhost.

As more code is generated by AI, scaffolded from templates, or assembled from SDK snippets, this gap gets worse. The code often looks reasonable. The PR diff appears small. The logic may even be correct in isolation. But reliability does not live inside isolated functions. Reliability lives in whether a real user can finish a real task.

That is the core mistake behind many “it passed CI” incidents: teams are validating code paths, not workflows.

The failure pattern behind green PRs and broken auth

OAuth and other external auth systems fail in ways that traditional test suites are structurally bad at catching.

A typical login flow is not one action. It is a chain:

1. User clicks a button.
2. Browser opens a popup or redirects away.
3. Third-party auth page loads on another origin.
4. User consents or signs in.
5. Provider redirects back to a callback URL.
6. App exchanges a code or token server-side.
7. Cookies or storage get written.
8. Frontend reads authenticated state.
9. User is routed to the intended post-login destination.
10. Downstream gated features now need to work with that session.

A failure at any step can produce a broken experience while every test below the workflow level remains green.

Common examples:

• The callback URL is correct in development but wrong in preview deployments.
• The provider allows one redirect domain, but CI uses another.
• A SameSite or secure cookie setting blocks session persistence in one environment.
• A popup-based flow works in headed mode but fails in headless CI.
• The app returns from auth successfully but loses the original intended route.
• Middleware rewrites an authenticated user back to /login because a session cookie is not visible on the expected subdomain.
• The frontend assumes session hydration completes before rendering a gated step, causing a redirect loop.
• A third-party checkout or identity flow opens a new tab your tests never followed.
• Staging and production use slightly different provider configurations, scopes, or callback handlers.

None of these failures are exotic. They are routine. They happen because real applications are distributed state machines wearing the costume of a web page.

Why current approaches fail

Most teams rely on some mix of unit tests, integration tests, manual QA, and CI/CD checks. All of them are useful. None of them are enough by themselves.

Unit tests validate logic, not lived behavior

Unit tests answer questions like:

• Does this function generate the right auth URL?
• Does this reducer store the token correctly?
• Does this callback handler parse the provider response?
• Does this middleware redirect unauthenticated requests?

Those are good questions. They are not the user’s question.

The user’s question is simpler: Can I sign in and continue what I was trying to do?

A unit test can prove your URL builder includes the right query params while still missing that the provider dashboard points to an outdated callback path. A unit test can confirm the session parser works while missing that the cookie was never set in a browser context.

This is why teams with excellent unit coverage still ship broken onboarding, login, and purchase flows.

Integration tests stop at the system boundary that matters most

Integration tests usually improve on unit tests by exercising more of the application stack. Maybe they hit a test database. Maybe they call your backend. Maybe they render pages in a test browser.

But many integration suites still fake the hard part:

• Mock the auth provider response
• Bypass login via test helpers
• Inject a session cookie directly
• Stub token exchange APIs
• Skip popup behavior entirely

This keeps tests fast and deterministic. It also removes the exact source of production risk.

Mocked auth is useful for broad app coverage, but it creates false confidence when teams mistake it for workflow validation.

If your tests log in by doing this:

js
await page.context().addCookies([
  {
    name: 'session',
    value: 'test-session',
    domain: 'localhost',
    path: '/',
    httpOnly: true,
    secure: false,
    sameSite: 'Lax'
  }
])

you are not testing login. You are testing the application after login.

That distinction matters.

CI/CD reports pass/fail on the wrong abstractions

Most CI/CD pipelines are optimized around static analysis, unit suites, build success, maybe a handful of browser tests, and deploy previews.

That setup is good at answering:

• Did the code compile?
• Did tests pass in isolation?
• Did we break known component behavior?
• Can the app boot?

It is weak at answering:

• Can a user start unauthenticated, traverse multiple origins, and come back with a valid session?
• Does a checkout flow survive redirects to payment providers and back?
• Does onboarding preserve state across auth and tab transitions?
• Does session state behave the same in preview, staging, and production-like domains?

Teams often treat a green pipeline as a proxy for release readiness. That is the false confidence trap.

A CI pipeline only proves what it actually checked. If it never followed the redirect chain, never observed cookie behavior, never tracked popup state, and never validated the final user outcome, “green” means very little.

Manual QA is too late and too inconsistent

Human QA can catch workflow failures, but it has hard limits:

• It is expensive to run on every PR.
• It is hard to make consistent across environments.
• It rarely covers every combination of browser behavior, popup handling, and redirect state.
• It tends to happen after merge or right before release.

For auth and payments, late discovery is particularly costly. These are not cosmetic bugs. They are conversion bugs.

Broken login means users never start. Broken checkout means revenue stops. Broken onboarding means activation drops and support volume rises.

The core insight: test the action, not the implementation

If the business-critical outcome is “user signs in and reaches dashboard,” then your most important test should validate exactly that.

Not token parsing. Not callback helper behavior. Not session reducer logic. Not whether the login button renders.

The outcome.

This is where action-level verification becomes essential.

Action-level verification means your CI runs tests that follow the same path a user takes:

• Start logged out
• Click the real sign-in button
• Follow redirects or popup flows
• Complete the auth step against a realistic provider surface or controlled equivalent
• Return to the app
• Verify cookies, session state, route state, and downstream functionality
• Confirm the next user goal still works

This approach changes what your tests are modeling. Instead of asserting pieces of implementation, you assert that a business-critical workflow survives the browser, the network, third-party domains, and your app’s own state transitions.

That is what reliability actually means.

Why agent-written code makes this worse

AI-assisted development increases output, but it also increases the number of changes that look plausible while hiding workflow risk.

An agent can:

• Refactor auth middleware
• Rename callback routes
• Update route guards
• Replace a session library
• Add onboarding redirects
• Change state persistence strategy
• Swap popup login for full-page redirect

All of these changes can be locally coherent and still break the end-to-end flow.

The problem is not that the code is “bad.” The problem is that generated code often optimizes for local correctness and syntactic completion. It does not inherently understand your deployed auth provider settings, your preview domain conventions, your cookie policy, or the exact route restoration behavior users depend on.

Traditional testing does not compensate for this. In many teams, AI speeds up shipping while validation remains scoped to the same low-level checks as before. That means more surface area changes with the same blind spots.

This is one reason developer productivity discussions need to include debugging and testing quality, not just generation speed. Writing code faster does not matter if broken workflows escape faster too.

A concrete example: the callback route changed, everything stayed green

Consider a Next.js app using OAuth.

A developer or coding agent changes the callback path from /api/auth/callback/google to /auth/callback/google to align with a new routing structure.

The PR includes:

• Updated frontend login button
• Updated backend callback handler
• Passing unit tests
• Passing route tests
• Passing mocked authenticated dashboard tests

But the Google provider configuration in staging and preview environments still points to the old callback path.

What happens?

• Local testing might pass if the developer uses a different provider app or a localhost-specific config.
• Unit tests pass because URL construction was updated correctly.
• Integration tests pass because auth is mocked.
• CI passes because no browser test actually goes through the provider redirect.

After merge, users authenticate with Google successfully, get redirected back to the old path, hit a 404 or invalid state handler, and fail to log in.

This is not a code bug in the narrow sense. It is a workflow failure caused by a config-runtime mismatch. Those are exactly the failures green PRs regularly miss.

What real workflow validation looks like in Playwright

Playwright is a strong fit for this kind of testing because it can observe multiple pages, browser contexts, redirects, storage state, and popup behavior with enough precision to test actual user flows.

A minimal example of a redirect-based login flow might look like this:

ts
import { test, expect } from '@playwright/test'

test('user can sign in and reach dashboard', async ({ page }) => {
  await page.goto('https://preview.example.com')

  await page.getByRole('button', { name: /continue with google/i }).click()

  await page.waitForURL(/accounts\.google\.com|auth\.example-provider\.com/)

  // In a real test environment, use a dedicated test account or provider sandbox.
  await page.getByLabel(/email/i).fill(process.env.E2E_AUTH_EMAIL!)
  await page.getByRole('button', { name: /next|continue/i }).click()
  await page.getByLabel(/password/i).fill(process.env.E2E_AUTH_PASSWORD!)
  await page.getByRole('button', { name: /next|sign in/i }).click()

  await page.waitForURL(/preview\.example\.com/)
  await expect(page).toHaveURL(/dashboard/)
  await expect(page.getByText(/welcome/i)).toBeVisible()

  const cookies = await page.context().cookies()
  expect(cookies.some(c => c.name === 'session')).toBeTruthy()
})

That already validates far more than mocked login. But many real flows use popups or intermediate pages.

Here is a popup version:

ts
import { test, expect } from '@playwright/test'

test('oauth popup returns authenticated user to onboarding', async ({ page, context }) => {
  await page.goto('https://preview.example.com/pricing')
  await page.getByRole('button', { name: /start free trial/i }).click()

  const popupPromise = page.waitForEvent('popup')
  await page.getByRole('button', { name: /continue with github/i }).click()
  const popup = await popupPromise

  await popup.waitForLoadState()
  await expect(popup).toHaveURL(/github\.com\/login|github\.com\/sessions/)

  await popup.getByLabel(/username or email/i).fill(process.env.E2E_GITHUB_USER!)
  await popup.getByLabel(/password/i).fill(process.env.E2E_GITHUB_PASSWORD!)
  await popup.getByRole('button', { name: /sign in/i }).click()

  // Sometimes consent appears only on first run.
  const authorizeButton = popup.getByRole('button', { name: /authorize|continue/i })
  if (await authorizeButton.isVisible().catch(() => false)) {
    await authorizeButton.click()
  }

  await popup.waitForEvent('close')

  await expect(page).toHaveURL(/onboarding/)
  await expect(page.getByRole('heading', { name: /set up your workspace/i })).toBeVisible()

  // Verify original intent survived auth.
  await expect(page.getByText(/free trial/i)).toBeVisible()
})

The important part is not the exact selector strategy. It is that the test follows the user journey across tabs, origins, and session transitions.

Validate session state explicitly

One of the biggest sources of auth bugs is assuming success because the UI changed. Do not stop at “dashboard loaded.” Verify the state that matters.

In Playwright, that can mean checking cookies, local storage, and authenticated API responses.

ts
import { test, expect } from '@playwright/test'

test('session persists after oauth redirect', async ({ page, request }) => {
  await page.goto('https://preview.example.com/login')
  await page.getByRole('button', { name: /continue with google/i }).click()

  // Complete provider flow...

  await page.waitForURL(/dashboard/)

  const cookies = await page.context().cookies()
  const sessionCookie = cookies.find(c => c.name === 'session')

  expect(sessionCookie).toBeDefined()
  expect(sessionCookie?.secure).toBeTruthy()

  const response = await page.request.get('https://preview.example.com/api/me')
  expect(response.ok()).toBeTruthy()

  const me = await response.json()
  expect(me.email).toBe(process.env.E2E_AUTH_EMAIL)
})

That catches a class of bugs where the page appears to recover, but authenticated backend access is missing or inconsistent.

Multi-step flows are where the real bugs hide

Login alone is often not the right test target. The more valuable validation is the user goal connected to login.

Examples:

• User begins checkout, signs in, returns to checkout, completes payment.
• User clicks invite link, authenticates, lands inside the correct team.
• User starts onboarding, completes SSO, returns to the right step with preserved workspace data.
• User attempts to access a gated report, signs in, returns to the original report view.

These are state restoration problems as much as auth problems.

Here is a simplified example for route restoration:

ts
import { test, expect } from '@playwright/test'

test('protected route is restored after login', async ({ page }) => {
  await page.goto('https://preview.example.com/reports/quarterly?team=acme')

  await expect(page).toHaveURL(/login/)
  await page.getByRole('button', { name: /continue with google/i }).click()

  // Complete auth flow...

  await expect(page).toHaveURL(/reports\/quarterly\?team=acme/)
  await expect(page.getByRole('heading', { name: /quarterly report/i })).toBeVisible()
})

That single test may be more valuable than dozens of isolated auth-related unit tests because it validates the thing users care about.

CI configuration that supports workflow testing

If you want these tests to matter, they have to run in CI before merge, not only in nightly suites no one trusts.

A practical GitHub Actions setup might look like this:

yaml
name: e2e-workflows

on:
  pull_request:
  workflow_dispatch:

jobs:
  workflow-tests:
    runs-on: ubuntu-latest
    timeout-minutes: 30
    env:
      BASE_URL: ${{ secrets.E2E_BASE_URL }}
      E2E_AUTH_EMAIL: ${{ secrets.E2E_AUTH_EMAIL }}
      E2E_AUTH_PASSWORD: ${{ secrets.E2E_AUTH_PASSWORD }}
      E2E_GITHUB_USER: ${{ secrets.E2E_GITHUB_USER }}
      E2E_GITHUB_PASSWORD: ${{ secrets.E2E_GITHUB_PASSWORD }}
    steps:
      - uses: actions/checkout@v4

      - uses: actions/setup-node@v4
        with:
          node-version: 20

      - run: npm ci
      - run: npx playwright install --with-deps
      - run: npm run build
      - run: npm run start &
      - run: npx wait-on http://localhost:3000
      - run: npm run test:e2e:critical

      - if: failure()
        uses: actions/upload-artifact@v4
        with:
          name: playwright-artifacts
          path: |
            playwright-report
            test-results

A few important notes:

• Run only critical workflow tests on every PR. Do not dump hundreds of flaky browser tests into merge protection.
• Use provider sandbox or dedicated test tenants where possible.
• Capture screenshots, traces, videos, and network logs on failure. Good debugging depends on artifacts.
• Keep test environments domain-realistic enough to exercise cookies and redirects correctly.

If your preview environment has a unique URL per PR, make sure your auth provider supports dynamic or wildcard redirect strategies safely, or create a broker pattern that normalizes callbacks.

When real providers are hard to use in CI

There is a legitimate objection here: many third-party auth surfaces are brittle, rate-limited, protected by anti-bot measures, or simply painful to automate.

That does not mean you should retreat to full mocking. It means you need a layered strategy.

Use three levels:

1. Mocked auth for broad app coverage

Use fast mocked login for the majority of tests that are not about auth itself.

This keeps your suite efficient.

2. Controlled provider-equivalent environments for merge-critical paths

Where possible, stand up a test identity provider or sandbox tenant that behaves like production enough to validate:

• Redirect handling
• Cookie/session exchange
• Route restoration
• Popup or tab transitions
• Consent/scope handling

This is the sweet spot for CI reliability.

3. Periodic production-like smoke validation

Run a smaller set of production-like auth checks against staging or a locked-down prod-equivalent environment on a schedule and before major releases.

This catches provider config drift and environment mismatches that PR-level tests may miss.

The goal is not purity. The goal is risk reduction around workflows that break the business.

JavaScript and Python examples for session-aware assertions

Teams often mix stacks, so it is useful to standardize the behavior you verify even if the test framework differs.

A JavaScript helper for preserving intended destination:

js
export async function assertRestoredAfterLogin(page, expectedPathPattern) {
  await page.waitForLoadState('networkidle')
  await expect(page).toHaveURL(expectedPathPattern)

  const response = await page.request.get(`${process.env.BASE_URL}/api/me`)
  expect(response.ok()).toBeTruthy()

  const user = await response.json()
  expect(user.id).toBeTruthy()
}

A Python example using Playwright for Python:

python
from playwright.sync_api import expect
import os

def test_user_returns_to_checkout_after_auth(page):
    base_url = os.environ["BASE_URL"]
    page.goto(f"{base_url}/checkout")

    expect(page).to_have_url(lambda url: "/login" in url or "/checkout" in url)

    page.get_by_role("button", name="Continue with Google").click()

    # Complete auth flow in your controlled environment here.

    expect(page).to_have_url(lambda url: "/checkout" in url)
    expect(page.get_by_role("heading", name="Checkout")).to_be_visible()

    response = page.request.get(f"{base_url}/api/me")
    assert response.ok
    assert response.json()["email"] == os.environ["E2E_AUTH_EMAIL"]

The point is consistent verification semantics:

• Did the user get back to the intended place?
• Is the session actually active?
• Can the next action succeed?

Tools comparison: what each approach is good at

No single tool solves reliability. Different layers catch different failures.

Unit test frameworks: Jest, Vitest, pytest

Best for:

• Pure logic validation
• Fast feedback during development
• Edge-case handling in helpers and middleware

Weak for:

• Redirect chains
• Browser session behavior
• Cross-origin auth workflows
• Popup/tab management

Verdict: necessary, insufficient.

API integration testing: Supertest, pytest + requests, contract tests

Best for:

• Backend callback handlers
• Token exchange logic
• Auth-related API contracts
• Session endpoint behavior

Weak for:

• Browser cookie policies
• Route restoration
• User-visible flow breakage

Verdict: useful for backend confidence, not enough for workflow confidence.

Browser automation: Playwright, Cypress

Best for:

• Real user workflows
• Multi-page and popup handling
• Storage and cookie inspection
• Debugging with traces and screenshots

Caveat:

• Requires discipline to avoid brittle tests
• Cross-origin behavior is easier in Playwright than in many alternatives

Verdict: strongest layer for critical auth, onboarding, and checkout verification.

Manual QA

Best for:

• Exploratory coverage
• Visual anomalies
• Rare edge cases and usability issues

Weak for:

• Repeatability
• Per-PR enforcement
• Speed

Verdict: valuable support layer, not a primary merge gate.

Actionable practices that actually reduce these failures

Here is the short list that matters most.

1. Define critical user workflows explicitly

Every product has a handful of flows that must never break:

• Sign up
• Log in
• Password reset or SSO join
• Checkout or upgrade
• Invite acceptance
• Onboarding completion

Write them down. Treat them as merge-critical.

If a flow affects activation, revenue, or access, it deserves action-level validation.

2. Separate “authenticated app coverage” from “auth workflow coverage”

Keep mocked login for broad UI and feature coverage. But do not confuse it with login validation.

Label tests clearly:

• app-authenticated-mocked
• workflow-oauth-realistic
• checkout-redirect-critical

That naming alone improves team reasoning.

3. Test the full path from intent to outcome

Do not stop your test at “user is logged in.” Test the continuation:

• User wanted to buy
• User wanted to access a team
• User wanted to view a report
• User wanted to finish onboarding

The bug often appears after auth, not during it.

4. Verify state, not just UI

Assert:

• Final URL
• Session cookie presence and attributes
• Authenticated API response
• Preserved route/query state
• Ability to perform the next protected action

This makes debugging much faster because you can pinpoint whether the break is in redirect, session exchange, or app hydration.

5. Make CI artifacts first-class debugging tools

On workflow test failure, capture:

• Playwright trace
• Screenshot
• Video
• Browser console logs
• Network logs or HAR when feasible

Good debugging is not just about detecting failure. It is about making failure explainable.

6. Protect against environment drift

Many auth failures come from config drift, not code regressions.

Audit regularly:

• Redirect URIs
• Allowed origins
• Cookie domain settings
• Secure and SameSite flags
• Preview/staging domain behavior
• Provider scopes and consent screens

A passing local flow tells you almost nothing about deployed correctness if environment settings differ.

7. Keep the critical suite small and trusted

Do not gate merges on a giant flaky end-to-end suite. Gate merges on 5–15 highly curated workflow tests that teams trust.

If a critical test flakes, fix the test or the environment immediately. Merge-gating tests must have social credibility.

8. Use dedicated test accounts and resettable tenants

Auth and onboarding tests need stable identities. Avoid shared human credentials or semi-manual test data.

Invest in:

• Dedicated provider sandbox accounts
• Tenant reset scripts
• Test user lifecycle helpers
• Isolated workspaces for onboarding and invite flows

Stable data is part of reliable testing.

9. Review auth-related PRs as workflow changes, not code diffs

When a PR touches:

• auth middleware
• redirects
• cookies
• route guards
• onboarding state
• checkout/session handoff

ask one question: Which user workflow could this break?

Then require evidence that workflow still works.

10. Optimize for trust, not just speed

A fast CI pipeline that misses revenue-killing bugs is not efficient. It is misleading.

Developer productivity improves when engineers trust that green means something real. That trust comes from validating outcomes users depend on.

The deeper shift teams need to make

A lot of engineering organizations still think about testing as coverage over code. That model is too narrow for modern web apps.

The more accurate model is coverage over user-critical state transitions.

Code matters, but users do not experience your code in isolated modules. They experience:

• redirects
• cookies
• browser policies
• tab focus changes
• hydration timing
• provider handshakes
• route restoration
• backend session exchange

That is the system.

If your testing strategy excludes those transitions, then your confidence is mostly administrative. You have proof that your tools ran, not proof that your product works.

This is especially important now that AI can generate implementation faster than teams can reason about all consequences by inspection. The answer is not to distrust generated code categorically. The answer is to raise the quality of verification to the level of actual user behavior.

Conclusion

The PR looked fine because the PR was never testing the thing that broke.

Green checks on unit tests, mocked integration tests, and basic CI/CD steps do not guarantee that a user can cross an auth boundary, survive redirects and popup behavior, preserve session state, and continue the task they came to complete.

That is why login, checkout, onboarding, and invite flows still fail in production even on disciplined teams.

The fix is not more generic testing. It is more specific testing.

Test the exact action the user takes. Follow the real workflow across origins, redirects, cookies, tabs, and provider surfaces. Verify not just that authentication technically happened, but that the user landed where they needed to be with a working session and a usable next step.

If you do that before merge, your CI starts measuring reality instead of ceremony.

And when AI or humans change auth code next week, “green” might finally mean something worth trusting.