Your PR Review Can’t See the Broken Checkout

A checkout can be completely broken while a pull request looks clean, the tests are green, and the reviewer approves it in under ten minutes.

That sounds dramatic until you’ve seen it happen in production. The cart page loads. The product page works. The backend tests pass. The payment API health check is fine. Even the diff looks harmless: a small refactor in a shared component, a package bump, a selector cleanup, maybe some AI-generated “improvements” to form state. Nothing in the PR screams revenue outage.

Then a customer tries to pay. The shipping step never enables the Continue button. The card iframe mounts too late and the submit handler binds to the wrong element. A coupon recalculation wipes the address state. Apple Pay disappears only on WebKit. The review was technically careful and operationally useless.

That is the core problem: pull request review is a terrible interface for validating workflow correctness.

Line-by-line review is good at catching code style issues, obvious mistakes, naming problems, and occasionally logic bugs. It is not good at proving that a real user can complete a money-critical path in a real browser with real timing, real DOM updates, and real third-party dependencies. Yet many teams still treat PR approval, green CI/CD, and a few screenshots as enough evidence that the product works.

It isn’t.

And the situation is getting worse because AI now produces larger, faster, and often less-readable changes than many teams can meaningfully inspect. You can merge a PR that “looks safe” while silently shipping a broken checkout, failed signup, dead booking flow, or non-functional upgrade path. In other words: the exact workflows the business actually depends on.

The failure doesn’t live in the diff

When checkout breaks, the root cause often does not look like a “checkout change.” That is why review fails.

A reviewer sees a diff. A user experiences a workflow.

Those are not the same interface.

The reviewer sees:

• a renamed prop
• a refactored async function
• a form library upgrade
• a CSS class change
• a DOM structure cleanup
• a new loading state
• a dependency bump
• an AI-generated patch touching twelve files

The user experiences:

• whether the address autocomplete overlays the submit button
• whether the tax recalculation stalls the page
• whether the payment widget actually becomes interactive
• whether the button remains disabled after valid input
• whether browser autofill corrupts state
• whether an error toast steals focus and blocks the next step
• whether the browser navigates before the order request finishes

Most production failures happen in that second list.

Not because teams are careless, but because modern frontend systems are stateful, async, componentized, third-party-heavy, and increasingly generated or modified by tooling that optimizes for code completion, not workflow reliability.

A broken checkout usually emerges from interaction effects:

• selector drift between components and tests
• race conditions around data fetching and rendering
• stale client state after retries or navigation
• analytics or feature flag code changing execution order
• visual overlays intercepting clicks
• iframes from payment providers loading differently per browser
• server responses that are technically valid but operationally mishandled

None of this is legible in a normal PR review unless the reviewer manually pulls the branch, runs the app, seeds the right state, uses the right browser, executes the exact flow, and notices the subtle failure. That rarely happens consistently, especially on fast-moving teams.

PR review optimizes for readability, not truth

There is a widespread but flawed assumption in software teams: if a change is understandable in review, then the risk is manageable.

That was never fully true, but it was more tolerable when changes were smaller, interfaces were simpler, and teams shipped more slowly. It is much less true now.

Review is built around textual artifacts:

• diffs
• comments
• screenshots
• logs
• test output
• summaries from the author

Those artifacts are indirect evidence. They tell you what changed in code. They do not tell you whether the workflow still works.

This is the key mismatch. We ask reviewers to approve user-facing behavior through a line-oriented code interface.

That forces trust in proxies:

• “The test suite passed.”
• “I ran it locally.”
• “The screenshot looks right.”
• “The AI agent said it updated the flow safely.”
• “The component story still renders.”
• “The backend contract didn’t change.”

But if the actual question is “Can a user complete checkout in Chromium, WebKit, and mobile viewport under realistic conditions?” none of those proxies are sufficient.

A screenshot can’t show that a button never becomes enabled. A diff can’t show that a spinner never resolves. A passing unit test can’t show that a payment iframe swallowed focus. A PR comment can’t show that a click hit the wrong element because of a z-index regression.

This is why teams get blindsided by “how did this pass review?” incidents. Review did what it is designed to do. The team expected it to prove something it fundamentally cannot prove.

Why AI-generated changes make this much worse

AI code generation changes the economics of shipping.

More code gets written. More refactors get attempted. More “safe cleanups” happen. More files get touched. More glue logic appears. More patterns are copied from nearby code without understanding all the runtime assumptions.

That increases output. It also increases review surface area faster than reviewer attention can scale.

A senior engineer might have been able to deeply inspect a 120-line handcrafted change. That same engineer cannot reliably reason through a 1,200-line AI-assisted PR spanning UI components, async actions, tests, and configuration while also holding the whole checkout workflow in their head.

This is not an argument against AI. It is an argument against pretending review scales linearly with generated code volume.

AI-generated changes often have a specific risk profile:

• they are syntactically plausible
• they align with local code style
• they pass narrow tests
• they preserve types
• they look consistent in diff view
• they subtly break behavior at integration boundaries

That is exactly the kind of change that survives review.

For example, an AI agent may “improve” form handling by consolidating state updates. The code becomes shorter and arguably cleaner. Unit tests around reducers pass. The component renders correctly in isolation. But under real use, the shipping address update now races with payment method initialization, and the submit button remains disabled after autofill.

Nothing in the review necessarily surfaces this. In fact, the improved readability can make the reviewer feel more confident.

That confidence is false.

As AI output increases, teams need stronger executed evidence, not better prose in PR descriptions.

Green CI/CD is often green for the wrong reasons

Teams lean on CI/CD because they need automation. That part is correct. The problem is what most pipelines actually validate.

A typical pipeline checks some combination of:

• type safety
• unit tests
• linting
• formatting
• build success
• API contract tests
• snapshots
• maybe a few integration tests

Useful? Absolutely. Sufficient? Not even close.

A checkout flow can fail while all of those remain green.

Why? Because CI usually validates code paths, not user workflows.

Unit tests check functions and components in controlled environments. Integration tests often mock the unstable parts. Snapshot tests confirm structure, not behavior. Build checks tell you the app compiles. Linting tells you almost nothing about runtime correctness.

Even many browser tests give false confidence because they are too synthetic:

• they mock payment providers n- they skip real waiting conditions
• they click elements by brittle selectors
• they don’t verify intermediate states
• they only run on one browser
• they don’t attach traces or video to the PR
• they are isolated from realistic user data

Worse, some teams respond to flaky end-to-end tests by reducing coverage around critical flows rather than improving test design. The result is a pipeline that is stable but strategically blind.

That blindness matters most at the money step.

If the checkout is the business-critical path, then “all unit tests passed” is not the same as “we can still take payment.” Those are radically different claims.

Why manual QA alone doesn’t solve it

The usual response is: that’s what QA is for.

No. Not by itself.

Manual QA is valuable, especially for exploratory testing and weird edge cases. But it is not a scalable substitute for workflow verification attached to each PR.

Manual QA breaks down because:

• it is slower than merge velocity
• it is inconsistent across testers and environments
• it often happens after review, not during review
• it creates queueing delays
• it relies on documentation that lags reality
• it does not produce durable evidence by default
• it struggles to cover browser/device combinations continuously

Most importantly, manual QA does not change the reviewer’s interface. The reviewer still sees a diff and a statement that someone tested it.

That is still indirect trust.

For modern delivery, especially where AI agents generate a meaningful amount of code, you need direct evidence embedded into the development loop: this branch executed the user workflow, in a real browser, against a realistic environment, and here is the action-level proof.

The core insight: test the workflow the way the user experiences it

If the business depends on checkout, then the artifact attached to the PR should not just be code. It should be evidence that checkout still works.

Not a screenshot of the page. Not a green checkmark from a generic pipeline. Not a comment saying “tested locally.”

Evidence.

Specifically:

• the sequence of actions taken
• the browser used
• the network behavior during the flow
• the DOM states at key steps
• the result of each assertion
• traces, screenshots, or video where useful
• a pass/fail signal tied to the exact user journey

This is the level where workflow correctness becomes reviewable.

A reviewer should be able to answer:

• Did the branch add an item to cart?
• Did shipping information submit successfully?
• Did the payment widget load and become actionable?
• Did the place-order action complete?
• Was confirmation shown?
• If it failed, where exactly did it fail?

That is a dramatically better interface than “Here’s a 700-line diff touching billing, address forms, and button state management. LGTM?”

A concrete broken-checkout example

Here is a simplified React example. It looks fine in review. It might even pass unit tests.

jsx
import { useEffect, useState } from 'react';

export function CheckoutButton({ isFormValid, paymentReady, submitOrder }) {
  const [enabled, setEnabled] = useState(false);
  const [submitting, setSubmitting] = useState(false);

  useEffect(() => {
    setEnabled(isFormValid && paymentReady);
  }, [isFormValid]);

  async function handleClick() {
    if (!enabled || submitting) return;
    setSubmitting(true);
    try {
      await submitOrder();
    } finally {
      setSubmitting(false);
    }
  }

  return (
    <button disabled={!enabled || submitting} onClick={handleClick}>
      {submitting ? 'Processing...' : 'Place order'}
    </button>
  );
}

The bug is subtle: paymentReady is used inside the effect but omitted from the dependency array.

In some flows, especially when the payment widget initializes after the form becomes valid, the button never enables. The UI looks fine. The tests may still pass if they initialize props in the “ready” state. A reviewer can easily miss it.

A narrow unit test might catch some behavior:

js
import { render, screen } from '@testing-library/react';
import { CheckoutButton } from './CheckoutButton';

test('disables button when form is invalid', () => {
  render(
    <CheckoutButton
      isFormValid={false}
      paymentReady={true}
      submitOrder={async () => {}}
    />
  );

  expect(screen.getByRole('button', { name: /place order/i })).toBeDisabled();
});

This test passes. It tells you almost nothing about the real failure.

A better browser-level Playwright test exercises the actual sequence:

ts
import { test, expect } from '@playwright/test';

test('user can complete checkout', async ({ page }) => {
  await page.goto('/product/sku-123');
  await page.getByRole('button', { name: /add to cart/i }).click();
  await page.goto('/checkout');

  await page.getByLabel('Email').fill('buyer@example.com');
  await page.getByLabel('Address').fill('123 Market St');
  await page.getByLabel('City').fill('San Francisco');
  await page.getByLabel('ZIP').fill('94103');

  await page.getByTestId('card-number').fill('4242424242424242');
  await page.getByTestId('card-expiry').fill('12/30');
  await page.getByTestId('card-cvc').fill('123');

  const placeOrder = page.getByRole('button', { name: /place order/i });
  await expect(placeOrder).toBeEnabled();
  await placeOrder.click();

  await expect(page.getByText(/thank you for your order/i)).toBeVisible();
});

This is closer to what matters. Better still would be testing the real payment iframe integration in a safe sandbox environment, with trace capture enabled.

Action-level evidence changes the review conversation

Once executed workflow evidence is attached to a PR, the review changes from speculative to concrete.

Without action-level evidence, the conversation sounds like this:

• “I think this refactor is safe.”
• “The tests passed.”
• “Can you add a screenshot?”
• “I clicked around locally.”

With action-level evidence, it becomes:

• “The checkout flow passed in Chromium and WebKit.”
• “The shipping step took 420ms longer than main, likely due to tax recalculation.”
• “The payment widget timed out on mobile Safari.”
• “The Place order button remained disabled after autofill on this branch.”
• “Here is the trace showing the exact failing action.”

That is a radically more useful debugging and testing surface.

It improves developer productivity not because it removes review, but because it gives review the thing it was missing: observable behavior.

What this looks like in Playwright

Playwright is not the only option, but it is a strong fit because it gives you real browser automation plus traces, screenshots, videos, and robust assertions.

A more production-minded checkout test might look like this:

ts
import { test, expect } from '@playwright/test';

test.describe('checkout workflow', () => {
  test('guest user can purchase a product', async ({ page }) => {
    await page.goto('/product/sku-123');

    await page.getByRole('button', { name: 'Add to cart' }).click();
    await expect(page.getByText('1 item')).toBeVisible();

    await page.goto('/checkout');

    await page.getByLabel('Email').fill('buyer@example.com');
    await page.getByLabel('First name').fill('Taylor');
    await page.getByLabel('Last name').fill('Nguyen');
    await page.getByLabel('Address').fill('123 Market St');
    await page.getByLabel('City').fill('San Francisco');
    await page.getByLabel('State').selectOption('CA');
    await page.getByLabel('ZIP').fill('94103');

    await expect(page.getByRole('button', { name: 'Continue to payment' })).toBeEnabled();
    await page.getByRole('button', { name: 'Continue to payment' }).click();

    const cardFrame = page.frameLocator('iframe[title="Secure payment input frame"]');
    await cardFrame.getByPlaceholder('Card number').fill('4242 4242 4242 4242');
    await cardFrame.getByPlaceholder('MM / YY').fill('12 / 30');
    await cardFrame.getByPlaceholder('CVC').fill('123');

    const placeOrder = page.getByRole('button', { name: 'Place order' });
    await expect(placeOrder).toBeEnabled();
    await placeOrder.click();

    await expect(page).toHaveURL(/\/order-confirmation/);
    await expect(page.getByRole('heading', { name: /thank you/i })).toBeVisible();
  });
});

And the Playwright config should preserve useful evidence when failures happen:

ts
import { defineConfig, devices } from '@playwright/test';

export default defineConfig({
  testDir: './e2e',
  retries: 1,
  use: {
    baseURL: process.env.APP_URL || 'http://localhost:3000',
    trace: 'on-first-retry',
    screenshot: 'only-on-failure',
    video: 'retain-on-failure',
  },
  projects: [
    { name: 'chromium', use: { ...devices['Desktop Chrome'] } },
    { name: 'webkit', use: { ...devices['Desktop Safari'] } },
  ],
});

Notice the intent here: not just pass/fail, but debuggability. When the test fails, the evidence should make the failure inspectable.

That is what most CI/CD setups are missing.

CI should surface workflow evidence, not just status checks

A mature pipeline for critical user journeys should run browser-based workflow tests on every meaningful PR and publish artifacts that reviewers can inspect.

Here is a GitHub Actions example:

yaml
name: pr-workflow-tests

on:
  pull_request:
    branches: [main]

jobs:
  checkout-flow:
    runs-on: ubuntu-latest
    timeout-minutes: 20

    steps:
      - name: Checkout code
        uses: actions/checkout@v4

      - name: Setup Node
        uses: actions/setup-node@v4
        with:
          node-version: 20
          cache: npm

      - name: Install dependencies
        run: npm ci

      - name: Install Playwright browsers
        run: npx playwright install --with-deps

      - name: Start app
        run: npm run start:test &

      - name: Wait for app
        run: npx wait-on http://localhost:3000

      - name: Run checkout workflow tests
        run: npx playwright test e2e/checkout.spec.ts --reporter=line,html

      - name: Upload Playwright report
        if: always()
        uses: actions/upload-artifact@v4
        with:
          name: playwright-report
          path: playwright-report/

      - name: Upload test results
        if: always()
        uses: actions/upload-artifact@v4
        with:
          name: test-results
          path: test-results/

This is the minimum standard for a money path. The outcome of the workflow should be visible to reviewers, not buried in a separate test dashboard nobody checks.

If your CI/CD only reports “pass” or “fail” without preserving traces, screenshots, and action logs, you are making debugging much harder than it needs to be.

Python example for backend state setup

One reason browser tests become flaky is poor environment control. Critical workflow tests need stable setup and teardown, especially when orders, inventory, promo codes, and payment tokens are involved.

Here is a lightweight Python helper to seed deterministic checkout state before tests:

python
import requests

BASE_URL = "http://localhost:8000"


def create_test_cart(email: str, sku: str, quantity: int = 1):
    response = requests.post(
        f"{BASE_URL}/test-support/carts",
        json={
            "email": email,
            "items": [{"sku": sku, "quantity": quantity}],
            "currency": "USD",
            "country": "US"
        },
        timeout=10,
    )
    response.raise_for_status()
    return response.json()


def enable_test_payment_provider():
    response = requests.post(
        f"{BASE_URL}/test-support/payment-provider",
        json={"mode": "sandbox"},
        timeout=10,
    )
    response.raise_for_status()


if __name__ == "__main__":
    enable_test_payment_provider()
    cart = create_test_cart("buyer@example.com", "sku-123")
    print(cart)

The point is not the specific script. The point is that reliable workflow testing requires deliberate testability hooks. If your staging or preview environment cannot be seeded predictably, your team will eventually stop trusting the tests.

Common reasons checkout fails while review passes

Let’s make this concrete. These are failure classes that routinely survive both human review and conventional automated testing:

Selector drift

A refactor changes markup, labels, or button hierarchy. The UI still looks fine. The event binding or targeting logic changes just enough that automation, analytics, or internal state wiring breaks.

Disabled-state bugs

Validation and readiness depend on multiple async conditions. One state update is missed, memoized incorrectly, or reset during rerender. The user cannot continue.

Timing issues

A button is clickable before data is consistent. Or it becomes unclickable while recalculating totals. Or a third-party script loads slower in one browser and the app assumes immediate availability.

Third-party widget changes

Payment, tax, fraud, shipping, or address autocomplete providers change DOM behavior, loading characteristics, or event timing. Your code compiles. Checkout still breaks.

State corruption

A coupon apply/remove flow resets shipping method. Navigating back from payment clears cart metadata. Retrying after a failed payment leaves the order form in an impossible state.

Browser-specific regressions

Safari focus behavior differs. Mobile viewport changes layout. Autofill fires events in a different order. WebKit treats iframes differently than Chromium.

Overlay and visual interception

A loading mask, sticky header, consent banner, or chat widget blocks the button. Screenshots look normal. Clicks fail in reality.

These are workflow failures, not code-review failures. Review is simply the wrong tool for primary detection.

Tools comparison: what helps and what does not

There is no single perfect testing stack, but tools differ sharply in how well they support workflow truth.

Unit test frameworks: Jest, Vitest, pytest

Strengths:

• fast
• excellent for business logic
• useful for edge cases and regressions
• easy to run in CI/CD

Weaknesses:

• poor visibility into real browser behavior
• weak at third-party widget and timing issues
• easy to over-mock critical interactions

Use them heavily, but do not confuse them with workflow validation.

Component testing

Strengths:

• validates UI logic in isolation
• catches rendering and state issues earlier
• faster feedback than full E2E

Weaknesses:

• limited realism for multi-step flows
• often excludes true browser integration boundaries

Great middle layer. Still not enough for checkout confidence.

Cypress

Strengths:

• approachable developer experience
• strong ecosystem
• useful interactive debugging

Weaknesses:

• browser model and cross-browser story are weaker than Playwright for some teams
• some workflows, especially around multiple tabs or certain browser behaviors, can be more constrained

Still a solid option if your team already uses it effectively.

Playwright

Strengths:

• strong real-browser automation
• excellent traces, video, screenshots
• multi-browser support
• robust locators and assertions
• good fit for PR-attached evidence

Weaknesses:

• requires disciplined environment setup
• can become flaky if teams write brittle tests

For critical user workflows, Playwright is currently one of the most practical choices.

Manual QA

Strengths:

• strong for exploratory testing
• catches weird experiential issues automation may miss

Weaknesses:

• not continuous enough
• hard to attach durable proof to every PR
• expensive as primary defense

Use it to complement workflow automation, not replace it.

What teams should do now

If you want to reduce “merged clean, broke checkout” incidents, here are the practical changes that matter.

1. Identify your money flows

List the workflows that matter to the business:

• signup
• checkout
• booking
• upgrade
• password reset
• lead submission
• core dashboard task completion

Do not treat all paths equally. Start with the ones that directly affect revenue or retention.

2. Define one happy-path browser test per critical workflow

Not fifty brittle permutations. Start with one deterministic, high-signal test that proves the workflow is alive.

For checkout, that means: add item, enter details, complete payment, see confirmation.

3. Attach artifacts to every PR

At minimum:

• pass/fail status
• action log
• screenshots on failure
• trace on retry/failure
• video when useful

The reviewer should not have to rerun the branch locally to understand the failure.

4. Test on at least two browser engines

If the flow handles money, one browser is not enough. Chromium plus WebKit catches a surprising amount of real risk.

5. Use stable locators and assertions

Prefer roles, labels, and explicit UX assertions over fragile CSS selectors.

Bad:

ts
await page.click('.checkout-form > div:nth-child(4) button');

Better:

ts
await page.getByRole('button', { name: 'Place order' }).click();

6. Seed deterministic test data

Flaky test setup destroys trust faster than almost anything else. Invest in test-support endpoints, seed scripts, and sandbox integrations.

7. Treat workflow failures as first-class incidents

If checkout automation starts failing intermittently, do not just quarantine the test and move on. That is often a real product reliability signal.

8. Require evidence for AI-assisted high-risk changes

If an AI agent touched authentication, billing, onboarding, or other core flows, review should require executed workflow evidence before merge.

This is not anti-AI. It is basic operational discipline.

9. Optimize for debugging, not just detection

A failed test without context creates toil. A failed test with trace, DOM snapshot, console logs, and network history creates fast fixes.

That distinction has a direct effect on developer productivity.

10. Change what “LGTM” means

For critical flows, “looks good to me” should mean:

• the code is reasonable
• the architecture is acceptable
• the critical workflow executed successfully
• the evidence is attached

Not just the first two.

The real shift: from reviewing code to reviewing behavior

This is the larger point many teams have not internalized.

As systems become more dynamic and AI produces more implementation detail, reviewing source code alone becomes less effective as a reliability control point. You still need code review. You just cannot pretend it is enough.

The question a business cares about is not:

• Was the code style consistent?
• Did the reviewer understand every line?
• Did CI/CD stay green on unit tests?

The question is:

• Can the customer still complete the task that matters?

That requires a different review primitive.

For workflow-heavy applications, the primitive should be executed user actions with evidence.

When attached to each PR, that evidence closes the gap between “looks safe in review” and “fails on the money step.” Without it, teams are mostly hoping that code quality proxies correlate with runtime truth.

Often they do not.

Conclusion

Your PR review cannot see the broken checkout.

It can see code style. It can see naming. It can see obvious mistakes. It can sometimes see suspicious logic. What it cannot see, at least not reliably, is whether a real customer can complete a high-value workflow in a real browser under real conditions.

That gap existed before AI, but AI-generated changes make it wider by increasing change volume, reducing readability, and creating more plausible-looking diffs that break behavior at the integration layer.

So the answer is not “review harder.” The answer is to stop asking line-by-line review to validate something it was never designed to validate.

If checkout matters, test checkout. If signup matters, test signup. If upgrade matters, test upgrade.

Then attach the evidence to the PR so reviewers can approve behavior, not just text.

That is how modern teams should think about debugging, testing, CI/CD, and developer productivity: not as a pile of green checks, but as confidence grounded in executed workflows.

Because a clean diff does not process payments. A working checkout does.