File upload looks easy right up until users touch it.
Most teams test the happy path: call an API, get a presigned URL, PUT the file to storage, save metadata, return 200. Unit tests pass. CI/CD goes green. A manual QA pass uploads one PDF on a fast connection and everyone moves on.
Then production starts collecting the bugs that matter:
- the browser blocks the upload because the storage bucket CORS policy is wrong
- the presigned URL expires before a large file finishes uploading
- the progress bar reaches 100% but the file is still being virus scanned
- the backend marks the upload complete before post-processing finishes
- users get a success toast for files that are later rejected
- retries create duplicate records or orphaned objects
- the API works perfectly while the actual user workflow is broken
This is the exact gap between code correctness and workflow correctness.
In this article, we’re going to build a real file upload flow end to end, then verify it the way users actually experience it: in the browser, across async processing, with failure states included.
We’ll implement:
- a drag-and-drop frontend in React/TypeScript
- a backend that creates upload sessions and presigned URLs
- direct-to-storage uploads
- async virus scanning and post-upload processing
- explicit state transitions for
pending,uploading,uploaded,scanning,ready, andfailed - retry behavior for expired URLs and processing failures
- Playwright tests that upload real files and verify visible UI states
- CI/CD wiring so the workflow is checked before shipping
I’ll use Node.js, Express, React, PostgreSQL-style data modeling, and an S3-compatible storage flow. The same design works with AWS S3, Cloudflare R2, MinIO, GCS signed URLs, or Azure SAS tokens.
The point is not the cloud vendor. The point is that a file upload feature is only done when the whole user-visible workflow is reliable.
What we’re building
The upload flow will work like this:
- User drops a file into the browser.
- Frontend calls
POST /api/uploadswith file metadata. - Backend creates an upload record and returns:
- upload ID
- object key
- presigned PUT URL
- expiry timestamp
- Browser uploads the file directly to object storage.
- Frontend notifies backend with
POST /api/uploads/:id/complete. - Backend moves the upload into
scanningand emits a background job. - A scan worker checks the file and updates the record.
- Frontend polls
GET /api/uploads/:idand renders the true state. - User sees one of:
ReadyRejected: infectedFailed: processing errorUpload expired, retry
That design matters because it avoids the classic lie: “upload complete” when only the network transfer finished.
For users, the file isn’t uploaded until your product can actually use it.
Architecture and state model
Before code, define the state machine. This is where most reliability comes from.
Here’s a practical upload record:
tsexport type UploadStatus = | 'pending' | 'uploading' | 'uploaded' | 'scanning' | 'ready' | 'failed' | 'rejected'; export interface UploadRecord { id: string; filename: string; contentType: string; sizeBytes: number; objectKey: string; status: UploadStatus; errorCode: string | null; errorMessage: string | null; storageEtag: string | null; uploadUrlExpiresAt: string; createdAt: string; updatedAt: string; }
And the transition rules:
pending→ record exists, presigned URL createduploading→ browser has started PUTuploaded→ storage transfer completedscanning→ async scanner is runningready→ file is usablerejected→ malware or policy rejectionfailed→ processing broke, user can retry or support can inspect
Notice what’s missing: no vague boolean like isComplete. Those always rot.
A state machine forces you to model what users actually need to know.
Backend: create upload sessions
Let’s start with an Express server.
ts// server/src/index.ts import express from 'express'; import cors from 'cors'; import crypto from 'crypto'; import { S3Client, PutObjectCommand, HeadObjectCommand } from '@aws-sdk/client-s3'; import { getSignedUrl } from '@aws-sdk/s3-request-presigner'; const app = express(); app.use(cors()); app.use(express.json()); const s3 = new S3Client({ region: process.env.AWS_REGION, endpoint: process.env.S3_ENDPOINT || undefined, forcePathStyle: !!process.env.S3_ENDPOINT, credentials: { accessKeyId: process.env.AWS_ACCESS_KEY_ID!, secretAccessKey: process.env.AWS_SECRET_ACCESS_KEY!, }, }); const BUCKET = process.env.UPLOAD_BUCKET!; const MAX_FILE_SIZE = 10 * 1024 * 1024; // 10MB type UploadStatus = 'pending' | 'uploading' | 'uploaded' | 'scanning' | 'ready' | 'failed' | 'rejected'; type UploadRecord = { id: string; filename: string; contentType: string; sizeBytes: number; objectKey: string; status: UploadStatus; errorCode: string | null; errorMessage: string | null; storageEtag: string | null; uploadUrlExpiresAt: string; createdAt: string; updatedAt: string; }; const uploads = new Map<string, UploadRecord>(); function nowIso() { return new Date().toISOString(); } function makeId() { return crypto.randomUUID(); } app.post('/api/uploads', async (req, res) => { const { filename, contentType, sizeBytes } = req.body; if (!filename || !contentType || !sizeBytes) { return res.status(400).json({ error: 'missing_fields' }); } if (sizeBytes > MAX_FILE_SIZE) { return res.status(400).json({ error: 'file_too_large', maxBytes: MAX_FILE_SIZE }); } const id = makeId(); const objectKey = `uploads/${id}/${filename}`; const expiresInSeconds = 60; const uploadUrlExpiresAt = new Date(Date.now() + expiresInSeconds * 1000).toISOString(); const command = new PutObjectCommand({ Bucket: BUCKET, Key: objectKey, ContentType: contentType, }); const uploadUrl = await getSignedUrl(s3, command, { expiresIn: expiresInSeconds }); const record: UploadRecord = { id, filename, contentType, sizeBytes, objectKey, status: 'pending', errorCode: null, errorMessage: null, storageEtag: null, uploadUrlExpiresAt, createdAt: nowIso(), updatedAt: nowIso(), }; uploads.set(id, record); res.json({ upload: record, uploadUrl, maxFileSize: MAX_FILE_SIZE, }); });
A few practical points:
- We validate file size before generating a signed URL.
- We store expiry in the DB record so the UI can handle stale uploads honestly.
- We tie one upload record to one object key. That makes debugging easier.
If you skip this and just let the browser fire files directly at storage, you lose observability fast.
Backend: mark upload complete only after storage transfer
Now we need an endpoint for the browser to call after the PUT succeeds.
tsapp.post('/api/uploads/:id/complete', async (req, res) => { const upload = uploads.get(req.params.id); if (!upload) { return res.status(404).json({ error: 'not_found' }); } try { const head = await s3.send(new HeadObjectCommand({ Bucket: BUCKET, Key: upload.objectKey, })); upload.status = 'scanning'; upload.storageEtag = head.ETag || null; upload.updatedAt = nowIso(); uploads.set(upload.id, upload); enqueueScan(upload.id); return res.json({ upload }); } catch (err) { return res.status(409).json({ error: 'object_not_found_in_storage' }); } }); app.get('/api/uploads/:id', async (req, res) => { const upload = uploads.get(req.params.id); if (!upload) { return res.status(404).json({ error: 'not_found' }); } res.json({ upload }); });
That HeadObject check is important.
Without it, your app can accept “complete” from the browser even if the actual storage upload failed or got interrupted. That creates a false success state, which is one of the worst debugging problems in production because your API logs look fine.
Background scan and processing
In production you might use S3 event notifications, a queue, and a worker service. For this article, we’ll simulate that with an in-process queue so the flow is runnable.
tsconst scanQueue: string[] = []; let scanWorkerStarted = false; function enqueueScan(uploadId: string) { scanQueue.push(uploadId); } function startScanWorker() { if (scanWorkerStarted) return; scanWorkerStarted = true; setInterval(async () => { const uploadId = scanQueue.shift(); if (!uploadId) return; const upload = uploads.get(uploadId); if (!upload) return; try { await simulateVirusScan(upload); } catch (err: any) { upload.status = 'failed'; upload.errorCode = 'scan_processing_error'; upload.errorMessage = err.message || 'Scan processing failed'; upload.updatedAt = nowIso(); uploads.set(upload.id, upload); } }, 500); } async function simulateVirusScan(upload: UploadRecord) { await new Promise((r) => setTimeout(r, 1500)); if (upload.filename.includes('infected')) { upload.status = 'rejected'; upload.errorCode = 'virus_detected'; upload.errorMessage = 'The file failed security scanning'; upload.updatedAt = nowIso(); uploads.set(upload.id, upload); return; } if (upload.filename.includes('scanfail')) { throw new Error('Scanner backend unavailable'); } upload.status = 'ready'; upload.errorCode = null; upload.errorMessage = null; upload.updatedAt = nowIso(); uploads.set(upload.id, upload); } startScanWorker(); app.listen(4000, () => { console.log('server listening on :4000'); });
This is obviously simplified, but the architecture is real:
- transfer completes first
- background processing starts second
- user-visible readiness only comes after processing succeeds
That separation is what keeps the UI truthful.
Storage CORS: the bug users hit first
If you’ve shipped uploads before, you already know this: CORS is where “works locally” goes to die.
For S3, you need bucket CORS that allows the browser to PUT directly.
Example:
json[ { "AllowedHeaders": ["*"], "AllowedMethods": ["PUT", "GET", "HEAD"], "AllowedOrigins": ["http://localhost:3000"], "ExposeHeaders": ["ETag"] } ]
Common failure modes:
Content-Typeused in the browser doesn’t match the one expected when signingOriginmissing from allowed origins- browser needs
ETagbut it isn’t exposed - staging and prod buckets have different CORS configs
This is why browser-level testing matters. Your backend tests will never catch bucket CORS issues.
Frontend: drag-and-drop uploader with real states
Now let’s build the browser side.
tsx// web/src/Uploader.tsx import React, { useRef, useState } from 'react'; type UploadStatus = 'idle' | 'requesting_url' | 'uploading' | 'scanning' | 'ready' | 'failed' | 'rejected'; export function Uploader() { const [status, setStatus] = useState<UploadStatus>('idle'); const [message, setMessage] = useState(''); const [progress, setProgress] = useState(0); const [uploadId, setUploadId] = useState<string | null>(null); const inputRef = useRef<HTMLInputElement>(null); async function onFile(file: File) { setStatus('requesting_url'); setMessage('Preparing upload...'); setProgress(0); try { const createRes = await fetch('http://localhost:4000/api/uploads', { method: 'POST', headers: { 'Content-Type': 'application/json' }, body: JSON.stringify({ filename: file.name, contentType: file.type || 'application/octet-stream', sizeBytes: file.size, }), }); const createData = await createRes.json(); if (!createRes.ok) throw new Error(createData.error || 'create_failed'); const { upload, uploadUrl } = createData; setUploadId(upload.id); setStatus('uploading'); setMessage('Uploading file...'); await putFile(uploadUrl, file, (pct) => setProgress(pct)); const completeRes = await fetch(`http://localhost:4000/api/uploads/${upload.id}/complete`, { method: 'POST', }); const completeData = await completeRes.json(); if (!completeRes.ok) throw new Error(completeData.error || 'complete_failed'); setStatus('scanning'); setMessage('Upload finished. Scanning file...'); await pollUntilResolved(upload.id); } catch (err: any) { setStatus('failed'); setMessage(readableError(err.message)); } } function putFile(url: string, file: File, onProgress: (pct: number) => void) { return new Promise<void>((resolve, reject) => { const xhr = new XMLHttpRequest(); xhr.open('PUT', url); xhr.setRequestHeader('Content-Type', file.type || 'application/octet-stream'); xhr.upload.onprogress = (e) => { if (e.lengthComputable) { onProgress(Math.round((e.loaded / e.total) * 100)); } }; xhr.onload = () => { if (xhr.status >= 200 && xhr.status < 300) { onProgress(100); resolve(); } else { reject(new Error(`upload_http_${xhr.status}`)); } }; xhr.onerror = () => reject(new Error('network_upload_error')); xhr.send(file); }); } async function pollUntilResolved(id: string) { for (let i = 0; i < 30; i++) { const res = await fetch(`http://localhost:4000/api/uploads/${id}`); const data = await res.json(); const upload = data.upload; if (upload.status === 'ready') { setStatus('ready'); setMessage('File uploaded and ready'); return; } if (upload.status === 'rejected') { setStatus('rejected'); setMessage(upload.errorMessage || 'File rejected'); return; } if (upload.status === 'failed') { setStatus('failed'); setMessage(upload.errorMessage || 'Processing failed'); return; } await new Promise((r) => setTimeout(r, 1000)); } setStatus('failed'); setMessage('Processing timed out'); } function readableError(code: string) { if (code.includes('file_too_large')) return 'File is too large'; if (code.includes('upload_http_403')) return 'Upload URL expired or was rejected'; if (code.includes('object_not_found_in_storage')) return 'Upload did not reach storage'; return 'Upload failed. Please try again.'; } return ( <div> <div onDragOver={(e) => e.preventDefault()} onDrop={(e) => { e.preventDefault(); const file = e.dataTransfer.files?.[0]; if (file) onFile(file); }} onClick={() => inputRef.current?.click()} style={{ border: '2px dashed #999', padding: 32, borderRadius: 12, cursor: 'pointer' }} data-testid="dropzone" > Drop file here or click to browse </div> <input ref={inputRef} type="file" hidden onChange={(e) => { const file = e.target.files?.[0]; if (file) onFile(file); }} data-testid="file-input" /> <div data-testid="status">{status}</div> <div data-testid="message">{message}</div> {status === 'uploading' && <progress value={progress} max={100} data-testid="progress" />} {uploadId && <div data-testid="upload-id">{uploadId}</div>} </div> ); }
The important design choices here:
- We do not show success immediately after the PUT.
- We separate upload progress from processing progress.
- We expose status text that can be tested in Playwright.
- Errors are mapped into user-readable states.
This is a debugging feature as much as a UX feature. Ambiguous status text creates support load and slows incident response.
A real retry flow for expired URLs
Presigned URL expiry is one of those bugs that shows up only under real conditions: large file, slow network, tab backgrounded, laptop sleeping, user on hotel Wi-Fi.
The solution is not “make the URL last forever.” The solution is to support restartable upload sessions.
Add a retry endpoint:
tsapp.post('/api/uploads/:id/retry', async (req, res) => { const upload = uploads.get(req.params.id); if (!upload) { return res.status(404).json({ error: 'not_found' }); } if (!['pending', 'failed'].includes(upload.status)) { return res.status(409).json({ error: 'retry_not_allowed_for_status' }); } const command = new PutObjectCommand({ Bucket: BUCKET, Key: upload.objectKey, ContentType: upload.contentType, }); const expiresInSeconds = 60; const uploadUrl = await getSignedUrl(s3, command, { expiresIn: expiresInSeconds }); upload.uploadUrlExpiresAt = new Date(Date.now() + expiresInSeconds * 1000).toISOString(); upload.errorCode = null; upload.errorMessage = null; upload.updatedAt = nowIso(); uploads.set(upload.id, upload); res.json({ upload, uploadUrl }); });
In a production system, I’d usually create a new upload attempt record linked to the same logical document. That gives you better debugging and analytics than mutating a single row forever.
The silent breakages if you stop at unit tests
This is where most articles stop. They show a pretty upload widget and maybe one backend test.
But here’s what still breaks in production even if all of that code is “tested” in isolation:
1. Storage CORS mismatch
Your API test never sees it. The browser blocks the PUT before it starts.
2. Signed URL and actual request headers differ
You signed for image/png, the browser sent application/octet-stream, storage rejects it with 403.
3. Progress bar lies
The transport finished, but scanning hasn’t. Users leave the page thinking the file is ready.
4. Background processing fails after success UI
The app renders a success toast before the async pipeline completes. Now users think your product lost their file.
5. Complete endpoint races storage consistency or interrupted uploads
If you don’t verify the object exists, your DB says complete while storage does not.
6. Oversized files fail unclearly
Backend rejects metadata creation or storage rejects body length, but the user sees a generic network error.
7. Retry creates duplicates
Users click upload again, creating multiple objects and mismatched DB records.
These are workflow failures. They kill trust even when the code technically did what individual components expected.
Playwright: verify the flow in the browser
Now for the part that actually protects you.
We’ll write Playwright tests that use real file input, hit the real app, and verify the user-visible state transitions.
ts// tests/upload.spec.ts import { test, expect } from '@playwright/test'; import path from 'path'; const fixtures = (name: string) => path.join(__dirname, 'fixtures', name); test('uploads a valid file and reaches ready state', async ({ page }) => { await page.goto('http://localhost:3000'); await page.setInputFiles('[data-testid="file-input"]', fixtures('document.pdf')); await expect(page.getByTestId('status')).toHaveText('uploading'); await expect(page.getByTestId('message')).toContainText('Uploading'); await expect(page.getByTestId('status')).toHaveText('scanning', { timeout: 10000 }); await expect(page.getByTestId('message')).toContainText('Scanning'); await expect(page.getByTestId('status')).toHaveText('ready', { timeout: 15000 }); await expect(page.getByTestId('message')).toHaveText('File uploaded and ready'); }); test('shows rejected state for infected file', async ({ page }) => { await page.goto('http://localhost:3000'); await page.setInputFiles('[data-testid="file-input"]', fixtures('infected-sample.pdf')); await expect(page.getByTestId('status')).toHaveText('scanning', { timeout: 10000 }); await expect(page.getByTestId('status')).toHaveText('rejected', { timeout: 15000 }); await expect(page.getByTestId('message')).toContainText('failed security scanning'); }); test('shows failed state when scanning backend errors', async ({ page }) => { await page.goto('http://localhost:3000'); await page.setInputFiles('[data-testid="file-input"]', fixtures('scanfail.pdf')); await expect(page.getByTestId('status')).toHaveText('scanning', { timeout: 10000 }); await expect(page.getByTestId('status')).toHaveText('failed', { timeout: 15000 }); await expect(page.getByTestId('message')).toContainText('Scanner backend unavailable'); }); });
A detail worth calling out: these tests assert transitions, not just the final state.
That matters because a UI that jumps from “idle” to “ready” without ever showing “scanning” is lying about the actual workflow.
Simulating expired presigned URLs in Playwright
We also want to test a realistic expiry failure.
One clean approach is to support a test-only mode that issues a very short-lived URL.
Backend tweak:
tsapp.post('/api/uploads', async (req, res) => { const { filename, contentType, sizeBytes, testMode } = req.body; const expiresInSeconds = testMode === 'expire_fast' ? 1 : 60; // rest of handler unchanged });
Frontend tweak for tests is optional, but you can pass this based on environment. Then test it:
tstest('shows useful error for expired presigned URL', async ({ page }) => { await page.goto('http://localhost:3000?testMode=expire_fast'); await page.route('http://localhost:4000/api/uploads', async (route) => { const request = route.request(); const body = JSON.parse(request.postData() || '{}'); body.testMode = 'expire_fast'; await route.continue({ postData: JSON.stringify(body) }); }); await page.setInputFiles('[data-testid="file-input"]', fixtures('document.pdf')); await page.waitForTimeout(2000); await expect(page.getByTestId('status')).toHaveText('failed'); await expect(page.getByTestId('message')).toContainText('expired'); });
In a more advanced setup, you’d also throttle network speed so expiry happens during the upload rather than before it starts. Playwright plus Chromium devtools protocol can help there, but even this simpler version catches a real product behavior.
Testing oversized files and false success states
Add one more test for size handling.
tstest('rejects oversized file with user-visible message', async ({ page }) => { await page.goto('http://localhost:3000'); await page.setInputFiles('[data-testid="file-input"]', { name: 'large.bin', mimeType: 'application/octet-stream', buffer: Buffer.alloc(11 * 1024 * 1024), }); await expect(page.getByTestId('status')).toHaveText('failed'); await expect(page.getByTestId('message')).toContainText('too large'); });
And if you really want to catch false positives, add an assertion that “ready” never appears for rejected flows:
tstest('never shows ready for infected file', async ({ page }) => { await page.goto('http://localhost:3000'); await page.setInputFiles('[data-testid="file-input"]', fixtures('infected-sample.pdf')); await expect(page.getByTestId('status')).toHaveText('rejected', { timeout: 15000 }); await expect(page.getByTestId('message')).not.toHaveText('File uploaded and ready'); });
This sounds trivial, but it catches surprising UI bugs where a delayed success callback overwrites a later error state.
CI/CD: run the workflow, not just the unit suite
Here’s a GitHub Actions example that starts the app and runs Playwright.
yamlname: test on: push: pull_request: jobs: test: runs-on: ubuntu-latest steps: - uses: actions/checkout@v4 - uses: actions/setup-node@v4 with: node-version: 20 cache: npm - run: npm ci - run: npm run build - run: npx playwright install --with-deps - name: Start backend run: npm run dev:server & - name: Start frontend run: npm run dev:web & - name: Wait for services run: npx wait-on http://localhost:3000 http://localhost:4000/api/uploads - name: Run unit tests run: npm run test:unit - name: Run browser workflow tests run: npm run test:e2e
A few practical notes:
- Keep unit tests. They’re still useful.
- But do not stop there.
- Put the browser workflow tests in the same CI/CD gate as the rest of your suite.
- If uploads rely on real cloud infra, use an ephemeral test bucket or an S3-compatible local service like MinIO for CI.
If your CI only proves that handlers and reducers behave in isolation, it is giving you false confidence.
What to log for debugging in production
If you want this feature to be operable, log with workflow context.
At minimum, log:
upload_idobject_keyfilenamecontent_typesize_bytesstatus_fromstatus_tostorage_etagerror_codeerror_messagerequest_iduser_id
Example transition log:
tsconsole.log(JSON.stringify({ event: 'upload_status_changed', upload_id: upload.id, object_key: upload.objectKey, status_from: 'scanning', status_to: 'rejected', error_code: 'virus_detected', at: nowIso(), }));
This is basic debugging hygiene, but it matters more for async workflows because failures happen across boundaries: browser, API, storage, queue, scanner, database.
If you can’t reconstruct the lifecycle of one upload from logs, incidents will drag on for hours.
Tools comparison: what each layer catches
You should use multiple testing layers, but be honest about what each one can and cannot tell you.
Unit tests
Good for:
- file validation rules
- state reducer behavior
- scanner decision logic
- retry policy rules
Bad at catching:
- CORS
- signed URL mismatch
- broken browser upload behavior
- user-visible state lies
API integration tests
Good for:
- upload session creation
- DB writes
- state transitions in backend endpoints
- storage existence verification logic
Bad at catching:
- drag-and-drop/input wiring
- browser PUT behavior
- progress UI issues
- polling/render race conditions
Playwright/browser tests
Good for:
- the actual workflow users experience
- upload input wiring
- state transitions on screen
- expired URL behavior
- false success states
- end-to-end correctness
Bad at:
- pinpointing low-level logic bugs quickly
- replacing unit tests for all edge logic
This is the right testing stack for modern developer productivity: fast low-level tests plus a few high-value browser workflow tests that prove the product actually works.
Production hardening you probably want next
The demo here is intentionally simple. In production, I’d strongly consider:
- multipart upload for large files
- resumable upload protocols like tus for unstable networks
- a dedicated upload attempts table
- webhooks from a real malware scanning service
- signed GET/download URLs for processed assets
- idempotency keys on session creation
- queue-backed workers instead of in-process timers
- cleanup jobs for abandoned
pendinguploads and orphaned objects - metrics for
time_to_ready, rejection rate, expiry failures, and scan backlog
And if uploads matter to revenue or activation, add synthetic monitoring that runs the entire file-upload workflow regularly in production.
That’s not overkill. That’s what it means to test the product instead of admiring the code.
The core lesson: stop shipping upload features that only pass in theory
File upload is a perfect example of why traditional testing falls short.
You can have:
- a backend that returns 200
- unit tests that all pass
- clean CI/CD output
- strong type safety
- good code coverage
…and still fail the user at the exact moment they drop a file.
That’s the real standard for correctness: not whether your handler returned a valid JSON shape, but whether the complete workflow works in the browser, through storage, across async processing, all the way to a trustworthy final state.
If you take one thing from this article, let it be this:
Don’t mark upload features done when the request passes. Mark them done when a real browser can upload a real file and the UI truthfully reflects what happened after the bytes land.
That’s better testing. Better debugging. Better CI/CD. And, in practice, much better developer productivity because you catch the expensive failures before users do.
