File Uploads to S3
There are two ways to get a user's file into cloud storage. Route it through your server, which means every 200MB video ties up your server's memory and bandwidth on the way past, and on serverless platforms slams into request-size limits around a few megabytes. Or have the browser upload straight to S3, with your server's only job being to sign a short-lived permission slip first. The second way is the right way, it is what the presigned URL exists for, and it is what this spec builds.
The flow is three steps and worth having in your head before the agent writes anything. The browser asks your server "I want to upload a JPEG, 4MB." Your server decides yes or no, and if yes, generates a presigned URL: a link that permits exactly one PUT, to one key it chose, expiring in minutes. The browser PUTs the file to that URL, straight to S3, never touching your server. Then the browser tells your server "done," and the server verifies the object actually landed before recording it. Your AWS credentials never leave the server; the browser only ever holds the temporary slip.
What agents get wrong here is not the happy path, which every SDK tutorial covers. It is the defaults around it: buckets left publicly readable, filenames taken from the user and used as keys (which is how ../../etc/passwd and overwritten files happen), size limits enforced only in JavaScript where anyone with curl can ignore them, and no answer for the user who requests a URL and never finishes the upload. The spec below closes each of those, and the bucket stays private the entire time: downloads are presigned too.
Prerequisites
- An AWS account, an S3 bucket with all public access blocked (the console's default, keep it), and an IAM user or role scoped to that one bucket with only `s3:PutObject`, `s3:GetObject`, `s3:DeleteObject`.
- That IAM credential's key pair, going straight into environment variables and nowhere else.
# Project: Direct-to-S3 file uploads for [app name]
Users upload [what: profile photos / PDF invoices / audio files] from
the browser directly to S3 using presigned URLs. My server signs and
records; it never relays file bytes.
## Rules that override anything else in this spec
- The bucket is PRIVATE. Block-all-public-access stays on. Nothing is
ever served by making an object public; downloads use presigned GET
URLs.
- AWS credentials exist only in server env vars: `AWS_ACCESS_KEY_ID`,
`AWS_SECRET_ACCESS_KEY`, `AWS_REGION`, `S3_BUCKET`. Any design that
puts them in browser code is wrong, stop and rethink.
- The SERVER chooses the object key:
`uploads/[userId]/[uuid].[safe-extension]`. The user's filename is
stored as metadata in the database for display, never used as the
key. User-controlled keys mean path tricks and silent overwrites.
- Limits live server-side. Client checks are UX; the signing endpoint
is the enforcement.
## Allowed uploads
- Types: [e.g., image/jpeg, image/png, application/pdf]; allowlist,
not blocklist.
- Max size: [e.g., 10MB]. Enforced by checking the declared size at
signing AND by verifying the real object size at confirmation
(HeadObject), because the declaration was just a claim.
- Who may upload: [authenticated users only / role X]. The signing
endpoint requires auth; it is the gate.
## Endpoint 1: `POST /api/uploads/sign`
Body: `{ filename, contentType, sizeBytes }`
1. Require an authenticated user.
2. Validate contentType against the allowlist and sizeBytes against
the max. Reject with a clear 400 otherwise.
3. Generate the key (server-side, per the rule above).
4. Create a presigned PUT URL, expiry 5 minutes, with the
content-type bound into the signature so the URL can only upload
what was declared.
5. Insert a DB row: `{ id, user_id, s3_key, original_filename,
content_type, size_bytes, status: "pending", created_at }`.
6. Return `{ uploadId, url, key }`.
```typescript
import { S3Client, PutObjectCommand } from "@aws-sdk/client-s3";
import { getSignedUrl } from "@aws-sdk/s3-request-presigner";
const url = await getSignedUrl(
s3,
new PutObjectCommand({
Bucket: process.env.S3_BUCKET,
Key: key,
ContentType: contentType, // bound into the signature
}),
{ expiresIn: 300 }
);
```
## Browser side
- `PUT` the file to the signed URL with the same Content-Type
declared at signing. Show progress; on failure allow retry, which
means requesting a fresh signed URL, not reusing the stale one.
- Bucket CORS: allow PUT and GET from [my app's origin(s)] only, not
`*`.
## Endpoint 2: `POST /api/uploads/confirm`
Body: `{ uploadId }`
1. Require the same authenticated user who owns the pending row.
2. `HeadObject` the key: confirm it exists and its actual size is
within the limit. Object missing or oversize: mark the row
"failed", delete the object if present, return 400.
3. Mark the row "complete". Only "complete" rows exist as far as the
rest of the app is concerned.
## Downloads: `GET /api/uploads/:id`
- Check the requesting user is allowed to see this file: [owner only /
team members / public-to-logged-in]. This check is the whole
security model, be explicit.
- Return a presigned GET URL, expiry 15 minutes, and redirect or hand
it to the client. Never proxy the bytes through the server.
## Cleanup
- A scheduled job (daily) deletes "pending" rows older than 24h and
their objects if any. Abandoned uploads must not accumulate as
orphaned storage I pay for.
## Done means
- Upload works end to end from the browser; file bytes never appear
in my server's logs or memory.
- Signing rejects a disallowed type and an oversize declaration.
- Uploading MORE bytes than declared gets caught at confirm and the
object is deleted.
- The raw S3 object URL (no signature) returns AccessDenied.
- An expired presigned URL fails; the retry path issues a fresh one.Adaptation notes:
- S3-compatible storage (Cloudflare R2, Backblaze B2, MinIO, Supabase Storage): same spec, same SDK calls, plus an endpoint URL env var. R2 is a strong default when the files get downloaded often, because egress is free.
- Files over ~100MB or flaky mobile connections: switch from single PUT to S3 multipart upload. The sign/confirm/cleanup skeleton holds; the signing endpoint grows to hand out per-part URLs.
- User-generated images shown to other users need processing (resize, strip EXIF, which contains GPS coordinates people don't know they're sharing). Do it after confirm, write results to a separate
processed/prefix, and serve only processed objects. - Public assets like blog images can skip presigned GETs by putting a CDN in front of the bucket. That is a deliberate architecture change, not a shortcut: keep the upload path exactly as specced.
- The mistake: trusting the confirm call. Users close laptops mid-upload, and attackers call confirm without uploading. That is why confirm verifies with HeadObject instead of believing the browser, and why the cleanup job exists. Skip either and your database and your bucket drift apart forever.