A practical map from browser upload to reliable image and video delivery.
Map the lifecycle
A media pipeline starts before the file reaches storage. The client needs a safe way to send bytes, the server needs an identity for the asset, workers need repeatable jobs, and delivery needs URLs with predictable behavior. Treat the original as an immutable input. Derivatives are outputs that can be recreated when requirements change.
Choose boundaries that survive change
Separate upload authorization, object storage, processing, metadata, and delivery. A product that only accepts small profile photos may process synchronously. Large video uploads, multiple renditions, or moderation usually justify a queue and an explicit processing state. Store the asset ID, owner, source key, checksum, status, dimensions, and derivative references in a database; keep binary bytes in object storage.
Design for retries
Give every processing job an idempotency key tied to the asset and recipe version. A retry should find the existing output or safely replace it. Publish an asset only after required derivatives exist, then make the delivery URL visible to the application. Keep failure states observable so a missing thumbnail does not silently become a broken page.
A concrete acceptance test
Upload a source image, force one worker failure, and confirm that a retry produces one ready asset with no duplicate derivative. Then change the recipe version and verify that the original remains unchanged. Run this test with a realistic asset and record the observed state, output URL, and failure reason.
Implementation check
- Keep the source immutable and give each derivative a recipe version.
- Move the asset to ready only after required outputs pass validation.
- Correlate upload, processing, and delivery logs with one asset ID.
Related guides
Primary reference: Official documentation.
Related field note
Continue with Designing queue backpressure for human and agent operators.