Developer Media Tools

Tag: Media API

  • Plan adaptive video renditions before you encode

    Plan adaptive video renditions before you encode

    Choose a playback ladder from source quality and viewer needs.

    Start with the source ceiling

    Do not upscale a low-resolution upload just to fill a standard ladder. Choose output sizes and bitrates from the source, target devices, and measured playback conditions. Keep segment duration and keyframe alignment consistent across renditions so the player can switch cleanly.

    Package a complete playback asset

    The job should produce segments, variant playlists, a master playlist, and metadata that names each rendition. Verify playlists reference files that actually exist before changing the asset to ready. Serve manifests and segments with correct MIME types and cache rules.

    Test the edges

    Seek near the end, switch bandwidth during playback, and try a short or portrait video. A valid encoder exit code does not prove the package plays. Preserve an accessible fallback when a viewer’s browser or network cannot use the preferred stream.

    A concrete acceptance test

    Play a short portrait clip and a long landscape clip while throttling network speed. Check that the rendition list respects each source and that seeking works near the end. Run this test with a realistic asset and record the observed state, output URL, and failure reason.

    Implementation check

    • Do not create renditions above the source ceiling.
    • Verify every playlist reference resolves.
    • Test seeking and switching under throttled bandwidth.

    Related guides

    Primary reference: Official documentation.

  • Build a reliable queue for video transcoding

    Build a reliable queue for video transcoding

    Turn expensive video jobs into an observable, retryable pipeline.

    Create a job from an immutable source

    Video conversion can outlast an HTTP request, so create a processing job after upload validation. Include source asset ID, recipe version, desired outputs, and an idempotency key. Workers should write to temporary keys and publish finished outputs only after probing them.

    Control concurrency

    Transcodes consume CPU, memory, disk, and network. Limit worker concurrency by workload, not simply by queue depth. Large 4K inputs may need a different pool from short clips. Record input duration, output size, processing time, and failure reason so capacity decisions use evidence.

    Retry only what can recover

    A transient storage failure may justify a retry; a corrupt input or unsupported codec needs a terminal state and a user-facing explanation. Clean up partial outputs and keep the original available for later reprocessing. Test worker crashes at each stage to ensure duplicate jobs do not publish mixed renditions.

    A concrete acceptance test

    Kill a transcoding worker after it writes one temporary rendition. Restart the job and verify that only complete, probed outputs become visible and no mixed recipe versions appear. Run this test with a realistic asset and record the observed state, output URL, and failure reason.

    Implementation check

    • Use a source-and-recipe idempotency key.
    • Probe output files before publishing URLs.
    • Classify corrupt inputs separately from transient failures.

    Related guides

    Primary reference: Official documentation.

    Related field note

    Continue with How metadata changes queue backpressure.

  • How to deliver a hero image without hurting LCP

    How to deliver a hero image without hurting LCP

    Make the likely largest image easy to discover and quick to render.

    Find the actual LCP element

    First measure the page. The largest contentful paint element may be text on one viewport and an image on another. If the hero image is the LCP element, place its URL in discoverable HTML and avoid hiding it behind a late JavaScript fetch or CSS background when a normal image works.

    Give it the right priority

    Supply a properly sized responsive source, intrinsic dimensions, and an appropriate fetch priority where supported. Do not lazy-load an image that appears immediately in the viewport. Compressing the image helps only one part of LCP; render delay and late discovery can dominate.

    Check under realistic conditions

    Measure cold-cache mobile loading and inspect the network waterfall. Compare transfer size, request start, decode, and final render. A faster local Wi-Fi screenshot does not prove an improvement for a user on a slower connection.

    A concrete acceptance test

    Measure a cold mobile load and identify the actual LCP element. Change only the hero delivery rule, then compare request start time, transferred bytes, and render delay. Run this test with a realistic asset and record the observed state, output URL, and failure reason.

    Implementation check

    • Measure the real LCP element at each viewport.
    • Make the hero URL discoverable in HTML.
    • Compare request start and render delay after the change.

    Related guides

    Primary reference: Official documentation.

  • Image metadata: what to preserve and what to strip

    Image metadata: what to preserve and what to strip

    Handle orientation, color, copyright, and location data deliberately.

    Read metadata before transforming

    Orientation and embedded color profiles affect how an image appears. Normalize orientation before sizing or cropping, and test color conversion against representative assets. Dimensions measured before orientation correction can lead to the wrong derivative shape.

    Keep private fields private

    Camera metadata may include location, device, and capture details that a user did not intend to publish. Define a publication policy: retain only fields needed for display or rights management, and strip the rest from public derivatives. The original may need a different retention rule under the product’s own privacy policy.

    Make the policy testable

    Use sample files with orientation, ICC profiles, GPS coordinates, and transparency. Verify both the rendered pixels and metadata of the downloaded derivative. A successful encoder exit code is not proof that color or privacy handling was correct.

    A concrete acceptance test

    Upload samples with rotated orientation, an ICC profile, and GPS data. Download the public derivatives and verify their pixels, color, dimensions, and removed private metadata. Run this test with a realistic asset and record the observed state, output URL, and failure reason.

    Implementation check

    • Normalize orientation before measuring or cropping.
    • Inspect public derivatives for location metadata.
    • Test color profiles against visible reference images.

    Related guides

    Primary reference: Official documentation.

  • Focal points and aspect ratios in automated cropping

    Focal points and aspect ratios in automated cropping

    Keep the subject in frame across cards, heroes, and mobile layouts.

    Save intent with the asset

    A center crop is convenient until a face, product, or text sits near an edge. Store an optional focal point as normalized coordinates on the source asset. A transformation can then center the requested crop around that point while clamping to the image bounds.

    Separate display and derivative decisions

    CSS object-position is useful when the browser displays an image in a fixed box, while server-side cropping saves bytes and ensures a consistent composition in shared previews. The same focal point can inform both, but test each target ratio: a very narrow banner may still need a manually approved crop.

    Give editors an escape hatch

    Automatic focal detection is a suggestion, not an editorial guarantee. Preview the common placements together and allow an override per important slot. Keep the original intact so a changed layout does not require re-uploading the image.

    A concrete acceptance test

    Preview a portrait subject in square, wide, and narrow slots. Move the focal point once and confirm all three placements update without changing the preserved source image. Run this test with a realistic asset and record the observed state, output URL, and failure reason.

    Implementation check

    • Store focal coordinates with the original asset.
    • Preview square, landscape, and narrow slots.
    • Allow a manual crop override for important placements.

    Related guides

    Primary reference: Official documentation.

  • Designing an image transformation API

    Designing an image transformation API

    Make resize and format URLs predictable, safe, and cacheable.

    Constrain the recipe

    A transformation URL should identify an immutable source plus a small allowed set of operations: width, height, fit, quality, and format. Normalize equivalent requests to one canonical recipe. Cap output dimensions and pixel count before starting a decoder so one request cannot consume excessive memory.

    Version intentional changes

    When crop behavior, sharpening, or encoder settings change, include a recipe version in the derivative identity. Otherwise a cached URL can mean different bytes before and after deployment. Record transformation errors separately from missing source assets.

    Protect the origin

    If anyone can request any width for any source, a crawler can create thousands of variants. Use named presets for common UI slots, sign private transformations, and rate-limit expensive misses. Warm only genuinely important variants; on-demand generation is useful when the variant space is bounded.

    A concrete acceptance test

    Request the same transform with reordered parameters, then request a disallowed extreme width. The first two URLs should map to one output; the extreme request should fail before expensive decoding. Run this test with a realistic asset and record the observed state, output URL, and failure reason.

    Implementation check

    • Canonicalize equivalent transformation recipes.
    • Reject excessive dimensions before decoding.
    • Version the recipe when output bytes change.

    Related guides

    Primary reference: Official documentation.

  • AVIF, WebP, or JPEG: choose formats by asset

    AVIF, WebP, or JPEG: choose formats by asset

    A format decision based on content, compatibility, and operational cost.

    Start with the content

    Photographs usually tolerate lossy compression; diagrams and interface screenshots expose ringing around sharp edges. WebP and AVIF can reduce transfer size, but visual quality, encoding time, and decode behavior matter. Keep a widely supported fallback when the audience includes older clients or embedded browsers.

    Measure at equal visual quality

    Compare outputs at the same displayed dimensions and similar perceived quality, not at the same numerical quality setting. Include transparency, animation, color profile, and metadata needs. Save a small representative corpus of photos, text-heavy screenshots, and graphics for repeatable tests.

    Negotiate without multiplying chaos

    A picture element or a well-defined CDN content negotiation strategy can serve modern formats and a fallback. Ensure the CDN cache key distinguishes format variants. Do not allow arbitrary format and quality combinations to create an unbounded derivative library.

    A concrete acceptance test

    Encode one photo, one UI screenshot, and one transparent graphic in each candidate format. Compare the downloaded bytes and visible edges at the exact size users will see. Run this test with a realistic asset and record the observed state, output URL, and failure reason.

    Implementation check

    • Compare formats at matched visual quality.
    • Include text-heavy screenshots and transparent graphics in tests.
    • Cache each negotiated format under a distinct key.

    Related guides

    Primary reference: Official documentation.

  • Responsive images: get srcset and sizes working together

    Responsive images: get srcset and sizes working together

    Give browsers the right candidates and a truthful display size.

    Describe the actual slot

    A width-descriptor srcset lists candidate image widths; sizes tells the browser how wide the image will render at each breakpoint. If sizes claims a full viewport while CSS shows a half-width card, the browser can choose an unnecessarily large file. Write sizes from the layout, not from the source dimensions.

    Separate crop from resolution

    Use srcset for the same composition at several resolutions. Use picture with media conditions when mobile needs a different crop or framing. Keep a fallback img with intrinsic width and height so layout space is reserved before the image loads.

    Inspect the browser’s choice

    Test a real mobile viewport and a high-density desktop display. Look at currentSrc in DevTools and compare the delivered pixels to the rendered slot. A candidate set is useful only if the transformation service produces those widths and the CDN caches them predictably.

    A concrete acceptance test

    Render the same card at narrow and wide widths, then inspect currentSrc and transferred bytes. The chosen resource should have enough pixels for the slot without sending a desktop hero to mobile. Run this test with a realistic asset and record the observed state, output URL, and failure reason.

    Implementation check

    • Match sizes to the actual CSS slot width.
    • Use picture when the crop changes across layouts.
    • Inspect currentSrc on mobile and high-density screens.

    Related guides

    Primary reference: Official documentation.

  • Signed media URLs: access control at delivery time

    Signed media URLs: access control at delivery time

    Use expiring delivery links without making private assets public.

    Decide what is private

    Private originals, paid downloads, and temporary review files need a delivery authorization step. The application checks the viewer’s rights and issues a short-lived URL for the exact object or transformation. The URL is a bearer capability: anyone holding it can use it until it expires.

    Keep caches aligned with permissions

    Choose an expiration long enough for a download but short enough for the access policy. If authorization can be revoked instantly, a plain presigned URL may not meet the requirement. A CDN token, proxy, or session-bound approach may be needed, with explicit cache-key and purge behavior.

    Do not log the secret link

    Avoid putting signed query strings in analytics, error reports, or third-party referrers. Test clock skew and range requests for large video files. Make the application able to refresh a link without asking users to restart an otherwise valid download.

    A concrete acceptance test

    Open a private URL before authorization, during its valid window, and after expiry. Repeat with a byte-range request and inspect logs to ensure the signed query string was not captured. Run this test with a realistic asset and record the observed state, output URL, and failure reason.

    Implementation check

    • Authorize the exact asset before signing a delivery URL.
    • Test expiration and byte-range requests.
    • Remove signed query strings from routine telemetry.

    Related guides

    Primary reference: Official documentation.

    Related field note

    Continue with A decision matrix for delivery URL signing.

  • Design object storage keys for a growing media library

    Design object storage keys for a growing media library

    A naming scheme that supports ownership, versions, cleanup, and migration.

    Use opaque identities

    An object key is an address, not a database record or a trustworthy filename. Use an asset ID and a stable prefix such as originals/{asset-id}/source. Keep the user’s filename, title, and owner in application metadata. This avoids collisions and makes renaming an asset independent of moving bytes.

    Separate source from derivatives

    Put generated files under a versioned recipe prefix, for example derivatives/{asset-id}/{recipe-version}/. A new crop rule or encoder can then coexist with the old output during rollout. Record each derivative’s key, dimensions, format, and status so the application never guesses what exists.

    Plan deletion as a workflow

    Removing an asset requires more than deleting one URL. Mark it unavailable, revoke public references, enqueue deletion of source and derivative keys, and confirm completion. Inventory and lifecycle rules help find abandoned uploads, but do not replace an application-level deletion record.

    A concrete acceptance test

    Rename an asset, regenerate one derivative, and delete the asset. Confirm that the opaque source key remains stable through renaming and that deletion removes every recorded derivative. Run this test with a realistic asset and record the observed state, output URL, and failure reason.

    Implementation check

    • Use an opaque asset ID instead of the original filename.
    • Version derivative prefixes separately from source keys.
    • Track every key that deletion must remove.

    Related guides

    Primary reference: Official documentation.

  • When to use resumable uploads for media

    When to use resumable uploads for media

    Decide when a one-request upload is enough and when recovery matters.

    Match the protocol to the file

    For small images on stable connections, a direct single-request upload is simple. For long videos or mobile networks, a failed request should not force the user to resend the entire file. A resumable protocol tracks an upload identifier and the committed byte offset so the client can continue from a known point.

    Persist state on both sides

    The server must bind the upload session to a user, expected size, expiration, and destination. The client should save the upload ID and query the server’s current offset after reconnecting instead of trusting its local counter. Progress means bytes accepted by the server, not just bytes sent by the browser.

    Finish exactly once

    After the final chunk, verify length and checksum, then create or finalize one asset record. Expire incomplete uploads and reclaim their storage. Test interrupted requests, duplicate chunks, browser refreshes, and two tabs resuming the same session before relying on the happy path.

    A concrete acceptance test

    Interrupt a large upload halfway, restart the browser, and resume from the server-reported offset. Repeat the last chunk and confirm that the final checksum and asset count remain correct. Run this test with a realistic asset and record the observed state, output URL, and failure reason.

    Implementation check

    • Resume from the server offset after reconnecting.
    • Bind each session to an owner, size, and expiration.
    • Check the completed length and checksum before publication.

    Related guides

    Primary reference: Official documentation.

    Related field note

    Continue with The maintenance burden behind resumable upload protocols.

  • A secure validation path for image and video uploads

    A secure validation path for image and video uploads

    Validate uploads in stages before turning untrusted bytes into public assets.

    Check more than an extension

    An extension and browser-supplied Content-Type are hints, not proof. Restrict allowed formats, limit byte size, inspect the file signature, and decode or probe the content with maintained libraries. Apply authorization and quota checks before accepting the upload. Store the original outside a directly executable web path.

    Quarantine before publication

    A useful state sequence is pending, uploaded, validating, processing, ready, and rejected. Keep an untrusted upload private until validation and any required scanning complete. Re-encode images or transcode video into formats you serve rather than passing arbitrary uploaded bytes through to visitors.

    Keep failure messages useful

    Return a clear reason such as unsupported format, corrupt file, or size limit, without leaking internal storage paths. Log the asset ID and validation stage for operators. An upload that passes type checks may still fail dimension, duration, or decoder limits; model those as expected product outcomes.

    A concrete acceptance test

    Rename a text file to .jpg, spoof its MIME header, and send a corrupt image with a valid signature. Each should receive a distinct, safe rejection before becoming publicly readable. Run this test with a realistic asset and record the observed state, output URL, and failure reason.

    Implementation check

    • Inspect actual bytes and decode the claimed format.
    • Keep untrusted files private until checks finish.
    • Give users a safe rejection reason for expected failures.

    Related guides

    Primary reference: Official documentation.