Developer Media Tools

Tag: Developer Guide

  • What to measure in a media pipeline

    What to measure in a media pipeline

    Track user outcomes, worker health, and delivery cost with a small metric set.

    Follow one asset through the system

    Give an upload and every downstream job a shared asset ID in logs and traces. Measure time from upload completion to ready state, failure rate by stage, retry count, and queue age. Separate user-caused validation failures from service failures so an increase in rejected files does not look like a broken worker pool.

    Watch delivery, not just processing

    Track derivative generation requests, cache hit ratio, bytes delivered, and playback or image errors from the client. A queue can be healthy while users still see broken URLs. Sample client telemetry with privacy and cost limits rather than collecting every raw URL.

    Put thresholds on decisions

    An alert should indicate a response: growing oldest-job age may call for worker capacity; rising corrupt-output failures may require a recipe rollback. Review the cost per processed asset and per delivered gigabyte alongside reliability, then retire metrics that no one uses.

    A concrete acceptance test

    Trace one upload through processing and delivery, then simulate a corrupt source and a stalled queue. Confirm that dashboards separate user rejection, worker failure, and age of the oldest job. Run this test with a realistic asset and record the observed state, output URL, and failure reason.

    Implementation check

    • Trace one asset across each processing stage.
    • Separate validation rejection from service failure.
    • Alert on queue age and delivery errors with clear owners.

    Related guides

    Primary reference: Official documentation.

  • CDN cache keys for transformed media

    CDN cache keys for transformed media

    Cache the right derivative without multiplying variants endlessly.

    Define identity first

    A cache key for transformed media needs the source version and every option that changes bytes: dimensions, crop, format, quality, and sometimes content negotiation. Normalize option order and defaults. Two equivalent requests should hit one cache entry; different outputs must never collide.

    Bound the variant space

    Arbitrary widths and quality values let a single asset produce thousands of expensive misses. Use named presets or round widths to a short approved list. Monitor unique variants per source and the ratio of cache hits to transformation requests.

    Make invalidation explicit

    Immutable versioned URLs are simpler than purging every edge when source bytes or recipes change. If mutable URLs are necessary, document purge timing and stale behavior. Test a replacement end to end on a plain URL after caches have been populated.

    A concrete acceptance test

    Warm a transform in the CDN, request equivalent options in a different order, then publish a new source version. Verify a shared hit for the equivalent URL and fresh bytes for the new version. Run this test with a realistic asset and record the observed state, output URL, and failure reason.

    Implementation check

    • Include every byte-changing option in the cache key.
    • Limit allowed dimensions and quality values.
    • Use source versions to avoid ambiguous purges.

    Related guides

    Primary reference: Official documentation.

    Related field note

    Continue with A migration plan for CDN cache keys.

  • Media processing webhooks that survive retries

    Media processing webhooks that survive retries

    Notify applications when assets become ready without duplicate side effects.

    Publish state changes as events

    A processing system can emit uploaded, ready, failed, or deleted events after committing the corresponding state. Give each event a unique ID, asset ID, event type, timestamp, and schema version. The receiver should be able to fetch the current asset state rather than treating an event payload as the only truth.

    Verify and deduplicate

    Sign webhook requests and verify the signature over the raw request body. The consumer records event IDs and handles repeat deliveries idempotently. A fast acknowledgment followed by queued work keeps transient downstream slowness from causing unnecessary retries.

    Expect disorder

    Events can arrive late or out of order. Make handlers compare versions or retrieve current state before moving a record backward. Provide a replay or reconciliation path so a temporary endpoint outage does not leave an asset permanently stuck in processing.

    A concrete acceptance test

    Deliver ready twice, then deliver an older processing event. The consumer should acknowledge all three requests while the application keeps one ready asset state. Run this test with a realistic asset and record the observed state, output URL, and failure reason.

    Implementation check

    • Verify signatures against the raw body.
    • Deduplicate by event ID before side effects.
    • Reconcile state when events arrive out of order.

    Related guides

    Primary reference: Official documentation.

  • Audio waveform previews without storing giant JSON files

    Audio waveform previews without storing giant JSON files

    Create small, useful amplitude data for audio timelines.

    Decide what the UI needs

    A waveform is a visual summary, not a second audio file. For a timeline, calculate peak or RMS amplitude over fixed time windows and store a compact array or rendered image. Choose the number of samples from the largest expected display width, not the number of source audio samples.

    Generate once per version

    Decode the audio in a background job and associate the output with the exact source version. For stereo, decide whether to show separate channels or a combined magnitude. Normalize carefully: per-file normalization makes quiet files visible but can hide relative loudness differences between files.

    Test seek alignment

    The waveform’s duration and sample count must map to the player’s time axis, including leading silence or trimmed segments. Verify a click at the midpoint seeks to roughly the midpoint. A beautiful waveform that drifts from playback is worse than a plain progress bar.

    A concrete acceptance test

    Use an audio file with leading silence and a loud event near the midpoint. Compare the waveform to playback and test a click at several points along the timeline. Run this test with a realistic asset and record the observed state, output URL, and failure reason.

    Implementation check

    • Set waveform resolution from display width.
    • Keep sample timestamps aligned with playback.
    • Test files with silence and channel imbalance.

    Related guides

    Primary reference: Official documentation.

    Related field note

    Continue with Cost-aware transformations without duplicate files.

  • Caption tracks in a video publishing workflow

    Caption tracks in a video publishing workflow

    Treat captions as a versioned deliverable with real review steps.

    Keep tracks separate from the video

    Store a caption file as a related asset with language, label, kind, version, and review status. The HTML track element can attach a WebVTT track to a video. Keep captions editable without re-encoding the video, and make the default language a deliberate product choice.

    Review timing and meaning

    Automatic speech recognition provides a draft, not a guaranteed accurate caption. Review names, technical terms, speaker changes, and timing against playback. Include meaningful non-speech audio where it affects understanding. Record who approved a track without inventing a public author identity.

    Handle changes together

    If a video is trimmed, replace or retime its captions before publishing the new cut. Validate the track URL, MIME type, and CORS behavior in the actual player. A caption file that exists in storage but never appears in controls has not completed the workflow.

    A concrete acceptance test

    Publish a revised cut with a caption track, then trim several seconds from the opening. Confirm that the old track is withdrawn and the revised cues remain aligned. Run this test with a realistic asset and record the observed state, output URL, and failure reason.

    Implementation check

    • Version each track with its language and video cut.
    • Review names, timing, and important non-speech audio.
    • Test the track in the real player controls.

    Related guides

    Primary reference: Official documentation.

  • Choosing a useful video poster frame

    Choosing a useful video poster frame

    Generate a poster that tells viewers what the clip contains.

    Choose meaning, not just a timestamp

    The first frame may be black, a fade, or a title card with no context. Sample candidate frames after probing duration, then reject frames that are blank, blurred, or misleading. For product demos, an editor-selected still is often better than any automatic sample.

    Generate for the actual slot

    Create a poster at the player’s display ratio and an appropriate pixel density. Keep the original frame or source time in metadata so the image can be regenerated. A poster is part of the video asset’s delivery state, not an unrelated decorative upload.

    Check loading behavior

    A poster can be the page’s largest image, so give it the same sizing and priority attention as a hero image. Test the transition from poster to playback and confirm controls remain clear over light and dark frames.

    A concrete acceptance test

    Generate poster candidates for a clip with a black opening frame and a fade. Check the chosen image at mobile size and verify that playback replaces it cleanly. Run this test with a realistic asset and record the observed state, output URL, and failure reason.

    Implementation check

    • Reject blank or misleading candidate frames.
    • Generate at the player slot ratio.
    • Check poster loading priority on the actual page.

    Related guides

    Primary reference: Official documentation.

  • Plan adaptive video renditions before you encode

    Plan adaptive video renditions before you encode

    Choose a playback ladder from source quality and viewer needs.

    Start with the source ceiling

    Do not upscale a low-resolution upload just to fill a standard ladder. Choose output sizes and bitrates from the source, target devices, and measured playback conditions. Keep segment duration and keyframe alignment consistent across renditions so the player can switch cleanly.

    Package a complete playback asset

    The job should produce segments, variant playlists, a master playlist, and metadata that names each rendition. Verify playlists reference files that actually exist before changing the asset to ready. Serve manifests and segments with correct MIME types and cache rules.

    Test the edges

    Seek near the end, switch bandwidth during playback, and try a short or portrait video. A valid encoder exit code does not prove the package plays. Preserve an accessible fallback when a viewer’s browser or network cannot use the preferred stream.

    A concrete acceptance test

    Play a short portrait clip and a long landscape clip while throttling network speed. Check that the rendition list respects each source and that seeking works near the end. Run this test with a realistic asset and record the observed state, output URL, and failure reason.

    Implementation check

    • Do not create renditions above the source ceiling.
    • Verify every playlist reference resolves.
    • Test seeking and switching under throttled bandwidth.

    Related guides

    Primary reference: Official documentation.

  • Build a reliable queue for video transcoding

    Build a reliable queue for video transcoding

    Turn expensive video jobs into an observable, retryable pipeline.

    Create a job from an immutable source

    Video conversion can outlast an HTTP request, so create a processing job after upload validation. Include source asset ID, recipe version, desired outputs, and an idempotency key. Workers should write to temporary keys and publish finished outputs only after probing them.

    Control concurrency

    Transcodes consume CPU, memory, disk, and network. Limit worker concurrency by workload, not simply by queue depth. Large 4K inputs may need a different pool from short clips. Record input duration, output size, processing time, and failure reason so capacity decisions use evidence.

    Retry only what can recover

    A transient storage failure may justify a retry; a corrupt input or unsupported codec needs a terminal state and a user-facing explanation. Clean up partial outputs and keep the original available for later reprocessing. Test worker crashes at each stage to ensure duplicate jobs do not publish mixed renditions.

    A concrete acceptance test

    Kill a transcoding worker after it writes one temporary rendition. Restart the job and verify that only complete, probed outputs become visible and no mixed recipe versions appear. Run this test with a realistic asset and record the observed state, output URL, and failure reason.

    Implementation check

    • Use a source-and-recipe idempotency key.
    • Probe output files before publishing URLs.
    • Classify corrupt inputs separately from transient failures.

    Related guides

    Primary reference: Official documentation.

    Related field note

    Continue with How metadata changes queue backpressure.

  • How to deliver a hero image without hurting LCP

    How to deliver a hero image without hurting LCP

    Make the likely largest image easy to discover and quick to render.

    Find the actual LCP element

    First measure the page. The largest contentful paint element may be text on one viewport and an image on another. If the hero image is the LCP element, place its URL in discoverable HTML and avoid hiding it behind a late JavaScript fetch or CSS background when a normal image works.

    Give it the right priority

    Supply a properly sized responsive source, intrinsic dimensions, and an appropriate fetch priority where supported. Do not lazy-load an image that appears immediately in the viewport. Compressing the image helps only one part of LCP; render delay and late discovery can dominate.

    Check under realistic conditions

    Measure cold-cache mobile loading and inspect the network waterfall. Compare transfer size, request start, decode, and final render. A faster local Wi-Fi screenshot does not prove an improvement for a user on a slower connection.

    A concrete acceptance test

    Measure a cold mobile load and identify the actual LCP element. Change only the hero delivery rule, then compare request start time, transferred bytes, and render delay. Run this test with a realistic asset and record the observed state, output URL, and failure reason.

    Implementation check

    • Measure the real LCP element at each viewport.
    • Make the hero URL discoverable in HTML.
    • Compare request start and render delay after the change.

    Related guides

    Primary reference: Official documentation.

  • Image metadata: what to preserve and what to strip

    Image metadata: what to preserve and what to strip

    Handle orientation, color, copyright, and location data deliberately.

    Read metadata before transforming

    Orientation and embedded color profiles affect how an image appears. Normalize orientation before sizing or cropping, and test color conversion against representative assets. Dimensions measured before orientation correction can lead to the wrong derivative shape.

    Keep private fields private

    Camera metadata may include location, device, and capture details that a user did not intend to publish. Define a publication policy: retain only fields needed for display or rights management, and strip the rest from public derivatives. The original may need a different retention rule under the product’s own privacy policy.

    Make the policy testable

    Use sample files with orientation, ICC profiles, GPS coordinates, and transparency. Verify both the rendered pixels and metadata of the downloaded derivative. A successful encoder exit code is not proof that color or privacy handling was correct.

    A concrete acceptance test

    Upload samples with rotated orientation, an ICC profile, and GPS data. Download the public derivatives and verify their pixels, color, dimensions, and removed private metadata. Run this test with a realistic asset and record the observed state, output URL, and failure reason.

    Implementation check

    • Normalize orientation before measuring or cropping.
    • Inspect public derivatives for location metadata.
    • Test color profiles against visible reference images.

    Related guides

    Primary reference: Official documentation.

  • Focal points and aspect ratios in automated cropping

    Focal points and aspect ratios in automated cropping

    Keep the subject in frame across cards, heroes, and mobile layouts.

    Save intent with the asset

    A center crop is convenient until a face, product, or text sits near an edge. Store an optional focal point as normalized coordinates on the source asset. A transformation can then center the requested crop around that point while clamping to the image bounds.

    Separate display and derivative decisions

    CSS object-position is useful when the browser displays an image in a fixed box, while server-side cropping saves bytes and ensures a consistent composition in shared previews. The same focal point can inform both, but test each target ratio: a very narrow banner may still need a manually approved crop.

    Give editors an escape hatch

    Automatic focal detection is a suggestion, not an editorial guarantee. Preview the common placements together and allow an override per important slot. Keep the original intact so a changed layout does not require re-uploading the image.

    A concrete acceptance test

    Preview a portrait subject in square, wide, and narrow slots. Move the focal point once and confirm all three placements update without changing the preserved source image. Run this test with a realistic asset and record the observed state, output URL, and failure reason.

    Implementation check

    • Store focal coordinates with the original asset.
    • Preview square, landscape, and narrow slots.
    • Allow a manual crop override for important placements.

    Related guides

    Primary reference: Official documentation.

  • Designing an image transformation API

    Designing an image transformation API

    Make resize and format URLs predictable, safe, and cacheable.

    Constrain the recipe

    A transformation URL should identify an immutable source plus a small allowed set of operations: width, height, fit, quality, and format. Normalize equivalent requests to one canonical recipe. Cap output dimensions and pixel count before starting a decoder so one request cannot consume excessive memory.

    Version intentional changes

    When crop behavior, sharpening, or encoder settings change, include a recipe version in the derivative identity. Otherwise a cached URL can mean different bytes before and after deployment. Record transformation errors separately from missing source assets.

    Protect the origin

    If anyone can request any width for any source, a crawler can create thousands of variants. Use named presets for common UI slots, sign private transformations, and rate-limit expensive misses. Warm only genuinely important variants; on-demand generation is useful when the variant space is bounded.

    A concrete acceptance test

    Request the same transform with reordered parameters, then request a disallowed extreme width. The first two URLs should map to one output; the extreme request should fail before expensive decoding. Run this test with a realistic asset and record the observed state, output URL, and failure reason.

    Implementation check

    • Canonicalize equivalent transformation recipes.
    • Reject excessive dimensions before decoding.
    • Version the recipe when output bytes change.

    Related guides

    Primary reference: Official documentation.