You've probably been here. Someone on the product team asks for automatic thumbnail generation. Then it's "can we compress uploads before they hit S3?" Then a customer wants their vertical video cropped to square for a feed. Each request sounds small. Together they turn into a media pipeline you now own, operate, and babysit.
FFmpeg can do all of it. FFmpeg has been able to do all of it for years. The problem was never the encoding. The problem is everything around the encoding — the servers, the queues, the crash recovery, the containers you rebuild every time a dependency breaks, the 2 a.m. page because a worker got stuck on a malformed file.
A JSON-based FFmpeg API changes that arrangement. Instead of running FFmpeg on a machine you manage, you send a JSON payload with your inputs and your FFmpeg arguments to an endpoint, and the job runs somewhere else. You get back a result. That's the whole interaction model.
This post walks through what that looks like in practice: what belongs in the payload, how filter_complex works when it's not sitting in a shell script, what "compute seconds" pricing really means, the mistakes people make when they move off local FFmpeg, and where a service like FFmpeGo fits into a modern media workflow.
If you're an indie developer shipping a video feature this week, an automation engineer gluing tools together for an ops team, or a SaaS company burning money on idle transcoding instances, this is written for you.
What a JSON-Based FFmpeg API Actually Is
Start with the plain version: it's FFmpeg running on someone else's infrastructure, controlled over HTTP instead of a terminal.
You POST a JSON body. The body contains the URLs of the files you want to process and the FFmpeg arguments you want applied. The service spins up a worker, runs your command, and hands back the output — usually a signed URL you can download or push straight to your own storage.
That's it. No Dockerfile. No apt-get install ffmpeg. No "which build has libvpx-vp9 in it?"
The command line, minus the server
The important part is what doesn't change. You still write FFmpeg arguments the way you always have. -c:v libx264, -crf 23, -preset fast, -vf scale=1280:-2. If you've spent any time reading the official FFmpeg documentation, you already know how to use this API. The learning curve is mostly about the payload shape and a few JSON-specific escaping quirks.
This is why the "arbitrary command" model matters. A lot of video APIs give you a dropdown: convert to MP4, convert to WebM, maybe generate a thumbnail. The moment your requirements go past those presets — a custom overlay that fades in at 3 seconds, a loudness normalization pass, a two-input picture-in-picture composite — you're stuck. Either you pay for an enterprise tier or you go back to running servers.
With an arbitrary-command API, the full surface area of FFmpeg stays available. Filters, codecs, muxers, stream mapping, complex graphs. If it works on your laptop, it works in the cloud.
Why the JSON part matters
You could theoretically build a REST API where you send the whole command as one long string. Plenty of people do that internally. It's brittle. Shell quoting, space splitting, argument order assumptions — all of it becomes a parsing problem you have to solve on both ends.
JSON sidesteps that by keeping arguments as a structured list. Each flag and each value is its own string in an array, exactly as the process would receive them in argv. There's no exec() with shell=true, no quoting surprises, no "did the filename with a space in it break something?" guesswork.
There's a second benefit that shows up later: structured payloads are easy for machines to generate. Your app, your queue worker, your n8n flow, your CI job — they all produce JSON without thinking about it. Turning a job description into a request body is trivial when the job description is already JSON.
Why Media Processing Breaks When You Self-Host
Everyone starts with the same architecture. One script. One worker. Maybe a cron job that picks up files from a folder.
It works fine until it doesn't.
CPU spikes and the queue you didn't want to build
Video encoding is bursty. A 1080p transcode with libx264 will happily pin every core you give it for as long as the file takes. Ten minutes of source footage can mean several minutes of a fully saturated machine — and if three uploads land at once, three encodes fight for the same CPU.
So the first thing you build is a queue. Fair enough. But a queue implies concurrency limits, which implies retries, which implies a way to detect hung jobs, which implies a timeout policy, which implies a dead-letter queue, which implies dashboards so you know when the DLQ is filling up.
None of that is media work. It's distributed systems work, and it's real work, and it never quite ends.
Meanwhile your CPU utilization graph looks like a heart monitor. Peaks at 100% during a batch of uploads, near zero at 3 a.m. You're paying for capacity sized to the peak and using it for a fraction of the day.
The container treadmill
FFmpeg is a moving target. New releases add codecs, deprecate flags, change default behaviors. Builds differ: the ffmpeg on Debian stable is not the ffmpeg on Alpine, is not the one Homebrew installs. If you need libfdk_aac or a specific build of libvpx, you're compiling from source, and now you own a custom base image.
Then a security patch lands in a system library, and you're rebuilding. Then your base image tag gets updated upstream and something breaks. Then a worker somewhere is still running the old image because the deploy didn't roll cleanly.
This is a tax that gets paid in small increments forever. Most teams don't notice how much of it there is until they stop paying it.
You pay for failures
Here's the part that stings. On your own hardware, a failed job costs the same as a successful one. A corrupt input file, a filter typo, a codec that doesn't exist in your build — the process still burns CPU and wall-clock time before it dies. You pay for the electricity and the instance hours either way.
And failed jobs are more common than people expect in the first month of a new pipeline. You're tuning CRF values, testing presets, discovering that your source files sometimes have no audio stream at all and your -c:a aac flag makes FFmpeg exit with an error. Every one of those iterations is a real cost.
Anatomy of a Request: Inputs, Arguments, Output
Let's get concrete. The exact field names on FFmpeGo live in the docs, but the shape of a request is consistent across this class of API and looks roughly like this.
A minimal transcode payload
{
"inputs": [
{ "url": "https://cdn.example.com/raw/interview.mp4" }
],
"args": [
"-c:v", "libx264",
"-preset", "veryfast",
"-crf", "23",
"-vf", "scale=1280:-2",
"-c:a", "aac",
"-b:a", "128k",
"-movflags", "+faststart"
],
"output": {
"format": "mp4"
}
}
Read the args array out loud and you have a valid command. That's the design goal: the arguments are the command.
A few notes on what's happening in there:
-preset veryfasttrades compression efficiency for speed. For user-generated content where you'll re-encode storage tiers later, that's usually the right call.-crf 23is x264's default quality. Lower is better quality and bigger files. Most web video lives between 18 and 26.scale=1280:-2forces width to 1280 and lets FFmpeg compute height. The-2(rather than-1) keeps the result divisible by two, which H.264 with 4:2:0 chroma requires. Use-1and you'll occasionally hit an odd height and get a cryptic error.-movflags +faststartmoves the moov atom to the front of the file so browsers can start playing before the whole file downloads. Skip it and your progressive playback is worse for no reason.
Multi-input jobs
The interesting stuff starts when you have more than one input. Say you want to overlay a logo on a video and mix in a background music track:
{
"inputs": [
{ "url": "https://cdn.example.com/raw/segment-a.mp4" },
{ "url": "https://cdn.example.com/brand/logo.png" },
{ "url": "https://cdn.example.com/audio/bed.m4a" }
],
"args": [
"-filter_complex",
"[0:v]scale=1920:-2[base];[1:v]scale=180:-1[logo];[base][logo]overlay=W-w-40:40[v];[2:a]volume=0.25[music];[0:a][music]amix=inputs=2:duration=first[a]",
"-map", "[v]",
"-map", "[a]",
"-c:v", "libx264",
"-crf", "21",
"-preset", "medium",
"-c:a", "aac",
"-b:a", "192k"
],
"output": { "format": "mp4" }
}
Three inputs. A filter graph that scales the base, scales the logo, overlays it in the top-right corner, attenuates the music bed, and mixes it with the original audio. One request.
This is the capability that separates a real FFmpeg API from a converter tool. Notice that none of this required a new endpoint or a preset. The graph is just text inside a JSON string.
What comes back
The response typically gives you a job identifier and a status. Depending on the service, you either poll a status endpoint or receive a webhook when the job finishes. On success you get a URL to the rendered file. On failure you get an error message and, if the service is good about it, the tail of FFmpeg's stderr so you can see which filter choked.
Two things worth checking before you commit to any provider:
- Does the stderr come back? Debugging a filter graph without FFmpeg's own error output is miserable.
- Are failed jobs billed? More on this below, but it changes your testing habits entirely.
Working Examples You Can Adapt
Let's look at how this lands in actual code.
cURL
curl -X POST https://api.ffmpego.com/v2/run \
-H "Authorization: Bearer $FFMPEGO_API_KEY" \
-H "Content-Type: application/json" \
-d '{
"inputs": [
{ "url": "https://cdn.example.com/raw/upload-8821.mov" }
],
"args": [
"-c:v", "libx264",
"-preset", "veryfast",
"-crf", "23",
"-pix_fmt", "yuv420p",
"-c:a", "aac",
"-b:a", "128k",
"-movflags", "+faststart"
],
"output": { "format": "mp4" }
}'
The -pix_fmt yuv420p is worth calling out. Phone cameras and screen recorders love producing files in yuv444p or yuvj420p, which H.264 technically supports but many browsers won't decode. If your video uploads play fine in VLC and show a black frame in Safari, this flag is usually the fix.
Node.js
const res = await fetch("https://api.ffmpego.com/v2/run", {
method: "POST",
headers: {
Authorization: `Bearer ${process.env.FFMPEGO_API_KEY}`,
"Content-Type": "application/json"
},
body: JSON.stringify({
inputs: [{ url: uploadUrl }],
args: [
"-vf", "scale='min(1920,iw)':-2",
"-c:v", "libx264",
"-crf", "22",
"-preset", "fast",
"-c:a", "aac",
"-b:a", "160k",
"-movflags", "+faststart"
],
output: { format: "mp4" }
})
});
const job = await res.json();
That scale='min(1920,iw)':-2 expression is a nice one for UGC platforms. It downscales anything wider than 1920 and leaves smaller videos alone, so you never upscale a 720p upload into a 1080p blur-fest.
Python
import os, requests
resp = requests.post(
"https://api.ffmpego.com/v2/run",
headers={"Authorization": f"Bearer {os.environ['FFMPEGO_API_KEY']}"},
json={
"inputs": [{"url": src_url}],
"args": [
"-af", "loudnorm=I=-16:TP=-1.5:LRA=11",
"-c:v", "copy",
"-c:a", "aac",
"-b:a", "128k"
],
"output": {"format": "mp4"}
},
timeout=30,
)
resp.raise_for_status()
This one normalizes audio to roughly -16 LUFS (a common target for web and podcast delivery) and copies the video stream untouched. Because the video is copied rather than re-encoded, the job finishes in a fraction of the time a full transcode would take. On a per-compute-second pricing model, that difference shows up directly on your bill.
Automation platforms
If you're wiring this into n8n, Make, or Zapier, an HTTP request node is all you need. Drop the JSON body into the node's raw body field, reference your API key from a credential store, and you're done.
A pattern that works well: watch a folder in Google Drive or Dropbox, pass the shared link as the input URL, and route the finished file to a client-facing folder. No code, no server, no maintenance. Media pipelines for teams that don't have engineers on staff become genuinely feasible this way.
Putting filter_complex to Work
The filter_complex flag is where FFmpeg stops being a converter and starts being a non-linear editor. Here are patterns that come up constantly, written so you can paste them into a request.
Watermarking with a fade
A static watermark is easy. A watermark that fades in after a few seconds and out before the end is only slightly harder:
-filter_complex "[1:v]format=rgba,fade=in:st=2:d=1:alpha=1,fade=out:st=8:d=1:alpha=1[wm];[0:v][wm]overlay=W-w-30:30"
The format=rgba gives the logo an alpha channel, the two fade filters animate its opacity, and overlay positions it 30 pixels from the top-right. Change st= values to move the timing around.
Concatenating clips
Two ways to join files, and picking the wrong one wastes time.
If the clips already share the same codec, resolution, and frame rate — for example, segments exported from the same camera — use the concat demuxer. It's stream copy, so it's fast:
-f concat -safe 0 -i inputs.txt -c copy
If your inputs differ in any way, you need the concat filter, which decodes and re-encodes:
-filter_complex "[0:v][0:a][1:v][1:a][2:v][2:a]concat=n=3:v=1:a=1[v][a]" -map "[v]" -map "[a]"
The n=3:v=1:a=1 reads as "three segments, one video output, one audio output." Get the counts wrong and you get a graph error. It happens to everyone at least once.
Side-by-side and picture-in-picture
Split-screen comparison:
-filter_complex "[0:v]scale=960:540[l];[1:v]scale=960:540[r];[l][r]hstack=inputs=2[out]"
Vertical stack instead of horizontal — the format most social feeds want:
-filter_complex "[0:v]scale=1080:960[top];[1:v]scale=1080:960[bottom];[top][bottom]vstack=inputs=2[out]"
Picture-in-picture with a rounded mask is more involved, but the building blocks are the same: scale each input, then compose with overlay.
Loudness normalization done properly
Podcasters and audio engineers will tell you that a single-pass loudnorm is fine for casual work and not fine for delivery specs that matter. The two-pass version measures the file first, then applies corrections based on the measured values.
On a single-request API, two passes means two requests: one to get the measurement from stderr, one to apply it. It's a bit more plumbing, but the difference is audible when you're pushing to platforms with real loudness standards.
HLS packaging
Adaptive streaming output is just another set of arguments:
-i input.mp4 -c:v libx264 -c:a aac -f hls -hls_time 6 -hls_playlist_type vod -hls_segment_filename "seg_%03d.ts" index.m3u8
Running this yourself means managing a directory of segments and uploading them all. Running it through an API means you point the output at a bucket and let the service handle the fan-out. Worth checking how your provider handles multi-file outputs before you build on it.
Thumbnails and preview sprites
Extracting a frame is cheap:
-ss 00:00:05 -i input.mp4 -frames:v 1 -vf scale=640:-2 thumb.jpg
Note that -ss before -i is a fast seek, and after -i is an accurate seek. For a thumbnail 5 seconds in, the fast version is fine. For frame-exact extraction — say, matching a specific timestamp in a transcript — put -ss after -i.
A contact sheet of frames is also a single command:
-i input.mp4 -vf "fps=1/10,scale=320:-1,tile=4x4" -frames:v 1 sheet.jpg
That gives you 16 frames sampled every 10 seconds. Good for hover previews and content moderation triage.
Compute Seconds: What You're Actually Paying For
Pricing models for media APIs fall into a few buckets: per minute of output, per gigabyte processed, per job, or per compute second. The first three have a common flaw — they bill you for something other than the work done.
Per compute second is the honest one. The meter measures wall-clock execution time of the FFmpeg process. A job that takes 40 seconds costs 40 units. A job that takes 4 seconds costs 4 units.
The math, roughly
Say your rate is R per compute second. A 10-minute 1080p clip transcoded to 720p with -preset veryfast typically finishes in under two minutes on a decent multi-core worker. Call it 90 seconds. That job costs 90 × R.
Now compare that to the self-hosted version. A mid-size instance capable of handling a couple of concurrent 1080p encodes runs somewhere in the $50–70/month range on the major clouds, and it's billed whether or not anything is running. If your actual encoding load is three hours a month, you're paying for roughly 720 hours of idle time to get those three hours of work.
That gap is the whole argument. Bursty workloads and always-on servers don't mix.
Why failed jobs shouldn't cost you anything
If a service only bills for successful encodes — HTTP 2xx responses — your development loop changes shape. You can iterate on filter graphs freely. You can test edge cases without watching a meter. You can point a staging environment at the API and let it fail loudly and often.
Compare that to your own worker, where a misconfigured filter burns the same CPU as a good one. There's an argument that this is minor, but in practice the psychology matters. When failures are free, people test more. When testing costs money, people ship untested filter graphs and find out in production.
Caps, free tiers, and not getting surprised
Two features to look for regardless of provider:
- A monthly hard cap. Once you hit a spending limit, jobs stop. Not "we'll email you" — actually stop. This is what keeps a runaway batch job from turning into a $4,000 invoice.
- A real free tier. Not a 60-second trial. Enough compute seconds that you can build a proof of concept, run it against real files, and decide based on evidence rather than a sales call.
FFmpeGo structures both around compute seconds and only bills successful encodes, which makes the cost model something you can predict before you commit, not something you reverse-engineer from a monthly invoice.
Use Cases: Where This Fits
Different teams arrive at the same API from different directions.
User-generated content platforms
The classic case. Users upload whatever their phone produced — HEVC, HDR, odd resolutions, rotated metadata, sometimes with no audio track. You need a consistent, web-playable MP4 at the end.
The pipeline: normalize to H.264 with yuv420p, cap the resolution, strip rotation metadata after applying it, extract a poster frame. All of it fits in one or two requests per upload. Because jobs only bill when they succeed, a flood of weird files in your early days doesn't cost you the same as a flood of good ones.
Podcast and video podcast production
Two hosts on separate tracks recorded locally, plus a mixed stereo feed. Before publishing you want: normalized loudness, level-matched host tracks, a compressed mono mix for the feed, and a video version with a waveform visualization.
This is a filter_complex job with three or four inputs. Running it by hand in Audacity takes ten minutes. Running it as an API call takes no minutes and produces identical output every week.
E-learning and course platforms
Uploaded lectures need chapter markers, thumbnails, transcripts, and multiple quality renditions. The thumbnails and renditions come from FFmpeg. The interesting part is that these jobs are bursty — a course gets uploaded in one afternoon and then the platform processes nothing for a week. Serverless billing matches that pattern exactly; a fixed instance doesn't.
Marketing and creative operations
Creative teams generate dozens of aspect-ratio variants per campaign: 16:9, 1:1, 4:5, 9:16, each with safe-zone padding and a caption burn-in variant. That's the same source file and a different filter graph each time. With a JSON API, the whole matrix is a loop over an array of arguments.
Internal automation
Ops teams building "when a file lands here, do this" workflows. A recruiting team that wants to strip candidate names from screen recordings. A support team that wants compressed clips attached to tickets. None of these need a media engineering team. They need an HTTP node and a JSON body.
Common Mistakes (and Cheap Fixes)
Most of the problems people hit in their first week are predictable. Here's the list.
Forgetting the even-dimension rule. scale=1280:-1 will eventually produce an odd height and fail with libx264. Use -2. Costs nothing, prevents a class of errors.
Skipping -pix_fmt yuv420p. Your output plays in VLC and shows black in Safari or QuickTime. This is the first thing to check when a video "works but doesn't."
Not using +faststart. Progressive playback is slower and your CDN's range requests do less work. One flag.
Re-encoding when you should copy. If you only need to change the container or normalize audio, -c:v copy saves the entire video encode. On compute-second pricing, that's most of the cost.
High CRF values on already-compressed input. Generational loss stacks. If you're transcoding a file that was already compressed, use a lower CRF (18–21) than you would for a camera original, or the artifacts pile up.
Escaping filter syntax wrong in JSON. Inside a JSON string, a backslash has to be escaped as \\, and quotes inside filter expressions need escaping too. If FFmpeg reports a filter parsing error on a graph that works in your terminal, this is usually why. Test graphs in the terminal first, then paste them in and fix the escaping.
Ignoring stderr on failure. When a job fails, read the last ten lines of FFmpeg's output. It almost always names the exact filter or codec that broke. Skipping this turns a two-minute fix into an hour of guessing.
Running everything at -preset slow. Slower presets buy you smaller files at the cost of compute seconds. For archival masters, worth it. For a preview clip that gets deleted in a week, absurd. Match the preset to the lifespan of the output.
No timeout on your HTTP client. Some jobs take minutes. Set your client timeout generously, or use the provider's job-status endpoint and poll. A 30-second client timeout on a 4-minute transcode produces a lot of confusing "the API is broken" reports that are actually your own code giving up.
Building a queue on top of a service that already has one. If your provider handles concurrency and retries, don't add your own single-threaded worker in front of it. You'll recreate the bottleneck you were trying to escape.
A Migration Checklist
If you're moving an existing local FFmpeg pipeline to a JSON API, work through this in order.
- Inventory your commands. Every
ffmpeginvocation in your codebase, your cron jobs, and your runbooks. Write them all down. - Classify each one. Transcode, remux, extract, filter graph, packaging. Remux jobs are the cheapest and easiest wins — start there.
- Confirm your inputs are reachable. The API fetches from URLs. If your files live behind authentication, you need signed URLs, and those expire. Generate them at request time, not ahead of time.
- Test the graph locally first. Get the arguments right on your laptop, then move them into the payload. Debugging is faster when you can see the terminal.
- Convert one job end to end. Pick your most common operation. Get it producing identical output to your old pipeline.
- Compare outputs byte-by-byte or by SSIM. Not just "it looks right." A frame-rate mismatch or a colorspace shift can be subtle enough to miss by eye.
- Add error handling around the response. Log the job ID and stderr. Your future self will want both.
- Set your monthly cap. Before you route production traffic through it.
- Run both pipelines in parallel for a week. Route a fraction of traffic to the new path, compare results, and check that your cost estimate holds.
- Delete the old worker. This is the step people skip, and it's the one that actually delivers the benefit.
FAQ
Do I need to know FFmpeg to use an FFmpeg API?
Practically, yes. The API runs arbitrary arguments, which means you're still writing FFmpeg commands. If you can't build a working command in a terminal, you can't build one in a JSON payload. The good news is that the API removes the operational work, not the media knowledge — so you only have to learn one thing instead of two.
How is this different from a preset-based video conversion API?
Preset APIs expose a fixed menu: convert to MP4, convert to WebM, make a thumbnail. They're fast to integrate and get stuck quickly. An arbitrary-command API exposes FFmpeg itself. When you need a two-input overlay with audio ducking, you write the filter graph instead of hunting for a plan that supports it.
What happens if my job fails halfway through?
The job returns a non-2xx response with the error output from FFmpeg. On FFmpeGo, failed jobs aren't billed — you only pay for successful encodes. Fix the argument or the input and resend.
Can I run jobs that take a long time?
Yes, within the platform's maximum job duration. Long jobs mean long compute-second counts, so if you're processing hours of 4K footage, look at whether -c:v copy or a faster preset gets you there before committing to a slow encode. And always use the status or webhook flow rather than holding an HTTP connection open for ten minutes.
Where do output files go?
Typically to a URL the service returns. Most teams immediately transfer the result to their own bucket or CDN. Check whether your provider supports direct-to-bucket output — it saves a download-and-reupload step.
How do I control costs?
Three levers. Faster presets reduce compute seconds. Stream copying instead of re-encoding removes the video encode entirely. And a monthly hard cap makes sure a bug can't turn into a surprise invoice.
Is this a replacement for a full media pipeline like Mux or AWS MediaConvert?
Not exactly. Those are opinionated end-to-end products with their own models of what a video is. An FFmpeg API is lower-level: it runs the commands you'd run, at scale, without the servers. If you want a managed player and analytics suite, you want the former. If you want the flexibility of FFmpeg in a cloud function, you want the latter.
Can I use it from a serverless function?
Yes, and it's a good fit. Your Lambda or Cloudflare Worker fires off the request and returns. The encoding happens elsewhere, so you don't hit the execution-time limits that make in-function FFmpeg painful. If you've ever tried bundling an FFmpeg binary into a Lambda layer, you know how much easier this is.
Wrapping Up: Stop Operating, Start Building
The reason FFmpeg servers accumulate cruft isn't that anyone planned it that way. It's that every media feature looks small in isolation. One more filter. One more preset. One more worker. Five years later there's a Kubernetes cluster running transcoding jobs and nobody remembers why.
The JSON API model flips the default. You keep the part that's actually yours — the encoding logic, the filter graphs, the business rules about what a "processed" file means — and hand off the part that isn't: capacity, queues, retries, containers, and paying for idle CPU.
If you want to see how the request-response shape feels in practice, FFmpeGo has a free tier and runs jobs from a single /v2/run endpoint. Send one payload with a real file from your own storage. If the output matches what your local FFmpeg produces, you've just validated replacing an entire server tier with a POST request.
Start with your simplest job — a remux, a thumbnail, a compression pass. Get it working end to end. Then migrate the filter graphs, one at a time, comparing outputs as you go. Most teams find the whole pipeline is converted in an afternoon and the servers get shut off that same week.