← All articles

Fix Audio Drift When Concatenating Videos With FFmpeg

Fix Audio Drift When Concatenating Videos With FFmpeg

Audio drift when concatenating videos with FFmpeg almost always traces back to the same handful of causes: mismatched sample rates, start timestamps that don't begin at zero, variable frame rates, or an AAC bitstream carrying its own encoder delay into the spliced file. The fix is either to make every input identical before you join them, or to use the concat filter with aformat and asetpts so FFmpeg rebuilds the audio timeline from scratch. This post walks through both methods, plus the ffprobe commands that tell you which problem you actually have.

No theory-first detour. Let's start with what breaks and why.

What Audio Drift Actually Looks Like

People use "drift" to describe two different problems, and they need different fixes.

Real drift means the offset grows. Clip one looks fine. By clip four, mouths are maybe a beat behind the words. By clip ten, it's a subtitled foreign film where nobody told you. That's accumulation — each join adds a little error, and the errors stack.

A fixed offset means the whole output is out of sync by the same amount from the first frame. That's not drift. That's a bad start timestamp or a channel layout mismatch, and it's usually easier to fix.

If your audio is perfectly synced at the start and progressively walks away, you're in the first bucket. Keep reading. If it's uniformly late or early, skip to the section on timestamp rebasing.

There's also a third symptom worth naming: the audio doesn't drift so much as it sputters — tiny skips or repeated fragments at each join point. That's a sample rate conversion happening badly, or an aresample filter doing gap-filling it shouldn't need to do.

Why Concatenation Breaks Audio Sync

FFmpeg gives you two ways to concatenate, and they behave completely differently under the hood.

The concat demuxer (-f concat) treats your files as a list of segments to staple together. With -c copy it doesn't decode or re-encode anything — it copies the compressed packets straight through. Fast, lossless, and only works if every input is already identical in ways most people never check.

The concat filter (concat=n=...:v=1:a=1) decodes everything first, then rebuilds a fresh continuous timeline. Slower because it re-encodes, but it doesn't care what your inputs looked like.

Almost every drift problem comes from expecting the demuxer to be forgiving. It isn't.

Here are the specific things that go wrong.

Mismatched sample rates

This is the big one. One clip is 44,100 Hz (a music track, say). Another is 48,000 Hz (phone video). A third is 32,000 Hz because someone recorded it on an old Android handset.

The concat demuxer will happily splice those together with -c copy. The output container reports one sample rate. Your player decodes the others at the wrong speed. The result is that audio plays slightly fast or slow relative to the video, and the error is proportional to segment length. A 30-second clip at the wrong rate can be off by more than a second.

Fix: pick 48,000 Hz, force everything to it, and re-encode the audio. There's no way around this one.

Non-zero start timestamps

Every media file has a start_time in its container metadata, and it isn't always zero. Some recorders, some editors, and a lot of files that have been trimmed or exported from NLEs start at something like 0.023 or 1.4 seconds. Or a negative value.

The concat demuxer copies those timestamps through. So segment one occupies 0.023s to 30.023s, segment two starts at 0.019s (because it was copied from a different file with its own offset), and the player has to guess what to do with overlapping timestamps. Audio can get pushed forward or clipped at each junction.

This is the classic "grows worse with every clip" cause.

AAC encoder delay and priming samples

AAC has a fixed encoder delay baked into every stream. The exact figure depends on the encoder — somewhere in the low thousands of samples, which works out to a few tens of milliseconds. Decoders are supposed to compensate for it, and most do... but only when each stream is decoded independently from a known starting point.

Staple a dozen AAC streams together with -c copy and you get a dozen priming delays that the final decode pass may or may not account for. Twenty milliseconds doesn't sound like much. Multiply by fifteen clips and you're a third of a second out.

Re-encoding the audio side-steps this entirely, because the new encode has one delay, at the very beginning, handled once.

Variable frame rate sources

Screen recordings, Zoom calls, phone video, and anything that's been through a web editor are frequently VFR — variable frame rate. The encoder wrote frames when they were available rather than on a fixed clock.

When you concatenate a VFR clip with a CFR clip, the frame counts don't map cleanly to time. Video duration and audio duration diverge. Players resync once at the start and then let it slide.

The fps filter (or -fps_mode cfr) fixes this by resampling to a constant rate.

Missing or extra audio streams

Some clips have no audio track at all — a silent drone shot, a screen recording with audio disabled, a stock clip with the audio stripped. If your concat filter graph expects [1:a] and there isn't one, the whole command errors out. If you're using the demuxer and one file has audio while others don't, you can get unpredictable results depending on the container.

You need to handle silent clips explicitly. More on that below.

Diagnose Before You Fix: The ffprobe Commands

Don't guess. Run this against each of your inputs first.

ffprobe -v error -select_streams a:0 \
  -show_entries stream=codec_name,sample_rate,channels,channel_layout,time_base,start_time,duration \
  -of default=noprint_wrappers=1 clip1.mp4

You want to see sample_rate=48000 on every file, channel_layout=stereo on every file, and start_time=0.000000 on every file. Any deviation is a suspect.

To check all of them at once:

for f in clip*.mp4; do
  printf '%s  ' "$f"
  ffprobe -v error -select_streams a:0 \
    -show_entries stream=sample_rate,channels,start_time \
    -of csv=p=0 "$f"
done

Sample output that's about to cause you pain:

clip1.mp4  48000,2,0.000000
clip2.mp4  44100,2,0.023000
clip3.mp4  48000,1,0.000000

Two problems in three lines: clip2 has the wrong sample rate and a start offset, clip3 is mono.

Now check the video side:

ffprobe -v error -select_streams v:0 \
  -show_entries stream=codec_name,width,height,r_frame_rate,avg_frame_rate,time_base,start_time \
  -of default=noprint_wrappers=1 clip1.mp4

If r_frame_rate and avg_frame_rate disagree, the file is VFR. If width/height/time_base differ across files, the demuxer is off the table.

Method 1: The Concat Demuxer, Done Properly

If every input genuinely matches — same codecs, same resolution, same SAR, same pixel format, same frame rate, same sample rate, same channel layout, zero start times — the demuxer is the cheapest option. It doesn't touch a single frame.

Create a list file:

file 'clip1.mp4'
file 'clip2.mp4'
file 'clip3.mp4'

Then:

ffmpeg -f concat -safe 0 -i list.txt -c copy output.mp4

A few notes on that command. -safe 0 lets you use absolute or ../-relative paths, which is required if your list file isn't in the same directory as the media. Paths inside list.txt are resolved relative to the list file itself, not your shell's working directory — a detail that trips people up constantly.

If FFmpeg complains about missing timestamps, add -fflags +genpts. If you see complaints about non-monotonous DTS, that's the demuxer telling you the timestamps overlap, which means you've got the start_time problem described above. You can paper over it with -avoid_negative_ts make_zero, but the honest answer is to normalize the files first.

The demuxer route breaks the moment two files differ in any of those parameters. It won't always warn you either. Sometimes it just produces a file where the audio is subtly wrong.

Method 2: The Concat Filter — The Reliable Fix

This is where the drift actually gets solved. The filter decodes every input, normalizes it, rebases its timeline to zero, and then lays everything end to end.

Here's a complete three-clip command:

ffmpeg \
  -i clip1.mp4 -i clip2.mp4 -i clip3.mp4 \
  -filter_complex "\
[0:v]scale=1280:720:force_original_aspect_ratio=decrease,pad=1280:720:-1:-1,setsar=1,fps=30,setpts=PTS-STARTPTS[v0];\
[0:a]aformat=sample_fmts=fltp:sample_rates=48000:channel_layouts=stereo,asetpts=PTS-STARTPTS[a0];\
[1:v]scale=1280:720:force_original_aspect_ratio=decrease,pad=1280:720:-1:-1,setsar=1,fps=30,setpts=PTS-STARTPTS[v1];\
[1:a]aformat=sample_fmts=fltp:sample_rates=48000:channel_layouts=stereo,asetpts=PTS-STARTPTS[a1];\
[2:v]scale=1280:720:force_original_aspect_ratio=decrease,pad=1280:720:-1:-1,setsar=1,fps=30,setpts=PTS-STARTPTS[v2];\
[2:a]aformat=sample_fmts=fltp:sample_rates=48000:channel_layouts=stereo,asetpts=PTS-STARTPTS[a2];\
[v0][a0][v1][a1][v2][a2]concat=n=3:v=1:a=1[v][a]" \
  -map "[v]" -map "[a]" \
  -c:v libx264 -crf 20 -preset medium -pix_fmt yuv420p \
  -c:a aac -b:a 192k -ar 48000 -ac 2 \
  -movflags +faststart \
  output.mp4

Let's break down why each piece is there.

scale=...:force_original_aspect_ratio=decrease,pad=1280:720:-1:-1 — fits every clip into a 1280×720 canvas without stretching it, then pads the leftover space with black. If you skip this and just use scale=1280:720, your anamorphic or 4:3 footage gets squashed.

setsar=1 — forces square pixels. Mismatched sample aspect ratios across clips make players letterbox inconsistently and can shift subtitle positioning. Set it and forget it.

fps=30 — resamples everything to a constant 30 fps. This is your VFR killer. Pick whatever rate matches your highest-quality source.

setpts=PTS-STARTPTS — the video timeline rebase. Every input's clock resets so its first frame is at time zero.

aformat=sample_fmts=fltp:sample_rates=48000:channel_layouts=stereo — the audio equivalent of a hard reset. Sample format, rate, and channel layout all forced to one target. This single filter handles the 44.1k/48k mismatch, the mono/stereo mismatch, and the float-vs-integer mismatch all at once.

asetpts=PTS-STARTPTS — this is the line that fixes drift. It rebases the audio timeline so the first sample of each segment is at time zero. Without it, each segment's original offset gets baked in and errors accumulate.

concat=n=3:v=1:a=1 — consumes three video and three audio streams in alternating order and emits one of each. The order matters: [v0][a0][v1][a1] not [v0][v1][a0][a1].

-ar 48000 -ac 2 on the output — belt and braces. Set the target rate at the encoder too, so nothing quietly renegotiates.

The belt-and-braces variant with aresample

If you've got clips where the audio track and video track have slightly different durations — common with phone footage and anything that's been through a lossy editor — add aresample before asetpts:

[0:a]aformat=sample_fmts=fltp:sample_rates=48000:channel_layouts=stereo,aresample=async=1:first_pts=0,asetpts=PTS-STARTPTS[a0]

aresample=async=1 stretches or squeezes audio to match the video duration, inserting silence if the audio runs short and dropping samples if it runs long. first_pts=0 forces the first output sample to land on time zero.

Use it when you need it. It's doing real work, and it can mask a truncated file rather than fixing it, so check your sources first.

Scaling this to more clips

Hand-writing filter chains gets old fast. Generate them.

files=(clip1.mp4 clip2.mp4 clip3.mp4)
n=${#files[@]}

inputs=()
graph=""
for i in "${!files[@]}"; do
  inputs+=(-i "${files[$i]}")
  graph+="[$i:v]scale=1280:720:force_original_aspect_ratio=decrease,pad=1280:720:-1:-1,setsar=1,fps=30,setpts=PTS-STARTPTS[v$i];"
  graph+="[$i:a]aformat=sample_fmts=fltp:sample_rates=48000:channel_layouts=stereo,asetpts=PTS-STARTPTS[a$i];"
done

for i in "${!files[@]}"; do graph+="[v$i][a$i]"; done
graph+="concat=n=$n:v=1:a=1[v][a]"

ffmpeg "${inputs[@]}" -filter_complex "$graph" \
  -map "[v]" -map "[a]" \
  -c:v libx264 -crf 20 -preset medium -pix_fmt yuv420p \
  -c:a aac -b:a 192k -ar 48000 -ac 2 \
  -movflags +faststart output.mp4

One caveat: a 20-input filter graph will chew through RAM and CPU, and a single bad input can blow up the whole job. For large batches, the two-pass approach below is more forgiving.

Normalize Once, Concat Many

If you're joining clips regularly — a product that assembles user uploads, a pipeline that stitches daily recordings — the smarter architecture is to normalize everything on ingest and keep a uniform library.

Step one: convert each raw file to identical parameters.

mkdir -p norm
i=0
for f in raw/*.mp4; do
  i=$((i+1))
  ffmpeg -y -i "$f" \
    -vf "scale=1280:720:force_original_aspect_ratio=decrease,pad=1280:720:-1:-1,setsar=1,fps=30" \
    -af "aformat=sample_fmts=fltp:sample_rates=48000:channel_layouts=stereo" \
    -c:v libx264 -crf 18 -preset fast -pix_fmt yuv420p \
    -c:a aac -b:a 192k -ar 48000 -ac 2 \
    -video_track_timescale 90000 \
    -movflags +faststart \
    "norm/$(printf '%03d' $i).mp4"
done

That -video_track_timescale 90000 flag forces every output to use the same MP4 timebase. Without it, FFmpeg picks a timescale based on the input frame rate, and your "identical" files end up with subtly different clock granularity — which the demuxer will notice.

The zero-padded filenames mean ls sorts them correctly, so your list file writes itself:

for f in norm/*.mp4; do echo "file '$f'"; done > list.txt
ffmpeg -f concat -safe 0 -i list.txt -c copy final.mp4

Now the concat is a straight stream copy. Fast, lossless, no drift, because every input was built to be concatenated.

Note the CRF difference: -crf 18 on the intermediate preserves quality for the final pass. Re-encoding twice at -crf 23 would visibly degrade the output.

Handling Clips With No Audio Track

The concat filter will fail outright if any input is missing an audio stream. You have two options.

Option A: add silence to the silent clips first.

ffmpeg -i silent.mp4 \
  -f lavfi -i anullsrc=channel_layout=stereo:sample_rate=48000 \
  -map 0:v -map 1:a -shortest \
  -c:v copy -c:a aac -b:a 192k out.mp4

anullsrc generates an infinite silent audio stream with the parameters you specify. -shortest cuts it at the video length. -c:v copy means the video isn't re-encoded, so this is quick.

Option B: build a video-only concat.

If you don't need audio at all for the final output, the concat filter handles video-only jobs cleanly. The FFmpeGo docs show the pattern:

[0:v][1:v]concat=n=2:v=1:a=0[outv]

The a=0 tells the filter there are no audio streams to consume. Useful for silent exports, GIF-style output, or when you're going to lay a single audio bed over the whole thing afterward anyway.

There's a third path if you're mixing and matching: map audio optionally with a trailing question mark. In a -map argument, -map 0:a? means "include the audio stream if it exists, and don't fail if it doesn't." FFmpeGo's write-up on video processing pitfalls calls this out as one of the most common concat failures, and it's easy to see why — -map 0:a against a .mp4 with no audio track is an instant error.

Common Mistakes That Cause Drift

A quick checklist of the ones I see most often.

  1. Using -map 0:a when a clip might be silent. Use -map 0:a? so the command doesn't die on audio-less inputs.
  2. Re-encoding audio you didn't need to touch. If nothing about the audio is changing, -c:a copy saves time and preserves the original quality. Only re-encode when you're changing sample rate, channel layout, or codec — which, in a concat, you usually are.
  3. Forgetting -ar on the output. Your aformat filter sets the internal rate, but the encoder needs to know what to write. Set both.
  4. Assuming -async 1 is the fix. It's a legacy CLI flag with confusing semantics. The modern replacement is aresample=async=1.
  5. Mixing mono and stereo without normalizing. A mono source mapped into a stereo-expected slot can play out of one speaker or at half volume. aformat with channel_layouts=stereo handles the upmix.
  6. Combining -shortest with a concat output. -shortest truncates everything to the shortest stream. On a concat that usually means your last clip gets cut short. Use it only on individual segments.
  7. Skipping setsar=1. A 1440×1080 anamorphic clip concatenated with a 1920×1080 square-pixel clip produces output where the SAR keeps flip-flopping. Some players handle this; most don't.
  8. Trusting the demuxer when the file counts differ. If one input has two audio streams and another has one, concat will pick something and not tell you what.

FAQ

Can I concatenate without re-encoding at all?

Yes, if — and only if — every input shares the same video codec, profile, resolution, SAR, pixel format, frame rate, and timebase, plus the same audio codec, sample rate, channel layout, and sample format, with zero start-time offsets. That's a narrow window. The normalizing pass described above gets you there in one step, after which every concat is a stream copy.

Why does the drift get worse with each additional clip?

Because most causes are per-join errors that accumulate. An AAC priming delay of ~20 ms per segment compounds to a quarter-second over twelve clips. A start-time offset of 0.02s per file does the same thing. Individually each error is invisible; stacked, they're a train wreck. Rebasing every input to zero with asetpts=PTS-STARTPTS resets the counter to zero at each join, so nothing accumulates.

Do I need the fps filter if all my clips are the same frame rate already?

If they're all genuinely constant frame rate at the same value, you can drop it. Keep it if any source came from a phone, a screen recorder, a browser-based editor, or a video call — those are frequently VFR, and the file will report a nominal frame rate that doesn't reflect what's actually inside.

My audio is ahead of the video instead of behind. What changed?

Probably a non-zero start_time on one of the inputs, or a segment whose container duration doesn't match its actual stream length. The first case is solved by asetpts=PTS-STARTPTS on every input. The second usually means the file was truncated mid-write — re-muxing it with ffmpeg -i broken.mp4 -c copy fixed.mp4 will often repair the container metadata, though if the underlying stream is genuinely short, only aresample=async=1 will paper over it.

When You'd Rather Not Run This Locally

The concat filter approach is CPU-heavy. A 15-clip re-encode with a filter graph that size will pin your laptop's cores for a while, and running it inside an API request handler is a good way to make your app feel broken under load.

That's the niche a serverless FFmpeg API fills. Instead of provisioning a worker, you POST a JSON payload containing your input URLs and your FFmpeg arguments to a single endpoint, and you get back the processed file. FFmpeGo works this way — the same filter graphs described above, running on managed infrastructure, billed by wall-clock compute second rather than by minute-of-output. Their docs cover the request format and a video-only concat example directly.

One design choice worth knowing about: FFmpeGo only charges for jobs that return a 2xx response. A malformed argument list, a dead input URL, an invalid filter graph — none of those cost you anything. For iterating on a concat graph where you're trying three different fps values to see which produces the cleanest output, that's a meaningful difference from platforms that bill for consumed CPU regardless of whether the job succeeded.

If your pipeline already runs on your own infrastructure and you like it that way, the commands in this post work exactly the same locally. Nothing here requires a third-party service.

The Short Version

Drift is accumulation. Kill it by eliminating what accumulates.

If you remember one command from this whole post, make it this: put aformat=sample_fmts=fltp:sample_rates=48000:channel_layouts=stereo,asetpts=PTS-STARTPTS on every audio input, and scale plus pad plus setsar=1 plus fps plus setpts=PTS-STARTPTS on every video input. Then concatenate with the filter. That combination solves the sample rate mismatch, the channel layout mismatch, the start-time offset, the VFR problem, and the aspect ratio mismatch in one pass.

Your next step is concrete: take the ffprobe loop from earlier in this post, run it against your actual source files, and look at what comes back. Ninety percent of the time, the fix will be obvious from the output — a stray 44100, a start_time that isn't zero, or a channels=1 sitting where you expected 2. Once you know which clip is the odd one out, you can either normalize it on ingest or push it through the concat filter and let FFmpeg handle it.

Then measure. Play the output, scrub to the last thirty seconds, and check that the audio still lines up. That's the only test that matters.