Ears for Claude Code

What it is and why it matters

If you make launch videos with code, for example with Remotion, or edit them with Claude Code, there is a gap: Claude writes the video, but it cannot hear it. It puts a hit where it thinks the cut lands, and it does not know if it arrives 80 milliseconds late. And anyone hears a sound that is off time.

earshot gives it ears. It is a free open source project on GitHub (MIT licence) that turns sound into things Claude can read: a sound described in words, what you hear in the video second by second, by how many milliseconds each hit misses the cut, what to fix first. And it builds the missing sounds in code.

It is for people who make their product videos alone, without a sound engineer. But it is very young: it came out on 15 October 2026, it is marked alpha and one person writes it, with Claude as co-author. Try it on a test video first, not on your launch video.

1. What you need

Everything runs on your computer. Before installing it, check that you have:

• Python 3.10 or newer.
• ffmpeg and ffprobe, two free programs that read audio and video.
• About 1.1 GB free for the two open models it downloads on first use.
• No graphics card and no key. There is an extra review with Gemini, but it runs only with your own key and your explicit consent to upload the file.

2. Have Claude Code install it

The install is five commands, but you do not need to type them. Open Claude Code in the folder where you keep your tools and paste this:

Install earshot from https://github.com/Leonxlnx/earshot following the Install section of the README. First check Python 3.10 or newer, ffmpeg and ffprobe: if something is missing, stop and tell me how to install it. Use a .venv virtual environment, the CPU-only build of torch and the mcp extra. At the end run earshot doctor --load and tell me in plain words whether everything is fine.

Claude creates a .venv folder, a virtual environment: everything goes there, without touching the system Python. The first time it downloads the two models and takes a while (to download them right away there is earshot doctor --fetch). Then they stay in the repository folder and it never looks for them online again.

3. Add it as a Claude Code tool

Each terminal command reloads the models: about 20-40 seconds before it answers, and a video needs several calls. That is why you connect it as an MCP server, a program that gives Claude Code new tools: the models load once and stay ready.

In the terminal type this, with the real folder path in place of the one between < >:

claude mcp add earshot --scope user -e EARSHOT_THREADS=3 -- <earshot folder>/.venv/bin/python -m earshot mcp

• --scope user makes it available in all your projects, only for you.
• EARSHOT_THREADS=3 says how many processor cores it may use: so the computer does not freeze while it listens.
• Then in Claude Code type /mcp: earshot must show as connected. If it does not show up on a slow computer, start Claude Code with MCP_TIMEOUT=60000 claude.

4. Let it listen to the video

Three tools do most of the work:

• describe: you play it a sound and it describes it in words, with measurements.
• timeline: you give it the whole video and it writes what you hear, second by second, with the picture cuts and the hit that answers each one. The offset is in milliseconds: negative means the sound arrives before the cut.
• sheet: an image with frames, waveform and events. Each cut has a colour: green if the sound lands within one frame, amber within 100 milliseconds, red if it is missing.

Use earshot to listen to out/video.mp4. Make the timeline and the sheet, open the image and tell me in plain words: what you hear at the main moments, which cuts have no sound, and which hits land before or after the cut, by how many milliseconds. Do not change anything yet.

If the video is made of animation more than hard cuts, have it pass the key moments as cues (the --cues option): a movement inside the same shot does not count as a cut.

5. The scorecard and the fixes

critique compares the soundtrack with a profile measured on seven professional launch films: for each measurement it says whether you are in range, above or below. Then it gives a verdict and the FIX FIRST list, the fixes to make first, with the exact second.

Use earshot to critique the soundtrack of out/video.mp4 and explain the verdict and FIX FIRST in plain words. Then fix the sync of hits with cuts in the video's code, one fix at a time: after each one render again, run timeline and critique again and show me the numbers before and after. Do not touch the music.

But the author chose those seven films and they describe one style only: software and AI launches, animated text, interface demos. A cinematic short will break rules that do not apply to it. If your style is different, earshot profile builds you a profile from the films you admire.

6. The missing sounds, made in code

If a cut stays silent, there is no need to hunt for sounds across a thousand sites. earshot has a bank of more than 250 effects written as code (whooshes, hits, clicks, pops, notes), which render in a few seconds, with no models and no network.

• sfx: takes a sound from the bank and aims it at the picture, with the length and the peak point you choose.
• kit: a set of sounds for the whole video, one per role, from a preset (calm, crisp, minimal or launch, for videos cut to music).
• forge: builds a new sound in code and improves it by listening. To do its best it wants real reference recordings, which you bring.

In the timeline of out/video.mp4 there are cuts with no sound. First look at the sheet to be sure there is not already a soft whoosh. Then, for each one, pick a fitting effect from the earshot bank, render it with sfx so it lands on the cut and add it to the video. Show me the list before you put it in.

On forge the author warns: the search tries to please earshot's models, and the models can be pleased even when the sound does not get better (it is Goodhart's law). Judge a sound made with forge by the measurements, by the spectrogram next to the references and with your own ears.

Common mistakes

• Using it for the first time on the real launch video. It is alpha and one person makes it: a test video first, then the one that matters.
• Trusting the labels inside a busy mix. Times and levels are reliable; the name of each hit under the music is a hint, right about one time in four.
• Taking the scorecard as law. It measures closeness to the style of seven films, not whether your video is good or the music is right for your product: you decide that.

Checklist

• earshot doctor --load says everything is fine
• In Claude Code, /mcp shows earshot connected
• First try done on a test video
• I looked at the sheet with my own eyes
• Every fix checked by running timeline and critique again

Frequently asked questions