MMAudio: auto-foley for silent AI clips

Try it in 60 seconds

pip install torch torchvision torchaudio --index-url https://download.pytorch.org/whl/cu118 --upgrade
git clone https://github.com/hkchengrex/MMAudio.git
pip install -e .

Install command copied from the project README. Check the README for the latest version.

What it does

Generates synced sound effects from a silent video, ideal for Seedance clips.

MMAudio Paper | Webpage | Models | Huggingface Demo | Colab Demo | Replicate Demo

Ho Kei Cheng, Masato Ishii, Akio Hayakawa, Takashi Shibuya, Alexander Schwing, Yuki Mitsufuji

Quoted from the README of hkchengrex/MMAudio.

Inside the repo

Step by step

1. Install per README.
2. Feed your silent clip.
3. Mux the audio back with ffmpeg.

Sources & credit

All credit to the original authors. We rewrite in our own words and link back. Something wrong? Report it.