Clone your own voice locally for voiceovers

Tools: Chatterbox (Resemble AI), Python
Price: Free (MIT licence)
Time: 30 min the first time
Level: Terminal, intermediate

You shot a video presenting your offer. A week later the price changes, and you have to re-record everything for one sentence: find the mic, the quiet room, the same tone. With a speech model that clones your voice, you type the new sentence and the AI says it in your voice. This tutorial uses Chatterbox, from Resemble AI: the code and the model weights are published under the MIT licence, and the multilingual version covers English, French, Spanish, German and about twenty other languages.

Consent first. Only clone your own voice. Cloning the voice of a client, a colleague or a public figure without their written consent exposes you to legal action (a voice is personal data and part of someone's identity) and can be used for scams. The EU AI Act (Article 50) also requires disclosure when generated audio realistically imitates a real person: add "AI-generated voice" under your posts. The MIT licence allows commercial use of the model, not of someone else's voice.

Who it is for

For a freelancer or a small team publishing short videos, tutorials or product demos: fixing a sentence, adapting a video, making a shorter cut. It is not meant to replace a real recording session for an important script: always listen to the result before publishing.

Install Chatterbox

Create an isolated Python environmentThe project was developed and tested with Python 3.11. An isolated environment keeps its dependencies away from the rest of your Mac.
Terminal
```
brew install [email protected] ffmpeg
python3.11 -m venv ~/chatterbox
source ~/chatterbox/bin/activate
```
Install the packageThe model downloads from Hugging Face on first run, so allow a few minutes.
Terminal
```
pip install chatterbox-tts
```
Record your reference voiceAbout ten seconds, somewhere quiet, in the tone of your videos (QuickTime is enough). Speak in the language of your scripts: according to the project, a reference in another language than the text adds an accent. Then convert it to WAV.
Terminal
```
ffmpeg -i my-voice.m4a my-voice.wav
```

Generate a voiceover

Save this script as voiceover.py, in the same folder as my-voice.wav. language_id="en" sets the language ("fr" for French); "mps" uses the chip in recent Macs.

voiceover.py

import torch
import torchaudio as ta
from chatterbox.mtl_tts import ChatterboxMultilingualTTS

device = "mps" if torch.backends.mps.is_available() else "cpu"
model = ChatterboxMultilingualTTS.from_pretrained(device=device, t3_model="v3")

text = "Hi, I'm Antonin. Today I'll show you how to automate your quote follow-ups."
wav = model.generate(text, language_id="en", audio_prompt_path="my-voice.wav")
ta.save("voiceover.wav", wav, model.sr)

Terminal

python voiceover.py

If loading fails on a Mac, the repository includes an example_for_mac.py file showing the fix, or set device to "cpu": slower, but reliable.

The full example: Claude writes, Chatterbox reads

A script written to be heard does not look like a text written to be read. I first have Claude rewrite the text, then paste it into voiceover.py:

Prompt to copy

Rewrite this text so it can be read aloud in about 40 seconds. Short sentences, one idea per sentence. Write numbers, prices and acronyms out in full, the way they are pronounced. No brackets, no lists. Keep my direct tone. Here is the text: [paste your text]

Useful settings

Defaults: exaggeration=0.5 and cfg_weight=0.5 suit most texts, according to the project.
Speech too fast: lower cfg_weight towards 0.3.
More expressive tone: raise exaggeration towards 0.7 while keeping cfg_weight low. Add them inside model.generate(…).
Long scripts: generate paragraph by paragraph, then join them; a mistake can be fixed without redoing everything.

Limits

An invisible watermark: every generated file carries a Resemble AI audio watermark, designed to stay detectable after compression. That is a good thing for transparency.
Possible errors: names, foreign words, misread numbers. Listen to everything before publishing.
Speed: it depends on your machine. Try one sentence before starting a long script.
Your files stay with you: generation runs locally, only the model is downloaded.

Security. Installing downloads code and a model from the internet: only use the official resemble-ai/chatterbox repository and the chatterbox-tts package. Keep your my-voice.wav file out of shared folders: it is the key to your voice.

Sources

Want to produce your video content faster, with a pipeline that goes from script to video? That is the kind of automation I set up.

Voiceovers in your own voice, generated on your computer