Overview
The voice plugin gives your agent two things: hearing (speech to text, so it understands a voice note) and speaking (text to speech, so it can reply with one).
- Hearing runs on your box by default, with whisper, kept warm so a note is heard in seconds. The audio never leaves the box.
- Speaking uses Microsoft’s online voices by default, so the reply text goes to Microsoft. It is free.
If you want more accuracy, you can move both halves to OpenRouter. Then the audio and the reply text leave the box. Nothing changes until you choose it.
Install
5dive plugin add 5dive-ai/5dive-voice # the plugin
sudo 5dive voice setup # the engine on your box (one time)
5dive voice # is the engine installed and warm?voice setup installs ffmpeg, the whisper hearing service and the speaking client on your box, and tells your agent it can hear and speak. You run it yourself: installing a plugin never runs a setup command for you.
Use the full name 5dive-ai/5dive-voice, not a bare voice: on a box that knows more than one plugin list, the bare name is ambiguous.
Where it hears and speaks
| local (default) | openrouter | |
|---|---|---|
| Hearing | Whisper on your box. The audio never leaves it. | OpenRouter. The audio leaves the box. |
| Speaking | Microsoft's online voices. The reply text goes to Microsoft. | OpenRouter. The reply text goes to OpenRouter. |
| Accuracy | Whisper base by default. small is more accurate and slower. | whisper-large-v3-turbo by default, 99+ languages. |
| Cost | Free. | About $0.00006 to hear a 20-second note, and $0.0045 to speak a 300-character reply. |
The cost is small either way. What you are choosing is whether your voice notes go to a vendor.
Switch backends
5dive voice backend # which one is in force
sudo 5dive voice backend openrouter # hearing and speaking on OpenRouter
sudo 5dive voice backend local # back to the defaultAnyone on the box can read the setting. Changing it needs root, because it decides whether the box’s audio leaves it.
The OpenRouter key
OpenRouter needs a key, kept where the box keeps its other provider keys:
sudo 5dive-write-connector openrouter.env <<< "OPENROUTER_API_KEY=sk-or-..."Without a key, switching to openrouter is refused. If the key is removed later, the box falls back to hearing locally with a warning, so a voice note is never dropped. A box set to local never reaches the network, even with a key on disk.
Settings
See every setting with 5dive voice config, and change one with sudo 5dive voice config set <key> <value>.
Hearing on your box
These apply when hearing runs locally.
| Setting | Default | What it does |
|---|---|---|
| whisper_model | auto (base) | base is the fast model. small is more accurate, above all on mixed-language notes, and about 2-3x slower. small needs about 3 GB of memory. |
| stt_language | auto | auto detects the language of every note. A code such as en or ru pins it, so a short note is never heard as the wrong language. |
| stt_fast | 0 | 1 makes one reading instead of five: faster, slightly less accurate on unclear speech. |
With both stt_language and stt_fast set, a 20-second note went from about 5.2 to about 3.4 seconds.
OpenRouter models
These apply only when the backend is openrouter.
| Setting | Default | What it does |
|---|---|---|
| stt_model | openai/whisper-large-v3-turbo | The model that hears. 99+ languages. meta/muse-voice-transcribe-1.0 is more accurate in English, but pick it only if your audio is English. |
| tts_model | microsoft/mai-voice-2-flash | The model that speaks. |
| tts_voice | alloy | A voice name the speaking model offers. |
Each agent's voice
An agent made from an OpenAgent persona speaks in the voice its persona names. On openrouter it also takes the persona’s speaking style.
On local, every agent has its own voice. An agent without a persona voice gets one that no other agent on the box holds, as long as there are voices to go round, and a reply in Russian or Ukrainian is spoken by a voice for that language. To give every agent the same voice, uncomment the edge_voice= line in the voice config file.
Turning voice replies off
sudo 5dive plugin disable voice # agents answer voice notes in text
5dive plugin enable voice # voice replies back onHearing keeps working while replies are off, because the agent still has to understand the note it answers.
What it does not do
No live, real-time conversation: a note is heard after it is sent, and a reply is spoken as a note. No telling speakers apart in one recording.
Something missing or wrong? Open an issue on GitHub.