Voice infrastructure
What happens between the microphone and the answer
Realtime speech is a latency problem before it is an AI problem. This page is about where the milliseconds go, what the platform handles for you, and what you can still get wrong.
Speech to speech, not a pipeline
The classic voice-bot architecture is three services in a row: speech to text, a language model, text to speech. Every hop adds its own delay, and the model only ever sees a transcript - tone, hesitation and interruption are gone before it gets there.
bitpull runs on a realtime speech-to-speech model instead. Audio goes in, audio comes out, and the turn-taking happens inside the model rather than in glue code between three vendors. That is why a caller can interrupt mid-sentence and be understood, and why the reply arrives in the same language the caller used without anyone selecting one.
- Caller speaksphone or browser mic
- TransportSIP or WebRTC
- Realtime modelspeech in, speech out
- Answersame turn, same language
Transport: WebRTC and SIP
The conversation itself does not run over the REST API. Creating a session returns a wsUrl and a token for a LiveKit room, and the audio flows over WebRTC from there. On the phone side the same agent is reached over SIP - the transport differs, the agent does not.
| Web | Phone | |
|---|---|---|
| Transport | WebRTC over the LiveKit room | SIP to sip.bitpull.ai:5060 |
| Audio quality | Wideband, whatever the device gives | Narrowband - the network decides, not you |
| Permission | Microphone prompt, HTTPS only | None - the caller dialled |
| Failure mode | Blocked mic, no HTTPS, restrictive firewall | SIP status codes, carrier routing |
| Identity | Whatever your app knows | Caller number, which can be withheld |
The latency budget
Conversational speech has a rhythm. Human turn-taking gaps sit in the low hundreds of milliseconds; a gap around a second reads as hesitation, and past two seconds people start talking over the agent. Everything in a voice integration is spending against that budget.
| Segment | Who owns it | What moves it |
|---|---|---|
| Microphone capture and encode | Device and browser | Cheap headsets and aggressive noise suppression both add delay. |
| Network to the media server | The caller's connection | Distance and hop count. Measurable - see the latency test below. |
| End-of-turn detection | The platform | Deciding the caller has finished. Too eager interrupts; too patient feels slow. |
| Model response | The platform | Not yours to tune. This is the largest single block. |
| Tool call | You | The one segment you fully control - and the one that most often ruins a call. |
| Playback | Device | Bluetooth headsets add a surprising amount. |
A lookup against your CRM sits inside the caller's silence. The tool timeout can be set anywhere from 1000 to 60000 ms, but a 3-second database query is three seconds of nothing on a phone call. Cache aggressively, set a tight timeout, and design the prompt so the agent says something while it waits.
Turn taking, barge-in and silence
Three behaviours decide whether a voice agent feels natural, and all three are handled by the platform rather than by your code:
- End-of-turn detection. Knowing that a caller has finished, as opposed to pausing to think. Getting this wrong in either direction is the most common reason a voice agent feels wrong.
- Barge-in. A caller talking over the agent stops it. Without this, long answers become a battle and people hang up.
- Silence. What happens when nobody says anything. This one is partly yours: the prompt decides whether the agent re-engages, waits, or ends the call.
Voices and languages
The shipped voice list is language-independent - around thirty named voices, each with a described character, any of which can speak any supported locale. That is a property of the realtime model rather than a bitpull feature, and it removes the usual "which voices exist for Dutch" problem entirely.
What matters for your integration is that voices and languages are per agent, not global. Always read them from the agent:
const res = await fetch('https://api.bitpull.ai/api/sessions/languages', {
headers: { Authorization: `Bearer ${API_KEY}` }
})
const { languages, defaultLanguage, languageConfigs } = await res.json()
// languageConfigs[n].availableVoices → [{ id, label, gender, tone }, …]
// Cache this. It changes when someone edits the agent, not per request.A session may override the voice or the language, but only within what the agent is configured for. Asking for a language the agent does not have is an error, not a fallback.
Audio quality is a prompt problem too
Narrowband phone audio drops the frequencies that distinguish similar-sounding letters and digits. No model fixes that entirely, so write the prompt to work with it: confirm order numbers and email addresses back to the caller, spell out ambiguous characters, and never let a single misheard digit trigger an irreversible action.