Lesson 1 said every voice agent has five layers, and Audio I/O is the first and last. This lesson unpacks that layer. By the end, you'll understand exactly what happens between the moment a caller dials a number and the moment raw audio packets hit your STT engine — and the reverse path for TTS output.
There are two worlds: the public switched telephone network (PSTN) — copper wires, cell towers, 100-year-old infrastructure — and the IP network where your voice agent lives. Bridging them is the Audio I/O layer's entire job.
Two paths into the IP world: SIP (phone calls) and WebRTC (browser/app calls). They solve the same problem — getting real-time audio into your server — with different tradeoffs. Let's take them one at a time.
SIP (Session Initiation Protocol) is signaling only. It handles the lifecycle of a call — setup, modification, teardown — but never touches the audio itself. Think of it as the handshake before the conversation.
The key insight: SIP and RTP are separate. SIP runs on port 5060 and negotiates the call. Once both sides agree, RTP (Real-time Transport Protocol) opens a direct UDP stream for the audio. The SIP provider (Twilio, Telnyx) acts as a gateway — it speaks SIP to the phone carrier and RTP to your server.
A SIP trunk is a virtual phone line. Instead of a physical copper wire, you get an IP endpoint that can handle multiple simultaneous calls. Twilio gives you a phone number, routes incoming calls to your SIP server, and sends outgoing calls through their carrier network.
You don't handle raw SIP yourself. LiveKit Agents includes a built-in SIP bridge: it speaks SIP on port 5060, terminates the media, and converts the call into a room — the same abstraction it uses for WebRTC. From your agent's perspective, a phone call and a browser call look identical.
# livekit.yaml — enable SIP
sip:
enabled: true
listen_address: "0.0.0.0:5060"
# Each incoming call becomes a room
# Your agent joins it like any other room
await ctx.connect() # same whether call came via SIP or WebRTCThe dispatch rule is what routes specific phone numbers to specific agents — you set this once per trunk and LiveKit handles the rest.
WebRTC (Web Real-Time Communication) is the browser-native way to send audio and video peer-to-peer. It's what powers Google Meet, Discord, and every in-browser calling app.
Unlike SIP, WebRTC bundles signaling and media into one stack. The browser uses:
The browser needs help establishing a direct connection. ICE (Interactive Connectivity Establishment) tries multiple paths: direct connection first, then STUN (a server that tells each peer its public IP), then TURN (a relay server when firewalls block everything).
Direct peer-to-peer WebRTC doesn't scale. You need a server in the middle — a Selective Forwarding Unit (SFU) — that receives every stream and routes them intelligently. LiveKit is that SFU. Your voice agent connects to LiveKit as just another participant in the room, receiving audio from the caller and sending TTS audio back.
This is the most practically important section of this lesson. It explains why you can't just use any STT engine for phone calls.
| Codec | Sample Rate | Bitrate | Used In | STT Impact |
|---|---|---|---|---|
| G.711 μ-law | 8 kHz | 64 kbps | PSTN (landlines, old cell calls) | ⚠️ Narrowband — 2-3× higher WER with Whisper |
| G.711 a-law | 8 kHz | 64 kbps | PSTN (Europe/LatAm) | Same as μ-law |
| G.722 | 16 kHz | 48-64 kbps | HD Voice (modern VoIP) | Decent — wideband, 2× the frequency range |
| Opus | 8-48 kHz | 6-510 kbps | WebRTC, modern VoIP | ✅ Best — full-band option, adaptive |
The critical number: 8 kHz sample rate = 4 kHz maximum frequency. Human speech extends to ~8 kHz, but phone calls literally chop off the top half. The "s" and "f" sounds (sibilants) live in that 4-8 kHz range, which is why phone calls sometimes make it hard to distinguish "six" from "fix."
Let's trace a real phone call through every Audio I/O step:
1. Caller dials +57 312 345 6789
↓ PSTN signaling (SS7, carrier network)
2. Twilio receives the call at their Colombian POP
↓ SIP INVITE sent to your LiveKit server (port 5060)
3. LiveKit SIP bridge: 200 OK → ACK (call established)
↓ Codec negotiation: G.711 a-law selected (64 kbps)
4. RTP audio packets start flowing
↓ 20ms frames, 160 bytes each, UDP
5. LiveKit internal: audio decoded, resampled if needed
↓ Audio frames delivered to your agent session
6. STT engine receives clean audio frames
↓ Transcription starts (interim results flow back)
7. LLM generates response → TTS synthesizes audio
↓ Audio frames stream back through LiveKit
8. LiveKit encodes → RTP → Twilio → PSTN → caller hears response| If you need… | Use… | Because… |
|---|---|---|
| Real phone calls (any phone, any network) | SIP trunk (Twilio) + LiveKit | Only path into the PSTN. Non-negotiable for production. |
| Browser-based calls (web app widget) | WebRTC + LiveKit | Zero phone cost, full-band Opus audio, simple setup. |
| Testing during development | WebRTC (browser) | Open localhost, grant mic, done. No phone numbers needed. |
| Production with call recording | SIP trunk with recording enabled | Twilio recording: $0.0025/min. Worth it for debugging. |
| Multiple countries | Multi-region Twilio or Telnyx | One trunk per country. DNS-based routing per number. |
When setting up a voice agent, the Audio I/O configuration boils down to: