Textify Voice: Architecture
Companion to
SPEC.md(what and why). This file is how: components, boundaries, call flows, data model, APIs, events, infrastructure, security. Stack: Textify (Go) + Voice Service (Java Spring Boot) + LiveKit (self-hosted) + OpenAI Realtime (over SIP) + Africa's Talking. No Python. Anything marked [VERIFY] must be checked against current provider docs or Africa's Talking before coding against it.
1. System overview
┌────────────────────────────────────────┐
Caller (any TZ │ Africa's Talking (AT) │
network) ◄──PSTN──►│ TZ numbers · telco interconnect · SIP │
└───────────────────┬────────────────────┘
│ SIP trunk (5060 + RTP)
▼
┌───────────────────────── Textify Voice box (dedicated) ─────────────────────────┐
│ │
│ ┌─────────────┐ Redis ┌────────────────┐ ┌───────────────┐ │
│ │ livekit-sip │◄───────►│ livekit-server │◄─►│ livekit-egress│──► S3 storage │
│ └──────┬──────┘ └───────┬────────┘ └───────────────┘ (recordings) │
│ │ │ ┌───────────────┐ │
│ │ │◄──────────►│livekit-ingress│ (hold/consent │
│ │ │ └───────────────┘ audio files) │
│ │ │ server API + webhooks │
│ │ ▼ │
│ │ ┌──────────────────────┐ ┌────────────┐ │
│ │ │ VOICE SERVICE │◄────►│ PostgreSQL │ │
│ │ │ (Java Spring Boot) │ └────────────┘ │
│ │ │ numbers · routing · │ │
│ │ │ calls · AI control ·│ │
│ │ │ recordings · usage │ │
│ │ └───┬──────────┬───────┘ │
└─────────┼─────────────────┼──────────┼───────────────────────────────────────────┘
│ SIP (TLS 5061 │ REST + │ REST (client token) ▲ signed webhooks
│ + SRTP) │ WebSocket│ │
▼ ▼ (control)▼ │
┌────────────────────────────┐ ┌──────────────────────────┴──┐ ┌─────────────────┐
│ OpenAI Realtime │ │ TEXTIFY (Go) │◄───►│ Agent browser │
│ joins the call room as a │ │ users · tenants · wallet · │ app │ (Textify web UI,│
│ SIP participant; webhooks │ │ pricing · dashboard · │ │ LiveKit JS SDK) │
│ + control to Voice Service │ │ SMS/WhatsApp follow-ups │ └────────┬────────┘
└────────────────────────────┘ └─────────────────────────────┘ │ WebRTC
▼
livekit-server
Key idea: every phone call is one LiveKit room. The caller, the AI and human agents are all participants in that room:
- the caller arrives from AT over the SIP trunk;
- the AI is OpenAI Realtime, which Spring Boot dials into the room over SIP and controls through OpenAI's call-control REST + WebSocket;
- human agents join from the browser over WebRTC.
Spring Boot is the only "brain". Nothing else has business logic.
1.1 Responsibilities (hard boundary)
| Component | Owns | Must NOT own |
|---|---|---|
| Textify (Go) | Users, login, roles, tenants, partners/white-label, wallet, pricing, invoices, dashboard UI, SMS/WhatsApp follow-ups | Any telephony, LiveKit or OpenAI logic |
| Voice Service (Spring Boot) | Client apps, accounts (opaque ids), numbers + config, agents (opaque refs), ring groups, call lifecycle, AI call control (OpenAI accept/tools/hangup), recordings, transcripts, summaries, usage events | Users, passwords, roles, plans, prices, money |
| LiveKit (server + SIP + egress + ingress) | Real-time media, SIP termination, rooms, recording to storage, playing audio files into rooms | Business logic |
| OpenAI Realtime | Listening, thinking and speaking during AI calls (speech-to-speech); transcription + summaries after calls | State, routing decisions |
| Africa's Talking | Numbers, PSTN, SIP trunk | Anything else |
Rule of thumb: if a feature mentions people, plans, roles or money, it belongs in Textify. If it mentions calls, audio, numbers or AI behaviour on a call, it belongs in the Voice Service.
1.2 Why a separate Voice Service
- Reusable: JikoXpress, Mauzodukani and NexGate can become clients later (each is a
client_appwith accounts). - Failure isolation: a portal outage or deploy doesn't drop calls, and the other way round.
- Security: the box with SIP ports open holds no user credentials.
- One language for the call logic: Java (team strength). Textify stays Go.
2. Component details
2.1 Voice Service (Java Spring Boot)
Stack: Java 21, Spring Boot 3.x, PostgreSQL + Flyway, Redis (locks, presence, short-lived call state), @Scheduled jobs (no Kafka at this scale), virtual threads (one OpenAI WebSocket per active AI call is fine).
Libraries
- LiveKit server SDK for JVM:
io.livekit:livekit-server(Kotlin, usable from Java). Room service, SIP service (trunks, dispatch rules,CreateSIPParticipant,TransferSIPParticipant), egress, ingress,AccessToken,WebhookReceiver. [VERIFY] exact class/method names in the current version. - OpenAI: plain HTTPS (
RestClient) for call accept/reject/refer/hangup, transcription and summaries; Java WebSocket client (java.net.http.WebSocketor SpringWebSocketClient) for live call control. - AWS SDK v2 S3 client (S3-compatible storage) for recordings.
Package layout
com.qbitspark.voice
├── api // client-facing REST (/v1/...), client-token auth
├── hooks // inbound webhooks: /hooks/livekit, /hooks/openai (signature-verified)
├── livekit // SDK wrapper, tokens, trunk/dispatch bootstrap, ingress (hold audio), egress
├── ai // OpenAI call control: accept + session config, WebSocket session handler,
│ // tool execution, AI usage capture; post-call transcription + summary
├── calls // call lifecycle state machine, call timeline
├── routing // ring groups, agent presence, accept/claim logic
├── recording // egress start/stop, storage, retention job
├── events // outbox table + dispatcher (signed webhooks to client apps)
├── accounts // client apps, accounts, numbers, number config, agents
└── common // errors, ids, time, config
2.2 AI on calls: OpenAI Realtime over SIP, controlled from Java
Caller ──AT──► LiveKit room "call-…" (caller = SIP participant)
▲
│ (1) Spring Boot: CreateSIPParticipant(outbound trunk "openai",
│ sip_call_to = <OpenAI project SIP URI>,
│ header X-Voice-Call-Id = <callId>)
▼
OpenAI Realtime (SIP, TLS 5061 + SRTP)
│ (2) webhook realtime.call.incoming {call_id, sip_headers} ──► POST /hooks/openai
│ (3) Spring Boot: POST /v1/realtime/calls/{call_id}/accept
│ {model, voice, instructions, tools, transcription on}
│ (4) Spring Boot: WebSocket wss://api.openai.com/v1/realtime?call_id={call_id}
│ ← function calls, transcripts, usage
│ → tool results, "say this" responses
│ (5) Spring Boot: POST /v1/realtime/calls/{call_id}/hangup (AI leaves; room stays)
- Correlation: pass our
callIdas a custom SIP header onCreateSIPParticipant; read it back fromdata.sip_headersin OpenAI's webhook. [VERIFY] LiveKit supports custom headers on outbound SIP participants and OpenAI forwards them. Treat headers as untrusted: only accept calls whosecallIdexists in our DB with statusAI_DIALING. - Session config on accept (built from the number's config):
instructions: base receptionist rules (Swahili first, short spoken answers, never invent prices, offer a human when unsure) + business name +ai_instructions+ "start by saying the recording notice and greeting: …".voice: chosen after the Swahili test.tools:transfer_to_human,take_message(name, phone, reason),end_call. (Phase 2: account-defined tools proxied to the client app.)- Input transcription on (for live transcript + summaries).
- Tool handling (on the WebSocket, in Java):
transfer_to_human→ routing rings agents (§3.2) → tool result"ringing"→ AI tells the caller to hold → on accept, AI briefs the agent in one sentence → Spring Boot hangs up the OpenAI leg.take_message→ save tocalls.message_taken→ eventcall.message_taken.end_call→ wait for the AI to finish speaking → delete the LiveKit room.
- AI usage: capture usage from OpenAI response events and the leg duration (
participant_joined/leftof identityai) →calls.ai_seconds,calls.ai_usage. - OpenAI API variants: OpenAI documents both the Realtime API (
/v1/realtime/calls/...) and a newer Live API (/v1/live/sessions/..., webhooklive.transport.incoming). Pick one at build time [VERIFY]. The design is the same. - Requirements: LiveKit outbound trunk to OpenAI uses TLS (5061) + SRTP; OpenAI requires SRTP, and trunks must support Opus with SDES-SRTP. LiveKit SIP supports TLS/SRTP. Interop [VERIFY] in the PoC.
- Trade-off: locked to OpenAI's realtime voices and turn detection. If OpenAI's Swahili speech isn't good enough → Appendix A.
2.3 LiveKit (self-hosted, dedicated box)
| Service | Purpose |
|---|---|
livekit-server |
Rooms, WebRTC for browser agents, server API, webhooks to the Voice Service |
livekit-sip |
Terminates the AT trunk (caller = SIP participant); dials OpenAI over SIP |
livekit-egress |
Audio-only room recording → S3-compatible storage |
livekit-ingress |
Plays pre-recorded audio (consent notice, hold music, "no balance" message) into a room from a URL [VERIFY] URL input with audio-only files |
redis |
Required by server, SIP, egress and ingress |
Telephony objects (created once at bootstrap, reused for every call):
- Inbound trunk "at-inbound": our AT numbers + AT IPs only (
numbers,allowed_addresses). - Dispatch rule:
dispatch_rule_individual,room_prefix: "call-", no agent dispatch (Spring Boot reacts to webhooks instead). - Outbound trunk "at-outbound": AT SIP host + credentials, for calls to customers.
- Outbound trunk "openai": OpenAI SIP host, TLS transport, SRTP on, for dialing the AI into rooms.
SIP participant attributes used: sip.phoneNumber (caller), sip.trunkPhoneNumber (our dialed number), sip.callID. [VERIFY] names, and that they are included in LiveKit webhook participant info.
LiveKit SIP limitations: no SIP REGISTER (AT must send calls to our SIP URI directly); no video over SIP; DTMF, cold transfer (REFER), warm transfer, TLS/SRTP and noise cancellation supported. AT is not on LiveKit's tested-provider list, so the PoC must prove compatibility.
Room naming: the individual dispatch rule puts the caller's number in the room name (PII in logs). We can't control the To header on calls from AT, so this is accepted for Phase 1 with access-controlled logs.
2.4 Browser agent client (inside Textify's web app)
- LiveKit JS SDK. Room tokens are minted by the Voice Service and fetched through Textify's backend; the browser never holds Voice Service credentials.
- Lobby room per agent:
lobby-{accountId}-{agentRef}, connected (mic off) while the agent is available:- Presence: lobby
participant_joined/leftwebhooks = online/offline, automatically. - Ringing: the Voice Service sends a data message
{type:"ring", callId, caller, numberLabel}into the lobby → incoming-call UI + ringtone.
- Presence: lobby
- Accept → Textify backend →
POST /v1/calls/{id}/accept→ first accept wins → returns a call-room token → the browser joins with mic on. - Losers get
409and a{type:"ring_cancel"}data message.
3. Call flows
3.1 Inbound, AI_FIRST (main flow)
Caller AT LiveKit (sip/server) Voice Service (Java) OpenAI Textify
│ dial ─►│ │ │ │ │
│ │── INVITE ────►│ trunk ok → room call-… │ │ │
│ │ │── webhook participant_joined (SIP, to=our number) ───────►│ │
│ │ │ │ lookup number → account │ │
│ │ │ │── GET can-call ─────────────┼───────────────►│
│ │ │ │◄── {allowed:true} ──────────┼────────────────│
│ │ │ │ create call (RINGING), event call.started ──►│
│ │ │◄── CreateSIPParticipant(openai trunk, X-Voice-Call-Id) ───│ │
│ │ │── SIP INVITE (TLS/SRTP) ────┼────────────────────────────►│ │
│ │ │ │◄── webhook call.incoming ───│ │
│ │ │ │── accept {instructions,tools}► │
│ │ │ │── open control WebSocket ──►│ │
│ │ │ │ start egress (if recording) │ │
│◄═══════╪═══ audio ═════╪═══ AI: recording notice + greeting, conversation ═════════│ │
│ │ │ │◄── tool call: end_call ─────│ │
│ │ │◄── delete room ─────────────│ │ │
│ │◄── BYE ───────│── webhooks: participant_left, room_finished ─────────────►│ │
│ │ │ │ compute usage → call.ended ─┼───────────────►│ deduct wallet
If can-call says no: play the "insufficient balance" audio via ingress → delete the room → call REJECTED_NO_BALANCE (no AI cost).
If Textify doesn't answer within 1.5 s: allow (fail-open) and log it.
3.2 AI → human handover (warm transfer)
- AI calls tool
transfer_to_human(caller asked for a person, is upset, or the AI can't answer). - Voice Service: find the number's ring group → available agents (Redis presence) → send
ringdata messages to their lobbies (ALL_AT_ONCE or ROUND_ROBIN) → tool result{"status":"ringing"}. - AI tells the caller "nakuunganisha na mhudumu sasa hivi" ("I'm connecting you to a staff member now").
- Agent accepts → claim (
SET call:{id}:agent NX) → call-room token → agent joins. - Voice Service sends the AI a "brief the agent in one sentence" instruction over the WebSocket → after it speaks →
POST …/calls/{call_id}/hangupon OpenAI → AI leg leaves; caller and agent keep talking. AI seconds stop. - Timeout (default 30 s) with no accept → tool result
{"status":"no_agent"}→ AI takes a message →call.message_taken.
3.3 Inbound, HUMAN_ONLY
- Same start as 3.1 (webhook → number lookup → can-call → call record).
- No AI dialed. Voice Service starts ingress playing consent + hold audio into the room, and rings agents.
- Agent accepts → stop ingress → agent joins.
- Timeout →
noAnswerAction:AI: dial OpenAI into the room (as in 3.1)VOICEMAIL: play the prompt via ingress, record via egress (Phase 2)HANGUP: delete the room →MISSED
MISSED→call.missedevent → Textify sends the SMS/WhatsApp follow-up.
3.4 Inbound, AI_ONLY
Same as 3.1, but transfer_to_human isn't offered; the AI takes messages instead.
3.5 Outbound / click-to-call (agent → customer)
Agent browser → Textify → POST /v1/calls/outbound {accountId, numberId, toE164, agentRef}
Voice Service:
1. GET can-call (direction=OUTBOUND)
2. create call + room out-{callId}
3. return agent token → browser joins (hears ringback)
4. CreateSIPParticipant(at-outbound, sip_call_to=toE164, sip_number=our number, room, wait_until_answered)
AT rings the customer → answer → customer joins → talk → room_finished → call.ended
3.6 Transfer to an external mobile (fallback, off by default)
- Preferred:
TransferSIPParticipant(SIP REFER) of the caller totel:+255…. [VERIFY] whether AT honours REFER. - Alternative:
CreateSIPParticipantdials the mobile into the same room (extra outbound leg,externalTransferSeconds).
3.7 Recording + summary pipeline
call answered ──► start RoomComposite egress (audio_only) → s3://recordings/{accountId}/{callId}.ogg
room_finished ──► egress ends ──► webhook egress_ended ──► recordings row READY
│
▼ (@Scheduled job picks pending)
transcript: AI part already captured live from the OpenAI WebSocket;
full call (incl. human agent part) → OpenAI transcription (sw/en)
│
▼
OpenAI chat (fixed prompt, JSON output) ──► summary
│
▼
transcripts row DONE ──► event summary.ready ──► Textify
daily retention job ──► delete objects + rows where delete_after < now()
Summary JSON:
{
"summary": "Mteja aliuliza bei ya kifurushi cha mwezi na akaomba apigiwe kesho.",
"intent": "pricing_inquiry",
"resolved": false,
"followUps": ["Call back tomorrow morning with monthly package price"],
"sentiment": "neutral",
"topic": "sales",
"language": "sw"
}
(The example summary says: "The customer asked the price of the monthly package and asked to be called back tomorrow.")
3.8 Call state machine
RINGING ──► AI_DIALING ──► ANSWERED ──► COMPLETED
│ │
│── (no balance) ──► REJECTED_NO_BALANCE
│── (nobody handled / caller hung up) ──► MISSED
└── (SIP/system error) ──► FAILED (AI_DIALING/ANSWERED can also → FAILED)
handled_by: NONE | AI | AGENT | AI_THEN_AGENT.
If the OpenAI leg fails to connect within 5 s: fall back to ringing agents (HUMAN_ONLY path) and log AI_UNAVAILABLE.
4. Data model (PostgreSQL)
client_apps (
id uuid pk, name text, token_hash text,
webhook_url text, webhook_secret text, can_call_url text,
created_at timestamptz
)
accounts (
id uuid pk, client_app_id uuid fk, external_ref text, -- e.g. Textify tenant id
status text, -- ACTIVE | SUSPENDED
default_language text default 'sw',
recording_retention_days int default 90,
created_at timestamptz,
unique (client_app_id, external_ref)
)
phone_numbers (
id uuid pk, e164 text unique, provider text default 'AFRICAS_TALKING',
type text, -- REGULAR | TOLL_FREE
status text, -- UNASSIGNED | ACTIVE | RELEASED
account_id uuid fk null,
label text, -- "Sales line"
answer_mode text, -- AI_FIRST | AI_ONLY | HUMAN_ONLY
greeting text,
ai_instructions text, -- business info, FAQs, tone
ai_voice text null, -- override default voice
recording_enabled boolean default true,
ring_group_id uuid fk null,
ring_timeout_sec int default 30,
no_answer_action text, -- AI | VOICEMAIL | HANGUP
external_transfer_e164 text null,
updated_at timestamptz
)
agents (
id uuid pk, account_id uuid fk, external_ref text, -- e.g. Textify user id
display_name text,
status text, -- AVAILABLE | BUSY | OFFLINE (live presence also in Redis)
last_seen_at timestamptz,
unique (account_id, external_ref)
)
ring_groups (id uuid pk, account_id uuid fk, name text, strategy text) -- ALL_AT_ONCE | ROUND_ROBIN
ring_group_members (ring_group_id uuid fk, agent_id uuid fk, position int,
primary key (ring_group_id, agent_id))
calls (
id uuid pk, account_id uuid fk, number_id uuid fk,
direction text, -- INBOUND | OUTBOUND
from_e164 text, to_e164 text,
livekit_room text, sip_call_id text,
openai_call_id text null,
status text, handled_by text, agent_id uuid null,
started_at timestamptz, answered_at timestamptz, ended_at timestamptz,
end_reason text,
inbound_seconds int default 0, outbound_seconds int default 0,
trunk_seconds int default 0, ai_seconds int default 0,
external_transfer_seconds int default 0,
message_taken jsonb null, -- {name, phone, reason}
ai_usage jsonb null -- model, token/audio counters (cost measurement)
)
call_events (id bigserial pk, call_id uuid fk, type text, at timestamptz, data jsonb) -- timeline
recordings (
id uuid pk, call_id uuid fk unique, egress_id text, storage_key text,
duration_sec int, size_bytes bigint,
status text, -- RECORDING | READY | FAILED | DELETED
delete_after timestamptz
)
transcripts (
call_id uuid pk fk, status text, -- PENDING | DONE | FAILED
language text, text text, segments jsonb, summary jsonb, model text, created_at timestamptz
)
outbox_events (
id uuid pk, client_app_id uuid fk, type text, payload jsonb,
attempts int default 0, next_attempt_at timestamptz, delivered_at timestamptz null, last_error text
)
Redis keys:
presence:{accountId}:{agentRef}→AVAILABLE|BUSY(TTL, refreshed by lobby webhooks)call:{callId}:agent→ claim lock (SET NX, TTL)call:{callId}:ai→ OpenAI call id + WebSocket owner noderr:{ringGroupId}→ round-robin pointer
5. APIs
5.1 Client-facing API (Textify → Voice Service)
| Method & path | Purpose |
|---|---|
POST /accounts |
Create/upsert account {externalRef} |
PATCH /accounts/{id} |
Status, language, retention days |
GET /numbers?status=UNASSIGNED |
Numbers available to assign |
POST /numbers/{id}/assign |
{accountId, label} |
POST /numbers/{id}/release |
Unassign |
PUT /numbers/{id}/config |
answerMode, greeting, aiInstructions, aiVoice, recordingEnabled, ringGroupId, ringTimeoutSec, noAnswerAction, externalTransferE164 |
PUT /accounts/{id}/agents/{agentRef} |
Upsert agent {displayName} |
DELETE /accounts/{id}/agents/{agentRef} |
Remove agent |
PUT /accounts/{id}/agents/{agentRef}/status |
AVAILABLE / BUSY / OFFLINE |
POST /accounts/{id}/agents/{agentRef}/lobby-token |
LiveKit token for the agent's lobby |
POST/PUT/DELETE /accounts/{id}/ring-groups[/{gid}] |
Manage groups + members |
POST /calls/{id}/accept |
{agentRef} → {roomToken, wsUrl} or 409 |
POST /calls/{id}/hangup |
End a call |
POST /calls/{id}/transfer |
{toAgentRef} or {toE164} |
POST /calls/outbound |
{accountId, numberId, toE164, agentRef} → {callId, roomToken, wsUrl} |
GET /calls?accountId&from&to&status&page |
Call log |
GET /calls/{id} |
Detail incl. timeline, message, summary |
GET /calls/{id}/recording |
Short-lived signed URL |
GET /calls/{id}/transcript |
Transcript + summary |
Admin-only (QBIT ops token): POST /numbers (register a number bought from AT), POST /client-apps.
5.2 Voice Service → Textify: balance check
GET {can_call_url}?accountRef=…&direction=INBOUND|OUTBOUND → {allowed: bool, maxMinutes?: int}. Timeout 1.5 s → allow (fail-open), log it.
5.3 Webhooks: Voice Service → Textify
POST {webhook_url}; headersX-Voice-Event-Id,X-Voice-Event-Type,X-Voice-Timestamp,X-Voice-Signature: sha256=HMAC(secret, timestamp + "." + body).- Delivered via the outbox with exponential backoff (1 m, 5 m, 30 m, 2 h, … up to 24 h). The receiver must be idempotent on event id.
| Event | Payload highlights |
|---|---|
call.started |
callId, accountExternalRef, direction, from, to, numberId |
call.answered |
callId, handledBy (AI/AGENT), agentRef? |
call.ended |
callId, status, usage {inboundSeconds, outboundSeconds, trunkSeconds, aiSeconds, externalTransferSeconds, recorded}, endReason |
call.missed |
callId, from, to, numberLabel. Textify sends the SMS/WhatsApp follow-up |
call.message_taken |
callId, name, phone, reason |
recording.ready |
callId, durationSec |
summary.ready |
callId, summary JSON |
Example call.ended:
{
"id": "evt_01J...", "type": "call.ended", "occurredAt": "2026-10-21T12:30:05Z",
"data": {
"callId": "3f2c...", "accountExternalRef": "tenant_123",
"direction": "INBOUND", "from": "+2557XXXXXXXX", "to": "+2557YYYYYYYY",
"status": "COMPLETED", "handledBy": "AI_THEN_AGENT", "agentRef": "user_88",
"usage": { "inboundSeconds": 245, "outboundSeconds": 0, "trunkSeconds": 245,
"aiSeconds": 70, "externalTransferSeconds": 0, "recorded": true },
"endReason": "CALLER_HANGUP"
}
}
5.4 Inbound webhooks to the Voice Service
POST /hooks/livekit: verified with LiveKit WebhookReceiver.
| LiveKit event | Action |
|---|---|
participant_joined (SIP, room call-*) |
Start of an inbound call: number lookup → can-call → create call → dial AI or ring agents |
participant_joined (identity ai) |
AI leg up → ANSWERED, handled_by AI; start egress |
participant_left (identity ai) |
AI seconds end |
participant_joined (agent identity) |
answered_at/handled_by AGENT; start egress if not running |
participant_left (SIP caller) |
Caller hung up → delete room |
participant_joined/left (lobby-*) |
Agent presence |
room_finished |
Close call, compute durations, enqueue call.ended / call.missed |
egress_ended |
Recording READY/FAILED → enqueue transcription |
POST /hooks/openai: verify OpenAI webhook signature, dedupe by event id.
| OpenAI event | Action |
|---|---|
realtime.call.incoming (or live.transport.incoming) |
Match X-Voice-Call-Id header to a call in AI_DIALING → accept with session config → open control WebSocket. Unknown → reject |
Durations are always computed from server-side webhook timestamps.
6. Infrastructure
6.1 Box (Phase 0/1: single dedicated VPS, separate from NexGate)
Docker Compose, host networking for media services (SIP/RTP/WebRTC need real ports).
| Service | Image / runtime | Notes |
|---|---|---|
caddy (or nginx) |
reverse proxy | TLS for voice-api.… (Spring Boot, incl. /hooks/*) and wss://voice-rtc.… (LiveKit 7880) |
livekit |
livekit/livekit-server |
bind 127.0.0.1:7880 behind proxy; keys via LIVEKIT_KEYS env |
sip |
livekit/sip |
config via SIP_CONFIG_BODY (api_key, api_secret, ws_url, redis, sip_port 5060, rtp_port 10000-20000, use_external_ip); TLS enabled for the OpenAI trunk [VERIFY] SIP TLS config keys |
egress |
livekit/egress |
audio-only recordings to S3; CPU-heavy |
ingress |
livekit/ingress |
URL input for hold/consent/no-balance audio |
redis |
redis:7 |
password, localhost |
postgres |
postgres:16 |
Voice Service DB |
voice-service |
Spring Boot jar (Java 21) | public via proxy (API + webhooks) |
Storage: private S3-compatible bucket (e.g. Linode Object Storage or MinIO). Signed URLs only. Audio prompt files (consent, hold, no-balance) in a separate bucket/path, readable by ingress.
Sizing (start, then measure): 4–8 vCPU, 8–16 GB RAM. Choose the region with the lowest RTT to Africa's Talking's SIP servers (measure in the PoC); OpenAI SIP is GeoIP-routed (sip.api.openai.com, EU: sip-eu.api.openai.com).
6.2 Ports & firewall
| Port | Proto | Direction / open to | Purpose |
|---|---|---|---|
| 22 | TCP | in, admin IPs | SSH |
| 80, 443 | TCP | in, world | TLS issuance, API, webhooks, WSS |
| 7881 | TCP | in, world | WebRTC TCP fallback (browser agents) |
| 50000–60000 | UDP | in, world | WebRTC media (browser agents) |
| 5060 | UDP+TCP | in, AT IPs only | SIP signalling from AT |
| 10000–20000 | UDP | AT IPs + OpenAI media CIDRs only | SIP RTP media |
| 5061 | TCP | out, to sip.api.openai.com |
SIP over TLS to OpenAI |
| 7880, 6379, 5432, 8080 | TCP | localhost only | LiveKit API, Redis, Postgres, Spring Boot |
OpenAI SIP media CIDRs (from OpenAI's SIP guide, Oct 2026; re-check before go-live): 13.79.45.80/28, 23.98.140.64/28, 40.67.149.176/28, 40.83.204.240/28.
WebRTC range and SIP RTP range must not overlap.
6.3 Configuration (env)
LIVEKIT_URL=ws://127.0.0.1:7880
LIVEKIT_PUBLIC_WS_URL=wss://voice-rtc.textify.africa
LIVEKIT_API_KEY / LIVEKIT_API_SECRET
REDIS_URL / REDIS_PASSWORD
DB_URL / DB_USER / DB_PASSWORD
S3_ENDPOINT / S3_BUCKET / S3_ACCESS_KEY / S3_SECRET_KEY
OPENAI_API_KEY
OPENAI_WEBHOOK_SECRET
OPENAI_SIP_URI # project SIP URI from OpenAI [VERIFY format]
OPENAI_REALTIME_MODEL / OPENAI_DEFAULT_VOICE
AT_SIP_HOST / AT_SIP_USERNAME / AT_SIP_PASSWORD / AT_SIP_IPS
AUDIO_CONSENT_URL / AUDIO_HOLD_URL / AUDIO_NO_BALANCE_URL
Client app tokens, webhook secrets and can-call URLs live in client_apps (tokens hashed).
6.4 Bootstrap (one-time, idempotent on startup)
- Inbound trunk
at-inbound(numbers + AT IPs). - Dispatch rule (individual,
call-prefix, no agent dispatch). - Outbound trunk
at-outbound(AT host + credentials). - Outbound trunk
openai(OpenAI SIP host, TLS, SRTP). - Configure the OpenAI project webhook →
https://voice-api…/hooks/openai.
Store the ids. Never create trunks/rules per call.
7. Security
- SIP exposed only to AT IPs (inbound) and OpenAI media CIDRs (firewall and LiveKit
allowed_addresses). - OpenAI webhook signature verification + only accept OpenAI calls matching a call in
AI_DIALING. - Client app → Voice Service: bearer token (hashed at rest), scoped to its accounts.
- Voice Service → client app: HMAC-signed webhooks with timestamp (reject > 5 min old).
- LiveKit webhooks: signature verified.
- Browser gets only short-lived, room-scoped LiveKit tokens (~10 min to join).
- OpenAI API key and SIP credentials only on the server.
- Recordings: private bucket, signed URLs (~5 min), retention job, consent notice.
- Secrets in env only. Keep the CrowdSec/fail2ban practice from Textify's infra.
8. Observability
- Structured JSON logs with
callId,room,accountId,openaiCallIdon every line. - Metrics: active calls, active AI sessions, calls/min, answer rate, missed rate, AI handover rate, AI dial-in time (CreateSIPParticipant → accept), AI first-response latency, egress failures, webhook backlog, can-call timeouts.
- Alerts: SIP service down, OpenAI dial-in failures > N%, zero inbound calls for X minutes in business hours, outbox backlog > N, egress failure rate > N%.
9. Testing strategy
- PoC checklist
- AT number → trunk → room created →
participant_joinedwebhook received. - Spring Boot dials OpenAI into the room →
call.incomingwebhook → accept → AI greets in Swahili → caller hears it. - Tool call over the WebSocket (
end_call) works → room deleted → durations correct. - Handover: agent in browser joins, AI leg hangs up, caller stays.
- AT number → trunk → room created →
- Swahili quality test: 10–20 real callers (Swahili, English, mixed). Score understanding and voice. If the voice is weak → Appendix A.
- Cost: log
ai_usageper call; compute TZS per AI minute. - Unit: routing (ring strategies, claim race), state machine, outbox retry, signature verification, OpenAI tool dispatch.
- Integration: LiveKit dev server + SIPp/softphone from an allowed IP; OpenAI in a test project; Textify webhook mock.
10. Repository layout (suggested)
textify-voice/
├── docs/ SPEC.md, ARCHITECTURE.md
├── voice-service/ Spring Boot (Java 21)
├── deploy/ docker-compose.yaml, livekit.yaml, sip, egress.yaml, ingress.yaml, Caddyfile, firewall.sh, .env.example
├── telephony/ trunk + dispatch-rule JSON templates
└── audio/ consent / hold / no-balance prompt files (sw + en)
11. Open items to verify before coding
- AT trunk behaviour (SPEC §11): external SIP URI, outbound auth, IPs, billing, REFER, codecs.
- LiveKit SIP → OpenAI SIP interop (TLS + SRTP, Opus/G.711), custom header pass-through, OpenAI project SIP URI format.
- OpenAI Realtime API vs Live API choice; webhook event names; usage fields.
- LiveKit JVM SDK method names (SIP participant create/transfer, egress, ingress, tokens).
- LiveKit SIP participant attributes present in webhook payloads.
- Ingress URL input for short audio prompts (latency to start playing).
- OpenAI Swahili quality + per-minute cost; choose a voice.
- Egress CPU usage for audio-only recordings on the chosen box.
Appendix A: Fallback if OpenAI SIP fails the PoC (Python worker)
Use only if (a) LiveKit ⇄ OpenAI SIP can't interoperate, or (b) OpenAI's Swahili voice isn't good enough and we need a different TTS provider.
- Add a small Python service using LiveKit Agents (
livekit-agents[openai]), dispatched into each call room by the dispatch rule (room_config.agents). - It runs the voice loop (OpenAI Realtime, or STT + LLM + another Swahili TTS) and exposes the same three tools, which call the Voice Service over a localhost-only internal API (
/internal/calls/start | handoff | message | end | ai-usage). - No database, no business logic; about 150–300 lines. Spring Boot stays the brain; data model, client APIs and events above are unchanged.