Skip to main content

Textify Voice: Architecture

Companion to SPEC.md (what and why). This file is how: components, boundaries, call flows, data model, APIs, events, infrastructure, security. Stack: Textify (Go) + Voice Service (Java Spring Boot) + LiveKit (self-hosted) + OpenAI Realtime (over SIP) + Africa's Talking. No Python. Anything marked [VERIFY] must be checked against current provider docs or Africa's Talking before coding against it.

1. System overview

                     ┌────────────────────────────────────────┐
  Caller (any TZ     │          Africa's Talking (AT)         │
  network) ◄──PSTN──►│  TZ numbers · telco interconnect · SIP │
                     └───────────────────┬────────────────────┘
                                         │ SIP trunk (5060 + RTP)
                                         ▼
┌───────────────────────── Textify Voice box (dedicated) ─────────────────────────┐
│                                                                                  │
│  ┌─────────────┐  Redis  ┌────────────────┐   ┌───────────────┐                  │
│  │ livekit-sip │◄───────►│ livekit-server │◄─►│ livekit-egress│──► S3 storage    │
│  └──────┬──────┘         └───────┬────────┘   └───────────────┘   (recordings)   │
│         │                        │            ┌───────────────┐                  │
│         │                        │◄──────────►│livekit-ingress│ (hold/consent    │
│         │                        │            └───────────────┘  audio files)    │
│         │                        │ server API + webhooks                         │
│         │                        ▼                                               │
│         │             ┌──────────────────────┐      ┌────────────┐               │
│         │             │   VOICE SERVICE      │◄────►│ PostgreSQL │               │
│         │             │   (Java Spring Boot) │      └────────────┘               │
│         │             │  numbers · routing · │                                   │
│         │             │  calls · AI control ·│                                   │
│         │             │  recordings · usage  │                                   │
│         │             └───┬──────────┬───────┘                                   │
└─────────┼─────────────────┼──────────┼───────────────────────────────────────────┘
          │ SIP (TLS 5061   │ REST +   │ REST (client token)  ▲ signed webhooks
          │  + SRTP)        │ WebSocket│                      │
          ▼                 ▼ (control)▼                      │
   ┌────────────────────────────┐  ┌──────────────────────────┴──┐     ┌─────────────────┐
   │ OpenAI Realtime            │  │ TEXTIFY (Go)                │◄───►│ Agent browser   │
   │ joins the call room as a   │  │ users · tenants · wallet ·  │ app │ (Textify web UI,│
   │ SIP participant; webhooks  │  │ pricing · dashboard ·       │     │ LiveKit JS SDK) │
   │ + control to Voice Service │  │ SMS/WhatsApp follow-ups     │     └────────┬────────┘
   └────────────────────────────┘  └─────────────────────────────┘              │ WebRTC
                                                                                ▼
                                                                         livekit-server

Key idea: every phone call is one LiveKit room. The caller, the AI and human agents are all participants in that room:

  • the caller arrives from AT over the SIP trunk;
  • the AI is OpenAI Realtime, which Spring Boot dials into the room over SIP and controls through OpenAI's call-control REST + WebSocket;
  • human agents join from the browser over WebRTC.

Spring Boot is the only "brain". Nothing else has business logic.

1.1 Responsibilities (hard boundary)

Component Owns Must NOT own
Textify (Go) Users, login, roles, tenants, partners/white-label, wallet, pricing, invoices, dashboard UI, SMS/WhatsApp follow-ups Any telephony, LiveKit or OpenAI logic
Voice Service (Spring Boot) Client apps, accounts (opaque ids), numbers + config, agents (opaque refs), ring groups, call lifecycle, AI call control (OpenAI accept/tools/hangup), recordings, transcripts, summaries, usage events Users, passwords, roles, plans, prices, money
LiveKit (server + SIP + egress + ingress) Real-time media, SIP termination, rooms, recording to storage, playing audio files into rooms Business logic
OpenAI Realtime Listening, thinking and speaking during AI calls (speech-to-speech); transcription + summaries after calls State, routing decisions
Africa's Talking Numbers, PSTN, SIP trunk Anything else

Rule of thumb: if a feature mentions people, plans, roles or money, it belongs in Textify. If it mentions calls, audio, numbers or AI behaviour on a call, it belongs in the Voice Service.

1.2 Why a separate Voice Service

  • Reusable: JikoXpress, Mauzodukani and NexGate can become clients later (each is a client_app with accounts).
  • Failure isolation: a portal outage or deploy doesn't drop calls, and the other way round.
  • Security: the box with SIP ports open holds no user credentials.
  • One language for the call logic: Java (team strength). Textify stays Go.

2. Component details

2.1 Voice Service (Java Spring Boot)

Stack: Java 21, Spring Boot 3.x, PostgreSQL + Flyway, Redis (locks, presence, short-lived call state), @Scheduled jobs (no Kafka at this scale), virtual threads (one OpenAI WebSocket per active AI call is fine).

Libraries

  • LiveKit server SDK for JVM: io.livekit:livekit-server (Kotlin, usable from Java). Room service, SIP service (trunks, dispatch rules, CreateSIPParticipant, TransferSIPParticipant), egress, ingress, AccessToken, WebhookReceiver. [VERIFY] exact class/method names in the current version.
  • OpenAI: plain HTTPS (RestClient) for call accept/reject/refer/hangup, transcription and summaries; Java WebSocket client (java.net.http.WebSocket or Spring WebSocketClient) for live call control.
  • AWS SDK v2 S3 client (S3-compatible storage) for recordings.

Package layout

com.qbitspark.voice
├── api            // client-facing REST (/v1/...), client-token auth
├── hooks          // inbound webhooks: /hooks/livekit, /hooks/openai (signature-verified)
├── livekit        // SDK wrapper, tokens, trunk/dispatch bootstrap, ingress (hold audio), egress
├── ai             // OpenAI call control: accept + session config, WebSocket session handler,
│                  //   tool execution, AI usage capture; post-call transcription + summary
├── calls          // call lifecycle state machine, call timeline
├── routing        // ring groups, agent presence, accept/claim logic
├── recording      // egress start/stop, storage, retention job
├── events         // outbox table + dispatcher (signed webhooks to client apps)
├── accounts       // client apps, accounts, numbers, number config, agents
└── common         // errors, ids, time, config

2.2 AI on calls: OpenAI Realtime over SIP, controlled from Java

Caller ──AT──► LiveKit room "call-…"   (caller = SIP participant)
                  ▲
                  │ (1) Spring Boot: CreateSIPParticipant(outbound trunk "openai",
                  │                  sip_call_to = <OpenAI project SIP URI>,
                  │                  header X-Voice-Call-Id = <callId>)
                  ▼
          OpenAI Realtime (SIP, TLS 5061 + SRTP)
                  │ (2) webhook realtime.call.incoming {call_id, sip_headers} ──► POST /hooks/openai
                  │ (3) Spring Boot: POST /v1/realtime/calls/{call_id}/accept
                  │        {model, voice, instructions, tools, transcription on}
                  │ (4) Spring Boot: WebSocket wss://api.openai.com/v1/realtime?call_id={call_id}
                  │        ← function calls, transcripts, usage
                  │        → tool results, "say this" responses
                  │ (5) Spring Boot: POST /v1/realtime/calls/{call_id}/hangup   (AI leaves; room stays)
  • Correlation: pass our callId as a custom SIP header on CreateSIPParticipant; read it back from data.sip_headers in OpenAI's webhook. [VERIFY] LiveKit supports custom headers on outbound SIP participants and OpenAI forwards them. Treat headers as untrusted: only accept calls whose callId exists in our DB with status AI_DIALING.
  • Session config on accept (built from the number's config):
    • instructions: base receptionist rules (Swahili first, short spoken answers, never invent prices, offer a human when unsure) + business name + ai_instructions + "start by saying the recording notice and greeting: …".
    • voice: chosen after the Swahili test.
    • tools: transfer_to_human, take_message(name, phone, reason), end_call. (Phase 2: account-defined tools proxied to the client app.)
    • Input transcription on (for live transcript + summaries).
  • Tool handling (on the WebSocket, in Java):
    • transfer_to_human → routing rings agents (§3.2) → tool result "ringing" → AI tells the caller to hold → on accept, AI briefs the agent in one sentence → Spring Boot hangs up the OpenAI leg.
    • take_message → save to calls.message_taken → event call.message_taken.
    • end_call → wait for the AI to finish speaking → delete the LiveKit room.
  • AI usage: capture usage from OpenAI response events and the leg duration (participant_joined/left of identity ai) → calls.ai_seconds, calls.ai_usage.
  • OpenAI API variants: OpenAI documents both the Realtime API (/v1/realtime/calls/...) and a newer Live API (/v1/live/sessions/..., webhook live.transport.incoming). Pick one at build time [VERIFY]. The design is the same.
  • Requirements: LiveKit outbound trunk to OpenAI uses TLS (5061) + SRTP; OpenAI requires SRTP, and trunks must support Opus with SDES-SRTP. LiveKit SIP supports TLS/SRTP. Interop [VERIFY] in the PoC.
  • Trade-off: locked to OpenAI's realtime voices and turn detection. If OpenAI's Swahili speech isn't good enough → Appendix A.

2.3 LiveKit (self-hosted, dedicated box)

Service Purpose
livekit-server Rooms, WebRTC for browser agents, server API, webhooks to the Voice Service
livekit-sip Terminates the AT trunk (caller = SIP participant); dials OpenAI over SIP
livekit-egress Audio-only room recording → S3-compatible storage
livekit-ingress Plays pre-recorded audio (consent notice, hold music, "no balance" message) into a room from a URL [VERIFY] URL input with audio-only files
redis Required by server, SIP, egress and ingress

Telephony objects (created once at bootstrap, reused for every call):

  • Inbound trunk "at-inbound": our AT numbers + AT IPs only (numbers, allowed_addresses).
  • Dispatch rule: dispatch_rule_individual, room_prefix: "call-", no agent dispatch (Spring Boot reacts to webhooks instead).
  • Outbound trunk "at-outbound": AT SIP host + credentials, for calls to customers.
  • Outbound trunk "openai": OpenAI SIP host, TLS transport, SRTP on, for dialing the AI into rooms.

SIP participant attributes used: sip.phoneNumber (caller), sip.trunkPhoneNumber (our dialed number), sip.callID. [VERIFY] names, and that they are included in LiveKit webhook participant info.

LiveKit SIP limitations: no SIP REGISTER (AT must send calls to our SIP URI directly); no video over SIP; DTMF, cold transfer (REFER), warm transfer, TLS/SRTP and noise cancellation supported. AT is not on LiveKit's tested-provider list, so the PoC must prove compatibility.

Room naming: the individual dispatch rule puts the caller's number in the room name (PII in logs). We can't control the To header on calls from AT, so this is accepted for Phase 1 with access-controlled logs.

2.4 Browser agent client (inside Textify's web app)

  • LiveKit JS SDK. Room tokens are minted by the Voice Service and fetched through Textify's backend; the browser never holds Voice Service credentials.
  • Lobby room per agent: lobby-{accountId}-{agentRef}, connected (mic off) while the agent is available:
    • Presence: lobby participant_joined/left webhooks = online/offline, automatically.
    • Ringing: the Voice Service sends a data message {type:"ring", callId, caller, numberLabel} into the lobby → incoming-call UI + ringtone.
  • Accept → Textify backend → POST /v1/calls/{id}/accept → first accept wins → returns a call-room token → the browser joins with mic on.
  • Losers get 409 and a {type:"ring_cancel"} data message.

3. Call flows

3.1 Inbound, AI_FIRST (main flow)

Caller     AT        LiveKit (sip/server)         Voice Service (Java)               OpenAI          Textify
  │ dial ─►│               │                             │                             │                │
  │        │── INVITE ────►│ trunk ok → room call-…      │                             │                │
  │        │               │── webhook participant_joined (SIP, to=our number) ───────►│                │
  │        │               │                             │ lookup number → account     │                │
  │        │               │                             │── GET can-call ─────────────┼───────────────►│
  │        │               │                             │◄── {allowed:true} ──────────┼────────────────│
  │        │               │                             │ create call (RINGING), event call.started ──►│
  │        │               │◄── CreateSIPParticipant(openai trunk, X-Voice-Call-Id) ───│                │
  │        │               │── SIP INVITE (TLS/SRTP) ────┼────────────────────────────►│                │
  │        │               │                             │◄── webhook call.incoming ───│                │
  │        │               │                             │── accept {instructions,tools}►               │
  │        │               │                             │── open control WebSocket ──►│                │
  │        │               │                             │ start egress (if recording) │                │
  │◄═══════╪═══ audio ═════╪═══ AI: recording notice + greeting, conversation ═════════│                │
  │        │               │                             │◄── tool call: end_call ─────│                │
  │        │               │◄── delete room ─────────────│                             │                │
  │        │◄── BYE ───────│── webhooks: participant_left, room_finished ─────────────►│                │
  │        │               │                             │ compute usage → call.ended ─┼───────────────►│ deduct wallet

If can-call says no: play the "insufficient balance" audio via ingress → delete the room → call REJECTED_NO_BALANCE (no AI cost). If Textify doesn't answer within 1.5 s: allow (fail-open) and log it.

3.2 AI → human handover (warm transfer)

  1. AI calls tool transfer_to_human (caller asked for a person, is upset, or the AI can't answer).
  2. Voice Service: find the number's ring group → available agents (Redis presence) → send ring data messages to their lobbies (ALL_AT_ONCE or ROUND_ROBIN) → tool result {"status":"ringing"}.
  3. AI tells the caller "nakuunganisha na mhudumu sasa hivi" ("I'm connecting you to a staff member now").
  4. Agent accepts → claim (SET call:{id}:agent NX) → call-room token → agent joins.
  5. Voice Service sends the AI a "brief the agent in one sentence" instruction over the WebSocket → after it speaks → POST …/calls/{call_id}/hangup on OpenAI → AI leg leaves; caller and agent keep talking. AI seconds stop.
  6. Timeout (default 30 s) with no accept → tool result {"status":"no_agent"} → AI takes a message → call.message_taken.

3.3 Inbound, HUMAN_ONLY

  1. Same start as 3.1 (webhook → number lookup → can-call → call record).
  2. No AI dialed. Voice Service starts ingress playing consent + hold audio into the room, and rings agents.
  3. Agent accepts → stop ingress → agent joins.
  4. Timeout → noAnswerAction:
    • AI: dial OpenAI into the room (as in 3.1)
    • VOICEMAIL: play the prompt via ingress, record via egress (Phase 2)
    • HANGUP: delete the room → MISSED
  5. MISSED → call.missed event → Textify sends the SMS/WhatsApp follow-up.

3.4 Inbound, AI_ONLY

Same as 3.1, but transfer_to_human isn't offered; the AI takes messages instead.

3.5 Outbound / click-to-call (agent → customer)

Agent browser → Textify → POST /v1/calls/outbound {accountId, numberId, toE164, agentRef}
Voice Service:
  1. GET can-call (direction=OUTBOUND)
  2. create call + room out-{callId}
  3. return agent token → browser joins (hears ringback)
  4. CreateSIPParticipant(at-outbound, sip_call_to=toE164, sip_number=our number, room, wait_until_answered)
AT rings the customer → answer → customer joins → talk → room_finished → call.ended

3.6 Transfer to an external mobile (fallback, off by default)

  • Preferred: TransferSIPParticipant (SIP REFER) of the caller to tel:+255…. [VERIFY] whether AT honours REFER.
  • Alternative: CreateSIPParticipant dials the mobile into the same room (extra outbound leg, externalTransferSeconds).

3.7 Recording + summary pipeline

call answered ──► start RoomComposite egress (audio_only) → s3://recordings/{accountId}/{callId}.ogg
room_finished ──► egress ends ──► webhook egress_ended ──► recordings row READY
                                   │
                                   ▼ (@Scheduled job picks pending)
                 transcript: AI part already captured live from the OpenAI WebSocket;
                 full call (incl. human agent part) → OpenAI transcription (sw/en)
                                   │
                                   ▼
                 OpenAI chat (fixed prompt, JSON output) ──► summary
                                   │
                                   ▼
                 transcripts row DONE ──► event summary.ready ──► Textify
daily retention job ──► delete objects + rows where delete_after < now()

Summary JSON:

{
  "summary": "Mteja aliuliza bei ya kifurushi cha mwezi na akaomba apigiwe kesho.",
  "intent": "pricing_inquiry",
  "resolved": false,
  "followUps": ["Call back tomorrow morning with monthly package price"],
  "sentiment": "neutral",
  "topic": "sales",
  "language": "sw"
}

(The example summary says: "The customer asked the price of the monthly package and asked to be called back tomorrow.")

3.8 Call state machine

RINGING ──► AI_DIALING ──► ANSWERED ──► COMPLETED
   │                          │
   │── (no balance) ──► REJECTED_NO_BALANCE
   │── (nobody handled / caller hung up) ──► MISSED
   └── (SIP/system error) ──► FAILED      (AI_DIALING/ANSWERED can also → FAILED)

handled_by: NONE | AI | AGENT | AI_THEN_AGENT. If the OpenAI leg fails to connect within 5 s: fall back to ringing agents (HUMAN_ONLY path) and log AI_UNAVAILABLE.

4. Data model (PostgreSQL)

client_apps (
  id uuid pk, name text, token_hash text,
  webhook_url text, webhook_secret text, can_call_url text,
  created_at timestamptz
)

accounts (
  id uuid pk, client_app_id uuid fk, external_ref text,       -- e.g. Textify tenant id
  status text,                                                 -- ACTIVE | SUSPENDED
  default_language text default 'sw',
  recording_retention_days int default 90,
  created_at timestamptz,
  unique (client_app_id, external_ref)
)

phone_numbers (
  id uuid pk, e164 text unique, provider text default 'AFRICAS_TALKING',
  type text,                        -- REGULAR | TOLL_FREE
  status text,                      -- UNASSIGNED | ACTIVE | RELEASED
  account_id uuid fk null,
  label text,                       -- "Sales line"
  answer_mode text,                 -- AI_FIRST | AI_ONLY | HUMAN_ONLY
  greeting text,
  ai_instructions text,             -- business info, FAQs, tone
  ai_voice text null,               -- override default voice
  recording_enabled boolean default true,
  ring_group_id uuid fk null,
  ring_timeout_sec int default 30,
  no_answer_action text,            -- AI | VOICEMAIL | HANGUP
  external_transfer_e164 text null,
  updated_at timestamptz
)

agents (
  id uuid pk, account_id uuid fk, external_ref text,           -- e.g. Textify user id
  display_name text,
  status text,                      -- AVAILABLE | BUSY | OFFLINE (live presence also in Redis)
  last_seen_at timestamptz,
  unique (account_id, external_ref)
)

ring_groups (id uuid pk, account_id uuid fk, name text, strategy text)   -- ALL_AT_ONCE | ROUND_ROBIN
ring_group_members (ring_group_id uuid fk, agent_id uuid fk, position int,
                    primary key (ring_group_id, agent_id))

calls (
  id uuid pk, account_id uuid fk, number_id uuid fk,
  direction text,                   -- INBOUND | OUTBOUND
  from_e164 text, to_e164 text,
  livekit_room text, sip_call_id text,
  openai_call_id text null,
  status text, handled_by text, agent_id uuid null,
  started_at timestamptz, answered_at timestamptz, ended_at timestamptz,
  end_reason text,
  inbound_seconds int default 0, outbound_seconds int default 0,
  trunk_seconds int default 0, ai_seconds int default 0,
  external_transfer_seconds int default 0,
  message_taken jsonb null,         -- {name, phone, reason}
  ai_usage jsonb null               -- model, token/audio counters (cost measurement)
)

call_events (id bigserial pk, call_id uuid fk, type text, at timestamptz, data jsonb)   -- timeline

recordings (
  id uuid pk, call_id uuid fk unique, egress_id text, storage_key text,
  duration_sec int, size_bytes bigint,
  status text,                      -- RECORDING | READY | FAILED | DELETED
  delete_after timestamptz
)

transcripts (
  call_id uuid pk fk, status text,  -- PENDING | DONE | FAILED
  language text, text text, segments jsonb, summary jsonb, model text, created_at timestamptz
)

outbox_events (
  id uuid pk, client_app_id uuid fk, type text, payload jsonb,
  attempts int default 0, next_attempt_at timestamptz, delivered_at timestamptz null, last_error text
)

Redis keys:

  • presence:{accountId}:{agentRef} → AVAILABLE|BUSY (TTL, refreshed by lobby webhooks)
  • call:{callId}:agent → claim lock (SET NX, TTL)
  • call:{callId}:ai → OpenAI call id + WebSocket owner node
  • rr:{ringGroupId} → round-robin pointer

5. APIs

5.1 Client-facing API (Textify → Voice Service)

Auth: Authorization: Bearer <client_app_token>. Every request is scoped to that client app's accounts. JSON. Base https://voice-api.textify.africa/v1 [VERIFY domain].

Method & path Purpose
POST /accounts Create/upsert account {externalRef}
PATCH /accounts/{id} Status, language, retention days
GET /numbers?status=UNASSIGNED Numbers available to assign
POST /numbers/{id}/assign {accountId, label}
POST /numbers/{id}/release Unassign
PUT /numbers/{id}/config answerMode, greeting, aiInstructions, aiVoice, recordingEnabled, ringGroupId, ringTimeoutSec, noAnswerAction, externalTransferE164
PUT /accounts/{id}/agents/{agentRef} Upsert agent {displayName}
DELETE /accounts/{id}/agents/{agentRef} Remove agent
PUT /accounts/{id}/agents/{agentRef}/status AVAILABLE / BUSY / OFFLINE
POST /accounts/{id}/agents/{agentRef}/lobby-token LiveKit token for the agent's lobby
POST/PUT/DELETE /accounts/{id}/ring-groups[/{gid}] Manage groups + members
POST /calls/{id}/accept {agentRef} → {roomToken, wsUrl} or 409
POST /calls/{id}/hangup End a call
POST /calls/{id}/transfer {toAgentRef} or {toE164}
POST /calls/outbound {accountId, numberId, toE164, agentRef} → {callId, roomToken, wsUrl}
GET /calls?accountId&from&to&status&page Call log
GET /calls/{id} Detail incl. timeline, message, summary
GET /calls/{id}/recording Short-lived signed URL
GET /calls/{id}/transcript Transcript + summary

Admin-only (QBIT ops token): POST /numbers (register a number bought from AT), POST /client-apps.

5.2 Voice Service → Textify: balance check

GET {can_call_url}?accountRef=…&direction=INBOUND|OUTBOUND → {allowed: bool, maxMinutes?: int}. Timeout 1.5 s → allow (fail-open), log it.

5.3 Webhooks: Voice Service → Textify

  • POST {webhook_url}; headers X-Voice-Event-Id, X-Voice-Event-Type, X-Voice-Timestamp, X-Voice-Signature: sha256=HMAC(secret, timestamp + "." + body).
  • Delivered via the outbox with exponential backoff (1 m, 5 m, 30 m, 2 h, … up to 24 h). The receiver must be idempotent on event id.
Event Payload highlights
call.started callId, accountExternalRef, direction, from, to, numberId
call.answered callId, handledBy (AI/AGENT), agentRef?
call.ended callId, status, usage {inboundSeconds, outboundSeconds, trunkSeconds, aiSeconds, externalTransferSeconds, recorded}, endReason
call.missed callId, from, to, numberLabel. Textify sends the SMS/WhatsApp follow-up
call.message_taken callId, name, phone, reason
recording.ready callId, durationSec
summary.ready callId, summary JSON

Example call.ended:

{
  "id": "evt_01J...", "type": "call.ended", "occurredAt": "2026-10-21T12:30:05Z",
  "data": {
    "callId": "3f2c...", "accountExternalRef": "tenant_123",
    "direction": "INBOUND", "from": "+2557XXXXXXXX", "to": "+2557YYYYYYYY",
    "status": "COMPLETED", "handledBy": "AI_THEN_AGENT", "agentRef": "user_88",
    "usage": { "inboundSeconds": 245, "outboundSeconds": 0, "trunkSeconds": 245,
               "aiSeconds": 70, "externalTransferSeconds": 0, "recorded": true },
    "endReason": "CALLER_HANGUP"
  }
}

5.4 Inbound webhooks to the Voice Service

POST /hooks/livekit: verified with LiveKit WebhookReceiver.

LiveKit event Action
participant_joined (SIP, room call-*) Start of an inbound call: number lookup → can-call → create call → dial AI or ring agents
participant_joined (identity ai) AI leg up → ANSWERED, handled_by AI; start egress
participant_left (identity ai) AI seconds end
participant_joined (agent identity) answered_at/handled_by AGENT; start egress if not running
participant_left (SIP caller) Caller hung up → delete room
participant_joined/left (lobby-*) Agent presence
room_finished Close call, compute durations, enqueue call.ended / call.missed
egress_ended Recording READY/FAILED → enqueue transcription

POST /hooks/openai: verify OpenAI webhook signature, dedupe by event id.

OpenAI event Action
realtime.call.incoming (or live.transport.incoming) Match X-Voice-Call-Id header to a call in AI_DIALING → accept with session config → open control WebSocket. Unknown → reject

Durations are always computed from server-side webhook timestamps.

6. Infrastructure

6.1 Box (Phase 0/1: single dedicated VPS, separate from NexGate)

Docker Compose, host networking for media services (SIP/RTP/WebRTC need real ports).

Service Image / runtime Notes
caddy (or nginx) reverse proxy TLS for voice-api.… (Spring Boot, incl. /hooks/*) and wss://voice-rtc.… (LiveKit 7880)
livekit livekit/livekit-server bind 127.0.0.1:7880 behind proxy; keys via LIVEKIT_KEYS env
sip livekit/sip config via SIP_CONFIG_BODY (api_key, api_secret, ws_url, redis, sip_port 5060, rtp_port 10000-20000, use_external_ip); TLS enabled for the OpenAI trunk [VERIFY] SIP TLS config keys
egress livekit/egress audio-only recordings to S3; CPU-heavy
ingress livekit/ingress URL input for hold/consent/no-balance audio
redis redis:7 password, localhost
postgres postgres:16 Voice Service DB
voice-service Spring Boot jar (Java 21) public via proxy (API + webhooks)

Storage: private S3-compatible bucket (e.g. Linode Object Storage or MinIO). Signed URLs only. Audio prompt files (consent, hold, no-balance) in a separate bucket/path, readable by ingress.

Sizing (start, then measure): 4–8 vCPU, 8–16 GB RAM. Choose the region with the lowest RTT to Africa's Talking's SIP servers (measure in the PoC); OpenAI SIP is GeoIP-routed (sip.api.openai.com, EU: sip-eu.api.openai.com).

6.2 Ports & firewall

Port Proto Direction / open to Purpose
22 TCP in, admin IPs SSH
80, 443 TCP in, world TLS issuance, API, webhooks, WSS
7881 TCP in, world WebRTC TCP fallback (browser agents)
50000–60000 UDP in, world WebRTC media (browser agents)
5060 UDP+TCP in, AT IPs only SIP signalling from AT
10000–20000 UDP AT IPs + OpenAI media CIDRs only SIP RTP media
5061 TCP out, to sip.api.openai.com SIP over TLS to OpenAI
7880, 6379, 5432, 8080 TCP localhost only LiveKit API, Redis, Postgres, Spring Boot

OpenAI SIP media CIDRs (from OpenAI's SIP guide, Oct 2026; re-check before go-live): 13.79.45.80/28, 23.98.140.64/28, 40.67.149.176/28, 40.83.204.240/28. WebRTC range and SIP RTP range must not overlap.

6.3 Configuration (env)

LIVEKIT_URL=ws://127.0.0.1:7880
LIVEKIT_PUBLIC_WS_URL=wss://voice-rtc.textify.africa
LIVEKIT_API_KEY / LIVEKIT_API_SECRET
REDIS_URL / REDIS_PASSWORD
DB_URL / DB_USER / DB_PASSWORD
S3_ENDPOINT / S3_BUCKET / S3_ACCESS_KEY / S3_SECRET_KEY
OPENAI_API_KEY
OPENAI_WEBHOOK_SECRET
OPENAI_SIP_URI                   # project SIP URI from OpenAI   [VERIFY format]
OPENAI_REALTIME_MODEL / OPENAI_DEFAULT_VOICE
AT_SIP_HOST / AT_SIP_USERNAME / AT_SIP_PASSWORD / AT_SIP_IPS
AUDIO_CONSENT_URL / AUDIO_HOLD_URL / AUDIO_NO_BALANCE_URL

Client app tokens, webhook secrets and can-call URLs live in client_apps (tokens hashed).

6.4 Bootstrap (one-time, idempotent on startup)

  1. Inbound trunk at-inbound (numbers + AT IPs).
  2. Dispatch rule (individual, call- prefix, no agent dispatch).
  3. Outbound trunk at-outbound (AT host + credentials).
  4. Outbound trunk openai (OpenAI SIP host, TLS, SRTP).
  5. Configure the OpenAI project webhook → https://voice-api…/hooks/openai.

Store the ids. Never create trunks/rules per call.

7. Security

  • SIP exposed only to AT IPs (inbound) and OpenAI media CIDRs (firewall and LiveKit allowed_addresses).
  • OpenAI webhook signature verification + only accept OpenAI calls matching a call in AI_DIALING.
  • Client app → Voice Service: bearer token (hashed at rest), scoped to its accounts.
  • Voice Service → client app: HMAC-signed webhooks with timestamp (reject > 5 min old).
  • LiveKit webhooks: signature verified.
  • Browser gets only short-lived, room-scoped LiveKit tokens (~10 min to join).
  • OpenAI API key and SIP credentials only on the server.
  • Recordings: private bucket, signed URLs (~5 min), retention job, consent notice.
  • Secrets in env only. Keep the CrowdSec/fail2ban practice from Textify's infra.

8. Observability

  • Structured JSON logs with callId, room, accountId, openaiCallId on every line.
  • Metrics: active calls, active AI sessions, calls/min, answer rate, missed rate, AI handover rate, AI dial-in time (CreateSIPParticipant → accept), AI first-response latency, egress failures, webhook backlog, can-call timeouts.
  • Alerts: SIP service down, OpenAI dial-in failures > N%, zero inbound calls for X minutes in business hours, outbox backlog > N, egress failure rate > N%.

9. Testing strategy

  • PoC checklist
    1. AT number → trunk → room created → participant_joined webhook received.
    2. Spring Boot dials OpenAI into the room → call.incoming webhook → accept → AI greets in Swahili → caller hears it.
    3. Tool call over the WebSocket (end_call) works → room deleted → durations correct.
    4. Handover: agent in browser joins, AI leg hangs up, caller stays.
  • Swahili quality test: 10–20 real callers (Swahili, English, mixed). Score understanding and voice. If the voice is weak → Appendix A.
  • Cost: log ai_usage per call; compute TZS per AI minute.
  • Unit: routing (ring strategies, claim race), state machine, outbox retry, signature verification, OpenAI tool dispatch.
  • Integration: LiveKit dev server + SIPp/softphone from an allowed IP; OpenAI in a test project; Textify webhook mock.

10. Repository layout (suggested)

textify-voice/
├── docs/                 SPEC.md, ARCHITECTURE.md
├── voice-service/        Spring Boot (Java 21)
├── deploy/               docker-compose.yaml, livekit.yaml, sip, egress.yaml, ingress.yaml, Caddyfile, firewall.sh, .env.example
├── telephony/            trunk + dispatch-rule JSON templates
└── audio/                consent / hold / no-balance prompt files (sw + en)

11. Open items to verify before coding

  1. AT trunk behaviour (SPEC §11): external SIP URI, outbound auth, IPs, billing, REFER, codecs.
  2. LiveKit SIP → OpenAI SIP interop (TLS + SRTP, Opus/G.711), custom header pass-through, OpenAI project SIP URI format.
  3. OpenAI Realtime API vs Live API choice; webhook event names; usage fields.
  4. LiveKit JVM SDK method names (SIP participant create/transfer, egress, ingress, tokens).
  5. LiveKit SIP participant attributes present in webhook payloads.
  6. Ingress URL input for short audio prompts (latency to start playing).
  7. OpenAI Swahili quality + per-minute cost; choose a voice.
  8. Egress CPU usage for audio-only recordings on the chosen box.

Appendix A: Fallback if OpenAI SIP fails the PoC (Python worker)

Use only if (a) LiveKit ⇄ OpenAI SIP can't interoperate, or (b) OpenAI's Swahili voice isn't good enough and we need a different TTS provider.

  • Add a small Python service using LiveKit Agents (livekit-agents[openai]), dispatched into each call room by the dispatch rule (room_config.agents).
  • It runs the voice loop (OpenAI Realtime, or STT + LLM + another Swahili TTS) and exposes the same three tools, which call the Voice Service over a localhost-only internal API (/internal/calls/start | handoff | message | end | ai-usage).
  • No database, no business logic; about 150–300 lines. Spring Boot stays the brain; data model, client APIs and events above are unchanged.

No comments to display

Back to top