# Ringmaster Voice Service

# Ringmaster Voice: Architecture

> Companion to `SPEC.md` (what/why) and `FLOWS.md` (flow & action reference). This file is **how**: components, adapters, call paths, media & voice, knowledge base, data model, APIs, auth, communication, infra, tools. Stack: **Textify (Go) → Voice Engine (Java 21, Spring Boot) → Africa's Talking + LiveKit (self-hosted) + TTS (Basic/Premium) + OpenAI + MinIO**. No Python, no message broker. **\[VERIFY\]** = check against provider docs or Africa's Talking before coding against it.

---

## 1. Overview

```
                          ┌──────────────────────────────────────┐
  Phones (any TZ network) │        Africa's Talking (AT)          │
        ◄──── PSTN ──────►│ numbers · callbacks · XML (Play,      │
                          │ GetDigits, Record, Dial) · call API · │
                          │ recordings · SIP to LiveKit           │
                          └───────┬───────────────────▲──────────┘
               callbacks (POST)   │                   │ XML replies · call API · fetch audio URLs
               Dial sip:{callId}  │                   │
┌──────────────────────── Voice box (dedicated) ──────┼──────────────────────────────┐
│                                 ▼                   │                              │
│  ┌──────────────┐ SIP  ┌──────────────────────────────────────────────┐            │
│  │ livekit-sip  │◄─────┤              VOICE ENGINE (Spring Boot)       │──► Postgres│
│  └──────┬───────┘ (AT) │ accounts · numbers · flows · media · agents · │──► Redis   │
│  ┌──────▼─────────┐    │ calls · campaigns · endpoints · knowledge ·   │──► MinIO   │
│  │ livekit-server │◄──►│ recordings · events                           │            │
│  │ + ingress      │    │ Telephony│Media│Tts(Basic,Premium)│Ai adapters │──► tts-basic│
│  └──────▲─────────┘    └──────▲───────────────┬────────────────────────┘  (CPU model)│
└─────────┼─────────────────────┼───────────────┼────────────────────────────────────┘
          │ WebRTC              │ REST (API key) │ webhooks (HMAC)        ┌──────────────┐
  ┌───────┴────────┐     ┌──────┴───────────────▼──────────┐             │ OpenAI /     │
  │ Agent browser  │◄app►│ TEXTIFY (Go)                     │             │ Azure (TTS,  │
  │ (Textify UI +  │     │ users · tenants · wallet · prices│             │ transcribe,  │
  │ LiveKit JS)    │     │ UI · contacts · tickets · SMS/WA │             │ summary, AI) │
  └────────────────┘     └──────────────────────────────────┘             └──────────────┘
                                         customer systems ◄── webhook actions (signed) ── engine

```

**Rules that hold the design together**

1. **AT is the only road to the phone network**, both directions.
2. **Listening = `Play`, keys = `GetDigits`, voicemail = `Record`, live = `Dial` → LiveKit.** `Say` is not used.
3. **Waiting for an agent happens on AT** (hold loop); the call enters LiveKit only once an agent accepted.
4. **No TTS during live calls.** Text becomes audio files at save time / before dialing.
5. **The engine is blind**: `accountId`, `agentRef`, numbers, calls. Never users, roles, prices, tickets or contacts.

### 1.1 Responsibilities

<table id="bkmrk-component-owns-never"><thead><tr><th>Component</th><th>Owns</th><th>Never owns</th></tr></thead><tbody><tr><td>**Textify (Go)**</td><td>Users/login/roles, tenants, partners, wallet, prices, all UI (answer-mode templates, campaign screens, audio library UI, KB editor), contacts, **tickets**, screen-pop data, SMS/WhatsApp sending</td><td>Telephony / LiveKit / TTS / AI logic</td></tr><tr><td>**Voice Engine**</td><td>Client apps, accounts (opaque), numbers, flows + versions, media + generated voice, agents (opaque refs), departments, presence, waiting orders, calls, campaigns, opt-outs, endpoints, knowledge base, recordings, transcripts, usage events</td><td>People, money, tickets</td></tr><tr><td>**AT**</td><td>PSTN, numbers, XML execution, outbound dialing, recording</td><td>Anything else</td></tr><tr><td>**LiveKit**</td><td>Rooms, WebRTC, SIP from AT, agent-hold audio, Ph 3 AI participant</td><td>Business logic</td></tr><tr><td>**TTS providers**</td><td>Turning text into audio files</td><td>State</td></tr><tr><td>**OpenAI**</td><td>Ph 2 transcribe/summarize; Ph 3 live AI</td><td>State, routing</td></tr></tbody></table>

---

## 2. Inside the engine

**Stack**: Java 21, Spring Boot 3.x, PostgreSQL + Flyway, Redis, `@Scheduled` workers, virtual threads, ffmpeg in the image.

### 2.1 Adapters (plain interfaces; no plugin framework)

```java
interface TelephonyAdapter {                // impl: AfricasTalkingAdapter
  String render(List<XmlAction> actions);                    // → AT XML (Play, GetDigits, Record, Dial, Reject)
  List<OutboundResult> placeCalls(String from, List<String> to);
  InputStream downloadRecording(String url);
}

interface MediaAdapter {                    // impl: LiveKitAdapter
  String sipUriFor(UUID callId);                             // sip:{callId}@sip.voice…
  String roomToken(String room, String identity, boolean publish);
  void sendData(String room, Object msg);                    // ring / ring_cancel / connecting
  void startHoldAudio(String room, String url); void stopHoldAudio(String room);   // agent hold
  void removeParticipant(String room, String identity); void deleteRoom(String room);
  // Ph 3: dialAi(room, callId)
}

interface TtsAdapter {                      // impls: BasicTtsAdapter (self-hosted), PremiumTtsAdapter (OpenAI or Azure)
  byte[] synthesize(String text, String lang, String voice);
  List<VoiceOption> voices();
}

interface AiAdapter {                       // impl: OpenAiAdapter
  Transcript transcribe(InputStream audio, String langHint); // Ph 2
  Summary summarize(Transcript t, String accountContext);    // Ph 2
  // Ph 3: acceptLiveCall(openAiCallId, SessionConfig), control WebSocket, hangup
}

```

A `TtsRouter` picks the adapter by `tier`. Swapping a provider = new class + config.

### 2.2 Packages

```
com.qbitspark.voice
├── api          // client REST /v1/** (API-key auth)
├── hooks        // /hooks/at/{secret}/**, /hooks/livekit, /hooks/openai
├── telephony    // AT adapter, XML builder
├── media        // LiveKit adapter, tokens, lobby/ring, ingress (agent hold)
├── voice        // TtsRouter, Basic/Premium adapters, Swahili text normalizer, generation cache
├── library      // media assets: upload, validate, ffmpeg transcode, signed play URLs
├── flows        // model, validator, versions, NodeExecutor per node type (FLOWS.md)
├── waiting      // hold loop, waiting order per department, segment builder
├── routing      // departments, presence, ring strategies, claim (first accept wins)
├── calls        // lifecycle, state machine, context, trail
├── campaigns    // campaigns, recipients, pre-generation, dispatcher, retries, opt-outs
├── endpoints    // customer webhook endpoints, signer, SSRF guard, sync/async calls
├── knowledge    // KB storage, prompt builder (Ph 3), search tool (later)
├── ai           // OpenAI adapter
├── recording    // download → MinIO, retention
├── jobs         // Postgres job table + workers
├── events       // outbox + signed dispatcher to client apps
├── accounts     // client apps, accounts, numbers, agents
└── common

```

---

## 3. Call paths

### 3.1 Inbound → menu → waiting → browser agent

```
Caller  AT                  Engine                                    LiveKit           Agent browser
 │dial─►│── POST /voice ───►│ number → account → flow(open/closed)    │                    │
 │      │                   │ can-call? (Textify, 1.5 s, fail-open)    │                    │
 │      │◄── Play notice + Play greeting, GetDigits(menu) ───────────│                    │
 │ "2" ►│── POST /step/{id}/menu ────────────────────────────────────►│ connect dep_support │
 │      │                   │ add to waiting order; ring agents ───────────── ring ──────►│ popup
 │      │◄── GetDigits(timeout 1){ Play hold segment } → /wait/{id} ──│                    │
 │ ♪♪   │── POST /wait/{id} (segment ended) ─────────────────────────►│ not accepted yet    │
 │      │◄── next segment (music / comfort message) ──────────────────│◄──── accept ───────│ (via Textify)
 │      │                   │ claim (Redis SET NX) → room token ───────────────────────────►│ joins room
 │      │── POST /wait/{id} ─►│ accepted → Dial sip:{callId}@… record │                    │ "Connecting…"
 │      │══ SIP INVITE ═════════════════════════════════════════════════► room call-{id} ◄══│
 │◄═════╪═══════════════════════ live audio ═══════════════════════════╪════════════════════│
 │hangup│── POST /voice (isActive=0: duration, amount, recordingUrl) ►│ close → events, jobs │

```

- Agents accept while the caller is mid-segment; connection happens at the next callback (segments 10–15 s).
- `maxWaitSec` reached / key pressed → outcome (voicemail etc.) on AT. Unanswered → `call.missed`.
- Agent hold/transfer inside LiveKit: engine starts Ingress hold audio into the room.
- **\[VERIFY\]** wait-loop behaviour (FLOWS §6); fallback = Dial immediately, hold music via Ingress.

### 3.2 Outbound click-to-call

```
Agent → Textify → POST /v1/calls/outbound {accountId, fromNumberId, to, agentRef}
Engine: can-call → create call + room call-{callId} → agent token (agent joins, hears ringback in page)
        → AT call API (from, to) → store AT sessionId
Customer answers → AT callback (Outbound, sessionId) → XML Dial sip:{callId}@… → customer joins → talk
End → final callback → call.ended

```

### 3.3 Campaign

```
Textify → POST /v1/campaigns (+ recipients in chunks) → POST /start
Engine:
  1. validate flow (pinned version), opt-outs, variables vs recipient columns
  2. PRE-GENERATE voice: render each recipient's text, dedupe by hash, TtsRouter → files (jobs)
     cost estimate (premium chars) available before start; failure → campaign PAUSED (TTS_FAILED)
  3. Dispatcher (@Scheduled 1 s): window? not paused? active < maxConcurrent?
       SELECT … campaign_recipients WHERE status='PENDING' AND next_attempt_at<=now()
       ORDER BY id LIMIT :cps FOR UPDATE SKIP LOCKED → AT call API → DIALING
  4. Answer → AT callback → run campaign flow (FLOWS §5.2) with the recipient's variables
  5. Final callback → ANSWERED / NO_ANSWER / BUSY / FAILED; retries per policy
Events: campaign.progress (30 s) · campaign.recipient_results (batches) · campaign.paused · campaign.completed
Textify: SMS fallback to NO_ANSWER; pause on low wallet

```

### 3.4 After the call (jobs)

```
final callback recordingUrl → DOWNLOAD → MinIO voice-recordings → recording.ready
  → (Ph 2) TRANSCRIBE → SUMMARIZE → summary.ready
voicemail → voicemail.received (+ recording)
daily RETENTION → delete expired recordings

```

Recording comes from **AT** (`Dial record="true"`, `Record`). No LiveKit egress needed. **\[VERIFY\]** AT records Dial-to-SIP; fallback egress.

### 3.5 Phase 3 AI

`ai` node → `Dial` into LiveKit → engine dials OpenAI Realtime into the room over SIP → accepts with instructions built from the **knowledge base** + node instructions + tools → controls via WebSocket from Java → `handoff` rings a department (agent joins room, AI leg hung up) → summary job. Appendix A.

### 3.6 Call state machine

```
INBOUND:  RINGING → IN_FLOW → WAITING_AGENT → CONNECTING → ANSWERED → COMPLETED
OUTBOUND: DIALING → IN_FLOW (campaign) / CONNECTING (click-to-call) → ANSWERED → COMPLETED
any → MISSED | NO_ANSWER | BUSY | FAILED | REJECTED_NO_BALANCE
handled_by: FLOW_ONLY | VOICEMAIL | AGENT | AI | AI_THEN_AGENT

```

---

## 4. Browser agents

- **No browser extension or app install needed.** The agent UI is a normal page in Textify's web app using the LiveKit JS SDK; browsers have WebRTC built in (Chrome, Edge, Firefox, Safari). 
    - Requires: HTTPS, microphone permission, the Textify tab open (can be in the background).
    - Browsers block sound until the user interacts with the page, so agents click **"Start shift / Go available"** once; that click unlocks the ringtone.
    - Ring alerts when the tab is in the background: Web Notifications API + ringtone; optional **PWA** (installable web app) for a desktop-app feel. A Chrome extension or native/mobile app is **not** required; consider only later (e.g. ringing with the browser closed).
- **Lobby room** per agent `lobby-{accountId}-{agentRef}` (mic off) while available: 
    - presence = lobby join/leave webhooks + explicit status;
    - ringing = `sendData(lobby, {type:"ring", callId, from, numberLabel, department})`.
- Accept → browser → **Textify backend** → `POST /v1/calls/{id}/accept {agentRef}` → claim → call-room token → join (shows "Connecting caller…" until the caller arrives). Losers: `409` + `ring_cancel`.
- **Screen pop**: the browser uses `from` to ask Textify for contact + open tickets. The engine never sees them.
- **Contacts**: Textify's **existing contact module** (the one SMS/WhatsApp already use) is reused for the call center: campaign recipients, caller names, customer timeline. The engine stores no contacts; campaigns receive `{e164, vars}` only.
- In-call: mute (client-side), hold (engine starts Ingress music, mutes agent), transfer (engine re-rings target; caller stays in room with hold music), hang up.

---

## 5. Media library &amp; voice generation

### 5.1 Uploaded audio

```
Textify (file or browser recording) → POST /v1/accounts/{id}/media (multipart)
Engine: validate (≤ 10 MB, ≤ 5 min, magic bytes) → MinIO original/ → job TRANSCODE (ffmpeg:
        mono, 16 kHz, MP3, loudness-normalized, trimmed) → MinIO play/ → READY → media.ready

```

### 5.2 Text → voice (generated audio)

- Messages with `source: text` are generated **at flow save** (inbound) or **before dialing** (campaigns, per-recipient text).
- Pipeline: render variables → **Swahili normalizer** (numbers, currency, dates, times → words) → cache lookup `hash(text+lang+voice+tier)` → `TtsRouter.tier → adapter` → ffmpeg normalize → MinIO `generated/` → `media_assets` row (kind `GENERATED`).
- **Basic**: self-hosted light model in a small sidecar (`tts-basic`, CPU), e.g. Piper with a Swahili voice, **only if** quality + commercial licence pass the listening test. **\[VERIFY licence\]**
- **Premium**: OpenAI TTS or Azure Speech (Swahili neural voices), picked by listening test.
- **Costs predictable**: premium character count computed before generation → shown in Textify → reported (`tts.usage`).
- **No runtime fallback to AT `Say`.** Failure → flow save error (previous version stays live) / campaign paused. Call-time safety net: system file.
- On-demand generation exists only for **sync webhook responses** (Ph 2): `Play url=/media/tts/{hash}.mp3` generated on first fetch, then cached. **\[VERIFY\]** AT `Play` fetch timeout.

### 5.3 Playing to AT

- XML uses `<Play url="https://voice-api…/media/{id}.mp3?t={hmac,24h}">`; the engine streams from MinIO (cache headers). Buckets stay private.
- Same URLs feed LiveKit Ingress (agent hold).
- Deleting media used by an active flow / running campaign → 409. Per-account storage quota.

### 5.4 Buckets

```
voice-media/system/{lang}/{name}.mp3                 # notices, hold default, sorry, invalid option, numbers 1–20
voice-media/{accountId}/original/{mediaId}.{ext}
voice-media/{accountId}/play/{mediaId}.mp3
voice-media/generated/{hash}.mp3                      # shared cache for generated voice
voice-recordings/{accountId}/{yyyy}/{mm}/{callId}.mp3 # retention applies

```

---

## 6. Knowledge base (Phase 3)

- Stored **in the engine per account** (business facts for call handling; not user data). Edited in Textify, pushed with `PUT /v1/accounts/{id}/knowledge`.
- Structure: ```json
    { "profile": { "name": "ABC Ltd", "about": "...", "location": "...", "hours": "..." },
      "services": [ { "name": "Kifurushi cha mwezi", "price": "TZS 25,000", "notes": "..." } ],
      "faqs": [ { "q": "...", "a": "..." } ],
      "departments": [ { "id": "dep_sales", "description": "New orders and prices" } ],
      "rules": { "never": ["Promise refunds"], "handoverWhen": ["Caller asks for a person", "Complaint"] },
      "documents": [ "kb_doc_1" ] }
    
    ```
    
    (`Kifurushi cha mwezi` = "Monthly package".)
- v1 of AI: **prompt builder** puts the whole KB into the AI session instructions (small businesses fit easily).
- Large KB: chunk documents into Postgres (`kb_chunks`, full-text or pgvector) and give the AI a `search_knowledge` tool. Still just Postgres.
- Live data via the account's **endpoints** exposed as AI tools.
- Improvement loop: the summary job extracts unanswered questions → `kb.unanswered` event → Textify suggests FAQ additions.

---

## 7. Data model (PostgreSQL)

```sql
client_apps (id uuid pk, name, status,                       -- ACTIVE | SUSPENDED | REMOVED
             webhook_url, webhook_secret_hash, can_call_url, rate_limit_rps, features jsonb,
             suspended_reason null, created_at, removed_at null)
client_app_keys (id uuid pk, client_app_id fk, key_hash, prefix, created_at, last_used_at,
                 last_used_ip, revoked_at null)                 -- ≥1 active; 2 during rotation
audit_log (id bigserial pk, actor, action, target, data jsonb, ip, at)

accounts (id uuid pk, client_app_id fk, external_ref, status, default_language default 'sw',
          retention_days default 90, features jsonb, created_at, unique(client_app_id, external_ref))

phone_numbers (id uuid pk, e164 unique, provider default 'AT', type, status,
               client_app_id fk null,                          -- pool owner (null = global pool)
               account_id fk null,
               label, hours jsonb, current_flow_version_id fk null, updated_at)

flows (id uuid pk, account_id fk, kind,                       -- NUMBER | CAMPAIGN
       owner_id uuid, created_at)
flow_versions (id uuid pk, flow_id fk, version int, definition jsonb, created_at,
               premium_chars int default 0, unique(flow_id, version))

media_assets (id uuid pk, account_id fk null,                 -- null = system / shared generated
              kind,                                           -- UPLOAD | GENERATED | SYSTEM
              name, original_key null, play_key, text_hash null, tier null, lang, voice,
              mime, size_bytes, duration_sec, status, error null, created_at)

agents (id uuid pk, account_id fk, external_ref, display_name, status, last_seen_at,
        unique(account_id, external_ref))
departments (id uuid pk, account_id fk, name, strategy, wrap_up_sec default 0)
department_members (department_id fk, agent_id fk, position, pk(department_id, agent_id))

endpoints (id uuid pk, account_id fk, name, url, secret, active bool, created_at)

calls (id uuid pk, account_id fk, number_id fk, campaign_id fk null, recipient_id fk null,
       flow_version_id fk, direction, from_e164, to_e164, at_session_id, livekit_room,
       status, handled_by, agent_id null, department_id null, context jsonb,
       started_at, answered_at, ended_at, end_reason,
       pstn_seconds, trunk_seconds, ai_seconds, wait_seconds, at_amount numeric, at_currency,
       ai_usage jsonb null)
call_events (id bigserial pk, call_id fk, type, node_id null, at, data jsonb)   -- trail

campaigns (id uuid pk, account_id fk, from_number_id fk, flow_version_id fk, status,
           window jsonb, cps int, max_concurrent int, max_retries int, retry_delay_sec int,
           starts_at, premium_chars_estimate int, created_at, completed_at, pause_reason null)
campaign_recipients (id bigserial pk, campaign_id fk, e164, vars jsonb, media_id fk null,
                     status, attempts, next_attempt_at, last_call_id fk null, answered_seconds,
                     keys text, index(campaign_id, status, next_attempt_at))
opt_outs (account_id fk, e164, source, created_at, pk(account_id, e164))

knowledge_bases (account_id pk fk, data jsonb, updated_at)                     -- Ph 3
kb_chunks (id bigserial pk, account_id fk, doc_id, text, tsv tsvector, embedding vector null)  -- later

recordings (id uuid pk, call_id fk unique, kind,                              -- CALL | VOICEMAIL
            source_url, storage_key, duration_sec, size_bytes, status, delete_after)
transcripts (call_id pk fk, status, language, text, segments jsonb, summary jsonb, model, created_at)

jobs (id bigserial pk, type, ref_id, status, attempts, run_after, last_error)
      -- TRANSCODE | TTS | DOWNLOAD | TRANSCRIBE | SUMMARIZE | RETENTION
outbox_events (id uuid pk, client_app_id fk, type, payload jsonb, attempts, next_attempt_at,
               delivered_at null, last_error)
endpoint_deliveries (id uuid pk, endpoint_id fk, call_id fk, node_id, payload jsonb, mode,
                     attempts, next_attempt_at, status, response_code null, last_error)

```

Redis: `presence:{accountId}:{agentRef}`, `call:{id}:agent` (claim, SET NX), `wait:{departmentId}` (sorted set: callId by wait start), `rr:{departmentId}`, `campaign:{id}:active`, flow/number config cache invalidation channel.

---

## 8. Security (service authentication)

<table id="bkmrk-traffic-method-texti"><thead><tr><th>Traffic</th><th>Method</th></tr></thead><tbody><tr><td>Textify → engine</td><td>API key per client app (hashed; 2 active keys for rotation; scoped to its accounts; rate-limited)</td></tr><tr><td>Engine → Textify</td><td>HMAC-SHA256 signed webhooks + timestamp (reject &gt; 5 min)</td></tr><tr><td>Engine → customer endpoints</td><td>HMAC with the endpoint's secret, idempotency key, HTTPS only, **SSRF guard** (block private/loopback/link-local/metadata IPs after DNS resolution), 3 s timeout</td></tr><tr><td>AT → engine</td><td>Unguessable secret in path + AT IP allowlist + sanity checks (known number / call) **\[VERIFY AT IPs, any signature\]**</td></tr><tr><td>LiveKit → engine</td><td>Signed JWT (`WebhookReceiver`)</td></tr><tr><td>OpenAI → engine (Ph 3)</td><td>Webhook signature + dedupe</td></tr><tr><td>AT → LiveKit SIP</td><td>Firewall + inbound trunk `allowed_addresses` = AT IPs; unknown callId → reject</td></tr><tr><td>Browser → LiveKit</td><td>Short-lived room tokens minted by engine, requested **through Textify** (human auth happens there)</td></tr><tr><td>AT → audio URLs</td><td>HMAC-signed, expiring media URLs</td></tr></tbody></table>

No OAuth/Keycloak/mTLS between our services. Secrets in env only. Recordings in private buckets with signed URLs, retention job, recording notice. Keep CrowdSec/fail2ban.

### 8.1 Client apps: how a service gets onto the engine, uses it, and is stopped or removed

Any service that consumes the engine (Textify first; later JikoXpress, Mauzodukani, NexGate…) is a **client app**. The engine treats them all the same and keeps them fully isolated from each other.

**Who registers a service:** only the **engine owner (QBIT SPARK ops)**. There's no self-signup, and client apps can never create, see or manage other client apps. Textify is simply the first client app registered.

**How: CLI-first (`voice-admin`), no public ops API**

- The admin tool is a mode of the engine jar (`java -jar voice-engine.jar admin …`, wrapped as `voice-admin`), run **on the Voice box over SSH**. It talks directly to the engine database, so **no ops endpoint exists on the internet**.
- SSH: key-only login, root login disabled, port 22 open only to the owner's IPs (ufw), CrowdSec on.
- Optional later (web ops panel): the `/ops/*` API binds **only to localhost or a WireGuard VPN interface**, never the public domain, and requires an **ops token** stored only on the box (env), plus the admin IP allowlist.

```
$ voice-admin client-app create --name textify \
    --webhook-url https://portal.textify.africa/voice/events \
    --can-call-url https://portal.textify.africa/voice/can-call \
    --features inbound,outbound,campaigns --rate-limit 50
→ clientAppId:    ca_7f3…
→ API key:        vk_live_3f9a…   (shown ONCE)
→ Webhook secret: whsec_81c2…     (shown ONCE)

$ voice-admin numbers allocate +2557XXXXXXXX --client ca_7f3…
$ voice-admin client-app test-webhook ca_7f3…
$ voice-admin key rotate ca_7f3…          # new key; old one stays valid until revoked
$ voice-admin key revoke ca_7f3… key_12…
$ voice-admin client-app suspend ca_7f3… --reason "…" [--hard]
$ voice-admin client-app resume ca_7f3…
$ voice-admin client-app export ca_7f3…  # zip of calls, recordings index, flows
$ voice-admin client-app remove ca_7f3…  # grace period, then delete
$ voice-admin audit --client ca_7f3… --since 7d

```

**Security throughout**

<table id="bkmrk-stage-protection-who"><thead><tr><th>Stage</th><th>Protection</th></tr></thead><tbody><tr><td>Who can register</td><td>Only someone with SSH access to the Voice box (owner's key + IP)</td></tr><tr><td>Creating credentials</td><td>Random 256-bit secrets generated by the engine; **only hashes stored**; shown **once** in the terminal</td></tr><tr><td>Hand-over</td><td>Owner pastes them **directly into the consuming service's env / secret store** on its own server. Never email, chat, tickets or git</td></tr><tr><td>Identifiable keys</td><td>Prefix `vk_live_` / `vk_test_` + short key id, so a leaked key is easy to spot and revoke; full key never logged</td></tr><tr><td>Using the engine</td><td>HTTPS only; key → exactly one client app; all data filtered by `client_app_id`; per-client rate limit + feature flags</td></tr><tr><td>Webhooks back</td><td>Signed with that client's webhook secret; client verifies signature + timestamp</td></tr><tr><td>Leak / staff change</td><td>`key rotate` (both valid briefly) → `key revoke`: effective immediately</td></tr><tr><td>Stop / remove</td><td>`suspend` (reversible) / `remove` (export → release numbers → revoke keys → delete after grace)</td></tr><tr><td>Traceability</td><td>Every admin command (who, when, from which IP) and each key's first/last use + IP in `audit_log`</td></tr></tbody></table>

No approval steps: protection comes from **who can reach the CLI** plus **show-once, hashed, scoped, rotatable keys**.

**Lifecycle**

<table id="bkmrk-step-what-happens-ho"><thead><tr><th>Step</th><th>What happens</th><th>How</th></tr></thead><tbody><tr><td>1. Register</td><td>Create the client app: name, `webhook_url`, `can_call_url`, rate limit, allowed features (inbound, campaigns, AI)</td><td>`voice-admin client-app create …` → returns `clientAppId`, **API key** and **webhook secret**, shown **once** (only hashes stored)</td></tr><tr><td>2. Give it numbers</td><td>Assign AT numbers from the global pool to the client app's pool</td><td>`voice-admin numbers allocate <e164> --client <id>`</td></tr><tr><td>3. Client stores credentials</td><td>The client keeps the API key + webhook secret in its own secrets/env</td><td>Client side</td></tr><tr><td>4. Verify</td><td>Client calls `GET /v1/ping` (auth check); engine sends a signed `test.ping` webhook to check the client's receiver</td><td>`voice-admin client-app test-webhook <id>`</td></tr><tr><td>5. Consume</td><td>Client creates its accounts (`POST /v1/accounts {externalRef}`), assigns numbers from **its** pool to accounts, saves flows, adds agents, runs campaigns…</td><td>`/v1/**` with `Authorization: Bearer <key>`</td></tr><tr><td>6. Rotate key</td><td>Issue a second key; both work; client switches; old key revoked</td><td>`voice-admin key rotate <id>` → `voice-admin key revoke <id> <keyId>`</td></tr><tr><td>7. Suspend (stop, reversible)</td><td>API returns `403 CLIENT_SUSPENDED`; running campaigns **paused**; no new outbound calls; webhooks held in the outbox (delivered after resume). **Inbound calls keep working by default**, so callers aren't hurt. Option `hard: true` also answers inbound with a system "service unavailable" message</td><td>`voice-admin client-app suspend <id> --reason … [--hard]` · `… resume <id>`</td></tr><tr><td>8. Remove (permanent)</td><td>Suspend → cancel campaigns → release numbers back to the global pool (AT callback config untouched; numbers reusable) → revoke keys → stop webhooks → optional **export** (calls, recordings, flows as a zip) → data deleted after a grace period (default 30 days)</td><td>`voice-admin client-app export <id>` · `… remove <id>`</td></tr></tbody></table>

**Isolation rules**

- Every `/v1` request resolves to exactly one client app from the API key; every query is filtered by `client_app_id`. A key can never see another client app's accounts, numbers, calls or media.
- `external_ref` is unique **per client app** (two apps can both have "tenant\_1").
- Numbers belong to one client app's pool at a time; inbound calls are routed by number → client app → account.
- Each client app has its own webhook URL + secret, rate limit (e.g. 50 req/s) and feature flags.
- An account inside a client app can also be suspended (`PATCH /v1/accounts/{id} {status: SUSPENDED}`): same effect as above, but for one tenant. Textify uses this when a customer stops paying.

**Audit**: every admin command and key use (first/last seen, IP) is logged in `audit_log`; `voice-admin audit` reads it.

---

## 9. Communication

<table id="bkmrk-need-mechanism-texti"><thead><tr><th>Need</th><th>Mechanism</th></tr></thead><tbody><tr><td>Textify ⇄ engine commands/queries</td><td>REST</td></tr><tr><td>Engine → Textify facts</td><td>Webhooks via Postgres **outbox** + `@Scheduled` dispatcher</td></tr><tr><td>Engine → customer systems</td><td>`webhook` node: async via `endpoint_deliveries` (retries), sync direct (Ph 2)</td></tr><tr><td>AT call control</td><td>Callbacks + XML; AT REST call API</td></tr><tr><td>Ring agents</td><td>LiveKit data messages (lobby rooms)</td></tr><tr><td>Presence, claims, waiting orders, concurrency, cache invalidation</td><td>Redis</td></tr><tr><td>Background work</td><td>Postgres `jobs` + workers (`FOR UPDATE SKIP LOCKED`)</td></tr><tr><td>Broadcast queue</td><td>`campaign_recipients` table + dispatcher</td></tr></tbody></table>

Not used: MQTT, Kafka, RabbitMQ. Add **Redis Streams** first only if several engine instances + high event volume demand it.

---

## 10. APIs

### 10.1 Client API (Textify → engine) · `Authorization: Bearer <api key>` · base `https://voice-api.textify.africa/v1` **\[VERIFY domain\]**

<table id="bkmrk-area-endpoints-accou"><thead><tr><th>Area</th><th>Endpoints</th></tr></thead><tbody><tr><td>Accounts</td><td>`POST /accounts` · `PATCH /accounts/{id}`</td></tr><tr><td>Numbers</td><td>`GET /numbers?status=UNASSIGNED` · `POST /numbers/{id}/assign` · `POST /numbers/{id}/release` · `PUT /numbers/{id}/hours` · `PUT /numbers/{id}/flow` (validate + new version) · `GET /numbers/{id}/flow/versions` · `POST /numbers/{id}/flow/rollback {version}` · `POST /numbers/{id}/test-call {toE164}`</td></tr><tr><td>Flow tools</td><td>`POST /flows/validate` (dry run, returns errors + premium char estimate)</td></tr><tr><td>Media</td><td>`POST /accounts/{id}/media` (multipart) · `POST /accounts/{id}/media/generate {text, lang, voice, tier}` (preview) · `GET /accounts/{id}/media` · `GET /media/{id}` · `DELETE /media/{id}` · `GET /voices`</td></tr><tr><td>Agents</td><td>`PUT /accounts/{id}/agents/{ref}` · `DELETE …` · `PUT …/status` · `POST …/lobby-token`</td></tr><tr><td>Departments</td><td>`POST/PUT/DELETE /accounts/{id}/departments[/{did}]`</td></tr><tr><td>Calls</td><td>`POST /calls/outbound` · `POST /calls/{id}/accept` · `POST /calls/{id}/hold</td></tr><tr><td>Campaigns</td><td>`POST /campaigns` · `POST /campaigns/{id}/recipients` (chunks ≤ 10k) · `POST /campaigns/{id}/estimate` · `POST /campaigns/{id}/start</td></tr><tr><td>Opt-outs</td><td>`GET/POST/DELETE /accounts/{id}/opt-outs`</td></tr><tr><td>Endpoints</td><td>`POST/PUT/DELETE /accounts/{id}/endpoints[/{eid}]` · `POST /endpoints/{eid}/test`</td></tr><tr><td>Knowledge (Ph 3)</td><td>`PUT /accounts/{id}/knowledge` · `POST /accounts/{id}/knowledge/documents` · `GET /accounts/{id}/knowledge/unanswered`</td></tr><tr><td>Stats</td><td>`GET /numbers/{id}/stats` (per node) · `GET /campaigns/{id}/stats`</td></tr><tr><td>Health</td><td>`GET /ping` (auth check)</td></tr><tr><td>Ops: **CLI `voice-admin` on the box** (primary). Optional `/ops` HTTP API bound to localhost/VPN only, ops token + admin IPs</td><td>`POST /ops/client-apps` · `POST /ops/client-apps/{id}/keys` · `DELETE /ops/client-apps/{id}/keys/{keyId}` · `POST /ops/client-apps/{id}/test-webhook` · `POST /ops/client-apps/{id}/suspend</td></tr></tbody></table>

### 10.2 Engine → Textify

- Balance: `GET {can_call_url}?accountRef&direction&estimatedCalls?` → `{allowed, maxMinutes?}` (1.5 s, fail-open).
- Webhooks (outbox, signed, retried 1m/5m/30m/2h… ≤ 24 h, idempotent by event id):

<table id="bkmrk-event-data-call.star"><thead><tr><th>Event</th><th>Data</th></tr></thead><tbody><tr><td>`call.started` · `call.answered`</td><td>callId, accountRef, direction, from, to, number, handledBy, agentRef, department</td></tr><tr><td>`call.ended`</td><td>status, usage {pstnSeconds, trunkSeconds, waitSeconds, aiSeconds, atAmount}, recorded, endReason, flowVersion</td></tr><tr><td>`call.missed`</td><td>callId, from, to, numberLabel, department</td></tr><tr><td>`call.action_triggered`</td><td>callId, node, endpoint, status</td></tr><tr><td>`message.requested`</td><td>callId/campaignId, to, channel, template, params → Textify sends SMS/WhatsApp</td></tr><tr><td>`voicemail.received`</td><td>callId, from, mediaId/recording</td></tr><tr><td>`recording.ready` · `summary.ready`</td><td>callId (+ summary JSON)</td></tr><tr><td>`media.ready` · `media.failed`</td><td>mediaId, duration / error</td></tr><tr><td>`tts.usage`</td><td>accountRef, chars, tier, ref (flow version / campaign)</td></tr><tr><td>`campaign.progress` · `campaign.recipient_results` · `campaign.paused` · `campaign.completed`</td><td>counts / batches / reason / totals</td></tr><tr><td>`kb.unanswered` (Ph 3)</td><td>accountRef, questions\[\]</td></tr><tr><td>`test.ping`</td><td>sent on request to verify the client's webhook receiver</td></tr></tbody></table>

### 10.3 Hooks into the engine

<table id="bkmrk-endpoint-from-post-%2F"><thead><tr><th>Endpoint</th><th>From</th></tr></thead><tbody><tr><td>`POST /hooks/at/{secret}/voice` · `/step/{callId}/{node}` · `/wait/{callId}`</td><td>AT</td></tr><tr><td>`POST /hooks/livekit`</td><td>LiveKit</td></tr><tr><td>`POST /hooks/openai` (Ph 3)</td><td>OpenAI</td></tr><tr><td>`GET /media/{id}.mp3?t=…` · `GET /media/tts/{hash}.mp3?t=…`</td><td>AT / LiveKit Ingress</td></tr></tbody></table>

---

## 11. Infrastructure

### 11.1 Dedicated Voice box (Docker Compose; host networking for media)

<table id="bkmrk-service-notes-caddy-"><thead><tr><th>Service</th><th>Notes</th></tr></thead><tbody><tr><td>`caddy` / nginx</td><td>TLS for `voice-api.…` (API, hooks, media URLs) and `wss://voice-rtc.…` (LiveKit)</td></tr><tr><td>`livekit`</td><td>`livekit/livekit-server`, 7880 on localhost</td></tr><tr><td>`sip`</td><td>`livekit/sip`, 5060 + RTP 10000–20000, AT only</td></tr><tr><td>`ingress`</td><td>`livekit/ingress`, agent-hold audio **\[VERIFY URL input\]**</td></tr><tr><td>`redis`</td><td>shared by LiveKit + engine (separate DB index)</td></tr><tr><td>`postgres`</td><td>engine DB (pgvector extension later for KB)</td></tr><tr><td>`minio`</td><td>private buckets `voice-media`, `voice-recordings`; console on localhost</td></tr><tr><td>`tts-basic`</td><td>self-hosted light TTS model (CPU), HTTP on localhost **(if Basic passes the test)**</td></tr><tr><td>`voice-engine`</td><td>Spring Boot jar + ffmpeg</td></tr><tr><td>(fallback) `egress`</td><td>only if AT recording isn't sufficient</td></tr></tbody></table>

Start 4 vCPU / 8 GB, measure. Region with lowest RTT to AT SIP servers.

### 11.2 Ports

<table id="bkmrk-port-open-to-purpose"><thead><tr><th>Port</th><th>Open to</th><th>Purpose</th></tr></thead><tbody><tr><td>22</td><td>owner IPs only (key-only login)</td><td>SSH, `voice-admin` CLI</td></tr><tr><td>80/443</td><td>world</td><td>API, hooks, media, WSS</td></tr><tr><td>7881/tcp, 50000–60000/udp</td><td>world</td><td>WebRTC (browser agents)</td></tr><tr><td>5060 udp/tcp, 10000–20000/udp</td><td>AT IPs only</td><td>SIP + RTP</td></tr><tr><td>7880, 6379, 5432, 9000/9001, tts-basic, 8080</td><td>localhost</td><td>internal</td></tr><tr><td>(Ph 3) out 5061 + RTP to OpenAI CIDRs</td><td>OpenAI</td><td>AI over SIP</td></tr></tbody></table>

### 11.3 Env

```
AT_USERNAME / AT_API_KEY / AT_CALLBACK_SECRET / AT_IPS
LIVEKIT_URL / LIVEKIT_PUBLIC_WS_URL / LIVEKIT_API_KEY / LIVEKIT_API_SECRET / LIVEKIT_SIP_HOST
DB_URL / DB_USER / DB_PASSWORD · REDIS_URL / REDIS_PASSWORD
MINIO_ENDPOINT / MINIO_ACCESS_KEY / MINIO_SECRET_KEY
MEDIA_URL_SECRET
TTS_BASIC_URL · TTS_PREMIUM_PROVIDER (openai|azure) / keys
OPENAI_API_KEY (Ph 2) · OPENAI_WEBHOOK_SECRET / OPENAI_SIP_URI (Ph 3)

```

### 11.4 LiveKit bootstrap (once, idempotent)

1. Inbound trunk `at`: `allowed_addresses` = AT IPs, no number filter.
2. Dispatch rule `dispatch_rule_callee`, `room_prefix: "call-"` → `sip:{callId}@…` ⇒ room `call-{callId}` **\[VERIFY\]**.
3. (Ph 3) Outbound trunk `openai` (TLS + SRTP).

---

## 12. Observability &amp; testing

- JSON logs with `callId`, `accountId`, `atSessionId`, `campaignId`, `node`.
- Metrics: active calls, waiting callers + wait time, ring-to-accept, answer/missed rate, callback latency (p95 &lt; 500 ms), campaign dials/sec + answer rate, TTS jobs + premium chars, endpoint delivery failures, outbox backlog.
- Alerts: SIP down, callback errors, zero inbound in business hours, outbox backlog, campaign stalled, TTS failures, system-file fallback used.
- PoC = SPEC §10 Phase 0.
- Unit: flow validator, every NodeExecutor (node → XML), wait loop, claim race, ring strategies, campaign dispatcher (window/cps/retries/opt-out), Swahili normalizer, signers, SSRF guard.
- Integration: Testcontainers (Postgres, Redis, MinIO), WireMock (AT, Textify, endpoints, TTS, OpenAI), LiveKit dev + browser test page, real AT number.

## 13. Tech stack &amp; tools

<table id="bkmrk-layer-tool-used-for-"><thead><tr><th>Layer</th><th>Tool</th><th>Used for</th></tr></thead><tbody><tr><td>Engine</td><td>Java 21, Spring Boot 3.x (Web, Validation, JDBC/JPA, Scheduling, Actuator)</td><td>All engine logic</td></tr><tr><td>DB</td><td>PostgreSQL 16 + Flyway (pgvector later)</td><td>State, flows, outbox, jobs, campaign queue, KB</td></tr><tr><td>Cache/locks</td><td>Redis 7</td><td>Presence, claims, waiting orders, concurrency</td></tr><tr><td>Telephony</td><td>Africa's Talking Voice (callbacks + XML: Play, GetDigits, Record, Dial; call API); optional `africastalking-java` for REST</td><td>PSTN both directions</td></tr><tr><td>Media</td><td>LiveKit server, SIP, Ingress; `io.livekit:livekit-server` JVM SDK; LiveKit JS SDK in Textify UI</td><td>Rooms, browser agents</td></tr><tr><td>Voice (TTS)</td><td>Basic: self-hosted light model (e.g. Piper, CPU) **\[VERIFY licence/quality\]**; Premium: OpenAI TTS or Azure Speech</td><td>Text → audio files</td></tr><tr><td>AI</td><td>OpenAI REST (transcribe, summarize); Ph 3 Realtime over SIP + `java.net.http.WebSocket`</td><td>Summaries, live AI</td></tr><tr><td>Storage</td><td>MinIO via AWS SDK v2</td><td>Media library, generated voice, recordings</td></tr><tr><td>Audio processing</td><td>ffmpeg</td><td>Transcode, normalize, trim</td></tr><tr><td>HTTP</td><td>Spring `RestClient`</td><td>AT, Textify, endpoints, TTS, OpenAI</td></tr><tr><td>Security</td><td>Hashed API keys, HMAC-SHA256, LiveKit `WebhookReceiver`, SSRF guard, ufw, CrowdSec</td><td>Service auth</td></tr><tr><td>Edge</td><td>Caddy (or nginx)</td><td>TLS, reverse proxy</td></tr><tr><td>Deploy</td><td>Docker Compose, dedicated VPS</td><td>Everything</td></tr><tr><td>Observability</td><td>Logback JSON, Actuator + Micrometer (Prometheus)</td><td>Logs, metrics</td></tr><tr><td>Testing</td><td>JUnit 5, Testcontainers, WireMock, browser test page</td><td>Unit + integration</td></tr><tr><td>Not used</td><td>AT `Say`, MQTT, Kafka, RabbitMQ, OAuth/Keycloak, Python, browser extension</td><td>Not needed</td></tr></tbody></table>

## 14. Repository layout

```
textify-voice/
├── docs/          SPEC.md, FLOWS.md, ARCHITECTURE.md
├── voice-engine/  Spring Boot (Java 21)
├── tts-basic/     self-hosted TTS sidecar (if Basic passes)
├── deploy/        docker-compose.yaml, livekit.yaml, sip, ingress, Caddyfile, firewall.sh, .env.example
├── audio/         system prompts (sw/en): notices, hold default, sorry, invalid option, numbers
└── agent-test/    minimal HTML page with LiveKit JS for the PoC

```

## 15. Open items to verify

1. AT: Dial to `sip:{callId}@host`; IPs; codecs/DTMF; billing; Dial-to-SIP recording; CPS/concurrency; per-second vs per-minute; multi-number call API; callback signature (SPEC §11).
2. AT: GetDigits wrapping Play (key interrupt, timeouts, callbacks per call) for the wait loop; Play fetch timeout/streaming; XML continuation after Dial.
3. LiveKit: callee dispatch with arbitrary user part; trunk without number filter; Ingress URL input; JVM SDK method names.
4. TTS: Basic candidate quality + commercial licence; Premium choice; Swahili normalizer coverage.
5. Ph 3: LiveKit → OpenAI SIP interop; Realtime vs Live API; Swahili AI quality + cost.

---

## Appendix A: Phase 3 AI (Java-only)

- No Java SDK joins LiveKit rooms as an audio participant; Spring AI / LangChain4j are text-only. So **OpenAI joins the room as a SIP participant**.
- Engine: `CreateSIPParticipant(outbound trunk "openai", to = OpenAI project SIP URI, header X-Voice-Call-Id)` → OpenAI webhook `realtime.call.incoming` (or Live API `live.transport.incoming`) → match header to a call in `AI_DIALING` → `POST /v1/realtime/calls/{id}/accept` with instructions (KB prompt + node instructions; Swahili first; short answers; never invent prices), voice, tools (`transfer_to_human`, `take_message`, `end_call`, endpoint tools, `send_sms`) → WebSocket `wss://api.openai.com/v1/realtime?call_id=…` for tool calls/transcripts/usage → handoff: ring department, agent joins, `POST …/hangup` AI leg.
- Firewall: out TCP 5061 to `sip.api.openai.com`; RTP with OpenAI media CIDRs `13.79.45.80/28, 23.98.140.64/28, 40.67.149.176/28, 40.83.204.240/28` (OpenAI SIP guide, Oct 2026; re-check).
- Trade-off: OpenAI voices only. If Swahili isn't good enough → fallback: small Python LiveKit Agents worker with another TTS, calling engine internal endpoints; nothing else changes.

# Ringmaster Voice: Product & Functional Specification

> Status: draft for implementation · Owner: Joshua Saimon Sakweli (QBIT SPARK) · Updated 2026-11-08 Companion docs: `FLOWS.md` (flow &amp; action reference) · `ARCHITECTURE.md` (components, data model, APIs, infra). Written so a new engineer, or a new AI coding session, can pick up the work with no prior context.

---

## 1. Background

**Textify Africa** (textify.africa) is an existing bulk SMS + WhatsApp platform, **written in Go**, live in Tanzania, Uganda and Nigeria. Its main model is **white-label**: partners brand it as their own and onboard their own customers, who top up a **prepaid wallet**.

We're adding **Voice** as a new Textify module, so the **same Textify customers** use it from the same portal and wallet. It combines:

- a **call center** (one business number, menus, browser agents), similar in spirit to SemaCall (semacall.com), and
- Textify's strength, **bulk**: **voice broadcast campaigns** with keypad actions and SMS fallback.

Voice runs on a separate service, the **Voice Engine** (Java Spring Boot). It is an **engine**: blind to users, logins, roles and money. It is told *what to do with numbers and calls* and reports *what happened*. Textify consumes it like any other API. Other QBIT SPARK products (JikoXpress, Mauzodukani, NexGate) can consume it later.

Status: we already have a **working Tanzanian number from Africa's Talking**.

## 2. Goals

1. Customers get a number, choose how it answers (**Direct**, **Menu**, later **AI**), and calls reach **agents answering in the browser**.
2. Callers waiting for an agent hear **hold music**, never silence; nobody waits forever (voicemail).
3. **Click-to-call** outbound for agents.
4. **Broadcast campaigns**: voice message to thousands, keypad actions (talk to us, confirm via webhook, opt out…), retries, **SMS fallback** for non-answerers.
5. Customers create what callers hear by **uploading/recording audio or typing text** (Basic or Premium voice).
6. Keypresses can **trigger the customer's own systems** (webhooks with parameters).
7. Every call is **logged, optionally recorded**, and (Phase 2) **transcribed + summarized**.
8. **Missed calls / voicemails** become follow-ups in Textify (tickets, SMS/WhatsApp).
9. **Predictable costs**: per-minute call billing from the Textify wallet; voice-generation cost known before it's spent.
10. (Phase 3) **AI receptionist** with a per-business **knowledge base**, handing over to humans.

## 3. Non-goals

- Engine has no users, auth of people, roles, plans, prices, money, tickets or contacts.
- No approval workflows anywhere.
- No live text-to-speech during calls (no AT `<Say>`; all text becomes audio files first).
- No video over phone lines. No message broker (Kafka/RabbitMQ/MQTT) in v1. No Python.
- Countries other than Tanzania (later, same design).

## 4. Key terms

<table id="bkmrk-term-meaning-pstn-no"><thead><tr><th>Term</th><th>Meaning</th></tr></thead><tbody><tr><td>**PSTN**</td><td>Normal phone network (Vodacom, Airtel, Yas, Halotel, TTCL)</td></tr><tr><td>**AT**</td><td>Africa's Talking: our numbers, PSTN both directions, voice callbacks/XML</td></tr><tr><td>**Voice callback / XML**</td><td>AT POSTs to our URL; we reply with XML actions (`Play`, `GetDigits`, `Dial`, `Record`…)</td></tr><tr><td>**SIP / SIP trunk**</td><td>Internet call protocol / the AT → LiveKit connection carrying calls</td></tr><tr><td>**LiveKit room**</td><td>Live audio session; each connected call = one room (caller + agent, later AI)</td></tr><tr><td>**Browser agent**</td><td>Staff answering in the Textify web app</td></tr><tr><td>**Department**</td><td>A group of agents that calls ring (a ring group)</td></tr><tr><td>**Flow**</td><td>Graph of nodes deciding what happens on a call (FLOWS.md)</td></tr><tr><td>**Node / action**</td><td>One step: play, menu, collect, connect, voicemail, webhook, branch, sms, optout, hangup, ai</td></tr><tr><td>**Message**</td><td>Something callers hear: uploaded audio, or text turned into audio (Basic/Premium voice)</td></tr><tr><td>**Campaign**</td><td>A broadcast: a flow run for every answered call on a recipient list</td></tr><tr><td>**Endpoint**</td><td>A customer's saved webhook URL + secret, used by `webhook` actions</td></tr><tr><td>**Knowledge base (KB)**</td><td>Business facts the AI uses (Phase 3)</td></tr><tr><td>**DTMF**</td><td>Keypad presses</td></tr></tbody></table>

## 5. Product structure (what customers see in Textify)

```
Voice
├── Numbers          get/assign numbers
├── Call Center      inbound + agents
│   ├── Answer mode: Direct | Menu | AI (Ph 3)
│   ├── Departments & agents
│   ├── Business hours, closed message
│   ├── Waiting: hold music, comfort message, max wait → voicemail
│   ├── Recording on/off
│   └── Click-to-call (agents calling one customer)
├── Campaigns        bulk outbound
│   ├── Message (audio or text, variables)
│   ├── Keypad actions (talk to dept, webhook, sms, opt-out…)
│   ├── Recipients picked from Textify's EXISTING contacts / groups (same as SMS)
│   ├── Schedule, calling hours, retries
│   └── Results + SMS fallback to non-answerers
├── Audio library    uploads, recordings, generated voices
├── Integrations     webhook endpoints
├── Knowledge base   (Ph 3, for AI)
└── Calls & Tickets  call log, recordings, summaries; follow-up tickets (Textify-only feature)

```

Answer-mode templates generate flows; customers never see raw nodes (an advanced flow editor may come later; the format already supports it).

<table id="bkmrk-answer-mode-caller-e"><thead><tr><th>Answer mode</th><th>Caller experience</th><th>AT actions</th></tr></thead><tbody><tr><td>**Direct**</td><td>Greeting → ring department → wait with music → voicemail</td><td>Play → (wait loop) → Dial → Record</td></tr><tr><td>**Menu**</td><td>Greeting → "press 1 sales, 2 support" → that department → voicemail</td><td>Play + GetDigits → wait loop → Dial → Record</td></tr><tr><td>**AI** (Ph 3)</td><td>AI answers from the knowledge base, hands over to a department</td><td>Dial → LiveKit + OpenAI → agents</td></tr></tbody></table>

## 6. Providers

<table id="bkmrk-provider-role-africa"><thead><tr><th>Provider</th><th>Role</th></tr></thead><tbody><tr><td>**Africa's Talking**</td><td>Numbers, PSTN in/out, callbacks + XML, outbound call API, call recording, SIP to LiveKit</td></tr><tr><td>**LiveKit (self-hosted, dedicated box)**</td><td>Rooms, browser agents (WebRTC), agent-hold audio, Ph 3 AI participant. Separate from NexGate's LiveKit</td></tr><tr><td>**TTS: Basic**</td><td>Light open-source model self-hosted on CPU (candidate: Piper Swahili voice), **only if** quality and commercial licence pass the listening test</td></tr><tr><td>**TTS: Premium**</td><td>OpenAI TTS or Azure Speech (Swahili neural voices), chosen by listening test</td></tr><tr><td>**OpenAI**</td><td>Ph 2 transcription + summaries; Ph 3 live AI (Realtime over SIP, controlled from Java)</td></tr><tr><td>**MinIO** (or Linode Object Storage)</td><td>Audio library + recordings</td></tr></tbody></table>

## 7. Costs (AT Tanzania, africastalking.com/pricing, Oct 2026)

<table id="bkmrk-item-price-regular-n"><thead><tr><th>Item</th><th>Price</th></tr></thead><tbody><tr><td>Regular number (Airtel)</td><td>Setup TZS 100,000 + 18% VAT; TZS 35,000/month + 18% VAT</td></tr><tr><td>Toll-free</td><td>Same + TZS 170,000 deposit</td></tr><tr><td>Incoming / Outgoing</td><td>TZS 17 / TZS 60 per min</td></tr><tr><td>Toll-free incoming</td><td>TZS 200 / min</td></tr><tr><td>SIP trunk</td><td>TZS 8.61 / min</td></tr></tbody></table>

<table id="bkmrk-call-type-approx.-tz"><thead><tr><th>Call type</th><th>Approx. TZS / min</th></tr></thead><tbody><tr><td>Inbound, menu / waiting / voicemail (stays on AT)</td><td>17</td></tr><tr><td>Inbound talking to browser agent</td><td>17 + 8.61 ≈ **25.61**</td></tr><tr><td>Click-to-call</td><td>60 + 8.61 ≈ **68.61**</td></tr><tr><td>Broadcast message only</td><td>60 per answered minute (unanswered likely free, **confirm**)</td></tr><tr><td>Broadcast → talk to agent</td><td>60 + 8.61 while talking</td></tr></tbody></table>

Voice generation (TTS): **Basic = fixed server cost**; **Premium ≈ $15–16 per 1M characters** (check current), always **calculated before spending** (at save or campaign creation). No AT `<Say>` usage. Waiting happens on AT (no trunk cost while waiting).

Agreed: cost follows **minutes, not agents** (unlimited agents); no mobile forwarding by default (77/min). Indicative selling prices (finalize after PoC): number 60–75k/month; inbound answered 40–50/min; outbound 90–100/min; broadcast 100–150 per answered call; premium voice per message/character; AI minute 150–250. **Prices live in Textify.**

## 8. Functional requirements

### 8.1 Numbers &amp; answer modes

- FR-N1: Ops registers AT numbers in the engine; Textify assigns a number to an account.
- FR-N2: Each number has an **open-hours flow** and a **closed-hours flow** (generated from the answer mode; FLOWS.md).
- FR-N3: Business hours per number (timezone Africa/Dar\_es\_Salaam), with holidays list (optional).
- FR-N4: Changes take effect on the next call; every save is a new **flow version**.
- FR-N5: **Test call**: engine calls the customer's own phone and runs the flow.

### 8.2 Flows &amp; actions (full reference: FLOWS.md)

- FR-F1: One action system for inbound, campaigns and (Ph 3) AI.
- FR-F2: Node types: `play`, `menu`, `collect`, `connect`, `voicemail`, `webhook`, `branch`, `sms`, `optout`, `hangup`, (Ph 3) `ai`.
- FR-F3: Variables: system, contact (campaigns), collected, webhook result.
- FR-F4: Validation on save (no dangling outcomes, size limits, loop guard, variables exist, media ready, targets belong to account). No approvals.
- FR-F5: Running campaigns are pinned to their flow version.
- FR-F6: Never fail the caller: any error → stored system message + graceful hangup.

### 8.3 Messages &amp; voice (what callers hear)

- FR-M1: Every message is **either** uploaded/recorded **audio** **or** **text**.
- FR-M2: Audio: mp3/wav/m4a/ogg, ≤ 10 MB, ≤ 5 min, or record in the browser → engine converts to phone-friendly MP3.
- FR-M3: Text: language (sw/en), voice, **tier Basic or Premium**; max 500 characters per message.
- FR-M4: Text becomes an audio file **when saved** (flows) or **before dialing** (campaigns, incl. per-recipient personalized text). **No TTS during live calls.**
- FR-M5: Generated audio is cached by text + language + voice + tier; identical text is generated once.
- FR-M6: Swahili number/currency/date normalization before TTS ("TZS 25,000" → "shilingi elfu ishirini na tano").
- FR-M7: Premium cost shown before saving/starting; reported in usage (`ttsCharacters`, tier).
- FR-M8: Generation failure: flow save shows an error and the previous version stays live; campaign stays unstarted or **pauses** (`TTS_FAILED`). Safety net at call time: a stored system file.
- FR-M9: Listen/preview in the browser for audio and generated text.

### 8.4 Inbound, waiting and agents

- FR-I1: AT callback → number → account → flow (open/closed by hours).
- FR-I2: **Balance check** with Textify; unreachable → allow (fail-open). No balance → system message, hang up.
- FR-I3: Recording notice when recording is on ("Simu hii inaweza kurekodiwa kwa ajili ya ubora wa huduma"; in English: "This call may be recorded for quality of service").
- FR-I4: `connect` to a department → engine rings available agents in their browsers → **caller waits on AT** with hold music → agent accepts → call dialed into LiveKit → connected.
- FR-I5: **Waiting v1**: hold music (system default or customer upload), optional comfort message, max wait → `noAnswer` (usually voicemail), optional keys while waiting (e.g. 1 = voicemail).
- FR-I6: Each department has a waiting order (oldest first). When an agent becomes available, the next waiting caller rings them.
- FR-I7: Agents: `agentRef` only in the engine; status `AVAILABLE / BUSY / OFFLINE`; presence automatic while the browser is connected.
- FR-I8: Ring strategy per department: `ALL_AT_ONCE` (default) or `ROUND_ROBIN`; **first to accept wins**.
- FR-I9: In-call: mute, **hold (with music)**, transfer to agent/department, hang up.
- FR-I10: Ring popup shows the caller number; Textify adds the contact name and open tickets (**screen pop**, Textify-side).
- FR-I12: Agents need **no browser extension or install**: Textify web page + microphone permission; one click "Go available" unlocks the ringtone; background-tab alerts via browser notifications; optional installable PWA.
- FR-I11 (later, designed): position announcements, estimated wait, callback request (virtual hold), whisper to agent, wrap-up time.

### 8.5 Outbound: click-to-call

- FR-O1: Agent clicks a number → engine calls via AT from the account's number → agent waits in the room → customer answers → connected.
- FR-O2: Balance check before dialing.

### 8.6 Campaigns (broadcast)

- FR-C1: Campaign = account, from-number, **flow** (message + keypad actions), recipients (with variable columns), start time, calling window (default 08:00–20:00 EAT), calls/sec + max concurrent, retries (default 2, 30 min apart, for no-answer/busy).
- FR-C1b: Recipients come from **Textify's existing contacts and groups** (the same ones used for SMS/WhatsApp); contact fields become campaign variables. Textify sends the engine only `{e164, vars}` per recipient. No separate call-center contact list.
- FR-C2: Personalized messages generated before dialing (FR-M4).
- FR-C3: Opt-out list per account, always checked before dialing.
- FR-C4: Start / pause / resume / cancel; Textify pauses automatically on low wallet; engine pauses on generation failure.
- FR-C5: Per-recipient results: status, seconds, keys pressed, actions triggered, cost.
- FR-C6: Progress + completion events; Textify can SMS/WhatsApp everyone who didn't answer.
- FR-C7: Throughput limited by AT CPS / concurrency (**confirm**); Textify shows ETA.

### 8.7 Webhook actions (customer integrations)

- FR-W1: Customers save **endpoints** (HTTPS URL + secret) in Textify; stored in the engine per account.
- FR-W2: `webhook` node sends parameters with variables (`{caller}`, contact columns, `{key}`, `{input.x}`, `{callId}`, `{campaignId}`…).
- FR-W3: **Async** (default, Phase 1): queued, retried up to 24 h; caller hears a fixed confirmation immediately.
- FR-W4: **Sync** (Phase 2): engine waits ≤ 3 s; the response can set variables (`{result.x}`) and a message to play (generated on demand, then cached).
- FR-W5: Signed (HMAC), idempotency key per call + node, HTTPS only, private/internal IPs blocked, failures never break the call.

### 8.8 Recording, transcript, summary

- FR-R1: Recording per number/campaign (default on for agent calls, with notice); recorded by AT, copied into our storage immediately.
- FR-R2 (Ph 2): transcript + summary JSON (summary, intent, resolved, follow-ups, sentiment, topic).
- FR-R3: Retention default 90 days (per account). Tanzania Personal Data Protection Act applies.

### 8.9 AI receptionist + knowledge base (Phase 3)

- FR-A1: Answer mode **AI** or `ai` node anywhere (e.g. menu option, campaign key, noAnswer).
- FR-A2: Swahili first, English if the caller speaks it; short spoken answers; never invents prices/policies.
- FR-A3: **Knowledge base per account**, edited in Textify, stored in the engine: business profile (name, about, location, hours), services &amp; prices, FAQs, departments with descriptions (for routing), rules (never do / when to hand over), optional documents (PDF/text).
- FR-A4: v1 of AI: KB placed directly into the AI's instructions (no vector DB). Large KBs: `search_knowledge` tool over Postgres full-text/pgvector.
- FR-A5: Live data via the customer's **endpoints** as AI tools (same endpoints as `webhook`).
- FR-A6: Handover to a department (warm transfer); no agent → take a message.
- FR-A7: **Improvement loop**: summaries flag questions the AI couldn't answer; Textify suggests adding them to the FAQ.

### 8.10 Follow-ups &amp; tickets (Textify-only)

- Built in Textify from engine events: missed call → "call back" ticket; voicemail → ticket with recording; unresolved AI call / follow-ups from summaries → ticket; manual "create ticket" by agents.
- Fields: phone (+ contact), subject, status `OPEN/PENDING/CLOSED`, assignee/department, notes, linked calls/recordings/SMS/WhatsApp, optional due date. Customer timeline per phone number. No SLAs/escalations in v1.
- **The engine has no ticket concept.**

### 8.11 Usage &amp; billing

- FR-B1: Per call: direction, `pstnSeconds`, `trunkSeconds`, `aiSeconds`, AT `amount`, recorded, actions triggered.
- FR-B2: Per generation: `ttsCharacters`, tier. Per campaign: totals.
- FR-B3: Textify prices and deducts; engine never handles money. Events idempotent, signed, retried.

## 9. Non-functional

<table id="bkmrk-area-requirement-ava"><thead><tr><th>Area</th><th>Requirement</th></tr></thead><tbody><tr><td>Availability</td><td>Calls continue if Textify is down (fail-open balance check, queued events)</td></tr><tr><td>Isolation</td><td>Dedicated Voice box (not shared with NexGate)</td></tr><tr><td>Latency</td><td>AT callback response &lt; 500 ms; agent ring &lt; 3 s; connect after accept within one wait segment (~10–15 s max)</td></tr><tr><td>Security</td><td>SIP only from AT; API keys + HMAC; verified webhooks; SSRF protection (ARCHITECTURE §8)</td></tr><tr><td>Privacy</td><td>Recording notice, retention, own storage, signed URLs</td></tr><tr><td>Predictable cost</td><td>No runtime TTS; premium cost known in advance</td></tr><tr><td>Simplicity</td><td>REST + webhooks + Postgres + Redis + LiveKit + MinIO; no broker, no OAuth, no approvals</td></tr></tbody></table>

## 10. Phases

**Phase 0: Proof of concept (on the working AT number)**

1. Callback → `Play` a file works.
2. `GetDigits` menu works; keys arrive.
3. Wait loop: `GetDigits` wrapping `Play` hold segment, callback every segment.
4. `Dial sip:{callId}@<livekit-sip-host>` → room `call-{callId}` → browser test page hears the caller.
5. Final callback fields (duration, amount, recordingUrl).
6. Outbound call API → answer → Dial into LiveKit.
7. **Voice listening test**: same Swahili sentences in Basic candidate (Piper), Premium candidates (OpenAI, Azure), and AT `Say` for reference, played on real phones to 5–10 Tanzanians.

**Phase 1: Call center MVP**Numbers; answer modes Direct + Menu; flow engine with nodes `play, menu, collect, connect, voicemail, webhook (async), branch, sms, optout, hangup`; messages (audio upload/record + text Basic tier); waiting v1; browser agents, departments, hold/transfer; click-to-call; recording to our storage; missed/voicemail events; usage events; balance check; test call; endpoints.

**Phase 2: Bulk + intelligence**Campaigns (all FR-C); Premium voice; sync webhooks; transcripts + summaries; stats per node/campaign.

**Phase 3: AI**`ai` node / AI answer mode; knowledge base; AI tools via endpoints; handover; improvement loop; AI outbound campaigns.

**Later (already designed):** position announcements, estimated wait, callback request, whisper, wrap-up time, advanced flow editor, partner white-label voice, other countries, other QBIT products as clients.

## 11. Questions for Africa's Talking (before Phase 1)

1. Can XML `Dial` target an **external SIP URI with a custom user part** (`sip:{callId}@our-host`)? Transport, port, codecs (G.711), DTMF method?
2. AT **SIP signalling + media IPs** (firewall, LiveKit allowlist).
3. Billing for Dial-to-SIP: 17 (or 60 outbound) **+ 8.61**? Anything else?
4. Does `Dial record="true"` record Dial-to-SIP legs? Format, how long AT keeps the file?
5. **CPS** and **max concurrent calls** (in + out) for our account; can they be raised?
6. Outbound billed **per second or per started minute**? Unanswered/busy free?
7. Outbound call API: **multiple numbers per request**? Returned session ids?
8. Callback **source IPs**; any **signature** on callbacks?
9. **`GetDigits` wrapping `Play`**: keys interrupt playback? Max timeout? Any limit on how many callbacks per call (our wait loop)?
10. **`Play`**: timeout for fetching the URL; does it stream while downloading; any caching/pre-upload of media?
11. After a `Dial` ends unanswered, does AT continue to the next XML (callback after Dial)?
12. Airtel number: any issues for callers from Vodacom, Yas, Halotel, TTCL?
13. **TCRA**: licence for reselling voice; rules for automated/marketing calls (consent, hours, caller ID). In writing.

## 12. Risks

<table id="bkmrk-risk-mitigation-at-c"><thead><tr><th>Risk</th><th>Mitigation</th></tr></thead><tbody><tr><td>AT can't Dial to our SIP URI</td><td>PoC step 4 first; fallback AT SIP softphones for agents</td></tr><tr><td>Wait loop limits (callbacks/timeouts)</td><td>PoC step 3; fallback: wait inside LiveKit with ingress music (no keys while waiting)</td></tr><tr><td>Basic TTS not good enough / licence not commercial</td><td>Listening test; Basic = Premium at lower price, or audio upload only</td></tr><tr><td>AT throughput slows broadcasts</td><td>Confirm Q5; throttle; ETA</td></tr><tr><td>Swahili AI quality (Ph 3)</td><td>Real-caller test; human handover always available</td></tr><tr><td>SIP attacks / webhook SSRF</td><td>AT-only firewall; endpoint IP blocking</td></tr><tr><td>Textify outage</td><td>Fail-open, outbox retries</td></tr><tr><td>Regulation (TCRA, PDPA)</td><td>Written answers; notices; opt-out; retention</td></tr></tbody></table>

## 13. Decisions log

<table id="bkmrk-%23-decision-d1-separa"><thead><tr><th>\#</th><th>Decision</th></tr></thead><tbody><tr><td>D1</td><td>Separate **Voice Engine** (Java 21, Spring Boot), blind to users/auth/money; Textify (Go) consumes it via REST + webhooks</td></tr><tr><td>D2</td><td>**AT is the only PSTN path, both directions**</td></tr><tr><td>D3</td><td>Greeting/menu/collect/voicemail/**waiting** run on AT XML; anything **live** goes `Dial` → LiveKit</td></tr><tr><td>D4</td><td>All agents answer in the **browser via LiveKit**</td></tr><tr><td>D5</td><td>Adapters: **Telephony (AT), Media (LiveKit), TTS (Basic/Premium), AI (OpenAI)**: plain Java interfaces</td></tr><tr><td>D6</td><td>**No Python**; Ph 3 AI = OpenAI Realtime over SIP controlled from Java (Python worker only as fallback)</td></tr><tr><td>D7</td><td>Service auth: API keys in, HMAC webhooks out, secret URL + IP allowlist for AT, built-in signatures for LiveKit/OpenAI</td></tr><tr><td>D8</td><td>Communication: REST + webhooks + Postgres outbox/jobs + Redis + LiveKit data messages; **no broker in v1**</td></tr><tr><td>D9</td><td>**Engine owns media** (MinIO, ffmpeg, signed play URLs); Textify references `mediaId`</td></tr><tr><td>D10</td><td>Messages = **audio or text**; text → audio file at save / before dialing; **no AT `<Say>`, no runtime TTS**</td></tr><tr><td>D11</td><td>Voice tiers **Basic (self-hosted light model) / Premium (OpenAI or Azure)**, decided by listening test</td></tr><tr><td>D12</td><td>**One flow/action system** (11 node types) for inbound, campaigns and AI (FLOWS.md)</td></tr><tr><td>D13</td><td>Customers pick **answer-mode templates** (Direct / Menu / AI), not a flowchart editor (later maybe)</td></tr><tr><td>D14</td><td>**Waiting happens on AT** (hold loop), not in LiveKit; LiveKit audio only for agent hold</td></tr><tr><td>D15</td><td>**Webhook actions** via saved endpoints; async in Ph 1, sync in Ph 2; called by the engine directly</td></tr><tr><td>D16</td><td>**Knowledge base lives in the engine** per account; prompt-stuffing first, search tool when large</td></tr><tr><td>D17</td><td>**Tickets/contacts/screen-pop live in Textify only**</td></tr><tr><td>D18</td><td>Dedicated Voice box, separate from NexGate's LiveKit</td></tr><tr><td>D19</td><td>No approval workflows</td></tr><tr><td>D20</td><td>Consumers are **client apps** with a managed lifecycle: registered **only by the engine owner via the `voice-admin` CLI over SSH** (no public ops API) → numbers allocated → API key + webhook secret shown once, stored hashed, handed over directly into the service's env → consume → rotate → suspend (inbound keeps working by default) → remove (export, release numbers, delete after grace) (ARCHITECTURE §8.1)</td></tr><tr><td>D22</td><td>**Contacts are Textify's existing contact module**, reused as-is (same lists/groups as SMS/WhatsApp). Call center adds no contact store. The engine only receives phone numbers + variable values per campaign and caller numbers on calls; Textify does name lookup and screen pop</td></tr><tr><td>D21</td><td>Browser agents need **no extension or app install**: a normal web page with WebRTC; optional PWA later</td></tr></tbody></table>

# Ringmaster Voice Engine: Flows & Actions Reference

> The single action system used by **inbound numbers**, **campaigns** and (Phase 3) **AI**. Companion to `SPEC.md` and `ARCHITECTURE.md`. **\[VERIFY\]** = confirm with Africa's Talking (AT) in the PoC.

---

## 1. Concepts

A **flow** is a small graph of **nodes**. Each node does one thing and has named **outcomes** that point to the next node (or `end`).

<table id="bkmrk-runs-in-starts-when-"><thead><tr><th>Runs in</th><th>Starts when</th><th>Flows</th></tr></thead><tbody><tr><td>Inbound number</td><td>A call arrives</td><td>`open` flow + `closed` flow (chosen by business hours)</td></tr><tr><td>Campaign</td><td>The recipient answers</td><td>One campaign flow</td></tr><tr><td>AI (Ph 3)</td><td>AI hands over / calls a tool</td><td>Uses the same `connect`, `webhook`, `sms` actions</td></tr></tbody></table>

Customers normally don't edit nodes. Textify templates generate flows:

- **Direct**: `play greeting → connect department → voicemail`
- **Menu**: `play greeting → menu → connect department(s) → voicemail`
- **AI** (Ph 3): `ai → connect department → voicemail`
- **Campaign screen**: `menu(message + keys) → actions`

An advanced flow editor may come later; this format already supports it.

### 1.1 How AT actions map

<table id="bkmrk-at-action-used-for-p"><thead><tr><th>AT action</th><th>Used for</th></tr></thead><tbody><tr><td>`Play`</td><td>Every message callers hear (uploaded audio, generated text, hold music)</td></tr><tr><td>`GetDigits`</td><td>Menus, collecting digits, keys during waiting</td></tr><tr><td>`Record`</td><td>Voicemail</td></tr><tr><td>`Dial`</td><td>Anything **live**: into LiveKit (`sip:{callId}@…`) for agents (and AI in Ph 3)</td></tr><tr><td>`Say`</td><td>**Not used** (predictable cost, consistent voice)</td></tr></tbody></table>

Rule: **one-way listening = `Play`**, **keys = `GetDigits`**, **live two-way = `Dial` → LiveKit**.

---

## 2. Message (what callers hear)

One shape everywhere:

```json
{ "source": "audio", "media": "med_123" }
{ "source": "text",  "text": "Karibu {name}", "lang": "sw", "voice": "female", "tier": "basic" }

```

- `audio`: an uploaded/recorded file from the audio library (must be `READY`).
- `text`: turned into an audio file **when the flow is saved** (or **before dialing** for campaigns with contact variables). Never synthesized during a live call. 
    - `tier`: `basic` (self-hosted light model, fixed cost) or `premium` (OpenAI/Azure, cost shown before saving).
    - Cached by `hash(renderedText + lang + voice + tier)`.
    - Swahili normalization first: numbers, currency, dates, times to words.
    - Max 500 characters.
- At call time the engine renders `<Play url="https://voice-api…/media/{id}.mp3?t=…"/>`.
- Safety net: if a file is missing at call time → play system file `sys_sorry_try_later` and hang up; alert.

---

## 3. Variables

Written as `{name}` inside message text and webhook params.

<table id="bkmrk-group-variables-avai"><thead><tr><th>Group</th><th>Variables</th><th>Available</th></tr></thead><tbody><tr><td>System</td><td>`{caller}`, `{number}`, `{callId}`, `{date}`, `{time}`, `{key}` (last key), `{businessName}`</td><td>Everywhere</td></tr><tr><td>Contact</td><td>Any recipient column: `{name}`, `{amount}`, `{appointment_id}`…</td><td>Campaign flows only</td></tr><tr><td>Collected</td><td>`{input.<saveAs>}` from `collect`</td><td>After that node</td></tr><tr><td>Webhook result</td><td>`{result.<field>}` from a **sync** `webhook`</td><td>After that node (Ph 2)</td></tr></tbody></table>

Missing variable at runtime → empty string + warning in the call trail (validation prevents most cases).

---

## 4. Node types (complete list)

Common fields: `type`, `on` (outcome → node id or `"end"`), optional `label` (for stats).

### 4.1 `play`

Play a message.

```json
{ "type": "play", "message": {…}, "on": { "next": "menu" } }

```

Outcomes: `next`. XML: `Play`.

### 4.2 `menu`

Message + wait for **one key**.

```json
{ "type": "menu", "message": {…},
  "timeoutSec": 8, "retries": 2,
  "invalidMessage": { "source": "audio", "media": "sys_invalid_option" },
  "on": { "1": "sales", "2": "support", "0": "menu", "noInput": "support", "invalid": "support" } }

```

Outcomes: keys `0–9`, `*`, `#`, `noInput`, `invalid` (after retries). XML: `GetDigits numDigits=1` wrapping `Play`.

### 4.3 `collect`

Ask for **several digits** and save them.

```json
{ "type": "collect", "message": {…}, "saveAs": "account",
  "minDigits": 6, "maxDigits": 10, "finishOnKey": "#", "pattern": "^[0-9]{6,10}$",
  "timeoutSec": 10, "retries": 2,
  "on": { "ok": "lookup", "failed": "agent" } }

```

Outcomes: `ok`, `failed`. XML: `GetDigits`. Saved as `{input.account}`.

### 4.4 `connect`

Put the caller through **live**.

```json
{ "type": "connect",
  "target": { "department": "dep_support" },        // or {"agent": "agentRef"} | {"external": "+2557…"} | {"ai": {…}} (Ph 3)
  "ringTimeoutSec": 30,
  "wait": {
    "music": { "source": "audio", "media": "sys_hold_default" },
    "comfortEverySec": 45,
    "comfortMessage": { "source": "text", "text": "Tafadhali endelea kusubiri, mhudumu atakupokea punde.", "tier": "basic" },
    "maxWaitSec": 300,
    "keys": { "1": "voicemail" },
    "announcePosition": false,     // later
    "announceWaitTime": false,     // later
    "callbackKey": null            // later: e.g. "2" = request a callback
  },
  "whisper": null,                  // later: message only the agent hears
  "record": true,
  "on": { "noAnswer": "vm", "voicemail": "vm", "callback": "cb_ok" } }

```

(The comfort message reads: "Please keep waiting, a staff member will answer you shortly.")

Outcomes: `noAnswer` (max wait reached / nobody available), plus any outcome named in `wait.keys`. Answered = the call is live; the flow ends there.

How it runs: see §6 "Waiting loop". `external` (forward to a mobile) is allowed but off by default (extra outbound leg cost).

### 4.5 `voicemail`

```json
{ "type": "voicemail", "message": {…}, "maxSec": 60, "beep": true, "on": { "next": "end" } }

```

XML: `Play` + `Record`. Emits `voicemail.received`.

### 4.6 `webhook`

Call the customer's saved endpoint.

```json
{ "type": "webhook", "endpoint": "ep_clinic", "mode": "async",
  "params": { "event": "appointment_confirmed", "id": "{appointment_id}", "phone": "{caller}", "key": "{key}" },
  "on": { "next": "thanks" } }

```

- `async` (Ph 1): queued in the outbox, retried up to 24 h; outcome `next` immediately.
- `sync` (Ph 2): wait ≤ `timeoutMs` (default 3000). Response JSON fields become `{result.*}`; an optional `"say"` field is played next (generated on demand, then cached). Outcomes `success`, `failure` (timeout / non-2xx).
- Request: `POST` JSON `{event params…, meta: {callId, campaignId, accountRef, node, timestamp}}`, headers `X-Voice-Signature` (HMAC with the endpoint secret), `X-Voice-Idempotency-Key` (= callId + node).
- HTTPS only; private/internal IP ranges blocked (SSRF); never breaks the call.
- Emits `call.action_triggered` to Textify (for logs/dashboard).

### 4.7 `branch`

Decide by condition (engine-side, instant).

```json
{ "type": "branch",
  "cases": [
    { "if": { "hours": "closed" }, "go": "closedMsg" },
    { "if": { "var": "result.status", "equals": "paid" }, "go": "paidMsg" },
    { "if": { "var": "input.account", "matches": "^99" }, "go": "vip" }
  ],
  "on": { "default": "menu" } }

```

Operators: `equals`, `in`, `matches` (regex), `exists`, `hours: open|closed`. First match wins.

### 4.8 `sms`

Ask Textify to send an SMS/WhatsApp to the caller ("we've sent you the payment link").

```json
{ "type": "sms", "channel": "sms", "template": "payment_link", "params": { "amount": "{amount}" },
  "on": { "next": "thanks" } }

```

Engine emits `message.requested` → **Textify sends and bills** (it owns messaging and templates).

### 4.9 `optout`

Add `{caller}` to the account's do-not-call list. Outcome `next`.

### 4.10 `hangup`

Optional goodbye message, then end.

```json
{ "type": "hangup", "message": { "source": "text", "text": "Asante, kwaheri." } }

```

(The goodbye reads: "Thank you, goodbye.")

### 4.11 `ai` (Phase 3)

```json
{ "type": "ai",
  "instructions": "Jibu maswali kuhusu bei na saa za kazi.",
  "knowledgeBase": true,
  "tools": { "endpoints": ["ep_orders"], "sms": ["payment_link"] },
  "handoff": { "department": "dep_support" },
  "maxMinutes": 10,
  "on": { "handoff": "agent", "ended": "end" } }

```

(The instructions read: "Answer questions about prices and working hours.")

Runs via `Dial` → LiveKit → OpenAI Realtime (ARCHITECTURE Appendix A). The AI uses the account knowledge base, can call endpoints/sms as tools, and hands over to a department (`handoff` outcome → usually a `connect`).

---

## 5. Examples

### 5.1 Menu answer mode (generated by the template)

```json
{
  "version": 7,
  "settings": { "record": true, "recordingNotice": true },
  "open": {
    "start": "greet",
    "nodes": {
      "greet": { "type": "play", "message": { "source": "audio", "media": "med_karibu" }, "on": { "next": "menu" } },
      "menu":  { "type": "menu", "message": { "source": "text", "text": "Bonyeza 1 mauzo, 2 huduma kwa wateja.", "tier": "basic" },
                 "on": { "1": "sales", "2": "support", "noInput": "support", "invalid": "support" } },
      "sales":   { "type": "connect", "target": { "department": "dep_sales" },   "wait": { "music": { "source": "audio", "media": "sys_hold_default" }, "maxWaitSec": 180 }, "on": { "noAnswer": "vm" } },
      "support": { "type": "connect", "target": { "department": "dep_support" }, "wait": { "music": { "source": "audio", "media": "sys_hold_default" }, "maxWaitSec": 300, "keys": { "1": "voicemail" } }, "on": { "noAnswer": "vm", "voicemail": "vm" } },
      "vm": { "type": "voicemail", "message": { "source": "text", "text": "Acha ujumbe baada ya mlio.", "tier": "basic" } }
    }
  },
  "closed": {
    "start": "closedMsg",
    "nodes": {
      "closedMsg": { "type": "play", "message": { "source": "text", "text": "Tumefunga kwa sasa. Tunafungua saa mbili asubuhi.", "tier": "basic" }, "on": { "next": "vm" } },
      "vm": { "type": "voicemail", "message": { "source": "text", "text": "Acha ujumbe baada ya mlio.", "tier": "basic" } }
    }
  }
}

```

(The Swahili messages read: "Press 1 for sales, 2 for customer service." · "Leave a message after the tone." · "We're closed right now. We open at eight in the morning." Tanzanian time "saa mbili asubuhi" is 08:00.)

### 5.2 Campaign: appointment reminder

```json
{
  "start": "msg",
  "nodes": {
    "msg": { "type": "menu",
             "message": { "source": "text", "text": "Habari {name}, una miadi kesho saa {time_slot}. Bonyeza 1 kuthibitisha, 2 kuongea na mhudumu, 3 kupata ujumbe wa SMS, 9 kujiondoa.", "tier": "premium" },
             "on": { "1": "confirm", "2": "agent", "3": "sms", "9": "out", "noInput": "msg", "invalid": "msg" } },
    "confirm": { "type": "webhook", "endpoint": "ep_clinic", "mode": "async",
                 "params": { "event": "confirmed", "id": "{appointment_id}", "phone": "{caller}" }, "on": { "next": "thanks" } },
    "thanks": { "type": "hangup", "message": { "source": "text", "text": "Asante, tumethibitisha miadi yako.", "tier": "basic" } },
    "agent":  { "type": "connect", "target": { "department": "dep_reception" }, "wait": { "music": { "source": "audio", "media": "sys_hold_default" }, "maxWaitSec": 120 }, "on": { "noAnswer": "vm" } },
    "vm":     { "type": "voicemail", "message": { "source": "audio", "media": "med_vm" } },
    "sms":    { "type": "sms", "channel": "sms", "template": "appointment_details", "params": { "id": "{appointment_id}" }, "on": { "next": "smsOk" } },
    "smsOk":  { "type": "hangup", "message": { "source": "text", "text": "Tumekutumia ujumbe. Kwaheri.", "tier": "basic" } },
    "out":    { "type": "optout", "on": { "next": "bye" } },
    "bye":    { "type": "hangup", "message": { "source": "text", "text": "Umeondolewa kwenye orodha. Kwaheri.", "tier": "basic" } }
  }
}

```

(The Swahili messages read: "Hello {name}, you have an appointment tomorrow at {time\_slot}. Press 1 to confirm, 2 to talk to a staff member, 3 to get an SMS, 9 to opt out." · "Thank you, we've confirmed your appointment." · "We've sent you a message. Goodbye." · "You've been removed from the list. Goodbye.")

---

## 6. Waiting loop (`connect` to a department)

Waiting happens **on AT**, not in LiveKit. Reasons: keys work while waiting, no trunk cost while waiting, announcements are just files.

```
connect node:
  1. engine marks call WAITING_AGENT, adds it to the department's waiting order (oldest first)
  2. rings available agents in their browsers (LiveKit lobby data message)
  3. returns XML:  <GetDigits numDigits="1" timeout="1" callbackUrl=".../wait/{callId}">
                      <Play url="{hold segment ~10–15 s}"/>
                   </GetDigits>
AT calls back after each segment (or on a key):
  - agent accepted?       → <Dial phoneNumbers="sip:{callId}@sip.voice…" record="true"/>   → live
  - key pressed?          → outcome from wait.keys (e.g. voicemail)
  - waited ≥ maxWaitSec?  → outcome noAnswer
  - else                  → next segment: music, or comfortMessage every comfortEverySec

```

- Agent accepts mid-segment → their screen shows "Connecting caller…" until the segment ends (≤ ~15 s; keep segments short).
- Agent becomes available (or finishes a call) → engine rings them with the oldest waiting caller.
- Later features slot into the same loop: position announcement (pre-generated number files), estimated wait, callback request, whisper, wrap-up time.
- **\[VERIFY\]** GetDigits-wrapping-Play interrupt behaviour, timeouts, callback count limits. Fallback if AT can't do this: dial into LiveKit immediately and play hold music via LiveKit Ingress (no keys while waiting).

In-call hold/transfer by the agent happens inside LiveKit (hold music via LiveKit Ingress).

---

## 7. Validation rules (on save; errors shown in Textify, no approvals)

1. Every outcome points to an existing node or `end`; `start` exists.
2. ≤ 50 nodes per flow; menus ≤ 12 options; ≤ 3 retries.
3. Variables exist in scope (contact columns only in campaigns; `{input.x}` only after its `collect`; `{result.x}` only after a sync `webhook`).
4. `connect` targets, endpoints, media, sms templates belong to the same account; media `READY`; text generation succeeded.
5. Message text ≤ 500 characters.
6. Inbound numbers must have both `open` and `closed` flows (template provides defaults).
7. `ai` nodes only when Phase 3 is enabled for the account.

Runtime guards: max **25 node visits per call** (loop guard → system goodbye + hangup); max call length per flow (default 30 min).

---

## 8. Versioning

- Every save → new immutable version (`flow_versions`). The number points to its current version.
- Calls store the version they ran.
- **Running campaigns are pinned** to the version they started with; edits apply to new campaigns only.
- Rollback = point the number to an older version.

---

## 9. Execution (engine)

- Each AT callback URL carries `callId` + node: `/hooks/at/{secret}/step/{callId}/{nodeId}` (and `/wait/{callId}` for the waiting loop).
- Call **context** (variables, retries, visit count) stored as JSON on the call row; no in-memory sessions.
- One small Java class per node type: `NodeExecutor.execute(node, ctx) → Xml | NextNode`. Engine-side nodes (`webhook` async, `branch`, `sms`, `optout`) run instantly and continue to the next node within the same callback.
- **Never fail the caller**: any exception → `Play sys_sorry_try_later` + hang up; log + alert. Never return 5xx to AT.
- **Call trail**: each node visited, key pressed and action result is recorded in `call_events` → per-node stats ("60% pressed 1, 12% pressed 9").
- **Test call**: `POST /v1/numbers/{id}/test-call {toE164}` or `/campaigns/{id}/test-call` runs the flow against the customer's own phone (contact variables from a sample row).

---

## 10. Cost per node

<table id="bkmrk-node-cost-play%2C-menu"><thead><tr><th>Node</th><th>Cost</th></tr></thead><tbody><tr><td>`play`, `menu`, `collect`, `voicemail`, `hangup`</td><td>Call minutes only (on AT)</td></tr><tr><td>Text messages</td><td>Basic: free · Premium: characters × rate, known at save / campaign creation</td></tr><tr><td>`connect` (waiting)</td><td>Call minutes only (still on AT)</td></tr><tr><td>`connect` (talking)</td><td>Minutes + SIP trunk 8.61/min; `external` = extra outbound leg</td></tr><tr><td>`webhook`, `branch`, `optout`</td><td>Free</td></tr><tr><td>`sms`</td><td>Textify's SMS/WhatsApp price</td></tr><tr><td>`ai`</td><td>AI per-minute price (Ph 3)</td></tr></tbody></table>

---

## 11. Phase availability

<table id="bkmrk-feature-phase-play%2C-"><thead><tr><th>Feature</th><th>Phase</th></tr></thead><tbody><tr><td>`play`, `menu`, `collect`, `connect` (department/agent), `voicemail`, `webhook` async, `branch`, `sms`, `optout`, `hangup`</td><td>1</td></tr><tr><td>Waiting: music, comfort message, max wait, keys while waiting</td><td>1</td></tr><tr><td>Basic voice tier</td><td>1</td></tr><tr><td>Premium voice, `webhook` sync, campaigns</td><td>2</td></tr><tr><td>`ai` node, knowledge base, AI tools</td><td>3</td></tr><tr><td>Position / wait-time announcements, callback request, whisper, wrap-up, `connect.external` on by default, advanced flow editor</td><td>Later</td></tr></tbody></table>