Scripts & Blocks Reference

Client app developers: read api/5-flow-reference.md instead. It is checked against the engine's code. This file is the engine's design notes (worker commands, execution, costs); some names here are older than the code (e.g. Condition test.kind, Go to target).

The one system that decides what happens on incoming calls, call-outs (broadcasts, campaigns) and AI conversations. Built by customers in the client app's Voice Studio (block canvas). Companion to SPEC.md, ENGINE.md, ARCHITECTURE.md, VOICE-WORKER.md, CLIENT-APP-ACTIONS.md, SCENARIOS.md, EXAMPLES.md. SIP trunk only (2026-11-12). Blocks run as commands for the voice worker inside the call's LiveKit room; no AT XML. The Voice API version of this file is in archive/.

1. Concepts

A script is a graph of blocks. Each block does one thing and has named exits wired to the next block. An unconnected exit ends the call (with the account's goodbye if the call is still up).

One direction per script. To reuse content in the other direction, duplicate the script, or put the shared part in its own script and use Transfer to script.

Not scripts: OTP (system message), click-to-call (live call), callback (a follow-up task).

1.1 Layers

Message type (Incoming / OTP / Broadcast / Campaign / Click-to-call)   → lane, recording rule, limits
  └─ Script version (blocks + settings)                                → what happens
      └─ Block (announcement, prompt, waiting room, exits)             → one step
          └─ Rendering (variables filled, text → cached audio)        → what the caller hears
              └─ Worker commands (play, ask, record, wait, connect, ai) → what the worker does in the room

1.2 How blocks become worker commands

Rules (full protocol: VOICE-WORKER.md):

• A turn reply is never empty: at least one command, or the system goodbye.
• An announcement plays before the prompt and can't be interrupted; the prompt can (barge-in).
• A prompt can have several parts (text and audio mixed); the worker plays them in order.
• Each turn has a number; a late or repeated result for an old turn changes nothing (the engine re-sends the current commands).

2. Messages

Same shape everywhere:

{ "src": "text",    "text": "Habari {name}", "lang": "sw" }
{ "src": "record",  "media": "med_123" }
{ "src": "upload",  "media": "med_456" }
{ "src": "library", "media": "sys_hold_default" }

(The Swahili example reads "Hello {name}".)

• text → filled, turned into words where needed (§3), then turned into audio by the engine's TTS and cached: the same text and voice is generated once and reused by every call (ARCHITECTURE §5.2). Max 500 characters.
• Voice and speed (who speaks, how fast). The client sets them where they want; the nearest wins:
  1. on the message: { "src": "text", "text": "…", "voice": "cedar", "speed": 1.2 }
  2. on the block: "voice": "cedar", "speed": 0.9 next to type (applies to every message the block says, including its retries and system lines)
  3. in the script: settings.voice ("cedar" or per language { "sw": "marin", "en": "cedar" }), settings.speed (number or per language)
  4. the engine default (ringmaster.tts.voice, speed 1.0)
• Speed runs 0.7 (slower) to 1.5 (faster), rounded to steps of 0.05; 1.0 is normal.
• Voices (OpenAI): female-sounding marin, coral, shimmer, sage; male-sounding cedar, ash, echo, ballad, verse.
• The audio cache keys on voice and speed (cedar@1.2), so changing either makes new audio; normal speed keeps the plain name.
• The AI Agent sets its own: agent.voice and agent.speed (agent.openaiVoice is the older name for the voice).
• record / upload / library → the worker plays the file; it must be READY.
• A prompt is a message or a list of messages: "msg": [ {…text…}, {…library…} ].
• Missing media at call time → system "sorry" message, then goodbye; alert.

3. Variables and metadata

Variables are typed values that blocks read aloud, send, or test. Written as {name}.

• Incoming calls only know {caller} (+ the client app's contact if found). An Incoming script must not depend on call-out variables.
• Missing value at runtime → empty string + warning in the call trail (validation prevents most cases; the call-out form refuses people without values).
• Variable types (vars[].type): text (default), money, number, date, digits. Spoken forms are made by the engine for the call's language before TTS: money 45000 → "shilingi elfu arobaini na tano" (sw) / "forty-five thousand shillings" (en); digits 0712… → read one by one; date → "tarehe 20 Novemba" (sw). In Send request URLs/bodies and App action params the raw value is used. [VERIFY: whether the TTS voice reads digits well enough to skip the conversion]

Metadata: opaque JSON the client attaches to a call or call-out ({"orderId": 991}). Never spoken, never sent to Send request unless mapped into a variable. Echoed back in every event as meta.client.

4. Script document

{
  "scriptId": "scr_debt", "version": 3, "direction": "OUTGOING",
  "settings": { "record": "policy", "maxCallMin": 15, "language": "sw", "voice": { "sw": "marin", "en": "marin" } },
  "vars": [ { "name": "name", "type": "text" }, { "name": "amount", "type": "money" } ],
  "start": { "out": "say_1" },
  "blocks": {
    "say_1":  { "type": "say", "msg": { "src": "text", "text": "Habari {name}. Deni lako ni {amount}." }, "x": { "next": "menu_1" } },
    "menu_1": { "type": "menu", "msg": { "src": "text", "text": "Bonyeza 1 au sema 'lipa' kulipa sasa, 2 kuongea na mhasibu, 9 kujiondoa." },
                "options": [ { "k": "1", "id": "pay",   "label": "Lipa",     "words": ["lipa", "kulipa", "malipo"] },
                             { "k": "2", "id": "staff", "label": "Mhasibu",  "words": ["mhasibu", "mtu", "kuongea"] },
                             { "k": "9", "id": "out",   "label": "Jiondoe",  "words": ["jiondoa", "acha"] } ],
                "speech": { "match": "words" }, "timeoutSec": 8,
                "x": { "pay": "pay_1", "staff": "connect_1", "out": "optout_1", "noInput": "bye_1", "notUnderstood": "menu_1" } },
    "pay_1":  { "type": "app_action", "action": "pay", "params": { "amount": "{amount}", "wallet": "M-Pesa" },
                "wr": { "preset": "Payment" }, "x": { "paid": "bye_2", "failed": "sms_1", "timeout": "sms_1" } }
  }
}

Each block: type, block settings, optional before (announcement message), optional wr (Waiting Room), x (exit → block id; missing = end). (The Swahili lines read: "Hello {name}. Your debt is {amount}." · "Press 1 or say 'pay' to pay now, 2 to talk to the accountant, 9 to opt out.")

5. Blocks (complete list: 17)

⚠ = needs approval (the whole script version).

Speak

Ask: keys, speech, or both

All Ask blocks share one rule set:

• Keys always work. A block also listens for speech when it has a speech setting. In the Studio this follows Twilio's pattern: connecting the block's "said something" exit turns speech on.
• Barge-in: the caller can press a key or start talking during the prompt; the prompt stops.
• Type-ahead: keys pressed during an interruptible Say carry into the next Ask.
• No input after timeoutSec (counted from the end of the prompt) → retry (up to retries, max 3) → noInput.
• Not understood (speech that matches nothing) → "Samahani…" ("Sorry…") retry → notUnderstood. Only on blocks that listen for speech.
• Speech is cut by the worker (VAD) and transcribed by the engine (gpt-transcribe, the block's language, the options' words as hints). The text is saved as {heard} and in the timeline (SPEECH).

Connect

Integrate

App action replaces "Textify action" and "Pay now": the engine has no built-in SMS or payment actions; each client app registers its own (CLIENT-APP-ACTIONS.md). An action the app marks risky (e.g. pay) makes the block ⚠.

Send request details:

• Request: method + URL with variables; header X-Voice-Signature (HMAC-SHA256 of timestamp + body with the account's signing secret), X-Voice-Timestamp, X-Voice-Idempotency-Key (= callId + block).
• Body variables filled per call; meta object always added (callId, callOutId, block, meta.client).
• Reply: 2xx = success (empty 200 = success, nothing mapped); 4xx/5xx = failure; no answer within timeout (default 10 s, max 15 s while the caller waits) = timeout. JSON paths mapped to variables (data.status → {status}).
• HTTPS only; DNS-resolved private / loopback / link-local / metadata IPs blocked; no redirects; response ≤ 64 KB.
• Send and continue: queued, retried up to 24 h; never delays the caller.
• Wait for callback (slow systems): the body includes {callback_url}, a one-time signed URL for this call + block. The caller waits in the Waiting Room until the business POSTs the result there (mapped like a reply) or the give-up time passes. If the caller already hung up, the result is still stored and reported (call.block).
• Reply conveniences: a top-level say string can be spoken as-is ({reply.say}, made into audio at the turn); a top-level items array feeds Pick from list. Recommended reply contract: EXAMPLES.md §0.1.
• Secrets (tokens, passwords) stored encrypted; never shown again in full.

Logic

Transfer to script works like a function call into a shared script (D52):

• A shared script has direction SHARED and start exit in. It can't be put on a number or a call-out.
• It declares inputs (e.g. ["account"]); the Transfer block must fill each one (checked at publish). Inside, they are {in.account}; system variables and live facts work as usual.
• It ends at Return blocks ({"type": "return", "exit": "done"}); the set of return exits is the script's interface and shows on the Transfer block as exits. Variables it sets ({input.*}, {answer.*}, reply variables) are visible to the caller after the return.
• Pinned version (editing the shared script changes nothing until the pin moves), nesting ≤ 5, and no script may reach itself (checked at publish).

End

5a. Outcome tag & sensitive flag (any block)

• Outcome tag (optional, block setting "Count as"): when a call reaches the block, the tag is added to the call (paid, booked, promised, wants_callback…). Used by call-out funnels and conversion. Automatic tags: App action outcomes the client app maps with countAs (e.g. paid), Opt out → opted_out, AI Agent goal → goal, Answered by machine → machine.
• Sensitive (Collect number): the value is usable in the call (e.g. sent to a verify request) but stored and shown as ••••, never in exports or webhooks. OTP codes are always sensitive. A sensitive Collect never listens for speech (no transcript of a PIN).

6. Announcement (any block)

• Optional message that plays once when the block starts ("Mazungumzo haya yanarekodiwa…", "This conversation is being recorded…"; "Tafadhali sikiliza kwa makini", "Please listen carefully").
• Added from block settings (+ Add announcement). Text, record, upload or library.
• Not played again when the block repeats after a wrong key.
• Plays before the prompt and can't be interrupted.

Automatic, not editable per block:

• Recording notice: once per call, right before the first recorded part (account wording).
• AI-voice disclosure (D53): OpenAI's usage policy requires telling callers that a voice is AI-generated, and every text prompt uses OpenAI's voice. So a call that will play generated voice starts with one short line, default "Habari, unazungumza na huduma ya kidijitali ya {business}." ("Hello, you're speaking with {business}'s digital service."), editable per account (settings.aiDisclosure). Played once, before the first generated prompt; skipped when the call plays only recorded audio. The AI Agent adds nothing extra.

7. Waiting Room (Connect, Send request "wait", App action "wait")

What the caller hears and can do while the block is busy. It runs in the call's room: the worker plays it, the engine decides when it ends.

How it runs:

engine → worker: wait { start, music (loop), messages [{audio, everySec}], keys allowed after N s }
worker: plays start, loops music, plays updates; keys → result
engine, when something changes:
  work done (agent accepted, reply arrived, outcome reported) → wake worker → next block's commands
  queue position changed                                       → wake worker → new wait (fresh position message)
  give-up timer (JobRunr) fires                                → wake worker → give-up exit
  option key (allowed only after "offer options after N s")    → that option exit

• Accept is instant: the caller is already in the room, so an agent who accepts is talking within a second (the Voice API needed up to 15 s).
• The Waiting Room says only what the author writes: announce the option keys in wr.start or wr.updates (e.g. "Bonyeza 9 kuacha ujumbe", "Press 9 to leave a message"). Options with neither give the warning WAIT_OPTIONS_UNANNOUNCED.

8. AI

8.0 AI language

Each AI Agent, and each Ask block with speech, has a language setting chosen by the customer:

• Fixed: listen and speak only the listed language(s), e.g. ["sw"] (Kiswahili only) or ["sw","en"]; a caller speaking another language is answered in the first one.
• Auto: detect the caller's language from their speech and answer in it, switching if the caller switches.

"language": { "mode": "FIXED", "languages": ["sw"] }      // or { "mode": "AUTO", "default": "sw" }

The detected language is saved as {ai.language} (and on the call), so later blocks and the client app know it. With AUTO and sw/en it also sets {lang}, so later system messages follow the caller.

8.1 Smart matching (Ask blocks)

• Menu, Poll and Pick with "speech": {"match": "smart"}: after word matching fails, the engine asks the AI which option the caller meant, given the options' labels and words. Answer below minConfidence (default 0.5) → notUnderstood.
• {heard} holds the text; the timeline records MATCH {option, by: key | words | smart, confidence}.
• Without an OpenAI key (openai.api-key=sim-not-set) the engine uses a simulated adapter: word matching only; the turn driver sends speech as text.

8.2 AI Agent

Designed in the Agent canvas (double-click the block).

{
  "goal": "Book an appointment", "persona": "…", "voice": "marin",
  "guard": "Never promise prices not in the knowledge base", "maxMin": 5,
  "kb": { "items": [ { "q": "Saa za kazi", "a": "…" } ], "docs": ["bei_2026.pdf"], "site": "https://…" },
  "fields": [ { "name": "patient_name", "type": "Text", "desc": "Jina lako ni nani", "req": "Required" },
              { "name": "date", "type": "Date", "req": "Required" } ],
  "tools": [ { "type": "Use saved request", "name": "book_appointment", "req": "request_1" },
             { "type": "App action", "name": "send_confirmation", "action": "send_sms", "params": { "text": "Miadi yako ni {ai.date}" } } ],
  "start": "s1",
  "anytime": [ { "kind": "person", "to": "s7" }, { "kind": "upset", "to": "s7" } ],
  "stages": {
    "s1": { "type": "greet", "say": "Habari! Nikusaidie nini leo?", "leave": [ { "kind": "always", "to": "s2" } ] },
    "s2": { "type": "understand", "leave": [ { "kind": "question", "to": "s3" }, { "kind": "intent", "to": "s4" } ] },
    "s3": { "type": "answer", "kb": true, "leave": [ { "kind": "always", "to": "s2" } ] },
    "s4": { "type": "collect", "collect": ["patient_name", "date"], "leave": [ { "kind": "done", "to": "s5" } ] },
    "s5": { "type": "act", "actions": ["book_appointment", "send_confirmation"], "leave": [ { "kind": "success", "to": "s6" }, { "kind": "failed", "to": "s7" } ] },
    "s6": { "type": "confirm", "leave": [ { "kind": "always", "to": "exit:goal" } ] },
    "s7": { "type": "handover", "ho": {
              "ok":     { "say": "Nakuunganisha na mhudumu sasa hivi.", "to": "exit:human" },
              "busy":   { "say": "Wahudumu wote wako kwenye simu…",     "to": "exit:human" },
              "none":   { "say": "Tutakupigia ndani ya saa moja.",       "to": "exit:cb" },
              "closed": { "say": "Tumefunga sasa… acha ujumbe.",         "to": "exit:vm" } } }
  }
}

(The Swahili reads: "What are your working hours" · "What is your name" · "Your appointment is on {ai.date}" · "Hello! How can I help you today?" · "I'm connecting you to an agent right now." · "All agents are on calls…" · "We'll call you back within an hour." · "We're closed now… leave a message.")

How it runs:

• The engine sends ai {instructions, voice, tools, stage, language}; the worker runs a LiveKit Agents session with the OpenAI Realtime model (gpt-realtime-2.1; gpt-live-1 to be tested). Tool calls come to the engine as turn results; the engine runs them and replies with the result. Stage changes update the session's instructions. ARCHITECTURE Appendix A.1.
• Tools the model gets: leave_stage(kind) (the stage's non-automatic kinds + anytime kinds), save_field(name, value) in Collect stages (→ {ai.<name>}; all collected → done), search_knowledge(query) in Answer stages, the stage's tools, set_language(code) with AUTO language, handover(), end_call(reason: goal_done | caller_done). A Take-action stage leaves success once all its actions worked, failed at the first failure.
• Handover uses live facts of the department behind the human exit: closed → closed, nobody online → none, someone free → ok, all busy → busy; each row {say, to} (defaults: ok/busy → human, none → cb, closed → vm).
• Limits: maxMin (1–30, default 5) → time; AI not starting within 20 s or failing → err. After the conversation: transcript (speaker-separated) + summary + collected fields → summary.ready.
• Knowledge: kb.items (Q/A), kb.docs (ids from POST /v1/accounts/{id}/kb-docs: .txt, .md, .pdf with text, ≤ 10 MB), kb.site (https page, read once when the version is saved). Built per script version in the background; searched with Postgres full-text (all words first, then any word).

Rule of thumb for tools vs exits: if the AI needs the result to keep talking (check balance, book) → a tool inside the agent; if the AI's part is finished (transfer, pay, goodbye) → an exit to a block.

8a. Block settings (exact JSON)

Behaviour:

• Signing: when the account has a request signing secret every request carries X-Voice-Timestamp, X-Voice-Signature = hex HMAC-SHA256(secret, timestamp + "." + body) (same scheme as webhooks), X-Voice-Idempotency-Key = {callId}:{block}:{visit}, X-Voice-Call-Id. JSON bodies get meta {callId, callOutId, block, client}; form bodies get meta.callId, meta.block.
• Reply: 2xx = success; other status, redirect (never followed), > 64 KB, blocked address or network error = failure; no reply in time = timeout. {reply.code} is always set; top-level say → {reply.say}; top-level items feeds Pick from list.
• Waiting: a reply within ~1 s is used at once, with no Waiting Room; otherwise the caller goes to the Waiting Room and the engine wakes the worker when the reply arrives.
• Wait for callback: the system posts the result to {callback_url} (POST /hooks/resume/{token}, one-time, valid until the give-up time). "result" (or "status") = failure / failed / error / declined / cancelled → failure exit, anything else → success; the body is mapped with map. After the caller left, the result is stored and sent as call.block with late: true. Answers: 200, 404 unknown, 409 already answered, 410 expired.
• Send and continue: queued (stored encrypted), retried on 5xx / 408 / 429 / timeout / network errors at 30 s, 2 min, 10 min, 30 min, 1 h, 2 h, 4 h… for up to 24 h; a final failure sends request.failed.
• Secrets: auth.token, auth.password, auth.value are encrypted when the version is saved (enc: prefix) and decrypted only to send.
• App actions: CLIENT-APP-ACTIONS.md (event, answer, outcomes, timeouts).
• Sensitive Collect: the value is kept encrypted in the call state, readable by later blocks ({input.pin}), shown as •••• in keys, timeline, webhooks and any spoken-text log.

9. Validation (on save / before publish)

Loops are allowed: any exit may point back to an earlier block or to the same block ("press 0 to repeat", story → back to the menu). The loop guard ends the call (LOOP_GUARD) only after maxBlockVisits (25) blocks in a row without the caller pressing a key, speaking or recording, so a broken say → goto → say loop stops, while a caller going round a menu is never cut off. Call length is capped separately (maxCallMinutes).

1. Start exits exist for the direction; every exit points to an existing block or is intentionally empty.
2. ≤ 60 blocks; menus ≤ 10 options; retries ≤ 3; text ≤ 500 characters.
3. Variables exist in scope: call-out variables only in Outgoing scripts; {input.x} after its Collect; {answer.x} after its Ask question; reply variables after their Send request; {ai.*} after the AI Agent.
4. Media READY; departments and saved requests belong to the account; App actions exist in the client app's list with every required parameter (CLIENT-APP-ACTIONS.md §4).
5. Send request: HTTPS URL, valid JSON body, auth complete.
6. Speech: a Menu, Poll or Pick with speech and match: words needs words on every option (warning OPTION_WITHOUT_WORDS); a sensitive Collect can't listen for speech.
7. Transfer to script: the target version exists, is runnable and SHARED; every declared input is filled; the block's exits match the target's return exits; no cycle; nesting ≤ 5. Return blocks only in SHARED scripts.
8. AI Agent: start stage set, every leave-when connected or intentionally empty, at least one exit reachable.
9. Approval flag computed: any ⚠ block → version status Waiting approval; else Ready.
10. Text to audio: every fixed text is listed in the version's texts so its audio is made at publish (ARCHITECTURE §5.2).

10. Versioning & approval

• Draft (editable) → Publish → immutable version vN with status Ready or Waiting approval → Approved / Rejected (reason).
• Numbers, call-outs and Transfer to script pin a version. Editing creates vN+1; running call-outs keep vN.
• A call-out on a version waiting approval is saved and starts automatically when approved.
• Rollback = point the number to an older approved/ready version.
• Approval lives in the client app; the engine only stores versions the app pushes as runnable (plus drafts for test calls to verified test numbers).

11. Execution (engine)

• Every result from the worker is one turn: POST /internal/calls/{callId}/turn (VOICE-WORKER §3). The engine locks the call, applies the result to the block that asked, runs blocks until one needs the caller, and answers with commands.
• Call context (variables, retries, visits, current block, current turn, AI state) stored as JSON on the call row; no in-memory sessions, so any engine instance can answer any turn, and a restarted worker resumes the call.
• One executor class per block type: BlockExecutor.execute(block, ctx) → Step (next, jump, wait, end) plus commands. Instant blocks (Condition, Go to, Transfer to script, Opt out, App action that doesn't wait, Send and continue) run inside the same turn and continue.
• Never fail the caller: exception → system "sorry" + goodbye; log + alert; never an empty turn.
• Call trail: every block visited, key, speech, match, request result and AI stage stored in call_events → per-block stats ("60% pressed 1", "25% said it").
• Guards: 25 block visits per call without caller input, max call length, max wait, voicemail length, 5 script transfers.

12. Cost per block