# NexGate — Private Chat & Calls Flow # Private Chat & Calls Phase 2 ### Production Architecture **NexGate / QBIT SPARK | Version 1.0** *Ejabberd · WebRTC Calls · Voice & Video · MessagePack · Coturn · Message Interactions* --- ## Table of Contents 1. [NexGate Chat Roadmap](#1-nexgate-chat-roadmap) 2. [What Is Phase 2](#2-what-is-phase-2) 3. [What We Are Building](#3-what-we-are-building) 4. [Full Architecture](#4-full-architecture) 5. [Ejabberd — The Transport Backbone](#5-ejabberd--the-transport-backbone) 6. [Ejabberd ↔ Spring Boot Bridge](#6-ejabberd--spring-boot-bridge) 7. [Authentication Flow](#7-authentication-flow) 8. [Message Flow — Phase 2](#8-message-flow--phase-2) 9. [Voice Calls](#9-voice-calls) 10. [Video Calls](#10-video-calls) 11. [Coturn — TURN Relay](#11-coturn--turn-relay) 12. [MessagePack Encoding](#12-messagepack-encoding) 13. [Broadcast Channels](#13-broadcast-channels) 14. [MQTT — Mini Apps Foundation](#14-mqtt--mini-apps-foundation) 15. [Message Interactions](#15-message-interactions) 16. [Docker Deployment](#16-docker-deployment) 17. [Database Schema](#17-database-schema) 18. [Commerce Stanzas & Custom Namespaces](#18-commerce-stanzas--custom-namespaces) 19. [Build Order](#19-build-order) --- ## 1. NexGate Chat Roadmap Before any code is written — understand the full journey. Three stages. Each builds on the previous. --- ### Stage 1 — Local Experiments (Terminal Only, No Coding) ``` Goal: understand the tools before building with them Duration: 1 week Output: confidence, not code Method: terminal only — Docker CLI, curl, sendxmpp NO Android app NO Android Studio NO Java project NO NexGate codebase Everything in this stage is throwaway Run it locally on your Xubuntu machine No production VPS involved ``` #### Tools to Install First ```bash # XMPP CLI client sudo apt install sendxmpp # WebSocket CLI client wget https://github.com/vi/websocat/releases/download/v1.12.0/websocat.x86_64-unknown-linux-musl chmod +x websocat.x86_64-unknown-linux-musl sudo mv websocat.x86_64-unknown-linux-musl /usr/local/bin/websocat # STUN test client sudo apt install stuntman-client # Network packet inspection sudo apt install tcpdump # Python + MessagePack (for experiment 7) pip3 install msgpack --break-system-packages # curl and docker — already installed ✅ ``` --- #### Experiment 1 — Ejabberd Running Locally ``` Goal: get Ejabberd running, send one message via terminal Success: message delivered, logs confirm routing ``` ```bash # Start Ejabberd docker run -d \ --name ejabberd \ -p 5222:5222 \ -p 5280:5280 \ -p 5285:5285 \ ghcr.io/processone/ejabberd # Wait for startup sleep 15 # Check it's running docker exec ejabberd ejabberdctl status # Expected: Node ejabberd@localhost is started # Create two test users docker exec ejabberd ejabberdctl register alice nexgate.com password123 docker exec ejabberd ejabberdctl register bob nexgate.com password123 # Verify users exist docker exec ejabberd ejabberdctl registered_users nexgate.com # Expected output: # alice # bob # Send message alice → bob (no app needed!) docker exec ejabberd ejabberdctl send_message \ chat alice@nexgate.com bob@nexgate.com \ "" "Habari Bob! Kutoka terminal" # Watch Ejabberd logs — see message routing docker logs ejabberd --tail 30 # Open dashboard in browser # http://localhost:5280/admin # admin / password (default) # See users, sessions, statistics ``` ``` What you learn: How Ejabberd starts and configures ejabberdctl is your management CLI Messages route without any app Dashboard shows what's happening Logs show every routing decision ``` --- #### Experiment 2 — REST API (How Spring Boot Will Talk to Ejabberd) ``` Goal: talk to Ejabberd via HTTP — same way Spring Boot will Success: curl commands work, responses received ``` ```bash # Send message via REST API (this is exactly what Spring Boot does) curl -s -X POST http://localhost:5285/api/send_message \ -H "Content-Type: application/json" \ -d '{ "from": "alice@nexgate.com", "to": "bob@nexgate.com", "body": "Kutoka curl — kama Spring Boot!" }' | python3 -m json.tool # Get all connected users curl -s -X POST http://localhost:5285/api/connected_users \ -H "Content-Type: application/json" \ -d '{}' | python3 -m json.tool # Get registered users curl -s -X POST http://localhost:5285/api/registered_users \ -H "Content-Type: application/json" \ -d '{"host": "nexgate.com"}' | python3 -m json.tool # Create a MUC group chat room curl -s -X POST http://localhost:5285/api/create_room \ -H "Content-Type: application/json" \ -d '{ "name": "nexgate-test-room", "service": "conference.nexgate.com", "host": "nexgate.com" }' | python3 -m json.tool # List active MUC rooms curl -s -X POST http://localhost:5285/api/muc_online_rooms \ -H "Content-Type: application/json" \ -d '{"service": "conference.nexgate.com"}' | python3 -m json.tool # Kick a user session curl -s -X POST http://localhost:5285/api/kick_session \ -H "Content-Type: application/json" \ -d '{ "user": "alice", "host": "nexgate.com", "resource": "test", "reason": "Test kick" }' | python3 -m json.tool ``` ``` What you learn: Every curl call = what Spring Boot RestTemplate does REST API is how NexGate backend controls Ejabberd Port 5285 = admin API (internal only in production) All operations possible without any mobile app ``` --- #### Experiment 3 — sendxmpp (Connect as XMPP User) ``` Goal: connect as a real XMPP user from terminal Success: send/receive messages between two terminal sessions ``` ```bash # Terminal 1 — send message as alice echo "Habari Bob! Ninatuma kutoka terminal" | sendxmpp \ --username alice \ --password password123 \ --host localhost \ --port 5222 \ --domain nexgate.com \ --tls-ca-path /dev/null \ --insecure \ bob@nexgate.com # Watch Ejabberd logs in another terminal: docker logs ejabberd -f # See stanzas flowing in logs: # Received message from alice@nexgate.com # Routing to bob@nexgate.com # Delivered ✅ # Send typing indicator (composing stanza) # sendxmpp handles this via --chat-state flag echo "Ninaandika..." | sendxmpp \ --username alice \ --password password123 \ --host localhost \ --port 5222 \ --domain nexgate.com \ --insecure \ --chat-state \ bob@nexgate.com ``` ``` What you learn: XMPP login flow from client perspective Stanza routing in Ejabberd logs How typing indicators flow What mobile app will do — terminal does it first ``` --- #### Experiment 4 — Watch Raw XMPP Stanzas ``` Goal: see actual XML stanzas flowing over the wire Success: raw XMPP XML visible in terminal ``` ```bash # Terminal 1 — watch all XMPP traffic sudo tcpdump -i lo -A port 5222 2>/dev/null | grep -A5 " # Test stanza # ``` ``` What you learn: What XMPP stanzas actually look like on wire Difference between connection, auth, message stanzas How namespaces appear in real traffic Visual confirmation of everything in the docs ``` --- #### Experiment 5 — Spring Boot Auth Bridge ``` Goal: Ejabberd calls Spring Boot to validate users Success: Spring Boot approves/rejects Ejabberd connections Note: minimal Spring Boot — one endpoint only, H2 in-memory DB ``` ```bash # Step 1: Create minimal Spring Boot project # ONE controller, ONE endpoint only: # POST /internal/ejabberd/auth # Body: { "user": "alice", "host": "nexgate.com", "pass": "password123" } # Returns: 200 (allow) or 401 (deny) # Step 2: Run Spring Boot on port 8080 ./mvnw spring-boot:run # Step 3: Configure Ejabberd to call Spring Boot # Create ejabberd.yml with: # auth_method: http # auth_opts: # url: "http://host.docker.internal:8080/internal/ejabberd/auth" # Restart Ejabberd with custom config docker stop ejabberd && docker rm ejabberd docker run -d \ --name ejabberd \ -p 5222:5222 \ -p 5280:5280 \ -p 5285:5285 \ -v $(pwd)/ejabberd.yml:/home/ejabberd/conf/ejabberd.yml \ ghcr.io/processone/ejabberd # Step 4: Test auth via sendxmpp echo "Test" | sendxmpp \ --username alice \ --password password123 \ --host localhost \ --domain nexgate.com \ --insecure \ bob@nexgate.com # Watch Spring Boot logs: # "Auth request received: alice@nexgate.com" # "Validated: allowed ✅" # Try wrong password echo "Test" | sendxmpp \ --username alice \ --password WRONG \ --host localhost \ --domain nexgate.com \ --insecure \ bob@nexgate.com # Spring Boot logs: # "Auth request received: alice@nexgate.com" # "Invalid credentials: rejected ❌" # Ejabberd logs: "Authentication failed" ``` ``` What you learn: Auth bridge works exactly as designed Spring Boot is the source of truth for auth Ejabberd trusts Spring Boot completely This is the same bridge NexGate will use Response time matters — must be < 200ms ``` --- #### Experiment 6 — Two Node Cluster + Erlang Dist ``` Goal: two Ejabberd nodes talking via Erlang distribution Success: message sent on node1 arrives at user on node2 ``` ```bash # Create Docker network for the cluster docker network create ejabberd-cluster # Start node 1 docker run -d \ --name ejabberd-node1 \ --hostname ejabberd-node1 \ --network ejabberd-cluster \ -e ERLANG_NODE=ejabberd@ejabberd-node1 \ -e ERLANG_COOKIE=nexgate_secret_cookie \ -p 5222:5222 \ -p 5280:5280 \ -p 5285:5285 \ ghcr.io/processone/ejabberd sleep 15 # Start node 2 docker run -d \ --name ejabberd-node2 \ --hostname ejabberd-node2 \ --network ejabberd-cluster \ -e ERLANG_NODE=ejabberd@ejabberd-node2 \ -e ERLANG_COOKIE=nexgate_secret_cookie \ -p 5223:5222 \ -p 5281:5280 \ -p 5286:5285 \ ghcr.io/processone/ejabberd sleep 10 # Join node2 to node1 cluster docker exec ejabberd-node2 \ ejabberdctl join_cluster ejabberd@ejabberd-node1 # Verify cluster is formed docker exec ejabberd-node1 ejabberdctl list_cluster # Expected: # ejabberd@ejabberd-node1 # ejabberd@ejabberd-node2 ✅ # Register alice on node1 docker exec ejabberd-node1 \ ejabberdctl register alice nexgate.com pass123 # Register bob on node2 docker exec ejabberd-node2 \ ejabberdctl register bob nexgate.com pass123 # Send message FROM node1 TO bob (who is on node2) docker exec ejabberd-node1 ejabberdctl send_message \ chat alice@nexgate.com bob@nexgate.com \ "" "Cross-node via Erlang dist!" # Check node2 logs — message arrived from node1 docker logs ejabberd-node2 --tail 20 # See: message routed from ejabberd@ejabberd-node1 ✅ # Verify cluster health docker exec ejabberd-node1 ejabberdctl mnesia_info | grep running_db_nodes # Shows both nodes sharing Mnesia DB ✅ ``` ``` What you learn: Erlang dist routing works across containers Same cookie = trusted cluster No Redis pub/sub needed for cross-node routing Mnesia shared across nodes automatically This is production-ready cluster behavior ``` --- #### Experiment 7 — RabbitMQ Events from Ejabberd ``` Goal: Ejabberd publishes events to RabbitMQ, read them in terminal Success: see chat events flowing to RabbitMQ queues ``` ```bash # Ensure RabbitMQ is running (already in your stack) docker ps | grep rabbit # Configure Ejabberd to publish to RabbitMQ # Add to ejabberd.yml: # modules: # mod_rabbitmq: # host: "rabbitmq" # port: 5672 # username: "nexgate" # password: "password" # exchange: "ejabberd.events" # Create the exchange in RabbitMQ docker exec rabbitmq rabbitmqadmin declare exchange \ name=ejabberd.events \ type=topic \ durable=true # Create queue and binding docker exec rabbitmq rabbitmqadmin declare queue \ name=chat.message.inbound \ durable=true docker exec rabbitmq rabbitmqadmin declare binding \ source=ejabberd.events \ destination=chat.message.inbound \ routing_key=chat.message.inbound # Send a message via ejabberdctl docker exec ejabberd ejabberdctl send_message \ chat alice@nexgate.com bob@nexgate.com \ "" "This should appear in RabbitMQ!" # Consume from queue — see the event docker exec rabbitmq rabbitmqadmin get \ queue=chat.message.inbound \ ackmode=ack_requeue_false # Watch queue depth in real time watch -n 1 'docker exec rabbitmq rabbitmqctl list_queues name messages' # Open RabbitMQ dashboard # http://localhost:15672 # See exchanges, queues, message rates ✅ ``` ``` What you learn: Ejabberd → RabbitMQ event pipeline works Event payload structure Queue depth monitoring This is exactly how Spring Boot Chat Service will receive Ejabberd events in production ``` --- #### Experiment 8 — Coturn STUN/TURN ``` Goal: TURN relay server running, STUN tested from terminal Success: STUN returns public IP, relay connection established ``` ```bash # Start Coturn docker run -d \ --name coturn \ --network host \ coturn/coturn \ -n \ --log-file=stdout \ --min-port=49152 \ --max-port=65535 \ --lt-cred-mech \ --user=nexgate:testpassword \ --realm=nexgate.com # Test STUN from terminal stunclient localhost 3478 # Expected output: # Binding test: success # Local address: 127.0.0.1:XXXXX # Mapped address: 127.0.0.1:XXXXX ✅ # Watch Coturn logs docker logs coturn -f # See STUN requests arriving and responses ✅ # Test WebRTC in browser (no Android needed!) # Open this URL in two browser tabs: # https://webrtc.github.io/samples/src/content/peerconnection/pc1/ # Configure TURN server: localhost:3478 # Credentials: nexgate / testpassword # Force TURN (disable direct connections in browser devtools) # Establish audio connection between tabs # Watch Coturn logs — see relay traffic ✅ ``` ``` What you learn: Coturn starts and runs correctly STUN works (public IP discovery) TURN relay works (audio through server) EA carrier NAT bypass confirmed Browser tabs = simpler than Android emulators ``` --- #### Experiment 9 — MessagePack Size Comparison ``` Goal: prove MessagePack saves 60% vs JSON on EA networks Success: numbers printed, saving confirmed ``` ```bash # Create test script cat > /tmp/test_msgpack.py << 'EOF' import json import msgpack # Real NexGate chat message message = { "type": "MSG_SEND", "conv_id": "conv-123456789", "sender_id": "usr-987654321", "body": "Habari yako Juma, vipi biashara leo?", "timestamp": 1719446400000, "temp_id": "abc-def-ghi-jkl-mno", "level": "NORMAL", "content_type": "TEXT" } # Commerce offer stanza metadata offer_message = { "type": "CUSTOM_PRICE_OFFER", "conv_id": "conv-123456789", "offer_id": "offer-uuid-abc-def", "product_id": "prod-samsung-a15", "public_price": 450000, "offer_price": 400000, "currency": "TZS", "valid_minutes": 30 } print("=" * 50) print("NEXGATE MESSAGE SIZE COMPARISON") print("=" * 50) for name, msg in [("Text message", message), ("Offer message", offer_message)]: json_bytes = json.dumps(msg).encode() msgpack_bytes = msgpack.packb(msg) reduction = round((1 - len(msgpack_bytes)/len(json_bytes)) * 100) print(f"\n{name}:") print(f" JSON: {len(json_bytes)} bytes") print(f" MessagePack: {len(msgpack_bytes)} bytes") print(f" Saving: {reduction}% smaller") # Daily usage estimate print("\n" + "=" * 50) print("EA DATA BUNDLE IMPACT (1000 messages/day)") print("=" * 50) avg_json = 160 avg_msgpack = 60 print(f" JSON: {avg_json * 1000 / 1024:.0f} KB/day") print(f" MessagePack: {avg_msgpack * 1000 / 1024:.0f} KB/day") print(f" Saving: {(avg_json - avg_msgpack) * 1000 / 1024:.0f} KB/day per user") print(f" ~{(avg_json - avg_msgpack) * 1000 * 30 / 1024 / 1024:.1f} MB saved per month") EOF python3 /tmp/test_msgpack.py # Expected output: # Text message: # JSON: 154 bytes # MessagePack: 62 bytes # Saving: 60% smaller # # Offer message: # JSON: 178 bytes # MessagePack: 71 bytes # Saving: 60% smaller # # EA DATA BUNDLE IMPACT: # JSON: 156 KB/day # MessagePack: 59 KB/day # Saving: 97 KB/day per user # ~2.8 MB saved per month ✅ ``` ``` What you learn: Real numbers — not estimates 60% confirmed on NexGate-specific messages Monthly saving per EA user calculated Justifies the MessagePack implementation effort ``` --- ### Experiment Success Checklist ``` Before moving to Stage 2 (building NexGate): All 9 must be green ✅ Exp 1 Ejabberd running locally ✅ / ❌ Exp 2 REST API working via curl ✅ / ❌ Exp 3 sendxmpp connects as XMPP user ✅ / ❌ Exp 4 Raw XMPP stanzas visible in tcpdump ✅ / ❌ Exp 5 Spring Boot auth bridge working ✅ / ❌ Exp 6 Two node cluster + Erlang dist ✅ / ❌ Exp 7 RabbitMQ events from Ejabberd ✅ / ❌ Exp 8 Coturn STUN/TURN + browser WebRTC ✅ / ❌ Exp 9 MessagePack saving confirmed ✅ / ❌ All green → Stage 2 starts Any red → fix it before moving forward surprises in experiments = learning surprises in production = problems ``` --- ### Stage 2 — Build NexGate Chat Phase 2 ``` Goal: production-ready chat on NexGate Duration: ~16 weeks Output: WhatsApp-class chat shipped to EA users Start coding HERE — not before Every experiment above maps to real code: Exp 1 → Ejabberd Docker in production compose Exp 2 → Spring Boot EjabberdClient (curl → RestTemplate) Exp 3 → Mobile app XMPP connection (sendxmpp → Smack SDK) Exp 5 → Real auth bridge with JWT validation Exp 6 → Two node cluster on Hetzner VPS Exp 7 → RabbitMQ consumers in Chat Service Exp 8 → Coturn on separate Hetzner CX11 Exp 9 → MessagePack in NexGate Chat SDK 16-week build order in Section 19 What ships: Text chat (1:1 + group) Voice notes Voice + video calls (+ switch audio↔video) Screen sharing Group calls (LiveKit) Commerce DMs (both flows) Offer sessions (full lifecycle) Message interactions (edit/delete/react/forward) Shop inbox with staff access Offline delivery + FCM + Textfy EA network optimized (Coturn + Opus + H.264) WhatsApp-class infrastructure Commerce-aware from day one Infrastructure: Ejabberd cluster (2 nodes, same VPS) Coturn (separate Hetzner CX11 ~€4/month) Spring Boot Chat Service (new microservice) All existing infra (Redis, RabbitMQ, PostgreSQL) File Thunder (already running) ✅ ``` --- ### Stage 3 — Eventually (WeChat EA) ``` Goal: full super app communication platform Timeline: after Phase 2 is live and growing VP Live (Video Streaming): SRS Media Server RTMP ingest → HLS → Cloudflare CDN Live comments (Ejabberd MUC) VOD after stream (File Thunder) VP Audio Spaces: LiveKit SFU Multi-speaker rooms (Twitter Spaces model) Radio mode (one broadcaster → millions) Raise hand system Group Calls: LiveKit already deployed for Audio Spaces Activate for group voice + video Up to 8 participants voice (3G compatible) Up to 4 video feeds simultaneously Mini Apps (MQTT): Ejabberd MQTT broker (already in Ejabberd config) Third-party apps subscribe to events JikoXpress integration Real-time order tracking NexGate developer platform WeChat EA: All of the above live NexGate = EA's daily life infrastructure Every transaction has a conversation Every conversation can become a transaction 🚀 ``` --- ### The Progression ``` NOW THEN EVENTUALLY ─────────────────── ─────────────────── ─────────────────── Terminal only NexGate chat live VP Live streaming Docker CLI Text + calls VP Audio Spaces curl + sendxmpp Commerce DMs Group calls ejabberdctl Offer sessions Mini Apps (MQTT) tcpdump + wireshark Shop inbox + staff NexGate developer 9 experiments 16 weeks to ship platform No app built yet WhatsApp-class WeChat EA vision ─────────────────── ─────────────────── ─────────────────── Confidence Product Platform ``` --- ## 2. What Is Phase 2 NexGate chat is built directly on Phase 2 architecture from scratch. There is no Phase 1 to migrate from. No Spring Boot WebSocket gateway was ever built. No Redis pub/sub routing to replace. Phase 2 is the starting point — not an upgrade. ``` Why start directly on Phase 2: Ejabberd handles 2M concurrent connections Spring Boot WS would need many pods to reach this Ejabberd does it on two Docker containers Voice + video calls needed from launch Ejabberd Jingle (XEP-0166) solves signaling natively Building WebRTC signaling from scratch = months wasted 25+ chat features free from Ejabberd XEPs Typing indicators, delivery ticks, read receipts, multi-device sync, message archive, push bridge All zero custom code — just Ejabberd config EA network demands carrier-grade infrastructure Stream Management (XEP-0198) = no message loss on 2G Cannot afford to rebuild this later Commerce-aware chat from day one Custom XMPP namespaces for product cards, offer sessions, event cards, group purchases Ejabberd routes them — Spring Boot handles business logic ``` NexGate chat is a greenfield Phase 2 build. --- ## 3. What We Are Building ``` Building from scratch: Ejabberd Cluster ← real-time transport Two nodes, same Hetzner VPS at launch Handles all WebSocket connections Routes all XMPP stanzas Manages presence, MUC, Jingle calls XEP-0198 stream management for EA networks Spring Boot Chat Service ← business brain Message persistence (PostgreSQL) Commerce context (offer sessions, product cards) Notification routing (FCM + Textfy) Shop inbox access control Call records + quality logs Offline escalation Spring Boot Main Backend ← platform API Auth (PONA Auth V3) + XMPP token issuance VP Shop, VP Feed, VP Events integration Commerce triggers to Chat Service Coturn TURN Server ← voice/video relay EA carrier NAT bypass Separate small Hetzner VPS NexGate Chat SDK ← mobile dev layer Android (Smack wrapper) iOS (XMPPFramework wrapper) Hides all XMPP complexity from mobile dev Clean Java/Swift API Infrastructure (already running): Redis ✅ presence cache, hot messages RabbitMQ ✅ offline queue, service events PostgreSQL ✅ persistence MinIO ✅ media storage Cloudflare ✅ CDN Vault ✅ secrets Traefik ✅ reverse proxy File Thunder ✅ media processing FCM + Textfy ✅ notifications ``` --- ## 4. Full Architecture ``` ┌──────────────────────────────────────────────────────────┐ │ NexGate Mobile App │ │ Android / iOS │ └────┬──────────────┬──────────────┬──────────────┬───────┘ │ │ │ │ WebSocket HTTP REST WebRTC HLS Player XMPP stanzas (unchanged) (calls + (streams — MessagePack Main Backend spaces) Phase 3) │ │ │ ▼ ▼ │ ┌─────────────────────────┐ │ │ Ejabberd Cluster │ │ │ │ │ │ Node 1 Node 2 │ │ │ ┌──────┐ ┌──────┐ │ │ │ │Erlang│◀─▶Erlang│ │ │ │ │ dist │ │ dist │ │ │ │ └──────┘ └──────┘ │ │ │ │ │ │ XMPP/WebSocket │ │ │ Presence (built-in) │ │ │ MUC rooms (XEP-0045) │ │ │ Jingle signaling │◀───────┘ │ (XEP-0166) │ (call signaling │ MQTT broker │ via WS) │ Push bridge XEP-0357 │ └──────────┬──────────────┘ │ │ HTTP (auth only — sync) │ RabbitMQ (all events — async) │ ┌──────────▼──────────────────────────────────────────────┐ │ Spring Boot Chat Service │ │ (unchanged from Phase 1) │ │ │ │ Messages · Receipts · Commerce Context │ │ Notification Router · Call Records │ │ Shop Inbox · Offline Escalation │ └──────────┬──────────────┬──────────────────────────────┘ │ │ ▼ ▼ ┌──────────────┐ ┌──────────────────────────────────────┐ │ PostgreSQL │ │ RabbitMQ │ │ Redis │ │ chat.message.inbound │ │ (unchanged) │ │ chat.presence │ │ │ │ chat.call.events │ └──────────────┘ │ chat.notify.push │ │ chat.notify.escalation │ └──────────────────────────────────────┘ │ ▼ ┌─────────────────────────────────┐ │ Coturn TURN Server │ │ (separate VPS) │ │ UDP relay for voice/video │ │ when P2P blocked by EA NAT │ └─────────────────────────────────┘ ``` --- ## 5. Ejabberd — The Transport Backbone ### What Ejabberd Owns in Phase 2 ``` ✅ All WebSocket connections (2M concurrent per node) ✅ XMPP stanza routing between users ✅ User presence — online/offline/away (built-in protocol) ✅ Typing indicators (XEP-0085) ✅ Message delivery receipts (XEP-0184) ✅ Multi-User Chat rooms — MUC (XEP-0045) ✅ Voice/video call signaling — Jingle (XEP-0166) ✅ Push notification bridge (XEP-0357 → FCM/APNs) ✅ MQTT broker (Mini Apps events) ✅ Stream management / reconnection (XEP-0198) ✅ Cross-node routing (Erlang distributed — no Redis pub/sub needed) ❌ Does NOT touch: PostgreSQL (NexGate's schema) Business logic Commerce context Payment processing File processing ``` ### Ejabberd Key Modules Enabled ``` mod_mam Message Archive Management Stores message history in its own DB Clients can sync history on reconnect mod_muc Multi-User Chat Group chats, live stream comment rooms Max 500 members per room (configurable) Persistent rooms survive server restart mod_ping Keepalive ping every 30 seconds Kills dead connections automatically Critical for EA mobile networks mod_push Push notification bridge Connects to FCM/APNs on user disconnect Replaces manual FCM calls from Chat Service mod_stun_disco STUN/TURN server discovery Tells clients where Coturn is Used for voice/video call setup mod_mqtt MQTT broker on port 1883 For Mini Apps real-time events (Phase 3) Zero extra infrastructure needed mod_http_api REST API on port 5285 Spring Boot calls this to send messages Admin operations (kick user, create room) ``` ### Ejabberd Config Highlights ```yaml hosts: - "nexgate.com" listen: - port: 5280 # WebSocket — mobile app connects here module: ejabberd_http request_handlers: /ws: ejabberd_ws /api: mod_http_api - port: 5285 # REST API — Spring Boot calls here (internal only) module: ejabberd_http ip: "127.0.0.1" request_handlers: /api: mod_http_api - port: 1883 # MQTT — Mini Apps (Phase 3) module: mod_mqtt - port: 3478 # STUN — voice/video setup transport: udp module: ejabberd_stun # Auth — Ejabberd calls Spring Boot auth_method: http auth_opts: url: "http://chat-service:8082/internal/ejabberd/auth" auth_header: "X-Internal-Secret" auth_header_value: "${EJABBERD_INTERNAL_SECRET}" # PostgreSQL — Ejabberd's own separate database sql_type: pgsql sql_server: "postgres" sql_database: "ejabberd" # NOT nexgate — separate DB default_db: sql modules: mod_mam: default: always db_type: sql mod_muc: db_type: sql max_users: 500 mod_ping: send_pings: true ping_interval: 30 timeout_action: kill mod_push: {} mod_stun_disco: credentials_lifetime: 3600 services: - host: "turn.nexgate.com" port: 3478 type: turn secret: "${COTURN_SECRET}" mod_mqtt: {} mod_http_api: {} ``` ### Two Separate PostgreSQL Databases ``` postgres instance (same server, two databases): nexgate ← NexGate application data messages, conversations, users, orders Spring Boot owns this entirely Ejabberd never touches this ejabberd ← Ejabberd's own operational data message archive (MAM) MUC room state roster data Spring Boot never touches this Why separate: Ejabberd manages its own schema migrations NexGate schema evolves independently Clean ownership — no shared tables Easy to backup independently ``` --- ## 6. Ejabberd ↔ Spring Boot Bridge ### Communication Rules ``` Ejabberd → Spring Boot: Auth events: HTTP (synchronous — must respond fast) Message events: RabbitMQ (async) Presence events: RabbitMQ (async) Call events: RabbitMQ (async) MUC events: RabbitMQ (async) Spring Boot → Ejabberd: Send message to user: Ejabberd REST API (port 5285) Create MUC room: Ejabberd REST API Kick user session: Ejabberd REST API Check user online: Ejabberd REST API Broadcast to room: Ejabberd REST API Rule: auth is the ONLY synchronous call Everything else is async via RabbitMQ ``` ### RabbitMQ Events from Ejabberd ``` Exchange: ejabberd.events (topic) Routing Key Fired When ────────────────────────────────────────────────────── chat.message.inbound User sends a message chat.message.group User sends to MUC room chat.presence.online User WS connects chat.presence.offline User WS disconnects chat.call.initiated Jingle session-initiate received chat.call.accepted Jingle session-accept received chat.call.declined Jingle session-declined received chat.call.ended Jingle session-terminate received chat.muc.created MUC room created chat.muc.joined User joined MUC room chat.muc.left User left MUC room ``` ### Spring Boot Internal Endpoints (Ejabberd calls these) ``` POST /internal/ejabberd/auth Called on every WebSocket connection Ejabberd sends: { username, token } Spring Boot responds: 200 (allow) or 401 (deny) Must respond in < 200ms (checked in Redis cache first) All other events arrive via RabbitMQ consumers No other synchronous HTTP endpoints needed ``` ### Spring Boot → Ejabberd REST API Examples ``` Send system message to user: POST http://ejabberd:5285/api/send_message { "from": "system@nexgate.com", "to": "usr-123@nexgate.com", "body": "", "extra": { "type": "ORDER_STATUS_UPDATE", "orderId": "ord-456", "status": "SHIPPED" } } Create live stream MUC room: POST http://ejabberd:5285/api/create_room { "name": "live-stream-abc", "service": "conference.nexgate.com", "host": "nexgate.com" } Kick expired session: POST http://ejabberd:5285/api/kick_session { "user": "usr-123", "host": "nexgate.com", "resource": "android", "reason": "Token expired" } ``` --- ## 7. Authentication Flow ### Two Tokens Issued at Login ``` User logs into NexGate │ ▼ POST /auth/login Main Backend (PONA Auth V3): Validate credentials Issue two tokens: REST JWT (7 days): Used for all HTTP API calls Standard Bearer token XMPP Token (24 hours): Used only for Ejabberd connection Contains: userId, JID, expiry Shorter lifetime — chat sessions refresh more often │ ▼ Both tokens returned to app ``` ### WebSocket Connection Auth ``` App connects WebSocket: wss://chat.nexgate.com/ws Header: Authorization: Bearer {XMPP_TOKEN} │ ▼ Ejabberd receives connection Extracts token from header │ ▼ HTTP POST (sync) → Spring Boot /internal/ejabberd/auth { username: "usr-123", token: "XMPP_TOKEN" } │ Spring Boot: Check Redis cache first (fast path): token:{hash} → valid/invalid (TTL 5min) If not cached: Validate JWT signature Check token type == XMPP Check user not banned/suspended Cache result in Redis Return: 200 { authorized: true, jid: "usr-123@nexgate.com" } or 401 { authorized: false, reason: "TOKEN_EXPIRED" } │ Ejabberd: 200 → allow connection register: usr-123@nexgate.com/android as ONLINE publish to RabbitMQ: chat.presence.online 401 → reject WebSocket app shows: "Session expired, please login again" ``` ### JID Structure ``` Every NexGate entity has a JID (Jabber ID): Personal user: usr-123@nexgate.com/android ← full JID (user + device) usr-123@nexgate.com ← bare JID (user only) Shop identity: techstore@shops.nexgate.com ← shop JID Multiple staff auth as this JID Customer sees "TechStore" — not the staff member System bot: system@nexgate.com ← order updates, notifications MUC rooms: live-abc@conference.nexgate.com ← live stream chat room group-xyz@conference.nexgate.com ← group chat room Multi-device: usr-123@nexgate.com/android ← phone usr-123@nexgate.com/ios ← tablet Both receive messages READ on one → Ejabberd notifies other to clear notification ``` ### XMPP Token Refresh ``` XMPP token expires every 24 hours App background service: At 23 hours → POST /auth/refresh-xmpp-token Header: Bearer {REST_JWT} (still valid — 7 days) Response: new XMPP token Re-auth without reconnecting: App sends new auth stanza on existing WS connection Ejabberd re-validates via Spring Boot No disconnection — seamless for user ``` --- ## 8. Message Flow — Phase 2 ### Inbound Message (User Sends) ``` [Client A — Android] │ │ WebSocket frame (MessagePack encoded): │ { │ type: MSG_SEND │ temp_id: "abc-123" │ to: "usr-456@nexgate.com" │ conv_id: "conv-789" │ body: "Habari" │ content_type: TEXT │ } │ ▼ [Ejabberd Node 1] │ ├── Validates session (already authed) ├── ACKs client immediately: │ { temp_id: "abc-123", status: ACK } ├── Routes to usr-456 (if online): │ Erlang looks up which node holds usr-456 │ If Node 1 → delivers directly │ If Node 2 → Erlang distributed message (no Redis needed) │ └── Publishes to RabbitMQ: chat.message.inbound { from: "usr-123@nexgate.com", to: "usr-456@nexgate.com", conv_id: "conv-789", body: "Habari", temp_id: "abc-123", timestamp: 1719446400 } . . (async) . [Spring Boot Chat Service] │ consumes chat.message.inbound │ ├── Authorization check (can A message B?) ├── Resolve message level ├── Write to PostgreSQL (messages table) ├── Write to Redis hot cache (last 50 per conv) ├── Update conversation last_message │ ├── usr-456 online? (check via Ejabberd REST API) │ YES → DELIVERED receipt after Ejabberd confirms │ NO → RabbitMQ offline queue + FCM + escalation timer │ └── Notify sender: tick update REST API → Ejabberd → WS push to Client A Client A: ✓✓ (delivered) ``` ### Cross-Node Routing — No Redis Pub/Sub Needed ``` Phase 1 (Spring Boot WS): Pod 1 holds Client A connection Pod 2 holds Client B connection Redis pub/sub needed to bridge pods Pod 1 publishes → Redis → Pod 2 delivers Phase 2 (Ejabberd cluster): Node 1 holds Client A connection Node 2 holds Client B connection Erlang distributed messaging bridges nodes Node 1 → Erlang dist → Node 2 delivers Redis pub/sub no longer needed for routing (Redis still used by Chat Service for hot cache) This is why Ejabberd can do 2M concurrent: Erlang process per connection (~2KB RAM each) Native cross-node routing built into the language No external message bus overhead ``` --- ## 9. Voice Calls ### Components ``` Signaling: Ejabberd Jingle (XEP-0166) coordinates call setup via XMPP stanzas STUN: Ejabberd built-in (mod_stun_disco) helps devices find their public IP behind NAT TURN: Coturn (separate VPS) relay when P2P impossible (EA carrier NAT) Transport: WebRTC in mobile app actual audio stream between devices Codec: Opus adaptive 6kbps (2G) → 64kbps (WiFi) echo cancellation + noise suppression built in non-negotiable for EA networks ``` ### Jingle Signaling Stanzas ```xml ``` ### Full Voice Call Flow ``` [Kibuti — taps Call] │ ▼ GET /chat/calls/turn-credentials Spring Boot returns: { iceServers: [ { urls: "stun:chat.nexgate.com:3478" }, { urls: "turn:turn.nexgate.com:3478", username: "usr-123:1719446400", credential: "hmac_token" } ], ttl: 3600 } │ ▼ Initialize WebRTC PeerConnection Add audio track (Opus codec) Gather ICE candidates (STUN discovery) │ ▼ Send Jingle session-initiate via Ejabberd WS Ejabberd routes to Juma Ejabberd fires RabbitMQ event: chat.call.initiated │ Spring Boot: Create call record: status: RINGING started_at: now If Juma offline → FCM HIGH priority: { type: INCOMING_CALL, callId, callerName, callType: VOICE } │ [Juma's phone rings] Juma taps Answer │ ▼ Juma sends Jingle session-accept via Ejabberd WS ICE negotiation begins between devices: │ ├── P2P possible? (good network) │ Direct connection established ✅ │ No Coturn bandwidth used │ └── P2P blocked? (EA carrier NAT) Both connect to Coturn relay Audio flows: Kibuti → Coturn → Juma │ Call live 🎉 RTCP monitors quality every 200ms: Good network → Opus 32-64kbps, clear voice 3G → Opus 16kbps, still good 2G → Opus 8kbps, slightly robotic but connected Very poor → Opus 6kbps, minimum viable │ Kibuti taps End │ ▼ Jingle session-terminate via Ejabberd WS Ejabberd fires: chat.call.ended Spring Boot: Update call record: status: COMPLETED ended_at: now duration_seconds: calculated relay_used: true/false ``` ### Call State Machine ``` IDLE │ user taps Call ▼ INITIATING │ getting TURN credentials │ creating WebRTC offer ▼ RINGING ──────────────────────▶ MISSED (45s timeout) │ Jume answers ▼ CONNECTING │ ICE negotiation │ finding best path ▼ CONNECTED ────────────────────▶ RECONNECTING (network drop) │ call live │ ICE restart │ │ 10s timeout → FAILED │ user ends ▼ ENDING │ Jingle terminate sent ▼ COMPLETED / MISSED / DECLINED / FAILED ``` ### Codec Ladder — Opus Adaptive ``` Network Bitrate Quality ───────────────────────────────────────────────── WiFi / 4G strong 64 kbps HD voice 4G standard 32 kbps Clear 3G 16 kbps Good enough 2G / Edge 8 kbps Robotic but connected Barely alive 6 kbps Minimum viable ───────────────────────────────────────────────── Opus switches automatically based on RTCP feedback No configuration needed — adaptive by design ``` --- ## 10. Video Calls ### Same Architecture as Voice + Camera ``` Everything from voice call applies Additional components: Video codec: H.264 (primary) Hardware accelerated on Tecno, Infinix, Samsung Low battery drain — GPU handles encoding Fallback: VP8 (software, more CPU) Camera: Front camera default (switchable) Device detects capability at call start Resolution ladder (adaptive): ───────────────────────────────────────────────── WiFi 720p 30fps 1.5 Mbps 4G strong 480p 24fps 800 kbps 3G 360p 15fps 400 kbps 2G 240p 10fps 150 kbps Very poor VIDEO OFF — audio only (Opus) ───────────────────────────────────────────────── Degradation order (never drops call): 1. Reduce color depth 2. Reduce resolution (720→480→360→240) 3. Reduce frame rate (30→24→15→10fps) 4. Reduce audio bitrate 5. Disable video entirely → audio only 6. Reduce audio to minimum (6kbps Opus) ``` ### Device Tier Detection ``` App detects device capability at call start: High-end (Pixel, Samsung S series): H.264 hardware encoder (GPU) Start at 720p 30fps Low battery impact Mid-range (Samsung A series): H.264 hardware encoder Start at 480p 24fps Medium battery impact Low-end (Tecno Spark, Infinix Hot): H.264 software encoder (CPU) Start at 360p 15fps High battery impact Show warning: "Video call may drain battery faster" Auto-disable video after 10min if battery < 20% ``` ### Jingle for Video — Additional Content Block ```xml ``` --- ## 11. Coturn — TURN Relay ### Why TURN is Mandatory for EA ``` Direct P2P (ideal): Both devices negotiate directly Audio/video flows device-to-device Ejabberd not involved in media No bandwidth cost on your servers EA reality — P2P often blocked: Vodacom, Airtel, Tigo use CGNAT Multiple users share one public IP P2P connection cannot be established Without TURN → call fails TURN relay (fallback): Both devices connect to Coturn Coturn relays audio/video between them Call works regardless of carrier NAT Bandwidth cost on your server (~50KB/min voice) ``` ### Coturn Config Highlights ``` listening-port=3478 tls-listening-port=5349 relay-ip=YOUR_COTURN_VPS_IP realm=nexgate.com lt-cred-mech # time-limited credentials use-auth-secret static-auth-secret=${COTURN_SECRET} # from Vault min-port=49152 max-port=65535 ``` ### TURN Credentials Generation ``` Credentials are time-limited HMAC tokens Generated by Spring Boot per call session Coturn validates them — prevents abuse Format: username: {userId}:{expiry_timestamp} credential: HMAC-SHA1(secret, username) ttl: 3600 seconds (1 hour per call) Only NexGate users can use your TURN server No credential → Coturn rejects connection ``` ### Bandwidth Estimation ``` Voice call via TURN: Opus 16kbps × 2 directions = ~4KB/min 1 hour call ≈ 240KB per participant Video call via TURN: 360p H.264 × 2 directions = ~6MB/min Force 360p max when on relay to control cost Coturn VPS sizing: Hetzner CX11 (€4/month, 1vCPU/2GB) 20TB bandwidth included Handles ~500 concurrent voice relay calls Upgrade to CX21 at scale ``` --- ## 12. MessagePack Encoding ### Why Switch from JSON ``` JSON message frame: {"type":"MSG_SEND","conv_id":"conv-123456","sender_id":"usr-789012", "body":"Habari","timestamp":1719446400000,"temp_id":"abc-def-ghi"} Size: ~140 bytes Every key repeated as string on every message Numbers encoded as ASCII characters Parsing: character by character MessagePack same message: [binary representation] Size: ~50 bytes Keys encoded as integers (schema registered) Numbers encoded as actual bytes (int32 = 4 bytes) Parsing: read fixed byte positions Result: 60-65% smaller on wire 3-5x faster to parse Critical for users on 2G/3G with limited data bundles ``` ### Migration Strategy (No Breaking Change) ``` Both formats supported simultaneously: Client sends header: Content-Type: application/msgpack → MessagePack Content-Type: application/json → JSON (default) Ejabberd detects Content-Type Routes to appropriate deserializer Migration flow: Old app version → sends JSON → works fine New app version → sends MessagePack → works fine No forced update required Gradual migration over 30-60 days Remove JSON support after 90%+ adoption ``` --- ## 13. Broadcast Channels ### What They Are ``` Creator → unlimited followers One-directional: creator posts, followers receive Like Telegram channels No replies from followers (unless creator enables Q&A) Use cases: Shop announcement channel ($techstore updates) Creator content channel (@kibuti posts) NexGate system channel (platform announcements) ``` ### Fan-out Strategy ``` Small channel (< 10,000 followers): Write-on-send — Chat Service pushes to each follower Same as group chat fan-out Large channel (10,000+ followers): Lazy fan-out — store message once Followers fetch on open (read-time delivery) No per-follower push for casual followers FCM push only to followers with notifications enabled Same celebrity bypass pattern as VP Feed: Hot channels → read-time merge Normal channels → write-time fan-out ``` ### Ejabberd MUC for Channels ``` Broadcast channel = MUC room with restrictions: Only owner/admins can send messages Members are read-only subscribers mod_muc handles this with role configuration: Role: moderator → can send Role: visitor → read only This means channels are built on the same MUC infrastructure as group chats No separate implementation needed ``` --- ## 14. MQTT — Mini Apps Foundation ### What MQTT Enables ``` Ejabberd runs MQTT broker on port 1883 No extra infrastructure — already in Ejabberd Mini Apps subscribe to topics: orders/{orderId} → real-time order updates delivery/{trackingId} → GPS delivery tracking live/{streamId}/viewers → viewer count updates jiko/{restaurantId} → JikoXpress kitchen events Spring Boot publishes events: Order shipped → publish to orders/{orderId} Mini App receives instantly No polling needed ``` ### MQTT vs XMPP for Mini Apps ``` XMPP (chat): Full protocol, complex stanzas Designed for human conversation Bidirectional, stateful sessions Right tool for chat MQTT (events): Lightweight pub/sub protocol Designed for IoT and event streams Minimal overhead (2-byte header) Right tool for Mini App events Works on very limited connections Both live inside Ejabberd: Same server, different protocols Mobile app uses XMPP for chat Mini Apps use MQTT for events Zero additional infrastructure ``` --- ## 15. Message Interactions All message interaction features are handled via standard XMPP XEPs. Ejabberd routes the stanzas automatically — Spring Boot handles persistence and business rules via RabbitMQ events. ### Overview — All Four Features ``` Feature XEP Status Ejabberd ──────────────────────────────────────────────────────── Edit message XEP-0308 Stable ✅ auto routed Delete for everyone XEP-0424 Stable ✅ auto routed Reactions XEP-0444 Stable ✅ auto routed Forwarding XEP-0297 Stable ✅ auto routed Reply to message XEP-0461 Experimental auto routed Stable stanza IDs XEP-0359 Stable ✅ auto assigned ──────────────────────────────────────────────────────── All routed by Ejabberd Spring Boot handles: validation, persistence, rules ``` ### XEP-0359 — Stable Stanza IDs (Foundation) Before the features — this XEP is the foundation all others depend on. Every message gets a stable server-assigned ID used by reactions, edits, retractions, and replies to reference the correct message. ```xml Habari ``` --- ### Message Editing — XEP-0308 ``` Who can edit: Original sender only Time window: 15 minutes after send What: Text body only Commerce cards: ❌ BLOCKED — financial records are immutable System messages: ❌ BLOCKED — never editable ``` ```xml Habari yako Juma, vipi biashara? ``` ``` Flow: Kibuti edits → stanza sent via Ejabberd WS Ejabberd routes to Juma (if online) Ejabberd fires RabbitMQ: chat.message.edited │ Spring Boot: Is sender original author? ✅ Within 15 minute window? ✅ Not a commerce/system message? ✅ Update messages.body = new text Update messages.edited_at = now Increment messages.edit_count │ Juma's app: Receives edit stanza Updates message in place (same position in thread) Shows "Edited" label under message Group chats: Same stanza sent to MUC room JID Ejabberd MUC broadcasts to all members All see updated message simultaneously ``` --- ### Delete for Everyone — XEP-0424 ``` Two delete modes: Delete for me: Local filter only No Ejabberd stanza needed Spring Boot records: message_deletions (scope: SELF) Recipient unaffected Delete for everyone: XEP-0424 retraction stanza Ejabberd routes to all recipients Time window: 15 minutes Commerce cards: ❌ BLOCKED System messages: ❌ BLOCKED ``` ```xml ``` ``` Flow: Kibuti retracts → stanza via Ejabberd WS Ejabberd routes to Juma Ejabberd fires RabbitMQ: chat.message.retracted │ Spring Boot: Is sender original author? ✅ Within 15 minute window? ✅ Not blocked message type? ✅ Soft delete: messages.deleted_at = now messages.deleted_by = usr-kibuti messages.delete_scope = EVERYONE body NOT removed (audit trail kept) │ Juma's app: Receives retraction stanza Replaces message with: "This message was deleted" Same position in thread Nothing is ever hard deleted from PostgreSQL: Legal compliance (EA regulations) Dispute resolution (order/payment disputes) Admin investigation (fraud cases) Soft delete always — hard delete never ``` --- ### Reactions — XEP-0444 ``` Model: One reaction per user per message Emoji set: Limited set at launch ❤️ 👍 😂 😮 😢 🙏 Change: Send new emoji → replaces old Remove: Send empty → removes reaction Commerce cards: ✅ ALLOWED (reactions don't modify content) System messages: ❌ BLOCKED ``` ```xml 👍 ❤️ ``` ``` Flow: Kibuti taps 👍 → reaction stanza via Ejabberd WS Ejabberd routes to Juma Ejabberd fires RabbitMQ: chat.message.reaction │ Spring Boot: Upsert in message_reactions: ON CONFLICT (message_id, user_id) → update emoji + timestamp Empty emoji received → delete reaction record │ Juma's app: Receives reaction stanza Updates reaction display below message: 👍 1 Kibuti's own reaction: highlighted Group chats: Stanza sent to MUC room Ejabberd MUC broadcasts to all members All screens update simultaneously: 👍 3 ❤️ 2 😂 1 Notification: Reaction on your message → FCM push "Juma reacted 👍 to your message" Level: NORMAL (FCM only — no SMS) Muted conversations → no reaction notification ``` --- ### Message Forwarding — XEP-0297 ``` What it is: Client creates NEW message in target conversation Original message wrapped inside as reference Server never "moves" anything Forwarded label shown with original sender name Forward chain tracking: chain = 1: "↪ Forwarded from Juma Mwangi" chain = 2-4: "↪ Forwarded" chain = 5+: "↪ Forwarded many times" (misinformation warning) Multi-forward: up to 5 conversations per action Max chain: no hard limit but UI degrades label Commerce rules: Product card: ✅ anyone can forward Custom price offer: ❌ private deal — blocked Order confirmation: ❌ private record — blocked Payment record: ❌ private record — blocked System messages: ❌ blocked ``` ```xml Angalia hii Habari yako rafiki! Juma Mwangi 1 ``` ``` Flow: Kibuti taps Forward on Juma's message Picks Amina's conversation App creates new message stanza (not routing original) Sends via Ejabberd WS to Amina Ejabberd routes normally as new message Fires RabbitMQ: chat.message.inbound (same as any message) │ Spring Boot: Validates forward is allowed (type check) Creates new messages record: is_forwarded: true original_sender_name: "Juma Mwangi" forward_chain: 1 media_ref: original fileId (no re-upload) │ Amina's app: Receives as new message Renders with forwarded label: ┌────────────────────────────────┐ │ ↪ Forwarded from Juma Mwangi │ │ │ │ Habari yako rafiki! │ │ 10:45 │ └────────────────────────────────┘ Media forwarding: References original fileId — no re-upload 10 people forward same image → 1 file in MinIO, 10 message records File Thunder serves same file to all ``` --- ### Message Replies — XEP-0461 ``` Reply to a specific message in thread Like WhatsApp/Telegram quote-reply Shows original message above reply Status: Experimental ⚠️ Not yet stable standard But widely implemented (Gajim, Monal, many others use it) Safe to implement — unlikely to change drastically ``` ```xml Nzuri sana, asante! ``` ``` UI renders: ┌────────────────────────────────┐ │ ┌──────────────────────────┐ │ │ │ Kibuti │ │ ← quoted original │ │ Habari yako Juma! │ │ │ └──────────────────────────┘ │ │ │ │ Nzuri sana, asante! │ │ 10:47 │ └────────────────────────────────┘ Tap on quote → scroll to original message ``` --- ### Commerce Messages — Interaction Rules Summary ``` Message type Edit Delete(all) React Forward ──────────────────────────────────────────────────────── Text message ✅ 15m ✅ 15m ✅ ✅ Voice note ❌ ✅ 15m ✅ ✅ Image / Video ❌ ✅ 15m ✅ ✅ Product card ❌ ❌ ✅ ✅ Custom price offer ❌ ❌ ✅ ❌ Order confirmation ❌ ❌ ✅ ❌ Payment confirmation ❌ ❌ ✅ ❌ System message ❌ ❌ ❌ ❌ ──────────────────────────────────────────────────────── Why commerce cards are protected: Immutable negotiation record Seller cannot change agreed price after the fact Buyer cannot claim different price was offered Full audit trail in thread — legally important ``` --- ### RabbitMQ Events — New in Phase 2 for Interactions ``` Exchange: nexgate.chat (topic) — additions: Routing Key Fired When ────────────────────────────────────────────────────── chat.message.edited XEP-0308 received chat.message.retracted XEP-0424 received chat.message.reaction XEP-0444 received chat.message.forwarded XEP-0297 received chat.message.delete_self delete for me (REST call) ``` --- ## 16. Docker Deployment ### docker-compose additions for Phase 2 ```yaml ejabberd: image: ghcr.io/processone/ejabberd:latest container_name: ejabberd restart: unless-stopped ports: - "5222:5222" # XMPP TCP - "5280:5280" # WebSocket + HTTP - "1883:1883" # MQTT - "3478:3478/udp" # STUN volumes: - ./ejabberd/ejabberd.yml:/home/ejabberd/conf/ejabberd.yml - ./ejabberd/data:/home/ejabberd/database - ./ejabberd/logs:/home/ejabberd/logs environment: - EJABBERD_BYPASS_WARNINGS=true depends_on: - postgres - rabbitmq networks: - nexgate-internal # Coturn on separate VPS — not in same compose # Deployed independently on Hetzner CX11 # Connects back to NexGate via internal network ``` ### Traefik — WebSocket Routing ```yaml # Ejabberd service labels for Traefik labels: - "traefik.enable=true" # App connects here for chat - "traefik.http.routers.chat.rule=Host(`chat.nexgate.com`)" - "traefik.http.routers.chat.tls=true" - "traefik.http.routers.chat.tls.certresolver=letsencrypt" - "traefik.http.services.chat.loadbalancer.server.port=5280" # Sticky sessions — CRITICAL for WebSocket # Same user must always hit same Ejabberd node - "traefik.http.services.chat.loadbalancer.sticky.cookie=true" - "traefik.http.services.chat.loadbalancer.sticky.cookie.name=ejabberd_node" ``` ``` Why sticky sessions: User connected to Ejabberd Node 1 Next request hits Node 2 → connection context lost → disconnected Sticky cookie ensures: usr-123 always → Node 1 usr-456 always → Node 2 WS sessions stable across load balancer ``` ### Ejabberd Cluster Config ```yaml # Second node joins cluster # On node 2's ejabberd.yml: hosts: - "nexgate.com" # Erlang cookie must match on all nodes # Set via environment variable # Both nodes discover each other automatically # Erlang distributed handles the rest # Result: # Message to usr-456 arrives on Node 1 # usr-456 connected to Node 2 # Erlang routes internally — transparent ``` --- ## 17. Database Schema ### calls (new in Phase 2) ``` calls ───────────────────────────────────────────── call_id UUID PK caller_id UUID receiver_id UUID conversation_id UUID FK → conversations type ENUM VOICE / VIDEO status ENUM RINGING / CONNECTED / COMPLETED / MISSED / DECLINED / FAILED started_at TIMESTAMPTZ answered_at TIMESTAMPTZ ended_at TIMESTAMPTZ duration_seconds INT relay_used BOOLEAN end_reason ENUM NORMAL / NETWORK / TIMEOUT / DECLINED ``` ### call_quality_logs (new in Phase 2) ``` call_quality_logs ───────────────────────────────────────────── log_id UUID PK call_id UUID FK → calls timestamp TIMESTAMPTZ direction ENUM OUTBOUND / INBOUND bitrate_kbps INT packet_loss_pct DECIMAL jitter_ms INT rtt_ms INT resolution TEXT "360p" "480p" "720p" or null codec_audio TEXT "opus" codec_video TEXT "h264" "vp8" or null ``` ### broadcast_channels (new in Phase 2) ``` broadcast_channels ───────────────────────────────────────────── channel_id UUID PK owner_id UUID userId or shopId owner_type ENUM USER / SHOP name TEXT description TEXT avatar_file_id UUID subscriber_count INT type ENUM PERSONAL / SHOP / SYSTEM created_at TIMESTAMPTZ ``` ### message_reactions (new in Phase 2) ``` message_reactions ───────────────────────────────────────────── id UUID PK message_id UUID FK → messages conversation_id UUID FK → conversations user_id UUID emoji TEXT "👍" "❤️" "😂" etc reacted_at TIMESTAMPTZ Unique constraint: (message_id, user_id) → one reaction per user per message → upsert on conflict replaces emoji ``` ### message_deletions (new in Phase 2) ``` message_deletions ───────────────────────────────────────────── id UUID PK message_id UUID FK → messages deleted_by UUID userId scope ENUM SELF / EVERYONE deleted_at TIMESTAMPTZ ``` ### messages table additions (Phase 2) ``` New columns added to existing messages table: edited_at TIMESTAMPTZ when last edited edit_count INT how many times edited original_body TEXT body before first edit (audit) deleted_at TIMESTAMPTZ soft delete timestamp deleted_by UUID who deleted delete_scope ENUM SELF / EVERYONE is_forwarded BOOLEAN was this forwarded forward_chain INT forwarding depth (1,2,3...) original_sender_name TEXT display name at forward time original_message_id UUID source message if forwarded reply_to_id UUID FK → messages (for replies) stanza_id TEXT Ejabberd XEP-0359 stable ID ``` --- ## 18. Commerce Stanzas & Custom Namespaces ### The Extensible Part of XMPP XMPP was designed to be extended by anyone for anything. The "X" in XMPP = Extensible. Any application can add custom XML elements inside standard XMPP stanzas using their own namespace. Ejabberd routes the entire stanza as-is — it never parses, validates, or modifies custom elements. Spring Boot reads them on the other side. ``` Standard stanza: Habari With NexGate custom element: Habari ... your custom data here ... Ejabberd: Routes whole stanza as-is ✅ Never touches nexgate-offer element ✅ Never validates it ✅ Just delivers it ✅ ``` ### NexGate Namespace Registry ``` All custom namespaces NexGate defines: urn:nexgate:commerce:1 product cards urn:nexgate:offer:1 price offer sessions urn:nexgate:groupbuy:1 Bei ya pamoja cards urn:nexgate:event:1 event cards urn:nexgate:feed:1 VP Feed post cards urn:nexgate:live:1 live stream cards urn:nexgate:audio:1 audio space cards urn:nexgate:system:1 system messages urn:nexgate:forward forwarding metadata urn:nexgate:states recording voice note state urn:nexgate:meta message metadata Versioning (:1, :2): Allows schema evolution Old app sees :1 → renders fine New app sees :2 → renders richer UI Old clients fall back to text No breaking changes ``` --- ### Product Card Stanza ``` Sent by: Spring Boot via Ejabberd REST API When: Buyer taps "Chat with Seller" on product page ``` ```xml Mteja anaomba habari: Samsung A15 PRODUCT_CARD usr-kibuti conv-789 prod-123 Samsung A15 450000 TZS https://cdn.nexgate.com/img.jpg 12 TechStore shop-456 2026-07-13T08:30:00Z ``` ``` Seller's app renders: ┌─────────────────────────────────────┐ │ 📦 Samsung A15 │ │ TZS 450,000 │ │ Inapatikana: Vipande 12 │ │ TechStore │ │ [Jibu] [Angalia Bidhaa] │ └─────────────────────────────────────┘ ``` --- ### Custom Price Offer Stanza ``` Sent by: Seller's app via Ejabberd WebSocket When: Seller attaches price offer from shop (both Flow 1 post-negotiation and Flow 2 direct attach) ``` ```xml Bei yako maalum: TZS 400,000 offer-uuid-abc conv-789 30 SELLER prod-123 Samsung A15 https://cdn.nexgate.com/img.jpg TechStore shop-456 450000 400000 TZS 50000 11 usr-amina ``` ``` Buyer's app renders: ┌─────────────────────────────────────┐ │ 💰 Bei Maalum Kwako │ │ Samsung A15 │ │ ~~TZS 450,000~~ │ │ TZS 400,000 (umepunguziwa 50,000) │ │ Inaisha: dakika 30 │ │ Idadi: [─ 1 +] │ │ [Kataa] [Endelea Kulipa →] │ └─────────────────────────────────────┘ ``` --- ### Offer Response Stanzas ```xml Nimekataa bei hii offer-uuid-abc DECLINED offer-uuid-abc EXPIRED offer-002 ``` --- ### Order Confirmation Stanza ``` Sent by: Spring Boot via Ejabberd REST API When: Buyer completes checkout successfully ``` ```xml Agizo limefanikiwa! ORDER_CONFIRMATION ord-xyz-789 conv-789 offer-uuid-abc Samsung A15 1 400000 TZS CONFIRMED ``` ``` Both buyer and seller see: ┌─────────────────────────────────────┐ │ ✅ Agizo Limethibitishwa │ │ Ord #ORD-XYZ-789 │ │ Samsung A15 × 1 │ │ TZS 400,000 imelipwa │ │ [Fuatilia Agizo] │ └─────────────────────────────────────┘ ``` --- ### Bei ya Pamoja Card Stanza ```xml Jiunge na group buy hii! gb-xyz prod-123 Samsung A15 https://cdn.nexgate.com/img.jpg 350000 450000 TZS 7 10 2026-07-13T18:00:00Z ``` --- ### Event Card Stanza ```xml Jiunge na event hii! evt-456 Dar Tech Summit 2026 2026-08-15T09:00:00Z Julius Nyerere ICC, Dar es Salaam 25000 TZS https://cdn.nexgate.com/evt.jpg 150 ``` --- ### VP Feed Post Card Stanza ```xml Angalia post hii post-789 Kibuti Mwangi https://cdn.nexgate.com/av.jpg Bidhaa mpya zimefika! 🔥 https://cdn.nexgate.com/post.jpg IMAGE 245 ``` --- ### Spring Boot — How It Handles Custom Stanzas ``` All stanzas arrive via RabbitMQ: chat.message.inbound Spring Boot parses XML and routes by namespace: Namespace detected Handler ────────────────────────────────────────────────── urn:nexgate:commerce:1 handleProductCard() urn:nexgate:offer:1 handleOfferSession() urn:nexgate:groupbuy:1 handleGroupBuy() urn:nexgate:event:1 handleEventCard() urn:nexgate:feed:1 handlePostCard() urn:nexgate:system:1 handleSystemMessage() none of the above handleTextMessage() ``` ### Offer Session — Spring Boot Processing ``` CUSTOM_PRICE_OFFER received: Spring Boot: Create message record (type: CUSTOM_PRICE_OFFER) Create commerce_offer_sessions record: offer_id: from stanza buyer_id: conversation partner shop_id: sender shop JID product snapshot: from stanza offer_price: from stanza (server authoritative) expires_at: now + valid_minutes status: PENDING Schedule RabbitMQ delayed job: delay: valid_minutes payload: { offerId, action: EXPIRE } Send FCM to buyer: "TechStore amekutumia bei maalum" Level: IMPORTANT Buyer taps "Endelea Kulipa": POST /checkout/initiate { offerId, quantity } Spring Boot: Validate: status=PENDING, not expired, buyer matches Update status: CHECKOUT Price from DB — never from client ✅ Return: { checkoutUrl, checkoutToken } Order completes: Update status: COMPLETED order_id: linked Send ORDER_CONFIRMATION stanza to conversation Expiry fires (RabbitMQ delayed job): Status still PENDING? → mark EXPIRED Status already changed? → do nothing Send OFFER_EXPIRED stanza to conversation ``` --- ### commerce_offer_sessions Table ``` commerce_offer_sessions ───────────────────────────────────────────────────── offer_id UUID PK conv_id UUID FK → conversations message_id UUID FK → messages shop_id UUID buyer_id UUID sent_by_staff UUID staff who sent (audit only) product_id UUID product_name TEXT product_image_url TEXT snapshot_json JSONB full product at offer time public_price BIGINT TZS offer_price BIGINT TZS (custom — server auth) currency TEXT TZS quantity_min INT quantity_max INT discount_amount BIGINT discount_pct DECIMAL status ENUM PENDING / ACCEPTED / DECLINED / EXPIRED / CHECKOUT / COMPLETED / CANCELLED / ABANDONED valid_minutes INT expires_at TIMESTAMPTZ initiated_by ENUM BUYER / SELLER notes TEXT created_at TIMESTAMPTZ responded_at TIMESTAMPTZ checkout_at TIMESTAMPTZ completed_at TIMESTAMPTZ order_id UUID FK → orders (after completion) ``` --- ## 19. Build Order NexGate chat is built from scratch — no migration, no Phase 1 to carry forward. This is the recommended sequence: ``` Week 1-2 — Local Experiments Ejabberd running in Docker locally Two containers (node1 + node2) clustered Auth bridge: Spring Boot validates XMPP tokens Send first message between two test JIDs Confirm Erlang dist working between nodes Confirm RabbitMQ events firing to Spring Boot Week 3-4 — PostgreSQL Schema + Chat Service All tables created (messages, conversations, receipts, calls, offer sessions, reactions etc) Spring Boot Chat Service: RabbitMQ consumers for all Ejabberd events Message persistence Receipt tracking Notification routing (FCM + Textfy) Week 5 — Ejabberd Staging Deployment Deploy to Hetzner staging VPS Two node cluster live Traefik sticky sessions configured Auth bridge connected to Chat Service Send first real message through staging Ejabberd Week 6-7 — Mobile SDK + Basic Chat NexGate Chat SDK (Android + iOS) Smack / XMPPFramework wrapper Clean send/receive API Auto-reconnect + Stream Management Text messages working end-to-end Typing indicators Delivery + read ticks Presence (online/offline) Week 8 — Message Interactions Reactions (XEP-0444) Edit messages (XEP-0308) Delete for everyone (XEP-0424) Forwarding (XEP-0297) Replies (XEP-0461) Week 9 — Voice Calls TURN credentials endpoint in Spring Boot Coturn deployed (Hetzner CX11) Jingle signaling through Ejabberd WebRTC on Android/iOS Opus audio confirmed on 2G test Coturn relay confirmed on EA network Week 10 — Video Calls H.264 video track added Adaptive resolution ladder Resolution ladder tested on 3G Week 11 — Commerce DMs Custom namespace stanzas: PRODUCT_CARD CUSTOM_PRICE_OFFER OFFER_DECLINED / OFFER_EXPIRED ORDER_CONFIRMATION Offer session lifecycle Both commerce flows (buyer initiates + seller attaches) Checkout redirect flow Shop inbox isolation + access control Week 12 — Group Chats + Broadcast MUC rooms (Ejabberd XEP-0045) Group message reactions Group typing indicators Broadcast channels (read-only MUC) Week 13 — Offline + Notifications RabbitMQ offline queue FCM HIGH priority integration Textfy SMS escalation Notification levels (NORMAL/IMPORTANT/CRITICAL) Catch-up banner on reconnect Week 14 — MessagePack MessagePack encoding in SDK Content-Type header detection in Ejabberd Both JSON + MessagePack supported simultaneously EA bandwidth savings confirmed Week 15 — Testing + EA Network Testing Test on actual Vodacom/Airtel SIM cards Test on Tecno/Infinix devices Test on 2G/3G networks Call quality on Coturn relay confirmed Commerce flow end-to-end confirmed Week 16 — Ship 🚀 Production deployment Two Ejabberd nodes live All features confirmed NexGate chat is live ``` --- ## Summary NexGate chat is built directly on Phase 2 architecture from scratch. No migration. No legacy code. Greenfield build on carrier-grade infrastructure from day one. **Ejabberd Cluster** (two Docker containers, same Hetzner VPS at launch) handles all WebSocket connections, XMPP stanza routing, presence, MUC group chats, Jingle voice/video signaling, and 25+ chat features via standard XEPs — all at zero custom code cost. Erlang Distribution connects the two nodes directly, routing messages between them in microseconds without Redis pub/sub. **Spring Boot Chat Service** owns all business logic — message persistence, commerce context, offer session lifecycle, shop inbox access control, notification routing, and call records. It communicates with Ejabberd asynchronously via RabbitMQ for all events except auth, which is synchronous HTTP because Ejabberd needs an immediate allow/deny decision. **Custom XMPP Namespaces** extend the protocol for NexGate's commerce features. Product cards, custom price offers, offer session responses, Bei ya pamoja cards, event cards, and post cards all travel as custom XML elements inside standard XMPP stanzas. Ejabberd routes them as-is — Spring Boot parses and handles them. Commerce messages are server-authoritative and immutable: offer prices come from the database, not the client. Public product prices are never touched. **WebRTC + Coturn** handles voice and video calls. Jingle stanzas through Ejabberd coordinate setup. Opus adapts audio from 64kbps on WiFi to 6kbps on 2G. H.264 hardware acceleration keeps video calls battery-friendly on EA phones. Coturn relay ensures calls work behind EA carrier NAT on Vodacom, Airtel, and Tigo. **MessagePack** reduces message frame size 60-65% — real saving for EA users on limited data bundles. Both JSON and MessagePack supported simultaneously during SDK rollout. The build is 16 weeks from local experiments to production. WhatsApp-class infrastructure. Commerce-aware from day one. EA network optimized throughout. --- *NexGate Chat Platform — Phase 2: Production Architecture v1.0* *QBIT SPARK | Ejabberd · Coturn · WebRTC · Commerce Stanzas · Edit · Delete · React · Forward* # VP Live & VP Audio Spaces #### Live Streaming Architecture **NexGate / QBIT SPARK | Version 1.0** *SRS · HLS · LiveKit · VP Live Video · VP Audio Radio · VP Audio Spaces* --- ## Table of Contents 1. [Overview](#1-overview) 2. [VP Live vs VP Audio — Key Differences](#2-vp-live-vs-vp-audio--key-differences) 3. [How Live Streaming Works](#3-how-live-streaming-works) 4. [VP Live — Video Streaming](#4-vp-live--video-streaming) 5. [VP Audio Radio — One Broadcaster Many Listeners](#5-vp-audio-radio--one-broadcaster-many-listeners) 6. [VP Audio Spaces — Multi Speaker Rooms](#6-vp-audio-spaces--multi-speaker-rooms) 7. [Live Chat — Ejabberd MUC](#7-live-chat--ejabberd-muc) 8. [Stream Key System](#8-stream-key-system) 9. [File Thunder Integration — VOD After Stream](#9-file-thunder-integration--vod-after-stream) 10. [Codecs & EA Network Strategy](#10-codecs--ea-network-strategy) 11. [Docker Deployment](#11-docker-deployment) 12. [Database Schema](#12-database-schema) 13. [Scale Path](#13-scale-path) --- ## 1. Overview VP Live and VP Audio Spaces live under **VP Feed** — the social pillar of NexGate. They are not separate products. They are the live expression layer of the social platform — where creators, merchants, and communities connect with their audiences in real time. ``` VP Feed ┌───────────────────────────────────────────────────┐ │ │ │ Social Posts Stories Reels Live │ │ │ │ ┌─────────────────┐ │ │ │ VP Live │ │ │ │ Video Stream │ │ │ ├─────────────────┤ │ │ │ VP Audio │ │ │ │ Radio │ │ │ ├─────────────────┤ │ │ │ VP Audio │ │ │ │ Spaces │ │ │ └─────────────────┘ │ └───────────────────────────────────────────────────┘ ``` All three modes share the same infrastructure foundation: SRS for ingest and transcoding, Cloudflare CDN for delivery, Ejabberd MUC for live chat, File Thunder for VOD processing, and Spring Boot for stream management and business logic. --- ## 2. VP Live vs VP Audio — Key Differences ``` VP Live VP Audio Radio VP Audio Spaces (Video) (Radio/Podcast) (Twitter Spaces) ────────────────────────────────────────────────────────────────────── Broadcasters 1 1 Multiple (up to 30) Viewers Unlimited Unlimited Unlimited listeners Direction One way One way Multi-speaker Broadcaster RTMP RTMP audio WebRTC (LiveKit) transport (video+audio) (audio only) Listener HLS video HLS audio HLS audio transport (adaptive) (adaptive) (listeners) WebRTC (speakers) Latency 6-15 seconds 6-15 seconds Speakers: <200ms Listeners: 6-15s Bandwidth High Very low Low (speakers) broadcaster (2-4 Mbps) (128 kbps) Very low (listeners) Bandwidth Medium Very low Very low listener (300kbps-2Mbps) (32-128 kbps) (32-128 kbps) Works on 2G? ❌ No ✅ Yes ✅ Listeners yes Live chat Ejabberd MUC Ejabberd MUC Ejabberd MUC Raise hand ❌ ❌ ✅ VOD after ✅ File Thunder ✅ File Thunder ✅ File Thunder New infra SRS SRS SRS + LiveKit ``` --- ## 3. How Live Streaming Works ### The Core Pattern — RTMP → HLS → CDN ``` Broadcasting (sending): Broadcaster's phone records camera + mic App encodes: H.264 video + AAC audio App streams via RTMP protocol to SRS server One stream upload from broadcaster Processing (server): SRS receives RTMP stream FFmpeg transcodes to multiple quality variants Packages into HLS format (2-second chunks) Writes chunks to MinIO storage every 2 seconds Delivery (viewing): Cloudflare CDN pulls chunks from MinIO Caches chunks at edge nodes globally Viewers request HLS playlist → adaptive player picks quality 10,000 viewers = 10,000 CDN requests, NOT 10,000 SRS requests SRS barely notices the viewer count Why HLS and not WebRTC for viewers: WebRTC to viewers: broadcaster uploads N streams (one per viewer) HLS via CDN: broadcaster uploads 1 stream → CDN serves all At 10,000 viewers: WebRTC = impossible, HLS = trivial ``` ### HLS — What It Actually Is ``` HLS (HTTP Live Streaming) — Apple's open standard SRS generates: master.m3u8 → playlist of all quality variants 360p/playlist.m3u8 → playlist for 360p variant 360p/seg_000.ts → 2-second video chunk 360p/seg_001.ts → next 2-second chunk 720p/playlist.m3u8 720p/seg_000.ts ... master.m3u8 looks like: #EXTM3U #EXT-X-STREAM-INF:BANDWIDTH=400000,RESOLUTION=640x360 360p/playlist.m3u8 #EXT-X-STREAM-INF:BANDWIDTH=1500000,RESOLUTION=1280x720 720p/playlist.m3u8 Player (ExoPlayer / AVPlayer): Downloads master.m3u8 first Measures current network speed Picks 360p if on 3G → plays seg_000.ts → seg_001.ts → ... Switches to 720p if network improves → seamless All automatic — zero app code needed for quality switching ``` --- ## 4. VP Live — Video Streaming ### Full Architecture ``` [Broadcaster Phone] │ │ RTMP stream │ rtmp://stream.nexgate.com/live/{streamKey} │ H.264 video + AAC audio │ ~2-4 Mbps upload ▼ [SRS Media Server] │ ├── Validates stream key: │ POST /internal/stream/validate │ { streamKey: "abc123" } │ Spring Boot: ✅ allow or ❌ reject │ ├── Receives raw RTMP stream │ ├── FFmpeg transcoding (real-time): │ 1080p H.264 → 3 Mbps (WiFi viewers) │ 720p H.264 → 1.5 Mbps (4G viewers) │ 480p H.264 → 600 kbps (3G viewers) │ 360p H.264 → 300 kbps (2G viewers) │ ├── Package as HLS: │ Segment every 2 seconds │ live/{streamKey}/master.m3u8 │ live/{streamKey}/360p/seg_NNN.ts │ live/{streamKey}/720p/seg_NNN.ts │ └── Write to MinIO: nexgate-live bucket New segments every 2 seconds │ ▼ [Cloudflare CDN] │ Pulls from MinIO automatically │ Caches at edge (Nairobi edge closest to EA) │ Short TTL: 10 seconds (live content) │ ▼ [Viewers — ExoPlayer (Android) / AVPlayer (iOS)] Requests master.m3u8 Player picks quality based on network Downloads .ts segments every 2 seconds Seamless adaptive quality switching ``` ### Stream Key Validation Flow ``` Broadcaster taps "Go Live" in app │ ▼ POST /live/start Spring Boot: Generate unique stream key Store in DB: stream_key: "abc123" user_id: usr-kibuti status: PENDING created_at: now Return stream key to app │ App connects RTMP: rtmp://stream.nexgate.com/live/abc123 │ SRS receives connection │ ▼ POST /internal/stream/validate (SRS webhook) Spring Boot checks: Key exists? ✅ User account active? ✅ User has live permission? ✅ No other active stream for this user? ✅ → 200 OK → SRS allows stream → Update DB: status: LIVE, started_at: now → Notify followers via FCM: "Kibuti anastreamu sasa! Tazama live" → Create Ejabberd MUC room: live-abc123@conference.nexgate.com ``` ### Broadcaster App — What Mobile Dev Implements ``` Android library: rtmp-rtsp-stream-client-java iOS library: HaishinKit (Swift) Steps for broadcaster app: 1. GET /live/start → receive stream key 2. Initialize camera + microphone 3. Connect RTMP to stream.nexgate.com/live/{key} 4. Start streaming — library handles everything: H.264 encoding (hardware) AAC audio encoding RTMP packet framing Network reconnection on drop 5. Show: viewer count (from Redis via REST poll) live comments (from Ejabberd MUC via WS) duration timer 6. Tap End → POST /live/end → cleanup Adaptive upload bitrate: Library monitors upload speed Reduces video quality if upload struggles Broadcaster's bad network → lower quality for viewers Never drops stream if avoidable ``` ### Viewer App — What Mobile Dev Implements ``` Android: ExoPlayer (Google's official video player) iOS: AVPlayer (built into iOS, zero setup) Steps for viewer app: 1. GET /live/{streamId}/url Response: { masterUrl, viewerCount, startedAt } 2. Feed masterUrl to ExoPlayer/AVPlayer 3. Player handles everything automatically: Downloads master.m3u8 Picks quality based on network Downloads segments every 2s Switches quality up/down seamlessly 4. Join Ejabberd MUC room → show live comments 5. Player shows: loading → buffering → playing That is genuinely all the viewer needs to implement. HLS + ExoPlayer/AVPlayer is the easiest viewer experience to build in all of mobile development. ``` --- ## 5. VP Audio Radio — One Broadcaster Many Listeners ### Why Audio Radio Matters for EA ``` VP Live video: Broadcaster needs: 2-4 Mbps upload Viewer needs: 300kbps minimum Data cost viewer: ~900MB per hour at 360p Works on: 4G and strong 3G only VP Audio Radio: Broadcaster needs: 64-128 kbps upload Listener needs: 32 kbps minimum Data cost listener: ~15MB per hour at 32kbps Works on: 2G, Edge, any connection For a farmer in rural Tanzania with 2G: VP Live video → impossible, too expensive VP Audio Radio → accessible, affordable Use cases: Live podcast / commentary Religious broadcasts (huge in EA) Political discussions Community announcements Sports commentary Language learning sessions Business webinars (audio only) ``` ### Architecture — Same SRS, Audio Only ``` [Broadcaster Phone] │ │ RTMP audio only (no video track) │ AAC codec, 128 kbps │ rtmp://stream.nexgate.com/audio/{streamKey} ▼ [SRS Media Server] │ ├── Same validation flow as VP Live │ ├── FFmpeg transcoding (audio only): │ AAC 128 kbps → good network listeners │ AAC 64 kbps → 3G listeners │ AAC 32 kbps → 2G listeners │ ├── Package as HLS audio: │ audio/{streamKey}/master.m3u8 │ audio/{streamKey}/128k/seg_NNN.aac │ audio/{streamKey}/32k/seg_NNN.aac │ └── Write to MinIO: nexgate-live bucket │ ▼ [Cloudflare CDN] │ ▼ [Listeners — ExoPlayer / AVPlayer] HLS audio playlist Adaptive bitrate: 128k → 32k automatically Same player, same code — just no video surface ``` ### Codec Choice — AAC Not Opus ``` Why AAC for HLS audio radio (not Opus): Opus is better quality at low bitrates — true But HLS has a compatibility requirement: Apple mandates AAC for HLS audio AVPlayer on iOS does not support Opus in HLS Using Opus → iOS listeners cannot play AAC → works on every device, every OS Opus is used for: Voice calls (WebRTC — different transport) Voice notes (file-based, not streaming) AAC is used for: VP Live audio track (in video stream) VP Audio Radio (HLS streaming) VP Audio Spaces listener HLS output AAC at 32kbps for EA: Acceptable speech quality ~15MB per hour Works on any 2G connection Universal device support ``` --- ## 6. VP Audio Spaces — Multi Speaker Rooms ### The Concept ``` Not one broadcaster → many listeners Multiple people in a shared audio room Some speak, many listen Listeners can raise their hand to speak Host controls who gets the mic Like Twitter Spaces, Clubhouse, Discord Stage Channels Key insight: Speakers need LOW LATENCY (<200ms) to have a natural conversation HLS (6-15s delay) is too slow for speakers Listeners just need to HEAR clearly HLS delay is fine — they're not responding HLS scales to millions via CDN Solution: TWO transport layers in one room Speakers → WebRTC (LiveKit SFU) → <200ms Listeners → HLS via CDN → 6-15s delay → millions scale ``` ### LiveKit SFU — What It Is ``` SFU = Selective Forwarding Unit Traditional conference (MCU): Server mixes ALL audio into one stream Sends mixed stream to everyone High CPU (server does all mixing) Simple client LiveKit SFU approach: Each speaker sends audio once to LiveKit LiveKit forwards each speaker's stream to all other speakers Speakers' apps mix locally (device CPU) Much lower server CPU Lower latency Better quality (no mixing artifacts) For listeners: LiveKit outputs a mixed HLS stream Goes through SRS → Cloudflare CDN Listeners get one mixed audio stream Same HLS pattern as Audio Radio Who built LiveKit: The same team that built Twitter Spaces Then open sourced it Actively maintained, Docker ready Official Android + iOS SDKs available ``` ### Full Architecture ``` [Speaker A phone] ──WebRTC──▶┐ [Speaker B phone] ──WebRTC──▶│ [Speaker C phone] ──WebRTC──▶│ ▼ [LiveKit SFU] │ ┌──────────┼──────────────┐ │ │ │ WebRTC fwd HLS output Room events to speakers (mixed audio) to Spring Boot │ │ [Speakers [SRS receives hear each HLS from LiveKit] other live] │ ▼ [Cloudflare CDN] │ ▼ [Thousands of listeners via HLS audio player] ExoPlayer / AVPlayer (same as Audio Radio) Room events (raise hand, join, leave): LiveKit → Spring Boot via webhook Spring Boot → Ejabberd MUC → all participants Ejabberd MUC → Listeners also see events (who joined as speaker etc) ``` ### Raise Hand Flow ``` Listener wants to speak: │ taps "Raise Hand" 🖐 │ sends via Ejabberd WS to MUC room: │ { type: RAISE_HAND, roomId: "space-abc" } │ ▼ Spring Boot: Records raise hand request Notifies host via Ejabberd WS: { type: HAND_RAISED, userId, displayName } Host sees list of raised hands in UI │ Host taps "Allow to speak" on a listener: │ ▼ Spring Boot: Calls LiveKit API: Update participant permissions: canPublish: true ← now allowed to send audio Generate new LiveKit token for this user (speaker token, not listener token) Send token to user via Ejabberd WS: { type: SPEAKER_PROMOTED, livekitToken: "..." } │ Former listener's app: Receives promotion event Stops HLS player (was listening at 15s delay) Connects WebRTC to LiveKit with speaker token Starts sending audio Now hears speakers at <200ms latency Other speakers hear them immediately │ Host can also: Lower someone's hand (dismiss) Mute a specific speaker Remove speaker (back to listener) End the space entirely ``` ### Speaker vs Listener — Connection Types ``` ┌──────────────────────────────────────────────────────┐ │ Audio Space Room │ │ │ │ Speakers (up to ~20-30): │ │ Connected via WebRTC to LiveKit │ │ Send and receive audio streams │ │ Latency: <200ms (real conversation) │ │ Connection: persistent WebRTC │ │ │ │ Listeners (unlimited): │ │ Connected via HLS to Cloudflare CDN │ │ Receive mixed audio only │ │ Latency: 6-15 seconds (fine — just listening) │ │ Connection: HTTP requests every 2s │ │ Scale: millions — CDN handles it │ │ │ │ All participants: │ │ Connected to Ejabberd MUC room │ │ Text chat, reactions, raise hand events │ │ Room membership awareness │ └──────────────────────────────────────────────────────┘ ``` ### LiveKit Token System ``` Spring Boot manages all LiveKit tokens (LiveKit has official Java SDK) Host token: canPublish: true canSubscribe: true roomAdmin: true → full control, can speak, manage Speaker token: canPublish: true canSubscribe: true roomAdmin: false → can speak, cannot manage room Listener token: canPublish: false ← cannot send audio canSubscribe: true ← can hear speakers roomAdmin: false → receive only Token generation: GET /audio-spaces/{spaceId}/join Spring Boot checks: Is user the host? → host token Is user an approved speaker? → speaker token Otherwise → listener token (gets HLS URL instead) ``` ### LiveKit Docker Config ```yaml livekit: image: livekit/livekit-server:latest container_name: livekit restart: unless-stopped ports: - "7880:7880" # HTTP API (Spring Boot calls here) - "7881:7881" # WebRTC TCP - "7882:7882/udp" # WebRTC UDP (primary) - "50000-60000:50000-60000/udp" # ICE relay ports volumes: - ./livekit/livekit.yaml:/etc/livekit.yaml command: --config /etc/livekit.yaml ``` ```yaml # livekit.yaml port: 7880 rtc: tcp_port: 7881 udp_port: 7882 use_external_ip: true redis: address: redis:6379 # reuses existing Redis ✅ turn: enabled: true domain: turn.nexgate.com tls_port: 5349 credential: "${COTURN_SECRET}" # reuses existing Coturn ✅ room: max_participants: 10000 empty_timeout: 300 ``` ``` LiveKit reuses: Redis → already deployed ✅ Coturn → already deployed for calls ✅ No new infrastructure beyond LiveKit container itself ``` --- ## 7. Live Chat — Ejabberd MUC All three live modes (VP Live, Audio Radio, Audio Spaces) use Ejabberd MUC rooms for real-time text interaction. ### Room Lifecycle ``` Stream / space starts: │ Spring Boot → Ejabberd REST API: POST /api/create_room { name: "live-{streamId}", service: "conference.nexgate.com" } Room created: live-abc@conference.nexgate.com │ Broadcaster / host auto-joined as moderator │ Viewers / listeners join room as participants: App connects Ejabberd WS Sends MUC join stanza: │ Comments sent as MUC messages: Mzuri sana! 🔥 │ All room members receive instantly No delay — Ejabberd MUC is real-time │ Stream / space ends: Spring Boot → Ejabberd REST API: POST /api/destroy_room { name: "live-abc", service: "conference.nexgate.com" } Room destroyed, members disconnected ``` ### Special Events in Live Chat ``` Beyond text comments, the MUC room carries: Reactions (emoji bursts): { type: REACTION, emoji: "🔥", userId, displayName } Client renders floating emoji animation Gifts: { type: GIFT, giftId, giftName, amount, userId, displayName } Client renders gift animation Spring Boot processes payment separately Raise hand (Audio Spaces only): { type: RAISE_HAND, userId, displayName } Host sees in management panel Speaker promoted (Audio Spaces only): { type: SPEAKER_PROMOTED, userId, displayName } All participants see "Amina joined as speaker" Viewer count updates: Broadcast every 30 seconds from Spring Boot { type: VIEWER_COUNT, count: 12453 } Product card dropped by broadcaster: { type: PRODUCT_CARD, productId, name, price } Viewers tap → go to VP Shop product page Commerce during live ✅ ``` ### Viewer / Listener Count ``` Two sources of truth: 1. Ejabberd MUC occupant count: GET ejabberd REST /api/get_room_occupants_count { room: "live-abc", host: "conference.nexgate.com" } → exact WebSocket-connected count 2. Redis counter (includes HLS-only listeners): INCR live:{streamId}:viewers → on HLS playlist request DECR → on playlist stop / timeout More accurate for Audio Radio/Spaces where many listeners never connect WS Display count = Redis counter (higher, more accurate) Spring Boot broadcasts to MUC every 30 seconds ``` --- ## 8. Stream Key System ### Stream Key Design ``` Stream key = single-use authentication token Broadcaster uses it to connect RTMP to SRS SRS validates with Spring Boot before accepting stream Format: random 32-character alphanumeric string Example: nx_live_a1b2c3d4e5f6g7h8i9j0k1l2m3n4 Lifecycle: PENDING → generated, not yet used LIVE → broadcaster connected, stream active ENDED → stream finished normally EXPIRED → generated but never used (24h TTL) REVOKED → manually stopped by admin One active stream per user at a time Attempting second stream → rejected by Spring Boot validation ``` ### SRS Webhooks to Spring Boot ``` SRS fires these events to Spring Boot: on_publish → broadcaster connected RTMP Spring Boot: validate key, update status LIVE, notify followers FCM, create Ejabberd MUC room, create LiveKit room (if audio space) on_unpublish → broadcaster disconnected Spring Boot: update status ENDED, trigger File Thunder for VOD, destroy Ejabberd MUC room, log stream duration + peak viewers on_play → viewer started watching HLS Spring Boot: increment Redis viewer counter on_stop → viewer stopped watching Spring Boot: decrement Redis viewer counter ``` --- ## 9. File Thunder Integration — VOD After Stream ### What Happens After Stream Ends ``` Stream ends (broadcaster taps End / disconnects) │ SRS fires on_unpublish webhook │ Spring Boot: Update stream record: status ENDED Trigger File Thunder for VOD processing SRS has saved full recording as .mp4 │ ▼ Spring Boot → File Thunder: POST /api/v1/upload/request (HMAC signed) { ownerId: broadcasterId, domain: POSTS, context: LIVE_RECORDING, filename: "stream_{streamId}.mp4", mimeType: "video/mp4" } Returns: presigned MinIO PUT URL │ Spring Boot pulls recording from SRS Uploads to MinIO via presigned URL POST /api/v1/confirm { fileId } │ ▼ File Thunder VideoWheel processes: HLS transcoding (all quality variants) Thumbnail extraction (best frame detection) Watermark: "@{broadcasterUsername}" NO outro — live recordings are long NO shortClip — full stream only Store in nexgate-public bucket │ ▼ File Thunder fires webhook: media ready Spring Boot: Creates VOD post on broadcaster's profile "Watch replay" button appears Appears in VP Feed for followers Stream record linked to VOD fileId ``` ### New File Thunder Contexts for Live ``` Existing contexts (unchanged): SOCIAL_VIDEO regular video posts DM_ATTACHMENT files sent in DMs DIGITAL_PRODUCT digital goods in VP Shop ... New contexts added for live: LIVE_RECORDING full stream VOD VideoWheel — no outro, no shortClip always HLS, always long AUDIO_RECORDING audio space / radio recording AudioWheel processes outputs: .m4a (AAC) podcast episode on profile waveform extracted (like voice notes) ``` ### nexgate-live MinIO Bucket ``` Existing buckets: nexgate-raw temp uploads nexgate-public social content nexgate-private DMs and private files nexgate-digital VP Shop digital products New bucket: nexgate-live live stream segments only Why separate: SRS writes directly here (not via File Thunder) Short TTL segments — deleted after stream ends + VOD ready Different CDN caching rules (10s TTL vs 1 year for VOD) Different access pattern (SRS writes, CDN reads) Easy to monitor storage growth separately Lifecycle: Stream starts → SRS creates live/{streamKey}/ folder During stream → .ts segments written every 2 seconds Stream ends → Spring Boot schedules cleanup job VOD confirmed → delete nexgate-live/{streamKey}/ folder Total life: stream duration + ~1 hour buffer ``` --- ## 10. Codecs & EA Network Strategy ### VP Live Video Codecs ``` Broadcaster encoding (phone → SRS): Video: H.264 (hardware encoder — mandatory) Software H.264 too slow for real-time on phones H.264 hardware support: every phone since 2013 Audio: AAC 128kbps (RTMP standard) Container: RTMP (streaming protocol) SRS transcoding (server-side): Receives H.264 + AAC Transcodes to HLS quality ladder: Quality Video bitrate Audio Resolution EA target ───────────────────────────────────────────────────────── 1080p 3 Mbps 128k 1920×1080 WiFi only 720p 1.5 Mbps 128k 1280×720 4G 480p 600 kbps 64k 854×480 3G 360p 300 kbps 48k 640×360 2G minimum ───────────────────────────────────────────────────────── ExoPlayer/AVPlayer auto-selects based on network ``` ### VP Audio Codecs ``` Audio Radio (broadcaster → SRS): Codec: AAC 128kbps Container: RTMP audio only Audio Radio (SRS → HLS): 128kbps → WiFi/4G listeners 64kbps → 3G listeners 32kbps → 2G listeners (15MB/hour — affordable) Audio Spaces (speaker → LiveKit): Codec: Opus (WebRTC standard) Adaptive: 32-64kbps per speaker Echo cancellation: mandatory (multiple people) Noise suppression: mandatory (EA background noise) Audio Spaces (LiveKit → HLS for listeners): LiveKit mixes speaker streams Outputs mixed audio → SRS → HLS Same AAC ladder as Audio Radio Listeners hear all speakers in one stream ``` ### Adaptive Streaming — EA Principle ``` The player always knows the network speed because it measures how fast segments download Segment download faster than playback → upgrade quality Segment download slower than playback → downgrade quality For a viewer in Dodoma on shaky 3G: Opens stream → starts at 360p (safe default) Network good → player tries 480p Stays stable → tries 720p Network drops → immediately back to 360p No rebuffering if switch is fast enough Buffer strategy: Player buffers 3-4 segments ahead (6-8 seconds) Gives time to switch quality before buffer empties Viewer may notice brief quality dip — never a freeze NexGate player config recommendation: Min buffer: 6 seconds Max buffer: 30 seconds Quality switch: aggressive downgrade, conservative upgrade → Prioritize uninterrupted playback over quality → EA networks fluctuate — better to be at 360p than buffering ``` --- ## 11. Docker Deployment ### Full docker-compose for Live Features ```yaml # SRS Media Server srs: image: ossrs/srs:5 container_name: srs restart: unless-stopped ports: - "1935:1935" # RTMP ingest (broadcaster connects here) - "8080:8080" # HTTP API + HLS output - "1985:1985" # SRS management API volumes: - ./srs/srs.conf:/usr/local/srs/conf/srs.conf - ./srs/logs:/usr/local/srs/logs - ./srs/recordings:/usr/local/srs/objs/recordings depends_on: - chat-service networks: - nexgate-internal # LiveKit SFU (Audio Spaces) livekit: image: livekit/livekit-server:latest container_name: livekit restart: unless-stopped ports: - "7880:7880" - "7881:7881" - "7882:7882/udp" - "50000-60000:50000-60000/udp" volumes: - ./livekit/livekit.yaml:/etc/livekit.yaml command: --config /etc/livekit.yaml depends_on: - redis networks: - nexgate-internal ``` ### SRS Config Highlights ``` listen 1935; # RTMP port max_connections 1000; vhost __defaultVhost__ { # Validate stream key with Spring Boot http_hooks { enabled on; on_publish http://chat-service:8082/internal/stream/validate; on_unpublish http://chat-service:8082/internal/stream/ended; on_play http://chat-service:8082/internal/stream/viewer-join; on_stop http://chat-service:8082/internal/stream/viewer-leave; } # HLS output for viewers hls { enabled on; hls_path ./objs/nginx/html; hls_fragment 2; # 2 second chunks hls_window 10; # keep last 10 chunks in playlist } # FFmpeg transcoding to multiple qualities transcode { enabled on; ffmpeg /usr/bin/ffmpeg; engine 360p { enabled on; vcodec libx264; vbitrate 300; vfps 15; vwidth 640; vheight 360; acodec aac; abitrate 48; output rtmp://localhost:1935/live360p/{stream}; } engine 720p { enabled on; vcodec libx264; vbitrate 1500; vfps 30; vwidth 1280; vheight 720; acodec aac; abitrate 128; output rtmp://localhost:1935/live720p/{stream}; } } } ``` ### Traefik — RTMP Does Not Go Through Traefik ``` Important: RTMP is TCP port 1935 Traefik handles HTTP/HTTPS only RTMP port 1935 exposed directly on VPS What Traefik does handle: stream.nexgate.com → SRS port 8080 (HLS output) TLS termination for HLS delivery RTMP broadcaster connects: rtmp://stream.nexgate.com:1935/live/{key} No TLS on RTMP (RTMPS is complex, not needed for launch) HLS viewers connect via Cloudflare CDN: https://cdn.nexgate.com/live/{key}/master.m3u8 Cloudflare pulls from SRS port 8080 Traefik handles TLS for this path ``` --- ## 12. Database Schema ### live_streams ``` live_streams ───────────────────────────────────────────── stream_id UUID PK broadcaster_id UUID FK → users type ENUM VIDEO / AUDIO_RADIO / AUDIO_SPACE title TEXT description TEXT cover_file_id UUID File Thunder fileId (stream thumbnail) stream_key TEXT UNIQUE, used for RTMP auth status ENUM PENDING / LIVE / ENDED / EXPIRED / REVOKED started_at TIMESTAMPTZ ended_at TIMESTAMPTZ duration_seconds INT peak_viewers INT total_viewers INT muc_room_id TEXT Ejabberd MUC room name vod_file_id UUID File Thunder fileId after processing created_at TIMESTAMPTZ ``` ### audio_spaces ``` audio_spaces ───────────────────────────────────────────── space_id UUID PK stream_id UUID FK → live_streams livekit_room_id TEXT LiveKit room name host_id UUID FK → users title TEXT status ENUM SCHEDULED / LIVE / ENDED max_speakers INT default 30 started_at TIMESTAMPTZ ended_at TIMESTAMPTZ ``` ### audio_space_participants ``` audio_space_participants ───────────────────────────────────────────── space_id UUID FK → audio_spaces user_id UUID role ENUM HOST / SPEAKER / LISTENER joined_at TIMESTAMPTZ left_at TIMESTAMPTZ hand_raised_at TIMESTAMPTZ promoted_at TIMESTAMPTZ when promoted from listener to speaker promoted_by UUID host who approved ``` ### stream_viewer_stats ``` stream_viewer_stats ───────────────────────────────────────────── stat_id UUID PK stream_id UUID FK → live_streams timestamp TIMESTAMPTZ viewer_count INT quality_360p_pct DECIMAL % of viewers on 360p quality_720p_pct DECIMAL % of viewers on 720p avg_watch_seconds INT ``` --- ## 13. Scale Path ### Current Architecture Limits ``` Single SRS node (Hetzner CPX31 — €19/month): Concurrent streams: ~200 (with transcoding) Concurrent viewers: ~50,000 (before CDN helps) Bandwidth: 20TB/month included With Cloudflare CDN: Concurrent viewers: Unlimited (CDN absorbs it) SRS only serves cache misses 99%+ cache hit rate → SRS barely loaded LiveKit single node: Concurrent spaces: ~500 Speakers per space: up to 30 Listeners per space: Unlimited (HLS via CDN) This is enough for NexGate launch and strong early growth — tens of thousands of users ``` ### Growth Stage — SRS Horizontal Scale ``` When 200 concurrent streams is not enough: SRS Origin node: Receives RTMP from all broadcasters Passes stream to Transcode Farm Transcode Farm (2-3 nodes): Each node handles FFmpeg transcoding Horizontal — add nodes as streams grow CPU-bound work distributed SRS Edge nodes: Serve HLS to viewers Pull from Origin Multiple edges → load distributed ┌──────────────────────────────────────────┐ │ Broadcaster → SRS Origin │ │ │ │ │ Transcode Farm │ │ (3 nodes, FFmpeg) │ │ │ │ │ ┌──────────┴──────────┐ │ │ SRS Edge 1 SRS Edge 2 │ │ │ │ │ │ Cloudflare CDN ────────────┘ │ │ │ │ │ All viewers (millions) │ └──────────────────────────────────────────┘ ``` ### WeChat EA Scale — Infrastructure ``` Broadcaster latency problem: Current: Broadcaster in Dar → stream goes to Hetzner Germany 150-300ms upload latency Acceptable but not ideal At scale: SRS nodes in EA region Google Cloud Johannesburg OR AWS Cape Town Broadcaster → nearby SRS → low latency upload Better broadcaster experience Storage cost at scale: Current: MinIO on Hetzner At scale: Cloudflare R2 Zero egress cost (unlike AWS S3 which charges per GB) S3 compatible → zero code change to migrate At millions of viewer-hours: massive cost saving Transcoding cost: CPU-heavy work At scale: GPU-accelerated FFmpeg nodes NVIDIA hardware encoding 5-10x faster than CPU Lower cost per stream transcoded ``` --- ## Summary VP Live and VP Audio Spaces are the live social layer of NexGate — living under VP Feed alongside regular posts, stories, and reels. All three modes (VP Live video, VP Audio Radio, VP Audio Spaces) share the same infrastructure foundation. SRS handles all RTMP ingest and transcoding. Cloudflare CDN distributes HLS to unlimited viewers and listeners. Ejabberd MUC powers live chat and room events for all modes. File Thunder processes every stream into a VOD after it ends. Spring Boot manages stream keys, webhooks, room lifecycle, and all business logic. VP Audio Spaces adds LiveKit SFU for the multi-speaker experience — speakers connect via WebRTC for real-time conversation while listeners receive the same HLS audio stream that Audio Radio uses, scaled to millions via Cloudflare CDN. The EA network strategy is woven into every decision: HLS adaptive streaming down to 32kbps means VP Audio Radio works on 2G in rural Tanzania. Video quality ladders from 1080p to 360p ensure VP Live is accessible on 3G. The entire viewer and listener experience requires only ExoPlayer or AVPlayer — the simplest possible mobile integration. For VOD, File Thunder's VideoWheel and new AudioWheel process every recording automatically after the stream ends — creating replay content with thumbnails, watermarks, and adaptive variants, stored in nexgate-public for CDN delivery. The live platform generates permanent content with zero extra work. --- *NexGate VP Live & VP Audio Spaces — Architecture v1.0* *QBIT SPARK | SRS · LiveKit · HLS · Ejabberd MUC · File Thunder VOD* # NexGate Chat — Phase 1 ## Foundation & Local Experiments **NexGate / QBIT SPARK | Version 1.0** *Spring Boot WebSocket · Commerce DMs · Offline Delivery · Local Experiments* --- ## Table of Contents 1. [Phase 1 Goals](#1-phase-1-goals) 2. [What Ships in Phase 1](#2-what-ships-in-phase-1) 3. [Architecture Overview](#3-architecture-overview) 4. [The Four Wheels](#4-the-four-wheels) 5. [Service Design](#5-service-design) 6. [Data Flows](#6-data-flows) 7. [Commerce DM Flows](#7-commerce-dm-flows) 8. [Offline Delivery & Notification Escalation](#8-offline-delivery--notification-escalation) 9. [Message Status System](#9-message-status-system) 10. [Database Schema](#10-database-schema) 11. [Inbox Model Implementation](#11-inbox-model-implementation) 12. [Local Experiments](#12-local-experiments) --- ## 1. Phase 1 Goals Phase 1 is about shipping fast and learning. The goal is not to build the perfect infrastructure from day one. The goal is to get NexGate chat into users' hands as quickly as possible — while running local experiments in parallel that prepare for Phase 2. ``` Two tracks running simultaneously: Track A — Production (ship it): Track B — Experiments (learn it): Spring Boot WebSocket Ejabberd local Docker setup Text chat + voice notes Spring Boot ↔ Ejabberd auth bridge Commerce DMs (both flows) WebRTC voice call on emulators Offline delivery + notifications Coturn TURN relay local test Message receipts MessagePack encoding test Isolated shop inbox ``` Phase 1 architecture is intentionally simpler than Phase 2. Everything designed here carries forward — the schema, the commerce logic, the notification system, the inbox model. Phase 2 only swaps the transport layer. --- ## 2. What Ships in Phase 1 ### Messaging Features ``` 1:1 Personal DMs text, voice notes, media cards Group chats up to 500 members Broadcast channels creator → fans (one-way) Voice notes Opus recorded, waveform rendered Rich content cards product, custom price, event, Bei ya pamoja, post, stream link ``` ### Commerce DM Features ``` Buyer initiates chat from product page → product card auto-appears Seller attaches product WhatsApp-style from inside any conversation Custom price offer private to buyer, public price unchanged Proceed to checkout button in chat → redirects to checkout Order updates in thread confirmation, shipping, delivery ``` ### Inbox Features ``` Isolated shop inbox separate tab per shop Personal inbox always private, owner only Shop inbox shared with authorized staff (Pro tier) Message receipts sent / delivered / read ticks Typing indicators ephemeral, Redis TTL based Online presence shown in conversation header ``` ### Notification Features ``` FCM push (Android + iOS) HIGH priority, bypasses Doze mode Textfy SMS escalation CRITICAL and IMPORTANT messages Offline queue RabbitMQ holds messages until delivery Catch-up banner summary on reconnect after offline ``` --- ## 3. Architecture Overview ``` ┌─────────────────────────────────────────────────┐ │ NexGate Mobile App │ └──────────────┬──────────────────────────────────┘ │ │ WebSocket (JSON over WS) │ wss://chat.nexgate.com │ ┌──────────────▼──────────────────────────────────┐ │ Spring Boot Chat Gateway │ │ │ │ Manages WebSocket connections │ │ Validates JWT on connect │ │ Routes incoming frames via Redis pub/sub │ │ Pushes outgoing frames to clients │ │ Registers presence in Redis │ │ Thin — zero business logic │ └──────────────┬──────────────────────────────────┘ │ │ Redis pub/sub │ ┌──────────────▼──────────────────────────────────┐ │ Spring Boot Chat Service │ │ │ │ Message validation + persistence │ │ Conversation + inbox management │ │ Commerce context handling │ │ Receipt tracking │ │ Notification routing │ │ Offline escalation │ │ Shop inbox role enforcement │ └──────┬──────────┬──────────┬────────────────────┘ │ │ │ ▼ ▼ ▼ ┌──────────┐ ┌────────┐ ┌───────────────────────┐ │PostgreSQL│ │ Redis │ │ RabbitMQ │ │ │ │ │ │ │ │messages │ │presence│ │ chat.offline.delivery │ │convs │ │hot msgs│ │ chat.notify.push │ │receipts │ │typing │ │ chat.notify.escalation│ │calls │ │pub/sub │ │ chat.commerce.events │ └──────────┘ └────────┘ └───────────┬───────────┘ │ ┌────────────┼────────────┐ ▼ ▼ ▼ ┌─────┐ ┌───────┐ ┌────────┐ │ FCM │ │Textfy │ │ Main │ │push │ │ SMS │ │Backend │ └─────┘ └───────┘ └────────┘ ``` ### Key Rule ``` Gateway never touches business logic Chat Service never manages WS connections Both communicate only via Redis pub/sub Gateway → Redis pub/sub → Chat Service (inbound) Chat Service → Redis pub/sub → Gateway (outbound) ``` --- ## 4. The Four Wheels Just as File Thunder has four processing wheels, the Phase 1 chat engine has four foundational components: ``` ┌─────────────────────────────────────────────────┐ │ NEXGATE CHAT ENGINE │ │ │ │ ┌──────────┐ ┌──────────┐ ┌────────────┐ │ │ │ Wheel 1 │ │ Wheel 2 │ │ Wheel 3 │ │ │ │ │ │ │ │ │ │ │ │ Netty │ │ Redis │ │ RabbitMQ │ │ │ │ WS │ │ State │ │ Queue │ │ │ └──────────┘ └──────────┘ └────────────┘ │ │ │ │ ┌────────────┐ │ │ │ Wheel 4 │ │ │ │ │ │ │ │ Textfy │ │ │ │ SMS │ │ │ └────────────┘ │ └─────────────────────────────────────────────────┘ ``` **Wheel 1 — Netty WebSocket** Spring Boot uses Netty under the hood for WebSocket connections. Handles connect, disconnect, heartbeat, and frame routing. Scales to 50k+ concurrent connections per pod. **Wheel 2 — Redis State** Tracks everything ephemeral and hot: online presence per user, typing indicators (5s TTL), last 50 messages per conversation (capped list), unread counts, cross-pod pub/sub routing, notification escalation timers. **Wheel 3 — RabbitMQ Queue** Handles everything async: offline message delivery queue, delayed SMS escalation jobs, commerce event publishing to Main Backend, receipt acknowledgment processing. **Wheel 4 — Textfy SMS** Critical and important message fallback. NexGate's own SMS platform — zero third-party cost. Swahili templates per message type. Deep links back to specific conversation. Full delivery audit log. --- ## 5. Service Design ### Chat Gateway Responsibilities ``` On WebSocket connect: Validate JWT token Register user presence in Redis: presence:{userId} → TTL 30s (refreshed by heartbeat) Drain offline queue via RabbitMQ trigger On frame received (inbound): Validate session Publish to Redis: chat:inbound ACK client immediately with temp_id On Redis pub/sub message (outbound): Find client connection for recipient Push WS frame to device On WebSocket disconnect: Remove presence from Redis Update last_seen_at via RabbitMQ event ``` ### Chat Service Responsibilities ``` On inbound message event (from Redis): Validate sender is conversation member Check conversation not blocked/archived Resolve message level (NORMAL / IMPORTANT / CRITICAL) Write to PostgreSQL Write to Redis hot cache Fan-out to recipients: Online → Redis pub/sub → Gateway → WS push Offline → RabbitMQ queue + FCM + escalation timer On commerce message: Attach product snapshot (frozen at send time) Emit commerce event to Main Backend via RabbitMQ On receipt ack: Update message_receipts in PostgreSQL Notify sender (tick update) via Redis pub/sub On presence event (user online): Drain RabbitMQ offline queue for user Send catch-up summary if messages missed ``` ### RabbitMQ Exchange Design ``` Exchange: nexgate.chat (topic) Routing Key Flow ────────────────────────────────────────────────────── chat.message.inbound Gateway → Chat Service chat.message.outbound Chat Service → Gateway chat.notify.push Chat Service → FCM Worker chat.notify.escalation Chat Service → SMS Worker chat.receipts.delivered Gateway → Chat Service chat.receipts.read App → Chat Service chat.commerce.initiated Chat Service → Main Backend chat.commerce.price.attached Chat Service → Main Backend chat.presence.online Gateway → Chat Service chat.presence.offline Gateway → Chat Service ``` --- ## 6. Data Flows ### Text Message — Full Sequence ``` [Client A] [Gateway] [Chat Service] [Client B] type "Habari" tap Send show pending ⏳ │ │ WS frame: │ { type: MSG_SEND │ temp_id: "abc" │ conv_id: "conv-123" │ body: "Habari" } │──────────────────▶│ │ publish Redis: │ chat:inbound │ ACK: { temp_id: "abc" } │◀────────────────────── │ single tick ✓ ◀───│ │ ...................▶ │ consume Redis │ validate sender │ write PostgreSQL │ write Redis cache │ check B online? ✅ │ publish outbound │◀ .................. │ │ push WS to B │─────────────────────────────────────▶│ │ │ show message │◀ ......................... DELIVERED ack ──│ │ write receipt │ notify A: │ double tick ✓✓ ◀──│◀ ................. │ . │ [B reads conv] . │◀─────── READ ack ────────────────────│ │ write receipt │ blue tick ✓✓ ◀────│◀ ................. │ ─── solid line = WebSocket (real-time) ... dotted line = Redis pub/sub (async, cross-pod) ``` ### Voice Note — Send Flow ``` User records audio (Opus codec, 16kHz) │ ▼ Upload to File Thunder GET /media/upload-request (DM_ATTACHMENT context) → presigned MinIO URL returned Upload .ogg file directly to MinIO POST /media/confirm { fileId } │ ▼ File Thunder processes: ClamAV scan Waveform extraction via FFmpeg → amplitude array (50 values for UI bars) → waveform.webp thumbnail Store in nexgate-private/messages/{convId}/{fileId}/ │ ▼ File Thunder returns: { fileId, waveformData[], durationSeconds } │ App sends WS frame: { type: MSG_SEND content_type: VOICE_NOTE media_ref: fileId duration_seconds: 15 waveform: [0.2, 0.8, 0.6, ...] } ← embedded for instant UI │ ▼ Same path as text message Recipient receives message with waveform data Waveform bars render instantly (no extra request) Tap play → GET /chat/media/{fileId}/url → signed URL (5 min TTL) → stream audio progressively ``` ### Offline Delivery Flow ``` Message arrives for offline user B │ Chat Service: Check Redis: B online? ❌ │ ├──▶ RabbitMQ: chat.offline.delivery │ { messageId, recipientId, level } │ ├──▶ FCM HIGH priority push │ { type: NEW_MESSAGE │ convId, senderName, preview } │ └──▶ RabbitMQ: chat.notify.escalation delay: CRITICAL=0min, IMPORTANT=10min . . (time passes) . Escalation consumer wakes: Check Redis: B online now? ✅ → cancel, done ❌ → send Textfy SMS │ Textfy SMS: "NexGate: Ujumbe mpya kutoka Juma. nexgate.app/chat/conv-123" │ User turns on WiFi / opens app: WS reconnects → Gateway registers presence Chat Service drains RabbitMQ queue (priority order) CRITICAL first → IMPORTANT → NORMAL Show catch-up banner: "Umekosa: 2 maagizo, 5 ujumbe" DELIVERED receipts fire for all drained messages ``` --- ## 7. Commerce DM Flows ### Flow 1 — Buyer Initiates from Product Page ``` Buyer on product page │ taps "Chat with Seller" ▼ POST /chat/commerce/initiate { productId, shopId } │ Chat Service: Find or create DM conversation conversation.type = COMMERCE conversation.owner_type = SHOP conversation.owner_id = shopId │ Fetch product snapshot from Main Backend: { name, price, images[0], stock, shopName } Snapshot frozen at this exact moment ✅ Public price change later → does not affect this card │ Create first message automatically: type: PRODUCT_CARD context_type: PRODUCT context_ref_id: productId snapshot_json: { frozen product data } │ ▼ Seller receives in shop inbox tab ┌─────────────────────────────────┐ │ 📦 Samsung A15 │ │ TZS 450,000 │ │ In stock: 12 units │ │ TechStore │ │ │ │ [Reply] [View Product] │ └─────────────────────────────────┘ │ Negotiation happens in thread │ Agreement reached ▼ Seller attaches custom price offer: ┌─────────────────────────────────┐ │ 💰 Special Price Offer │ │ Samsung A15 │ │ TZS 400,000 (was 450,000) │ │ Valid for you only │ │ │ │ Quantity: [─ 1 +] │ │ [Proceed to Checkout →] │ └─────────────────────────────────┘ │ Buyer taps Proceed → redirected to checkout Checkout outside inbox — at negotiated price Public product price: TZS 450,000 unchanged ✅ │ Order placed → confirmation back in thread: ┌─────────────────────────────────┐ │ ✅ Order Confirmed │ │ Order #ORD-789 │ │ Samsung A15 × 1 │ │ TZS 400,000 paid │ └─────────────────────────────────┘ ``` ### Flow 2 — Seller Attaches from Inside Chat ``` Seller inside any conversation │ taps attach (+) ▼ ┌─────────────────────────────────┐ │ Attach │ │ │ │ 📷 Image │ │ 🎵 Voice Note │ │ 📄 File │ │ 🏪 From My Shop ◀──── this one │ │ 📅 Event │ │ 👥 Group Purchase │ └─────────────────────────────────┘ │ taps "From My Shop" ▼ Seller browses their shop products Picks product Sets custom price for this buyer (optional) │ ▼ Sends as card in chat ┌─────────────────────────────────┐ │ 🏪 TechStore Offer │ │ Samsung A15 │ │ TZS 400,000 │ │ │ │ Quantity: [─ 1 +] │ │ [Proceed to Checkout →] │ └─────────────────────────────────┘ │ Buyer taps Proceed → checkout outside inbox ``` ### Commerce Message Types ``` message.type values for commerce: PRODUCT_CARD product shared in chat CUSTOM_PRICE_OFFER seller's private price for this buyer EVENT_CARD event shared in chat GROUP_PURCHASE_CARD Bei ya pamoja shared in chat POST_CARD VP Feed post shared in chat ORDER_CONFIRMATION system message after order placed ORDER_STATUS_UPDATE system message for shipping/delivery PAYMENT_CONFIRMATION system message after payment ``` --- ## 8. Offline Delivery & Notification Escalation ### Notification Levels ``` Message level resolved by Chat Service before fan-out: CRITICAL: Order placed / payment received / payment failed Order status changed / delivery update → FCM HIGH + Textfy SMS simultaneously (no waiting) IMPORTANT: Commerce DM from buyer Custom price offer received Bei ya pamoja threshold reached → FCM HIGH immediately → Textfy SMS after 10 minutes if no delivery ack NORMAL: Regular DMs, group messages Social cards, post shares → FCM HIGH only → No SMS escalation ``` ### Textfy SMS Templates ``` Order notification (Swahili): "NexGate: Agizo jipya kutoka [Buyer]! Kiasi: TZS [amount]. Kagua: nexgate.app/orders/[id]" Payment received: "NexGate: Malipo ya TZS [amount] yamepokelewa kutoka [Buyer]. nexgate.app/wallet" Commerce DM: "NexGate: [Buyer] anakuuliza kuhusu [Product]. Jibu: nexgate.app/chat/[convId]" Bei ya pamoja: "NexGate: Watu [n]/[target] wamejiunga! nexgate.app/group-buy/[id]" ``` ### Notification Delivery Log All notifications tracked for audit — critical for order disputes: ``` notification_log ───────────────────────────────────── id UUID user_id UUID message_id UUID level ENUM (NORMAL/IMPORTANT/CRITICAL) fcm_status ENUM (SENT/DELIVERED/FAILED) sms_status ENUM (SENT/DELIVERED/FAILED/SKIPPED) sms_provider TEXT sent_at TIMESTAMPTZ delivered_at TIMESTAMPTZ opened_at TIMESTAMPTZ ``` --- ## 9. Message Status System ``` Client shows status via tick indicators: ⏳ Pending message on device, not yet sent (no connection) ✓ Sent server received and persisted Gateway ACK returned with temp_id ✓✓ Delivered recipient device received WS delivery ack received by Chat Service ✓✓ Read recipient opened the conversation READ event sent by recipient app (blue ticks) Flow: [Send] → pending ⏳ [Gateway ACK] → sent ✓ [Recipient WS ack] → delivered ✓✓ [Recipient opens conv] → read ✓✓ (blue) ``` ### Typing Indicators ``` Ephemeral — never persisted to PostgreSQL User starts typing: App sends: { type: TYPING_START, convId } Chat Service sets Redis key: typing:{convId}:{userId} → TTL 5 seconds Redis pub/sub notifies conversation members Recipients see "Juma anaandika..." User stops typing (or TTL expires): Key auto-expires after 5 seconds Chat Service notifies: typing stopped Indicator disappears ``` --- ## 10. Database Schema ### conversations ``` conversations ───────────────────────────────────────────── id UUID PK type ENUM DM / GROUP / BROADCAST / COMMERCE owner_type ENUM USER / SHOP owner_id UUID userId or shopId title TEXT for groups and broadcast channels avatar_file_id UUID File Thunder fileId status ENUM ACTIVE / ARCHIVED / BLOCKED created_by UUID userId created_at TIMESTAMPTZ ``` ### conversation_members ``` conversation_members ───────────────────────────────────────────── conversation_id UUID FK → conversations user_id UUID role ENUM MEMBER / ADMIN / OWNER joined_at TIMESTAMPTZ last_read_at TIMESTAMPTZ last_read_seq BIGINT sequence ID (for gap detection) is_muted BOOLEAN muted_until TIMESTAMPTZ ``` ### messages ``` messages ───────────────────────────────────────────── id UUID PK conversation_id UUID FK → conversations sender_id UUID seq BIGINT monotonic per conversation type ENUM TEXT / IMAGE / VIDEO / VOICE_NOTE / FILE / PRODUCT_CARD / CUSTOM_PRICE_OFFER / EVENT_CARD / GROUP_PURCHASE_CARD / POST_CARD / ORDER_CONFIRMATION / ORDER_STATUS_UPDATE / PAYMENT_CONFIRMATION / SYSTEM body TEXT media_ref UUID File Thunder fileId context_type ENUM PRODUCT / ORDER / PAYMENT / EVENT / GROUP_PURCHASE context_ref_id UUID ref to relevant entity snapshot_json JSONB frozen context data at send time reply_to_id UUID FK → messages (thread replies) status ENUM SENT / DELIVERED / READ / FAILED level ENUM NORMAL / IMPORTANT / CRITICAL created_at TIMESTAMPTZ edited_at TIMESTAMPTZ deleted_at TIMESTAMPTZ ``` ### message_receipts ``` message_receipts ───────────────────────────────────────────── message_id UUID FK → messages user_id UUID status ENUM DELIVERED / READ device_id TEXT timestamp TIMESTAMPTZ ``` ### calls ``` calls ───────────────────────────────────────────── call_id UUID PK caller_id UUID receiver_id UUID conversation_id UUID FK → conversations type ENUM VOICE / VIDEO status ENUM RINGING / CONNECTED / COMPLETED / MISSED / DECLINED / FAILED started_at TIMESTAMPTZ answered_at TIMESTAMPTZ ended_at TIMESTAMPTZ duration_seconds INT relay_used BOOLEAN was TURN relay used? end_reason ENUM NORMAL / NETWORK / TIMEOUT / DECLINED ``` ### shop_conversation_access ``` shop_conversation_access ───────────────────────────────────────────── shop_id UUID user_id UUID staff member role ENUM MANAGER / SUPPORT_AGENT / READ_ONLY granted_by UUID owner userId granted_at TIMESTAMPTZ revoked_at TIMESTAMPTZ ``` --- ## 11. Inbox Model Implementation ### Conversation Ownership ``` Personal DM: owner_type = USER owner_id = usr-kibuti Access: only usr-kibuti Shop DM: owner_type = SHOP owner_id = shop-techstore Access: anyone with role in shop_conversation_access for shop-techstore ``` ### Tab Resolution (App Side) ``` App fetches inbox tabs on load: GET /chat/inbox/tabs Response: [ { type: PERSONAL, label: "Personal", unread: 3 }, { type: SHOP, shopId: "shop-techstore", label: "TechStore", unread: 12 }, { type: SHOP, shopId: "shop-clothinghub", label: "ClothingHub", unread: 0 } ] Chat Service resolves tabs by: 1. User's own personal conversations 2. All shops where user has role in shop_conversation_access ``` ### Access Control Check ``` Every request to open a shop conversation: Does requesting user own the shop? YES → allow NO → check shop_conversation_access: user_id = requester shop_id = conversation.owner_id revoked_at IS NULL Found? → allow with their role Not found? → 403 Forbidden ``` --- ## 12. Local Experiments These run in parallel with Phase 1 production work. All throwaway code — not NexGate quality. Goal is understanding, not production output. ### Experiment 1 — Ejabberd Local Docker ``` Goal: get Ejabberd running, send one message Steps: docker run -d --name ejabberd \ -p 5222:5222 \ -p 5280:5280 \ -p 5285:5285 \ ghcr.io/processone/ejabberd Open: http://localhost:5280/admin Create two test users: alice, bob Install Conversations app on Android Connect to localhost:5222 as alice Send message to bob See it arrive What you learn: How Ejabberd config works What XMPP stanzas look like in logs How the dashboard shows connections What errors look like and how to fix them Success: message delivered between two test users ``` ### XMPP Stanzas — What They Look Like When Alice sends "Habari" to Bob, this travels over the wire: ```xml Habari available Ninafanya kazi ``` These stanzas are what Ejabberd routes. In Phase 2, NexGate wraps its own data inside custom XMPP stanzas. ### Experiment 2 — Spring Boot Auth Bridge ``` Goal: Ejabberd calls Spring Boot to validate users Setup: Simple Spring Boot app (H2 in-memory DB) One endpoint: POST /internal/ejabberd/auth receives: { username, token } returns: 200 (allow) or 401 (deny) ejabberd.yml: auth_method: http auth_opts: url: "http://host.docker.internal:8080/internal/ejabberd/auth" Test: Connect Conversations app to Ejabberd Ejabberd calls Spring Boot for auth Spring Boot validates → returns 200 Connection allowed What you learn: Auth flow between Ejabberd and Spring Boot How fast Spring Boot must respond (< 200ms) What happens when auth fails How to structure the internal endpoint Success: Ejabberd rejects unknown users, allows users Spring Boot approves ``` ### Experiment 3 — WebRTC Voice Call on Emulators ``` Goal: voice call between two Android emulators Setup: Two Android Studio emulators running Simple Android app — just two buttons: [Call] and [Answer] WebRTC library: io.getstream:stream-webrtc-android No UI — just logcat output What to test: Create PeerConnection on both Exchange SDP offer/answer manually (copy-paste between logs) Exchange ICE candidates Hear audio between emulators What you learn: How WebRTC PeerConnection works in practice What SDP looks like What ICE candidates look like How long negotiation actually takes What errors appear and how to fix them Success: audio heard between two emulators ``` SDP offer example (what WebRTC generates): ``` v=0 o=- 123456 2 IN IP4 127.0.0.1 s=- t=0 0 a=group:BUNDLE 0 m=audio 9 UDP/TLS/RTP/SAVPF 111 c=IN IP4 0.0.0.0 a=rtcp:9 IN IP4 0.0.0.0 a=ice-ufrag:someRandomString a=ice-pwd:anotherRandomString a=fingerprint:sha-256 AA:BB:CC:... a=rtpmap:111 opus/48000/2 ← Opus codec negotiated here a=fmtp:111 minptime=10;useinbandfec=1 ``` ### Experiment 4 — Coturn TURN Relay ``` Goal: force audio through TURN relay, confirm it works Setup: docker run -d --network=host coturn/coturn \ -n --log-file=stdout \ --min-port=49152 --max-port=65535 \ --lt-cred-mech \ --user=test:password123 \ --realm=nexgate.com Test: Disable P2P in WebRTC config (force TURN only) Run voice call experiment Confirm audio still flows through relay What you learn: How Coturn logs relay connections Bandwidth used per call (check with iftop) How to generate HMAC credentials (not plain text) What happens when Coturn is unavailable Success: audio heard between emulators via TURN relay Coturn logs show relay traffic ``` ### Experiment 5 — MessagePack Encoding ``` Goal: compare JSON vs MessagePack on same message Simple Spring Boot test: Serialize same chat message object Once as JSON Once as MessagePack Measure: Byte size comparison Serialization speed Deserialization speed Expected result: JSON: ~180-200 bytes per message MessagePack: ~60-70 bytes per message ~65% size reduction confirmed Success: numbers prove EA bandwidth saving is real ``` ### Experiment Success Criteria Summary ``` Experiment 1 — Ejabberd local: ✅ Message delivered between two users ✅ Dashboard shows live connections Experiment 2 — Auth bridge: ✅ Ejabberd rejects unknown tokens ✅ Ejabberd allows Spring Boot approved users Experiment 3 — WebRTC emulators: ✅ Audio heard between two emulators ✅ SDP and ICE flow understood Experiment 4 — Coturn relay: ✅ Audio heard via forced TURN relay ✅ Bandwidth per call measured Experiment 5 — MessagePack: ✅ Size reduction confirmed ✅ Serialization speed measured All five done → ready to write Phase 2 doc → ready to build NexGate chat Phase 2 ``` --- ## Summary Phase 1 is the foundation. It ships real features — text chat, voice notes, commerce DMs, offline delivery, the isolated shop inbox — using a clean Spring Boot WebSocket architecture that runs on existing infrastructure. Everything designed here carries directly into Phase 2. The schema stays. The commerce logic stays. The notification system stays. The inbox model stays. Only the transport layer swaps — Spring Boot WS gateway out, Ejabberd in. The five local experiments running in parallel are not wasted time. They are the insurance policy that makes Phase 2 a confident execution rather than a risky exploration. Every surprise Ejabberd, WebRTC, and Coturn have in store — you want to find them in a throwaway experiment, not in your production chat system. --- *NexGate Chat Platform — Phase 1: Foundation & Local Experiments v1.0* *QBIT SPARK | Spring Boot WebSocket · Commerce DMs · Offline Delivery* # NexGate Messaging — Product Requirements & Feature Flows **NexGate / QBIT SPARK | Version 1.0** *What the messaging platform does — rules, flows, permissions, scenarios* --- ## Table of Contents 1. [Document Purpose](#1-document-purpose) 2. [User Identity & Discovery](#2-user-identity--discovery) 3. [Contact Sync](#3-contact-sync) 4. [Inbox Model](#4-inbox-model) 5. [Messaging Permissions](#5-messaging-permissions) 6. [Call Permissions](#6-call-permissions) 7. [1:1 Messaging Flows](#7-11-messaging-flows) 8. [Group Chat Flows](#8-group-chat-flows) 9. [Commerce DM Flows](#9-commerce-dm-flows) 10. [Offer Session Flows](#10-offer-session-flows) 11. [Shareable Content](#11-shareable-content) 12. [Message Interactions](#12-message-interactions) 13. [Voice & Video Call Flows](#13-voice--video-call-flows) 14. [Group Call Flows](#14-group-call-flows) 15. [Offline & Notification Flows](#15-offline--notification-flows) 16. [Privacy & Safety](#16-privacy--safety) 17. [Shop Inbox & Staff Access](#17-shop-inbox--staff-access) 18. [Notification Settings](#18-notification-settings) 19. [Edge Cases & Scenarios](#19-edge-cases--scenarios) --- ## 1. Document Purpose This document defines WHAT NexGate messaging does — the rules, user flows, permissions, and scenarios for every messaging feature. It is the product reference document. It does not describe HOW features are built technically. For technical architecture see: - Doc 3: Phase 2 Production Architecture - Doc 5: Private Chat & Calls Deep Dive **Audience:** Product, mobile developers, QA, design. --- ## 2. User Identity & Discovery ### Identity on NexGate ``` Every NexGate user has: @username unique, chosen at registration shown publicly on VP Feed used for messaging: @kibuti Display name full name shown in conversations can differ from username Phone number used for registration + auth NOT shown publicly by default discoverable only with user permission NexGate ID internal UUID never shown to users ``` ### How Users Find Each Other ``` Discovery mechanism Requires contact sync? ────────────────────────────────────────────────────── VP Feed follow ❌ No VP Shop (buyer/seller) ❌ No Username search (@kibuti) ❌ No QR code scan ❌ No Shareable profile link ❌ No Group chat membership ❌ No Phone number search ❌ No (type manually) Contact sync (phonebook) ✅ Yes (optional) ``` NexGate has multiple discovery paths. Contact sync is one option — not the only one. ### Discoverability Settings ``` Two independent settings: "Sync my contacts" Find NexGate users in my phonebook Default: OFF (user must opt in) "Let others find me via phone number" If someone has my number → can find me Default: ON (can turn off) These are INDEPENDENT: I can sync contacts without being findable I can be findable without syncing contacts ``` --- ## 3. Contact Sync ### Rules ``` ✅ Optional — never mandatory ✅ Can be enabled/disabled anytime ✅ Can delete synced data anytime from settings ✅ Only finds users who have discoverability ON ❌ Not required to use messaging ❌ Not required for commerce DMs ❌ Contacts data never shared with third parties ❌ Non-NexGate contacts never stored on server ``` ### Privacy-Preserving Implementation ``` How it works when user accepts: Step 1: Phone numbers hashed ON DEVICE sha256(+255712345678) → hash Actual numbers never leave the device Step 2: Hashes uploaded to NexGate server Not real phone numbers Server cannot reverse hashes Step 3: Server matches hashes Against registered users who have "Let others find me via phone number" = ON Returns: which hashes are NexGate users Step 4: App shows matched users "3 of your contacts are on NexGate" [Connect] buttons shown Non-NexGate contacts: Their hashes deleted from server immediately Never stored Never tracked ``` ### First Launch Prompt ``` Shown once at first launch after onboarding: ┌──────────────────────────────────────────────┐ │ 📱 Pata Marafiki Wako NexGate │ │ │ │ Ruhusu NexGate kutumia contacts zako │ │ kupata marafiki wanaotumia NexGate tayari │ │ │ │ ✓ Contacts zako hazitashirikiwa na mtu │ │ ✓ Unaweza kufuta ruhusa wakati wowote │ │ ✓ Nambari za simu zinabadilishwa kuwa │ │ msimbo kabla ya kupakiwa │ │ │ │ [Ruhusu Contacts] [Sasa Hivi Sio] │ └──────────────────────────────────────────────┘ If declined: App works fully ✅ Remind once after 7 days Never prompt again after second decline Always accessible in Settings ``` ### Sync Scenarios ``` Scenario 1 — User accepts sync: Hashes uploaded Matched users shown User can message/follow matched users Sync runs again when contacts change Scenario 2 — User declines sync: App works fully Can still find people via username/QR/VP Feed No contacts uploaded Scenario 3 — User accepts then disables: Settings → Privacy → Contact Sync → OFF All hash data deleted from server Matched contacts remain as connections (connections not removed — only future sync stops) Scenario 4 — User wants to delete contact data: Settings → Privacy → Delete Contacts Data All hashes deleted from server immediately Confirmation shown: "Contacts data imefutwa" Scenario 5 — Contact not on NexGate: Their hash deleted from server User sees option: "Invite Juma to NexGate" Tap → share NexGate invite link via SMS/WhatsApp ``` --- ## 4. Inbox Model ### Two Separate Inboxes ``` Every NexGate user has TWO completely separate inboxes: Personal Inbox: 1:1 DMs with other users (personal) Group chats (personal) Friend conversations Private — only the account owner sees this NEVER accessible to shop staff Shop Inbox (one per shop owned): All customer conversations for that shop Commerce DMs with buyers Shared with authorized shop staff Completely separate from personal inbox Customer sees shop identity — not personal name ``` ### Inbox Tabs ``` User with 2 shops sees: ┌─────────────────────────────────────────────┐ │ 💬 Inbox │ │ │ │ [Personal ●3] [TechStore ●12] [ClothingHub]│ │ │ │ Personal tab: │ │ Regular DMs + group chats │ │ Private — owner only │ │ │ │ TechStore tab: │ │ Customer commerce conversations │ │ Shared with assigned staff │ │ │ │ ClothingHub tab: │ │ Separate shop conversations │ │ Different staff assigned │ └─────────────────────────────────────────────┘ Staff member (Amina assigned to TechStore): ┌─────────────────────────────────────────────┐ │ [Personal] [TechStore ●12] │ │ │ │ ClothingHub: NOT visible (no access) │ │ Owner personal: NEVER visible │ └─────────────────────────────────────────────┘ ``` ### Conversation Types ``` Type Owner Participants ──────────────────────────────────────────────────── DM USER 2 users GROUP USER 3-500 users COMMERCE SHOP 1 shop + 1 buyer BROADCAST USER or SHOP 1 sender + N followers ``` --- ## 5. Messaging Permissions ### Who Can Message Who ``` Relationship Can message? How ──────────────────────────────────────────────────────────── Saved contacts (phone) ✅ Always Direct Mutual followers (VP Feed) ✅ Always Direct Commerce relationship ✅ Always Direct (shop inbox) Group chat members ✅ Always Via group only Strangers (no relationship) ⚠️ Request Message request first Blocked users ❌ Never Blocked ``` ### Message Request System ``` Stranger (no relationship) wants to message you: They can send ONE message request: ┌─────────────────────────────────────────┐ │ 📩 Maombi ya Ujumbe │ │ │ │ Juma Mwangi anataka kukutumia ujumbe │ │ @juma_mwangi · 245 wafuasi │ │ │ │ "Habari, nilikuona kwenye VP Shop..." │ │ │ │ [Kubali] [Kataa] [Zuia] │ └─────────────────────────────────────────┘ If Kubali (Accept): Full conversation opens Juma can now message freely Can call after conversation established If Kataa (Decline): Juma cannot send another request One attempt only per account Juma sees: "Ujumbe haukukubaliwa" If Zuia (Block): Juma blocked immediately Cannot contact again in any way Juma sees: nothing (as if you don't exist) If Ignored (no response): Request stays in request folder Juma cannot send more messages No call attempt allowed Request expires after 30 days ``` ### Message Request Limits ``` Per account per day: Max 10 message requests sent Prevents spam Per account to same person: 1 request only (ever) If declined → cannot request again Request folder: Separate from main inbox User checks it voluntarily No notification for low-priority requesters (unless they have mutual connections) ``` --- ## 6. Call Permissions ### Permission Tiers ``` Tier Voice call Video call ──────────────────────────────────────────────────────── Saved contacts ✅ ✅ Mutual followers ✅ ✅ Commerce relationship ✅ (shop) ✅ (shop) Active conversation partner ✅ ✅ (message request accepted) Group chat member ✅ (group) ✅ (group) Strangers ❌ ❌ Blocked users ❌ ❌ ``` ### Commerce Call Rules ``` Buyer can call: The SHOP JID — not the owner personally techstore@shops.nexgate.com Any available staff answers as TechStore Seller (shop) can call: Buyers they have active commerce DM with Call goes from shop JID to buyer Buyer sees "TechStore" calling — not a personal name Protecting personal contact: Owner's personal phone/JID never revealed Commerce calls always through shop identity Staff cannot reveal personal number to customers ``` ### Anti-Harassment Call Rules ``` Call rate limiting: Max 3 unanswered calls to same person in 24 hours 4th attempt → blocked for 24 hours automatically Resets daily Silence unknown callers: Setting: "Pumzisha simu kutoka kwa wasio contacts" Calls from non-contacts ring silently Notification appears after — not during First call warning: First EVER call from this person: ┌───────────────────────────────────────┐ │ 📞 Simu ya kwanza kutoka │ │ Juma Mwangi (@juma_mwangi) │ │ │ │ [Jibu] [Kataa] [Zuia] │ └───────────────────────────────────────┘ Quick block during call: Report button visible during call One tap → ends call + reports + blocks ``` ### Privacy Settings for Calls ``` Who can call me: ○ Mutual followers + contacts (default) ○ My contacts only ○ Nobody User can change anytime in Settings → Privacy → Calls ``` ### Call Permission Check Flow ``` Kibuti tries to call Juma: │ Spring Boot permission check: Is Juma blocked by Kibuti? ❌ → reject immediately Is Kibuti blocked by Juma? ❌ → reject immediately Are they saved contacts? ✅ → allow Are they mutual followers? ✅ → allow Active commerce relationship? ✅ → allow Active accepted conversation? ✅ → allow Does Juma's privacy setting allow? ✅ → allow None of above → ❌ reject │ If rejected: App shows: "Huwezi kupiga simu mtu huyu" No Jingle stanza sent No TURN credentials issued If allowed: TURN credentials generated Jingle session-initiate sent via Ejabberd Juma's device rings ``` --- ## 7. 1:1 Messaging Flows ### Send Text Message ``` Happy path: User types message Taps send Message shows: pending ⏳ Server receives → single tick ✓ Recipient receives → double tick ✓✓ Recipient reads → blue tick ✓✓ Offline path: User taps send Single tick ✓ (server received) Recipient offline → queued FCM push fires to recipient device Recipient comes online → message delivered Double tick ✓✓ appears ``` ### Send Voice Note ``` User holds mic button: "Kibuti anarekodia..." shown to recipient (if online) Recording in progress Timer shown: 0:01, 0:02... User releases: Voice note sent Waveform shown in thread Duration shown: 0:15 User swipes up while holding: Lock mode — records without holding Release sends User swipes left while holding: Cancel recording — nothing sent Limits: Basic users: max 2 minutes Pro users: max 15 minutes File size: max 16MB ``` ### Send Media ``` Images: Max 10 images per message Formats: JPG, PNG, WebP, HEIC Max 16MB per image Auto-compressed for EA networks Videos: Max 1 video per message Max 2 minutes (short video) Max 64MB Auto-transcoded to HLS by File Thunder Files/Documents: Max 100MB per file Formats: PDF, DOCX, XLSX, ZIP, etc ClamAV scanned before delivery Available for 30 days then archived ``` --- ## 8. Group Chat Flows ### Create Group ``` User creates group: Tap new group Add members (min 2, max 500) Set group name (required) Set group photo (optional) Tap create Group created: Members notified: "Kibuti amekuongeza kwenye kikundi" All members see welcome message Creator is automatically OWNER ``` ### Group Roles & Permissions ``` Role Add members Remove Send msg Delete msg Change info ────────────────────────────────────────────────────────────────── OWNER ✅ ✅ ✅ Any msg ✅ ✅ ADMIN ✅ ✅ ✅ Any msg ✅ ✅ MEMBER ❌ ❌ ✅ Own only ✅ ❌ ``` ### Group Size Limits ``` Free group: max 50 members Pro group: max 500 members At 500 members: Consider using Broadcast Channel instead Broadcast = one way (better for large audiences) Group = two way (better for community) ``` ### Leave / Remove ``` Member leaves: "Juma ameacha kikundi" shown in thread No longer receives messages Admin removes member: "Amina ameondolewa na Kibuti" shown Member notified: "Umeondolewa" Deleted group (owner only): All members removed All messages deleted (for everyone) Cannot be undone ``` --- ## 9. Commerce DM Flows ### Flow 1 — Buyer Initiates from Product Page ``` Step 1: Buyer on product page Sees: [Chat na Muuzaji] button Tap button Step 2: Conversation opens New DM opens OR existing conversation if already had one Conversation type: COMMERCE Conversation owner: SHOP (not personal) Product card auto-appears: ┌──────────────────────────────────────┐ │ 📦 Samsung A15 │ │ TZS 450,000 │ │ Inapatikana: Vipande 12 │ │ TechStore │ │ [Jibu] [Angalia Bidhaa] │ └──────────────────────────────────────┘ NOTE: Price in card is frozen at this moment Public price changes later → does NOT affect this card Step 3: Negotiation Buyer and seller exchange text messages Normal conversation — no restrictions on text Step 4: Agreement reached Seller taps attach (+) → From My Shop Selects product Sets custom price for THIS buyer ONLY Public product price: unchanged ✅ Sends price offer card Step 5: Buyer receives offer ┌──────────────────────────────────────┐ │ 💰 Bei Maalum Kwako │ │ Samsung A15 │ │ ~~TZS 450,000~~ │ │ TZS 400,000 (-TZS 50,000) │ │ Inaisha: dakika 28 │ │ Idadi: [─ 1 +] │ │ [Kataa] [Endelea Kulipa →] │ └──────────────────────────────────────┘ Step 6: Buyer taps "Endelea Kulipa" Redirected to checkout flow (OUTSIDE inbox) Checkout at custom offer price Quantity confirmed in checkout Step 7: Order placed Confirmation message appears in same thread: ┌──────────────────────────────────────┐ │ ✅ Agizo Limethibitishwa │ │ Ord #ORD-XYZ-789 │ │ Samsung A15 × 1 │ │ TZS 400,000 imelipwa │ │ [Fuatilia Agizo] │ └──────────────────────────────────────┘ ``` ### Flow 2 — Seller Attaches from Inside Any Chat ``` Step 1: Seller inside any DM conversation Taps attach (+) button Menu appears: 📷 Picha 🎵 Sauti 📄 Faili 🏪 Kutoka Dukani ← this one 📅 Tukio 👥 Ununuzi wa Pamoja Step 2: Seller taps "Kutoka Dukani" Their shop product list opens Seller browses and selects product Sets custom price (optional) Sets quantity limit (optional) Adds note (optional): "Bei hii ni leo tu" Taps Send Step 3: Buyer receives offer card Same offer card UI as Flow 1 Same checkout redirect Same order confirmation in thread ``` ### Commerce DM Rules ``` ✅ Buyer can initiate from any product page ✅ Seller can attach from inside any conversation ✅ Custom price is private to this buyer only ✅ Public product price never changes ✅ Checkout always happens outside inbox ✅ Order confirmation appears in thread ✅ Multiple offers allowed in one conversation ❌ Buyer cannot request specific price (only negotiate via text) ❌ Offer cannot be edited after sending (send new one instead) ❌ Offer cannot be forwarded to other conversations ❌ Order confirmation cannot be deleted or edited ``` --- ## 10. Offer Session Flows ### Offer Lifecycle ``` PENDING → offer sent, waiting for buyer response │ ├──▶ DECLINED buyer tapped "Kataa" │ seller notified │ seller can send new offer │ ├──▶ EXPIRED timer ran out │ both parties notified in thread │ seller can send new offer │ ▼ ACCEPTED → buyer tapped "Endelea Kulipa" │ buyer enters checkout ▼ CHECKOUT → buyer in payment flow │ ├──▶ ABANDONED buyer left checkout without paying │ offer returns to PENDING? NO │ offer marked ABANDONED │ seller must send new offer │ ▼ COMPLETED → order placed successfully │ cannot be reversed here (order system handles) ▼ CANCELLED → order cancelled (handled by order system) offer marked CANCELLED ``` ### Multiple Offers — Same Conversation ``` Seller can send multiple offers: Each is independent session Previous offers remain in thread with their status Thread shows history: ┌──────────────────────────────────────┐ │ 💰 Bei Maalum: TZS 430,000 │ │ ❌ Ilikataliwa │ └──────────────────────────────────────┘ ┌──────────────────────────────────────┐ │ 💰 Bei Maalum: TZS 410,000 │ │ ⏰ Imeisha muda │ └──────────────────────────────────────┘ ┌──────────────────────────────────────┐ │ 💰 Bei Maalum: TZS 400,000 │ │ ✅ Imekubaliwa │ └──────────────────────────────────────┘ ``` ### Offer Expiry Scenarios ``` Scenario 1 — Expires while buyer is reading: Offer card shows live countdown timer Timer hits 0:00 Card updates: "Imeisha muda" Proceed button disabled automatically Buyer sees: [Omba Bei Mpya] button Scenario 2 — Expires while buyer in checkout: Buyer was on checkout page when offer expired Checkout validates offer at payment time If expired: payment rejected Buyer returned to conversation "Bei yako imeisha muda. Omba bei mpya" Scenario 3 — Seller sends same price again: Allowed — new offer session created New 30-minute timer Both offers visible in thread (old + new) Scenario 4 — Buyer tries to use expired offer link: Deep link from notification opens expired offer Shows: "Bei hii imeisha muda" [Rudi kwa Mazungumzo] button ``` ### Price Security Rules ``` ✅ Offer price stored server-side only ✅ Checkout validates price from server (not client) ✅ Client cannot manipulate price ✅ Offer is single-use (cannot complete twice) ✅ Offer belongs to specific buyer (others cannot use) ❌ Seller cannot change price after offer sent (must send new offer) ❌ Buyer cannot change price ❌ Public product price never affected ``` --- ## 11. Shareable Content ### What Can Be Shared Into Any DM or Group ``` Content Who can share Restrictions ──────────────────────────────────────────────────────────── Text message Anyone None Voice note Anyone Max 2 min (basic) Image/Video Anyone Size limits Product card Anyone Shows public price Custom price offer Seller only Private — NOT forwardable Event card Anyone None Bei ya pamoja card Anyone None VP Feed post Anyone None VP Live stream Anyone None Audio Space Anyone None File/Document Anyone Max 100MB ``` ### Forwarding Rules ``` Forward chain tracking: chain 1: "↪ Imetumwa kutoka Juma Mwangi" chain 2-4: "↪ Imetumwa" chain 5+: "↪ Imetumwa mara nyingi" (misinformation warning — different icon) Multi-forward limit: Max 5 conversations per forward action Prevents spam broadcasting Cannot forward: Custom price offers (private deal) Order confirmations (private record) Payment confirmations (private record) System messages ``` --- ## 12. Message Interactions ### Editing Messages ``` Rules: ✅ Only original sender can edit ✅ Text messages only (not media captions yet) ✅ Within 15 minutes of sending ✅ Shows "Imehaririwa" label after edit ✅ Original send time stays the same ✅ Works in group chats (sender edits their own) ❌ Commerce cards cannot be edited (immutable) ❌ System messages cannot be edited ❌ Voice notes cannot be edited ❌ After 15 minutes: edit option disappears UI: Long press message → [Hariri] option (within 15 min) After edit: message updates in place "Imehaririwa" label appears below message Position in thread unchanged (no jump to bottom) ``` ### Deleting Messages ``` Delete for Me: ✅ Any message, any time, no limit ✅ Only removed from your view ✅ Recipient still sees it ✅ Works on any message type Delete for Everyone: ✅ Only original sender ✅ Within 15 minutes of sending ✅ Removed from all screens ✅ Shows: "Ujumbe huu umefutwa" ✅ Works in group chats ❌ Commerce cards: NOT allowed ❌ System messages: NOT allowed ❌ After 15 minutes: option disappears Nothing is ever permanently deleted: Soft delete only For legal compliance + dispute resolution Admin can view deleted messages for support cases Users cannot recover deleted messages ``` ### Reactions ``` Rules: ✅ Any message can be reacted to ✅ One reaction per user per message ✅ Change reaction: send new emoji (replaces old) ✅ Remove reaction: tap same emoji again ✅ Works on commerce cards (reactions don't modify content) ❌ System messages: no reactions Available emojis at launch: ❤️ 👍 😂 😮 😢 🙏 Expand to full emoji keyboard: Phase 3 Group reaction display: 1 type: "👍 3" 2 types: "👍 3 ❤️ 2" 3+ types: "👍 3 ❤️ 2 +2 zaidi" Tap reaction to see who reacted: Bottom sheet opens List of names per emoji ``` ### Replying to Messages ``` Rules: ✅ Anyone in conversation can reply to any message ✅ Reply shows quoted original above new message ✅ Tap quote → scrolls to original ✅ Works in group chats ✅ Works on any message type ❌ Cannot reply to deleted messages (shows: "Ujumbe umefutwa") UI: Swipe right on message → reply mode OR long press → [Jibu] option Quote appears in input field Send normally ``` ### Forwarding ``` Rules: ✅ Anyone can forward allowed content ✅ Max 5 conversations per forward action ✅ Forward chain tracked and shown ✅ Original sender name shown (chain 1 only) ✅ Media: references original file (no re-upload) ❌ Custom price offers: not forwardable ❌ Order/payment records: not forwardable ❌ System messages: not forwardable UI: Long press message → [Tuma] option Conversation picker opens Select up to 5 conversations Tap send ``` --- ## 13. Voice & Video Call Flows ### Initiating a Call ``` Kibuti taps call button on Juma's profile or conversation: │ Permission check: Relationship exists? ✅ Juma's privacy allows? ✅ Kibuti not blocked? ✅ │ TURN credentials generated (server) │ Juma's phone rings: ┌──────────────────────────────────────┐ │ │ │ 📞 Simu Inayoingia │ │ │ │ [Kibuti Mwangi] │ │ @kibuti │ │ │ │ [❌ Kataa] [✅ Jibu] │ │ │ └──────────────────────────────────────┘ Works even if: App is closed (FCM HIGH wakes it) Screen is locked (full screen notification) App is in background ``` ### During a Call — Controls ``` Voice call controls: 🔇 Mute/unmute microphone 🔊 Speaker on/off 📷 Enable camera (upgrade to video) ❌ End call Video call controls: 🔇 Mute/unmute microphone 📷 Camera on/off 🔄 Switch camera (front/rear) 🖥️ Share screen 🔊 Speaker on/off ❌ End call ``` ### Switching Audio ↔ Video ``` Voice → Video: Kibuti taps camera button during voice call Juma sees: "Kibuti anataka kuongeza video" Auto-accepted (based on Juma's settings) OR Juma taps Accept Video starts — same call session continues Audio uninterrupted during upgrade Video → Audio (manual): Kibuti taps camera OFF Video stops immediately for both Audio continues No renegotiation needed Video → Audio (automatic): Network degrades below video threshold Video disabled automatically Banner shown: "Video imezimwa — mtandao dhaifu" Audio continues Video resumes when network improves ``` ### Screen Sharing ``` Start screen share: Tap screen share icon during video call System permission dialog appears (Android/iOS) "Ruhusu NexGate kunasa skrini yako?" User accepts Screen share starts Other party sees your screen During screen share: Your camera: small PiP (picture in picture) Their view: your screen (large) + your face (small) Your view: normal call view + "Unaonyesha skrini" banner Stop screen share: Tap stop button Returns to normal video call ``` ### Call Quality Indicators ``` Signal bars shown during call: ████ Excellent (WiFi / 4G strong) ███░ Good (4G) ██░░ Fair (3G) — may show quality banner █░░░ Poor (2G) — video disabled, audio only ░░░░ Very poor — "Mtandao dhaifu sana" banner Quality banner examples: "Ubora wa sauti umepungua kwa sababu ya mtandao" "Video imezimwa — data ndogo" "Unaunganika tena..." ``` ### Call End Scenarios ``` Normal end: One party taps end Other party sees call ended Duration shown in conversation: "Simu ya dakika 4:32" Declined: Juma taps Kataa Kibuti sees: "Simu ilikataliwa" Missed call notification NOT sent (was declined) No answer (timeout 45 seconds): Kibuti sees: "Hakujibu" Juma sees: "Simu iliyokosekana kutoka Kibuti" notification Network failure: Both lose connection App attempts reconnect (10 seconds) If reconnect fails: Call marked as FAILED "Simu ilikatizwa" shown to both Kibuti can redial ``` --- ## 14. Group Call Flows ### Starting a Group Call ``` From a group chat: Tap call icon in group header Choose: Voice only OR Video All group members receive incoming call notification Members who join → enter call Members who don't → miss it (missed call shown) From a 1:1 conversation: Not supported directly Must create group first OR use "Add person" button during active 1:1 call Add person to active 1:1 call: During call → tap "Ongeza Mtu" Pick from allowed contacts They receive group call invitation They join → 1:1 becomes group call ``` ### Group Call Limits ``` Voice only group call: Max 8 participants (comfortable for 3G) Up to 12 possible but discouraged on EA networks Video group call: Max 4 video feeds shown simultaneously 5th person onwards: audio only tile shown Active speaker highlighted (larger tile) Tap any tile to pin/feature them Layout options: Grid view: all tiles equal size Speaker view: active speaker large, others small Auto: switches based on who is talking ``` ### Group Call Permission Rules ``` Within a group chat: Any member can start group call ✅ Any member can join ✅ No extra permission check needed (group membership = call permission) Adding someone outside the group: Same call permission check as 1:1 Must have relationship (contact/follower/commerce) If no relationship → cannot add ``` --- ## 15. Offline & Notification Flows ### Message Notification Levels ``` Level When FCM SMS after ────────────────────────────────────────────────────────────── CRITICAL Order placed/paid/failed ✅ 0 min Delivery update Payment confirmation IMPORTANT Commerce DM from buyer ✅ 10 min Custom price offer received Bei ya pamoja threshold reached Call missed (commerce context) NORMAL Regular DM ✅ Never Group message Reaction Follow notification ``` ### Offline Message Delivery ``` User offline — message sent to them: │ ├── FCM HIGH priority push fired immediately │ (all message levels) │ ├── CRITICAL: Textfy SMS sent simultaneously │ ├── IMPORTANT: Textfy SMS after 10 min │ (if FCM not acknowledged) │ └── Message queued in RabbitMQ User comes back online: │ ├── WS connects → presence registered ├── Queued messages drained (priority order) │ CRITICAL first → IMPORTANT → NORMAL │ └── Catch-up banner shown: "Umekosa: maagizo 2, ujumbe 8" [Angalia Maagizo] [Ona Ujumbe] ``` ### Textfy SMS Templates ``` Order placed (to seller): "NexGate: Agizo jipya kutoka [Buyer]! TZS [amount]. Kagua: nexgate.app/orders/[id]" Payment received: "NexGate: Malipo ya TZS [amount] yamepokelewa kutoka [Buyer]. nexgate.app/wallet" Commerce DM (to seller): "NexGate: [Buyer] anakuuliza kuhusu [Product]. Jibu: nexgate.app/chat/[id]" Bei ya pamoja: "NexGate: Watu [n]/[target] wamejiunga! nexgate.app/group-buy/[id]" All SMS: Swahili first ✅ Deep link included ✅ Shop name shown (not staff name) ✅ ``` --- ## 16. Privacy & Safety ### Privacy Settings Matrix ``` Setting Options Default ──────────────────────────────────────────────────────────────── Who can message me Everyone / Followers Followers + Contacts / Contacts + Contacts / Nobody Who can call me Followers+Contacts / Contacts Contacts / Nobody only Last seen Everyone / Contacts / Contacts Nobody Profile picture Everyone / Contacts / Everyone Nobody Be found by phone number On / Off On Contact sync On / Off Off Read receipts (blue ticks) On / Off On (off = others see delivered but not read) ``` ### Block System ``` User A blocks User B: User B cannot: ❌ See User A's profile ❌ Send messages to User A ❌ Call User A ❌ See User A in search ❌ See User A's VP Feed posts ❌ See User A's online status User B sees: Profile: "Mtumiaji huyu hayupo" Messages: appear sent but never delivered User A can: ✅ Unblock anytime from Settings ✅ See User B's profile still (A blocked B — not reverse) Group chats: If in same group: messages visible but cannot DM or call each other directly Admin can remove either from group ``` ### Report System ``` Report options: Spam Harassment / Vitisho Inappropriate content Fake account Scam / Udanganyifu Other Report flow: Long press message → [Ripoti] OR profile → [...] → [Ripoti] Choose reason Optional: add description Submit After report: User not notified they were reported Report goes to NexGate moderation queue Automatic temporary restrictions may apply for high-volume reporters Block on report: "Ripoti na Zuia" option available Blocks immediately + sends report ``` ### Safety for Women — Specific Features ``` This is important for EA platform trust: Silence unknown callers: ON by default for new accounts Calls from non-contacts ring silently User sees missed call — no disruption Call rate limiting: 3 unanswered calls per day to same person Automatic block after 4th attempt Quick block during call: One tap visible during any call Ends + reports + blocks in single action Message request system: Strangers cannot freely DM Must send request first User controls who enters their inbox Default privacy settings: New accounts: strict defaults Users open up if they choose Better to protect by default ``` --- ## 17. Shop Inbox & Staff Access ### Shop Tiers ``` Basic shop (free): Owner manages inbox alone No staff assignment One active stream at a time Standard features Pro shop (paid): Staff roles unlocked Multiple staff in inbox Advanced analytics Priority support Audit logs ``` ### Staff Roles ``` Role Inbox Products Analytics Settings Staff mgmt ────────────────────────────────────────────────────────────────── OWNER ✅ ✅ ✅ ✅ ✅ MANAGER ✅ ✅ ✅ ❌ ❌ SUPPORT_AGENT ✅ ❌ ❌ ❌ ❌ READ_ONLY 👁️ view ❌ ❌ ❌ ❌ ``` ### Staff Identity Rules ``` Staff responds to customer: Customer sees: "TechStore" (shop name) Customer NEVER sees: "Amina" (staff name) Staff identity is internal only Audit log (internal): Which staff member responded Timestamp Visible to OWNER and MANAGER only Not visible to customers Staff cannot: ❌ Reveal personal phone number to customers ❌ Switch conversation to personal DM ❌ Access owner's personal inbox ❌ Access other shops they're not assigned to ``` ### Multiple Shops — Inbox Separation ``` Owner with 3 shops: Each shop = separate inbox tab Each shop = separate conversation records Each shop = separate staff assignments Staff for Shop A cannot see Shop B Owner sees all (as owner of all) Commerce DM routing: Buyer messages about Product A (TechStore product) → Goes to TechStore inbox → NOT to ClothingHub inbox → NOT to owner's personal inbox Correctly routed by product ownership ``` --- ## 18. Notification Settings ### Per-Conversation Settings ``` Each conversation has: Notifications: ○ All messages ○ Mentions only (groups) ○ Muted (until: 8 hours / 1 week / forever) Media auto-download: ○ WiFi only ○ WiFi + Mobile data ○ Never ``` ### Global Notification Settings ``` Message notifications: On / Off Call notifications: On / Off Reaction notifications: On / Off Group notifications: On / Off Commerce notifications: On / Off (cannot turn off CRITICAL) Sound: Default / Custom / Silent Vibration: On / Off In-app preview: Show / Hide content ``` ### CRITICAL Notifications — Always On ``` These cannot be turned off by user: Order placed (seller receives) Payment confirmed (buyer receives) Payment failed (buyer receives) Order cancelled (both receive) Reason: Financial events User could miss critical money information Platform liability without guaranteed delivery Textfy SMS fallback ensures they always arrive ``` --- ## 19. Edge Cases & Scenarios ### Messaging Edge Cases ``` User deletes their account: Their messages remain visible in conversations Name shows: "Mtumiaji aliyefuta akaunti" Profile picture: default avatar Cannot be messaged or called User changes username: All conversations update automatically Old username links still work (redirect) No broken references Very long message (>4000 chars): Truncated in thread: first 200 chars + "...Soma zaidi" Tap to expand full message Not split into multiple messages Same message sent twice (duplicate): Detected by temp_id Only one stored in DB User sees single message (not duplicate) Message sent to blocked user: Appears sent (single tick) Never delivered (stays at single tick) User not informed of block ``` ### Commerce Edge Cases ``` Product goes out of stock while offer pending: Offer still valid (was frozen at send time) Checkout validates stock at payment time If out of stock at payment: Payment rejected "Bidhaa hii imeisha" Buyer returned to conversation Seller notified to resend offer or cancel Product price changes while offer pending: Offer price locked at creation ✅ Public price change does NOT affect offer Buyer pays the offer price always Seller deletes product while offer pending: Offer still valid (snapshot frozen) Checkout validates product existence If product deleted at payment: Payment rejected "Bidhaa hii haipatikani tena" Two buyers get offers for last item: Both offers exist simultaneously First to complete checkout gets the item Second buyer's payment rejected: "Bidhaa hii imeisha" Standard e-commerce race condition handling Seller sends offer to wrong person: Cannot retract offer Offer expires naturally Seller can contact NexGate support if urgent Buyer in checkout when offer expires: Payment attempt fails "Bei yako imeisha muda" Must request new offer from seller ``` ### Call Edge Cases ``` Call drops mid-conversation: WebRTC detects loss Auto-reconnect attempted (10 seconds) If reconnect success: call continues If reconnect fails: call ended Both see: "Simu ilikatizwa" Kibuti can redial immediately Both call each other simultaneously: Race condition — both see "Simu inayoingia" System picks one (first to reach server) Other cancelled automatically One call established Call during active call: Second call: goes to missed calls Busy signal not sent (no UX for this) After first call: notification of missed call Phone runs out of battery during call: WebRTC detects disconnect Same as network drop Other party sees: "Simu ilikatizwa" Someone calls during Do Not Disturb (DND): DND on device: depends on device DND rules NexGate mute: call goes to missed calls silently CRITICAL commerce calls: bypass mute (configurable) ``` ### Group Chat Edge Cases ``` Owner leaves group: Must transfer ownership first OR system auto-assigns to oldest admin OR if no admins: oldest member becomes owner Last person leaves group: Group archived automatically Messages preserved for 30 days Then permanently deleted Adding member who blocked you: Cannot add blocked users to groups System rejects silently No error shown (privacy) Member blocked in group: They stay in group (cannot remove via block) Cannot DM each other Can both still see group messages Admin can remove either from group ``` --- ## Summary NexGate messaging is built for East African social commerce — where every conversation can become a transaction, and every transaction has a conversation behind it. The permission system is designed to be safe by default: contacts and mutual followers can communicate freely, strangers go through message requests, and calls require an established relationship. Commerce contexts create natural communication permissions — buyers and sellers can always reach each other through shop identities that protect personal privacy. Contact sync is optional, privacy-preserving (hashed on device), and exists alongside multiple other discovery mechanisms. NexGate does not need contact sync to function — it is one convenience feature among many. Commerce offer sessions are the unique heart of NexGate messaging — a formal negotiated deal between a specific buyer and seller, with a locked price, expiry timer, and full lifecycle tracking. The public product price is never affected. Checkout always happens outside the inbox. Everything is immutable for legal and dispute reasons. The safety system protects users (especially women) through strict defaults, rate limiting, quick block options, and a message request system that gives users control over who enters their inbox. --- *NexGate Messaging — Product Requirements & Feature Flows v1.0* *QBIT SPARK | Rules · Flows · Permissions · Scenarios · Edge Cases*