Directly-message-nexgate-service(6)
NexGate — Private Chat & Calls Flow
NexGate — Private Chat & Calls Flow
Private Chat & Calls Arch Phase 2
NexGate Chat — Phase 2
Production Architecture
NexGate / QBIT SPARK | Version 1.0 Ejabberd · WebRTC Calls · Voice & Video · MessagePack · Coturn · Message Interactions
Table of Contents
- NexGate Chat Roadmap
- What Is Phase 2
- What We Are Building
- Full Architecture
- Ejabberd — The Transport Backbone
- Ejabberd ↔ Spring Boot Bridge
- Authentication Flow
- Message Flow — Phase 2
- Voice Calls
- Video Calls
- Coturn — TURN Relay
- MessagePack Encoding
- Broadcast Channels
- MQTT — Mini Apps Foundation
- Message Interactions
- Docker Deployment
- Database Schema
- Commerce Stanzas & Custom Namespaces
- Build Order
1. NexGate Chat Roadmap
Before any code is written — understand the full journey. Three stages. Each builds on the previous.
Stage 1 — Local Experiments (Terminal Only, No Coding)
Goal: understand the tools before building with them
Duration: 1 week
Output: confidence, not code
Method: terminal only — Docker CLI, curl, sendxmpp
NO Android app
NO Android Studio
NO Java project
NO NexGate codebase
Everything in this stage is throwaway
Run it locally on your Xubuntu machine
No production VPS involved
Tools to Install First
# XMPP CLI client
sudo apt install sendxmpp
# WebSocket CLI client
wget https://github.com/vi/websocat/releases/download/v1.12.0/websocat.x86_64-unknown-linux-musl
chmod +x websocat.x86_64-unknown-linux-musl
sudo mv websocat.x86_64-unknown-linux-musl /usr/local/bin/websocat
# STUN test client
sudo apt install stuntman-client
# Network packet inspection
sudo apt install tcpdump
# Python + MessagePack (for experiment 7)
pip3 install msgpack --break-system-packages
# curl and docker — already installed ✅
Experiment 1 — Ejabberd Running Locally
Goal: get Ejabberd running, send one message via terminal
Success: message delivered, logs confirm routing
# Start Ejabberd
docker run -d \
--name ejabberd \
-p 5222:5222 \
-p 5280:5280 \
-p 5285:5285 \
ghcr.io/processone/ejabberd
# Wait for startup
sleep 15
# Check it's running
docker exec ejabberd ejabberdctl status
# Expected: Node ejabberd@localhost is started
# Create two test users
docker exec ejabberd ejabberdctl register alice nexgate.com password123
docker exec ejabberd ejabberdctl register bob nexgate.com password123
# Verify users exist
docker exec ejabberd ejabberdctl registered_users nexgate.com
# Expected output:
# alice
# bob
# Send message alice → bob (no app needed!)
docker exec ejabberd ejabberdctl send_message \
chat alice@nexgate.com bob@nexgate.com \
"" "Habari Bob! Kutoka terminal"
# Watch Ejabberd logs — see message routing
docker logs ejabberd --tail 30
# Open dashboard in browser
# http://localhost:5280/admin
# admin / password (default)
# See users, sessions, statistics
What you learn:
How Ejabberd starts and configures
ejabberdctl is your management CLI
Messages route without any app
Dashboard shows what's happening
Logs show every routing decision
Experiment 2 — REST API (How Spring Boot Will Talk to Ejabberd)
Goal: talk to Ejabberd via HTTP — same way Spring Boot will
Success: curl commands work, responses received
# Send message via REST API (this is exactly what Spring Boot does)
curl -s -X POST http://localhost:5285/api/send_message \
-H "Content-Type: application/json" \
-d '{
"from": "alice@nexgate.com",
"to": "bob@nexgate.com",
"body": "Kutoka curl — kama Spring Boot!"
}' | python3 -m json.tool
# Get all connected users
curl -s -X POST http://localhost:5285/api/connected_users \
-H "Content-Type: application/json" \
-d '{}' | python3 -m json.tool
# Get registered users
curl -s -X POST http://localhost:5285/api/registered_users \
-H "Content-Type: application/json" \
-d '{"host": "nexgate.com"}' | python3 -m json.tool
# Create a MUC group chat room
curl -s -X POST http://localhost:5285/api/create_room \
-H "Content-Type: application/json" \
-d '{
"name": "nexgate-test-room",
"service": "conference.nexgate.com",
"host": "nexgate.com"
}' | python3 -m json.tool
# List active MUC rooms
curl -s -X POST http://localhost:5285/api/muc_online_rooms \
-H "Content-Type: application/json" \
-d '{"service": "conference.nexgate.com"}' | python3 -m json.tool
# Kick a user session
curl -s -X POST http://localhost:5285/api/kick_session \
-H "Content-Type: application/json" \
-d '{
"user": "alice",
"host": "nexgate.com",
"resource": "test",
"reason": "Test kick"
}' | python3 -m json.tool
What you learn:
Every curl call = what Spring Boot RestTemplate does
REST API is how NexGate backend controls Ejabberd
Port 5285 = admin API (internal only in production)
All operations possible without any mobile app
Experiment 3 — sendxmpp (Connect as XMPP User)
Goal: connect as a real XMPP user from terminal
Success: send/receive messages between two terminal sessions
# Terminal 1 — send message as alice
echo "Habari Bob! Ninatuma kutoka terminal" | sendxmpp \
--username alice \
--password password123 \
--host localhost \
--port 5222 \
--domain nexgate.com \
--tls-ca-path /dev/null \
--insecure \
bob@nexgate.com
# Watch Ejabberd logs in another terminal:
docker logs ejabberd -f
# See stanzas flowing in logs:
# Received message from alice@nexgate.com
# Routing to bob@nexgate.com
# Delivered ✅
# Send typing indicator (composing stanza)
# sendxmpp handles this via --chat-state flag
echo "Ninaandika..." | sendxmpp \
--username alice \
--password password123 \
--host localhost \
--port 5222 \
--domain nexgate.com \
--insecure \
--chat-state \
bob@nexgate.com
What you learn:
XMPP login flow from client perspective
Stanza routing in Ejabberd logs
How typing indicators flow
What mobile app will do — terminal does it first
Experiment 4 — Watch Raw XMPP Stanzas
Goal: see actual XML stanzas flowing over the wire
Success: raw XMPP XML visible in terminal
# Terminal 1 — watch all XMPP traffic
sudo tcpdump -i lo -A port 5222 2>/dev/null | grep -A5 "<message\|<presence\|<iq"
# Terminal 2 — connect via websocat (WebSocket)
websocat ws://localhost:5280/ws
# Type this in websocat terminal:
# (open XMPP stream)
# Terminal 3 — send message via sendxmpp
echo "Test stanza" | sendxmpp \
--username alice \
--password password123 \
--host localhost \
--domain nexgate.com \
--insecure \
bob@nexgate.com
# Watch Terminal 1 — see raw XML:
# <message from='alice@nexgate.com'
# to='bob@nexgate.com'
# type='chat'>
# <body>Test stanza</body>
# </message>
What you learn:
What XMPP stanzas actually look like on wire
Difference between connection, auth, message stanzas
How namespaces appear in real traffic
Visual confirmation of everything in the docs
Experiment 5 — Spring Boot Auth Bridge
Goal: Ejabberd calls Spring Boot to validate users
Success: Spring Boot approves/rejects Ejabberd connections
Note: minimal Spring Boot — one endpoint only, H2 in-memory DB
# Step 1: Create minimal Spring Boot project
# ONE controller, ONE endpoint only:
# POST /internal/ejabberd/auth
# Body: { "user": "alice", "host": "nexgate.com", "pass": "password123" }
# Returns: 200 (allow) or 401 (deny)
# Step 2: Run Spring Boot on port 8080
./mvnw spring-boot:run
# Step 3: Configure Ejabberd to call Spring Boot
# Create ejabberd.yml with:
# auth_method: http
# auth_opts:
# url: "http://host.docker.internal:8080/internal/ejabberd/auth"
# Restart Ejabberd with custom config
docker stop ejabberd && docker rm ejabberd
docker run -d \
--name ejabberd \
-p 5222:5222 \
-p 5280:5280 \
-p 5285:5285 \
-v $(pwd)/ejabberd.yml:/home/ejabberd/conf/ejabberd.yml \
ghcr.io/processone/ejabberd
# Step 4: Test auth via sendxmpp
echo "Test" | sendxmpp \
--username alice \
--password password123 \
--host localhost \
--domain nexgate.com \
--insecure \
bob@nexgate.com
# Watch Spring Boot logs:
# "Auth request received: alice@nexgate.com"
# "Validated: allowed ✅"
# Try wrong password
echo "Test" | sendxmpp \
--username alice \
--password WRONG \
--host localhost \
--domain nexgate.com \
--insecure \
bob@nexgate.com
# Spring Boot logs:
# "Auth request received: alice@nexgate.com"
# "Invalid credentials: rejected ❌"
# Ejabberd logs: "Authentication failed"
What you learn:
Auth bridge works exactly as designed
Spring Boot is the source of truth for auth
Ejabberd trusts Spring Boot completely
This is the same bridge NexGate will use
Response time matters — must be < 200ms
Experiment 6 — Two Node Cluster + Erlang Dist
Goal: two Ejabberd nodes talking via Erlang distribution
Success: message sent on node1 arrives at user on node2
# Create Docker network for the cluster
docker network create ejabberd-cluster
# Start node 1
docker run -d \
--name ejabberd-node1 \
--hostname ejabberd-node1 \
--network ejabberd-cluster \
-e ERLANG_NODE=ejabberd@ejabberd-node1 \
-e ERLANG_COOKIE=nexgate_secret_cookie \
-p 5222:5222 \
-p 5280:5280 \
-p 5285:5285 \
ghcr.io/processone/ejabberd
sleep 15
# Start node 2
docker run -d \
--name ejabberd-node2 \
--hostname ejabberd-node2 \
--network ejabberd-cluster \
-e ERLANG_NODE=ejabberd@ejabberd-node2 \
-e ERLANG_COOKIE=nexgate_secret_cookie \
-p 5223:5222 \
-p 5281:5280 \
-p 5286:5285 \
ghcr.io/processone/ejabberd
sleep 10
# Join node2 to node1 cluster
docker exec ejabberd-node2 \
ejabberdctl join_cluster ejabberd@ejabberd-node1
# Verify cluster is formed
docker exec ejabberd-node1 ejabberdctl list_cluster
# Expected:
# ejabberd@ejabberd-node1
# ejabberd@ejabberd-node2 ✅
# Register alice on node1
docker exec ejabberd-node1 \
ejabberdctl register alice nexgate.com pass123
# Register bob on node2
docker exec ejabberd-node2 \
ejabberdctl register bob nexgate.com pass123
# Send message FROM node1 TO bob (who is on node2)
docker exec ejabberd-node1 ejabberdctl send_message \
chat alice@nexgate.com bob@nexgate.com \
"" "Cross-node via Erlang dist!"
# Check node2 logs — message arrived from node1
docker logs ejabberd-node2 --tail 20
# See: message routed from ejabberd@ejabberd-node1 ✅
# Verify cluster health
docker exec ejabberd-node1 ejabberdctl mnesia_info | grep running_db_nodes
# Shows both nodes sharing Mnesia DB ✅
What you learn:
Erlang dist routing works across containers
Same cookie = trusted cluster
No Redis pub/sub needed for cross-node routing
Mnesia shared across nodes automatically
This is production-ready cluster behavior
Experiment 7 — RabbitMQ Events from Ejabberd
Goal: Ejabberd publishes events to RabbitMQ, read them in terminal
Success: see chat events flowing to RabbitMQ queues
# Ensure RabbitMQ is running (already in your stack)
docker ps | grep rabbit
# Configure Ejabberd to publish to RabbitMQ
# Add to ejabberd.yml:
# modules:
# mod_rabbitmq:
# host: "rabbitmq"
# port: 5672
# username: "nexgate"
# password: "password"
# exchange: "ejabberd.events"
# Create the exchange in RabbitMQ
docker exec rabbitmq rabbitmqadmin declare exchange \
name=ejabberd.events \
type=topic \
durable=true
# Create queue and binding
docker exec rabbitmq rabbitmqadmin declare queue \
name=chat.message.inbound \
durable=true
docker exec rabbitmq rabbitmqadmin declare binding \
source=ejabberd.events \
destination=chat.message.inbound \
routing_key=chat.message.inbound
# Send a message via ejabberdctl
docker exec ejabberd ejabberdctl send_message \
chat alice@nexgate.com bob@nexgate.com \
"" "This should appear in RabbitMQ!"
# Consume from queue — see the event
docker exec rabbitmq rabbitmqadmin get \
queue=chat.message.inbound \
ackmode=ack_requeue_false
# Watch queue depth in real time
watch -n 1 'docker exec rabbitmq rabbitmqctl list_queues name messages'
# Open RabbitMQ dashboard
# http://localhost:15672
# See exchanges, queues, message rates ✅
What you learn:
Ejabberd → RabbitMQ event pipeline works
Event payload structure
Queue depth monitoring
This is exactly how Spring Boot Chat Service
will receive Ejabberd events in production
Experiment 8 — Coturn STUN/TURN
Goal: TURN relay server running, STUN tested from terminal
Success: STUN returns public IP, relay connection established
# Start Coturn
docker run -d \
--name coturn \
--network host \
coturn/coturn \
-n \
--log-file=stdout \
--min-port=49152 \
--max-port=65535 \
--lt-cred-mech \
--user=nexgate:testpassword \
--realm=nexgate.com
# Test STUN from terminal
stunclient localhost 3478
# Expected output:
# Binding test: success
# Local address: 127.0.0.1:XXXXX
# Mapped address: 127.0.0.1:XXXXX ✅
# Watch Coturn logs
docker logs coturn -f
# See STUN requests arriving and responses ✅
# Test WebRTC in browser (no Android needed!)
# Open this URL in two browser tabs:
# https://webrtc.github.io/samples/src/content/peerconnection/pc1/
# Configure TURN server: localhost:3478
# Credentials: nexgate / testpassword
# Force TURN (disable direct connections in browser devtools)
# Establish audio connection between tabs
# Watch Coturn logs — see relay traffic ✅
What you learn:
Coturn starts and runs correctly
STUN works (public IP discovery)
TURN relay works (audio through server)
EA carrier NAT bypass confirmed
Browser tabs = simpler than Android emulators
Experiment 9 — MessagePack Size Comparison
Goal: prove MessagePack saves 60% vs JSON on EA networks
Success: numbers printed, saving confirmed
# Create test script
cat > /tmp/test_msgpack.py << 'EOF'
import json
import msgpack
# Real NexGate chat message
message = {
"type": "MSG_SEND",
"conv_id": "conv-123456789",
"sender_id": "usr-987654321",
"body": "Habari yako Juma, vipi biashara leo?",
"timestamp": 1719446400000,
"temp_id": "abc-def-ghi-jkl-mno",
"level": "NORMAL",
"content_type": "TEXT"
}
# Commerce offer stanza metadata
offer_message = {
"type": "CUSTOM_PRICE_OFFER",
"conv_id": "conv-123456789",
"offer_id": "offer-uuid-abc-def",
"product_id": "prod-samsung-a15",
"public_price": 450000,
"offer_price": 400000,
"currency": "TZS",
"valid_minutes": 30
}
print("=" * 50)
print("NEXGATE MESSAGE SIZE COMPARISON")
print("=" * 50)
for name, msg in [("Text message", message), ("Offer message", offer_message)]:
json_bytes = json.dumps(msg).encode()
msgpack_bytes = msgpack.packb(msg)
reduction = round((1 - len(msgpack_bytes)/len(json_bytes)) * 100)
print(f"\n{name}:")
print(f" JSON: {len(json_bytes)} bytes")
print(f" MessagePack: {len(msgpack_bytes)} bytes")
print(f" Saving: {reduction}% smaller")
# Daily usage estimate
print("\n" + "=" * 50)
print("EA DATA BUNDLE IMPACT (1000 messages/day)")
print("=" * 50)
avg_json = 160
avg_msgpack = 60
print(f" JSON: {avg_json * 1000 / 1024:.0f} KB/day")
print(f" MessagePack: {avg_msgpack * 1000 / 1024:.0f} KB/day")
print(f" Saving: {(avg_json - avg_msgpack) * 1000 / 1024:.0f} KB/day per user")
print(f" ~{(avg_json - avg_msgpack) * 1000 * 30 / 1024 / 1024:.1f} MB saved per month")
EOF
python3 /tmp/test_msgpack.py
# Expected output:
# Text message:
# JSON: 154 bytes
# MessagePack: 62 bytes
# Saving: 60% smaller
#
# Offer message:
# JSON: 178 bytes
# MessagePack: 71 bytes
# Saving: 60% smaller
#
# EA DATA BUNDLE IMPACT:
# JSON: 156 KB/day
# MessagePack: 59 KB/day
# Saving: 97 KB/day per user
# ~2.8 MB saved per month ✅
What you learn:
Real numbers — not estimates
60% confirmed on NexGate-specific messages
Monthly saving per EA user calculated
Justifies the MessagePack implementation effort
Experiment Success Checklist
Before moving to Stage 2 (building NexGate):
All 9 must be green ✅
Exp 1 Ejabberd running locally ✅ / ❌
Exp 2 REST API working via curl ✅ / ❌
Exp 3 sendxmpp connects as XMPP user ✅ / ❌
Exp 4 Raw XMPP stanzas visible in tcpdump ✅ / ❌
Exp 5 Spring Boot auth bridge working ✅ / ❌
Exp 6 Two node cluster + Erlang dist ✅ / ❌
Exp 7 RabbitMQ events from Ejabberd ✅ / ❌
Exp 8 Coturn STUN/TURN + browser WebRTC ✅ / ❌
Exp 9 MessagePack saving confirmed ✅ / ❌
All green → Stage 2 starts
Any red → fix it before moving forward
surprises in experiments = learning
surprises in production = problems
Stage 2 — Build NexGate Chat Phase 2
Goal: production-ready chat on NexGate
Duration: ~16 weeks
Output: WhatsApp-class chat shipped to EA users
Start coding HERE — not before
Every experiment above maps to real code:
Exp 1 → Ejabberd Docker in production compose
Exp 2 → Spring Boot EjabberdClient (curl → RestTemplate)
Exp 3 → Mobile app XMPP connection (sendxmpp → Smack SDK)
Exp 5 → Real auth bridge with JWT validation
Exp 6 → Two node cluster on Hetzner VPS
Exp 7 → RabbitMQ consumers in Chat Service
Exp 8 → Coturn on separate Hetzner CX11
Exp 9 → MessagePack in NexGate Chat SDK
16-week build order in Section 19
What ships:
Text chat (1:1 + group)
Voice notes
Voice + video calls (+ switch audio↔video)
Screen sharing
Group calls (LiveKit)
Commerce DMs (both flows)
Offer sessions (full lifecycle)
Message interactions (edit/delete/react/forward)
Shop inbox with staff access
Offline delivery + FCM + Textfy
EA network optimized (Coturn + Opus + H.264)
WhatsApp-class infrastructure
Commerce-aware from day one
Infrastructure:
Ejabberd cluster (2 nodes, same VPS)
Coturn (separate Hetzner CX11 ~€4/month)
Spring Boot Chat Service (new microservice)
All existing infra (Redis, RabbitMQ, PostgreSQL)
File Thunder (already running) ✅
Stage 3 — Eventually (WeChat EA)
Goal: full super app communication platform
Timeline: after Phase 2 is live and growing
VP Live (Video Streaming):
SRS Media Server
RTMP ingest → HLS → Cloudflare CDN
Live comments (Ejabberd MUC)
VOD after stream (File Thunder)
VP Audio Spaces:
LiveKit SFU
Multi-speaker rooms (Twitter Spaces model)
Radio mode (one broadcaster → millions)
Raise hand system
Group Calls:
LiveKit already deployed for Audio Spaces
Activate for group voice + video
Up to 8 participants voice (3G compatible)
Up to 4 video feeds simultaneously
Mini Apps (MQTT):
Ejabberd MQTT broker (already in Ejabberd config)
Third-party apps subscribe to events
JikoXpress integration
Real-time order tracking
NexGate developer platform
WeChat EA:
All of the above live
NexGate = EA's daily life infrastructure
Every transaction has a conversation
Every conversation can become a transaction 🚀
The Progression
NOW THEN EVENTUALLY
─────────────────── ─────────────────── ───────────────────
Terminal only NexGate chat live VP Live streaming
Docker CLI Text + calls VP Audio Spaces
curl + sendxmpp Commerce DMs Group calls
ejabberdctl Offer sessions Mini Apps (MQTT)
tcpdump + wireshark Shop inbox + staff NexGate developer
9 experiments 16 weeks to ship platform
No app built yet WhatsApp-class WeChat EA vision
─────────────────── ─────────────────── ───────────────────
Confidence Product Platform
2. What Is Phase 2
NexGate chat is built directly on Phase 2 architecture from scratch. There is no Phase 1 to migrate from. No Spring Boot WebSocket gateway was ever built. No Redis pub/sub routing to replace.
Phase 2 is the starting point — not an upgrade.
Why start directly on Phase 2:
Ejabberd handles 2M concurrent connections
Spring Boot WS would need many pods to reach this
Ejabberd does it on two Docker containers
Voice + video calls needed from launch
Ejabberd Jingle (XEP-0166) solves signaling natively
Building WebRTC signaling from scratch = months wasted
25+ chat features free from Ejabberd XEPs
Typing indicators, delivery ticks, read receipts,
multi-device sync, message archive, push bridge
All zero custom code — just Ejabberd config
EA network demands carrier-grade infrastructure
Stream Management (XEP-0198) = no message loss on 2G
Cannot afford to rebuild this later
Commerce-aware chat from day one
Custom XMPP namespaces for product cards,
offer sessions, event cards, group purchases
Ejabberd routes them — Spring Boot handles business logic
NexGate chat is a greenfield Phase 2 build.
3. What We Are Building
Building from scratch:
Ejabberd Cluster ← real-time transport
Two nodes, same Hetzner VPS at launch
Handles all WebSocket connections
Routes all XMPP stanzas
Manages presence, MUC, Jingle calls
XEP-0198 stream management for EA networks
Spring Boot Chat Service ← business brain
Message persistence (PostgreSQL)
Commerce context (offer sessions, product cards)
Notification routing (FCM + Textfy)
Shop inbox access control
Call records + quality logs
Offline escalation
Spring Boot Main Backend ← platform API
Auth (PONA Auth V3) + XMPP token issuance
VP Shop, VP Feed, VP Events integration
Commerce triggers to Chat Service
Coturn TURN Server ← voice/video relay
EA carrier NAT bypass
Separate small Hetzner VPS
NexGate Chat SDK ← mobile dev layer
Android (Smack wrapper)
iOS (XMPPFramework wrapper)
Hides all XMPP complexity from mobile dev
Clean Java/Swift API
Infrastructure (already running):
Redis ✅ presence cache, hot messages
RabbitMQ ✅ offline queue, service events
PostgreSQL ✅ persistence
MinIO ✅ media storage
Cloudflare ✅ CDN
Vault ✅ secrets
Traefik ✅ reverse proxy
File Thunder ✅ media processing
FCM + Textfy ✅ notifications
4. Full Architecture
┌──────────────────────────────────────────────────────────┐
│ NexGate Mobile App │
│ Android / iOS │
└────┬──────────────┬──────────────┬──────────────┬───────┘
│ │ │ │
WebSocket HTTP REST WebRTC HLS Player
XMPP stanzas (unchanged) (calls + (streams —
MessagePack Main Backend spaces) Phase 3)
│ │ │
▼ ▼ │
┌─────────────────────────┐ │
│ Ejabberd Cluster │ │
│ │ │
│ Node 1 Node 2 │ │
│ ┌──────┐ ┌──────┐ │ │
│ │Erlang│◀─▶Erlang│ │ │
│ │ dist │ │ dist │ │ │
│ └──────┘ └──────┘ │ │
│ │ │
│ XMPP/WebSocket │ │
│ Presence (built-in) │ │
│ MUC rooms (XEP-0045) │ │
│ Jingle signaling │◀───────┘
│ (XEP-0166) │ (call signaling
│ MQTT broker │ via WS)
│ Push bridge XEP-0357 │
└──────────┬──────────────┘
│
│ HTTP (auth only — sync)
│ RabbitMQ (all events — async)
│
┌──────────▼──────────────────────────────────────────────┐
│ Spring Boot Chat Service │
│ (unchanged from Phase 1) │
│ │
│ Messages · Receipts · Commerce Context │
│ Notification Router · Call Records │
│ Shop Inbox · Offline Escalation │
└──────────┬──────────────┬──────────────────────────────┘
│ │
▼ ▼
┌──────────────┐ ┌──────────────────────────────────────┐
│ PostgreSQL │ │ RabbitMQ │
│ Redis │ │ chat.message.inbound │
│ (unchanged) │ │ chat.presence │
│ │ │ chat.call.events │
└──────────────┘ │ chat.notify.push │
│ chat.notify.escalation │
└──────────────────────────────────────┘
│
▼
┌─────────────────────────────────┐
│ Coturn TURN Server │
│ (separate VPS) │
│ UDP relay for voice/video │
│ when P2P blocked by EA NAT │
└─────────────────────────────────┘
5. Ejabberd — The Transport Backbone
What Ejabberd Owns in Phase 2
✅ All WebSocket connections (2M concurrent per node)
✅ XMPP stanza routing between users
✅ User presence — online/offline/away (built-in protocol)
✅ Typing indicators (XEP-0085)
✅ Message delivery receipts (XEP-0184)
✅ Multi-User Chat rooms — MUC (XEP-0045)
✅ Voice/video call signaling — Jingle (XEP-0166)
✅ Push notification bridge (XEP-0357 → FCM/APNs)
✅ MQTT broker (Mini Apps events)
✅ Stream management / reconnection (XEP-0198)
✅ Cross-node routing (Erlang distributed — no Redis pub/sub needed)
❌ Does NOT touch:
PostgreSQL (NexGate's schema)
Business logic
Commerce context
Payment processing
File processing
Ejabberd Key Modules Enabled
mod_mam Message Archive Management
Stores message history in its own DB
Clients can sync history on reconnect
mod_muc Multi-User Chat
Group chats, live stream comment rooms
Max 500 members per room (configurable)
Persistent rooms survive server restart
mod_ping Keepalive ping every 30 seconds
Kills dead connections automatically
Critical for EA mobile networks
mod_push Push notification bridge
Connects to FCM/APNs on user disconnect
Replaces manual FCM calls from Chat Service
mod_stun_disco STUN/TURN server discovery
Tells clients where Coturn is
Used for voice/video call setup
mod_mqtt MQTT broker on port 1883
For Mini Apps real-time events (Phase 3)
Zero extra infrastructure needed
mod_http_api REST API on port 5285
Spring Boot calls this to send messages
Admin operations (kick user, create room)
Ejabberd Config Highlights
hosts:
- "nexgate.com"
listen:
- port: 5280 # WebSocket — mobile app connects here
module: ejabberd_http
request_handlers:
/ws: ejabberd_ws
/api: mod_http_api
- port: 5285 # REST API — Spring Boot calls here (internal only)
module: ejabberd_http
ip: "127.0.0.1"
request_handlers:
/api: mod_http_api
- port: 1883 # MQTT — Mini Apps (Phase 3)
module: mod_mqtt
- port: 3478 # STUN — voice/video setup
transport: udp
module: ejabberd_stun
# Auth — Ejabberd calls Spring Boot
auth_method: http
auth_opts:
url: "http://chat-service:8082/internal/ejabberd/auth"
auth_header: "X-Internal-Secret"
auth_header_value: "${EJABBERD_INTERNAL_SECRET}"
# PostgreSQL — Ejabberd's own separate database
sql_type: pgsql
sql_server: "postgres"
sql_database: "ejabberd" # NOT nexgate — separate DB
default_db: sql
modules:
mod_mam:
default: always
db_type: sql
mod_muc:
db_type: sql
max_users: 500
mod_ping:
send_pings: true
ping_interval: 30
timeout_action: kill
mod_push: {}
mod_stun_disco:
credentials_lifetime: 3600
services:
- host: "turn.nexgate.com"
port: 3478
type: turn
secret: "${COTURN_SECRET}"
mod_mqtt: {}
mod_http_api: {}
Two Separate PostgreSQL Databases
postgres instance (same server, two databases):
nexgate ← NexGate application data
messages, conversations, users, orders
Spring Boot owns this entirely
Ejabberd never touches this
ejabberd ← Ejabberd's own operational data
message archive (MAM)
MUC room state
roster data
Spring Boot never touches this
Why separate:
Ejabberd manages its own schema migrations
NexGate schema evolves independently
Clean ownership — no shared tables
Easy to backup independently
6. Ejabberd ↔ Spring Boot Bridge
Communication Rules
Ejabberd → Spring Boot:
Auth events: HTTP (synchronous — must respond fast)
Message events: RabbitMQ (async)
Presence events: RabbitMQ (async)
Call events: RabbitMQ (async)
MUC events: RabbitMQ (async)
Spring Boot → Ejabberd:
Send message to user: Ejabberd REST API (port 5285)
Create MUC room: Ejabberd REST API
Kick user session: Ejabberd REST API
Check user online: Ejabberd REST API
Broadcast to room: Ejabberd REST API
Rule: auth is the ONLY synchronous call
Everything else is async via RabbitMQ
RabbitMQ Events from Ejabberd
Exchange: ejabberd.events (topic)
Routing Key Fired When
──────────────────────────────────────────────────────
chat.message.inbound User sends a message
chat.message.group User sends to MUC room
chat.presence.online User WS connects
chat.presence.offline User WS disconnects
chat.call.initiated Jingle session-initiate received
chat.call.accepted Jingle session-accept received
chat.call.declined Jingle session-declined received
chat.call.ended Jingle session-terminate received
chat.muc.created MUC room created
chat.muc.joined User joined MUC room
chat.muc.left User left MUC room
Spring Boot Internal Endpoints (Ejabberd calls these)
POST /internal/ejabberd/auth
Called on every WebSocket connection
Ejabberd sends: { username, token }
Spring Boot responds: 200 (allow) or 401 (deny)
Must respond in < 200ms (checked in Redis cache first)
All other events arrive via RabbitMQ consumers
No other synchronous HTTP endpoints needed
Spring Boot → Ejabberd REST API Examples
Send system message to user:
POST http://ejabberd:5285/api/send_message
{
"from": "system@nexgate.com",
"to": "usr-123@nexgate.com",
"body": "",
"extra": {
"type": "ORDER_STATUS_UPDATE",
"orderId": "ord-456",
"status": "SHIPPED"
}
}
Create live stream MUC room:
POST http://ejabberd:5285/api/create_room
{
"name": "live-stream-abc",
"service": "conference.nexgate.com",
"host": "nexgate.com"
}
Kick expired session:
POST http://ejabberd:5285/api/kick_session
{
"user": "usr-123",
"host": "nexgate.com",
"resource": "android",
"reason": "Token expired"
}
7. Authentication Flow
Two Tokens Issued at Login
User logs into NexGate
│
▼ POST /auth/login
Main Backend (PONA Auth V3):
Validate credentials
Issue two tokens:
REST JWT (7 days):
Used for all HTTP API calls
Standard Bearer token
XMPP Token (24 hours):
Used only for Ejabberd connection
Contains: userId, JID, expiry
Shorter lifetime — chat sessions refresh more often
│
▼ Both tokens returned to app
WebSocket Connection Auth
App connects WebSocket:
wss://chat.nexgate.com/ws
Header: Authorization: Bearer {XMPP_TOKEN}
│
▼
Ejabberd receives connection
Extracts token from header
│
▼ HTTP POST (sync) → Spring Boot
/internal/ejabberd/auth
{ username: "usr-123", token: "XMPP_TOKEN" }
│
Spring Boot:
Check Redis cache first (fast path):
token:{hash} → valid/invalid (TTL 5min)
If not cached:
Validate JWT signature
Check token type == XMPP
Check user not banned/suspended
Cache result in Redis
Return: 200 { authorized: true, jid: "usr-123@nexgate.com" }
or 401 { authorized: false, reason: "TOKEN_EXPIRED" }
│
Ejabberd:
200 → allow connection
register: usr-123@nexgate.com/android as ONLINE
publish to RabbitMQ: chat.presence.online
401 → reject WebSocket
app shows: "Session expired, please login again"
JID Structure
Every NexGate entity has a JID (Jabber ID):
Personal user:
usr-123@nexgate.com/android ← full JID (user + device)
usr-123@nexgate.com ← bare JID (user only)
Shop identity:
techstore@shops.nexgate.com ← shop JID
Multiple staff auth as this JID
Customer sees "TechStore" — not the staff member
System bot:
system@nexgate.com ← order updates, notifications
MUC rooms:
live-abc@conference.nexgate.com ← live stream chat room
group-xyz@conference.nexgate.com ← group chat room
Multi-device:
usr-123@nexgate.com/android ← phone
usr-123@nexgate.com/ios ← tablet
Both receive messages
READ on one → Ejabberd notifies other to clear notification
XMPP Token Refresh
XMPP token expires every 24 hours
App background service:
At 23 hours → POST /auth/refresh-xmpp-token
Header: Bearer {REST_JWT} (still valid — 7 days)
Response: new XMPP token
Re-auth without reconnecting:
App sends new auth stanza on existing WS connection
Ejabberd re-validates via Spring Boot
No disconnection — seamless for user
8. Message Flow — Phase 2
Inbound Message (User Sends)
[Client A — Android]
│
│ WebSocket frame (MessagePack encoded):
│ {
│ type: MSG_SEND
│ temp_id: "abc-123"
│ to: "usr-456@nexgate.com"
│ conv_id: "conv-789"
│ body: "Habari"
│ content_type: TEXT
│ }
│
▼
[Ejabberd Node 1]
│
├── Validates session (already authed)
├── ACKs client immediately:
│ { temp_id: "abc-123", status: ACK }
├── Routes to usr-456 (if online):
│ Erlang looks up which node holds usr-456
│ If Node 1 → delivers directly
│ If Node 2 → Erlang distributed message (no Redis needed)
│
└── Publishes to RabbitMQ: chat.message.inbound
{
from: "usr-123@nexgate.com",
to: "usr-456@nexgate.com",
conv_id: "conv-789",
body: "Habari",
temp_id: "abc-123",
timestamp: 1719446400
}
.
. (async)
.
[Spring Boot Chat Service]
│ consumes chat.message.inbound
│
├── Authorization check (can A message B?)
├── Resolve message level
├── Write to PostgreSQL (messages table)
├── Write to Redis hot cache (last 50 per conv)
├── Update conversation last_message
│
├── usr-456 online? (check via Ejabberd REST API)
│ YES → DELIVERED receipt after Ejabberd confirms
│ NO → RabbitMQ offline queue + FCM + escalation timer
│
└── Notify sender: tick update
REST API → Ejabberd → WS push to Client A
Client A: ✓✓ (delivered)
Cross-Node Routing — No Redis Pub/Sub Needed
Phase 1 (Spring Boot WS):
Pod 1 holds Client A connection
Pod 2 holds Client B connection
Redis pub/sub needed to bridge pods
Pod 1 publishes → Redis → Pod 2 delivers
Phase 2 (Ejabberd cluster):
Node 1 holds Client A connection
Node 2 holds Client B connection
Erlang distributed messaging bridges nodes
Node 1 → Erlang dist → Node 2 delivers
Redis pub/sub no longer needed for routing
(Redis still used by Chat Service for hot cache)
This is why Ejabberd can do 2M concurrent:
Erlang process per connection (~2KB RAM each)
Native cross-node routing built into the language
No external message bus overhead
9. Voice Calls
Components
Signaling: Ejabberd Jingle (XEP-0166)
coordinates call setup via XMPP stanzas
STUN: Ejabberd built-in (mod_stun_disco)
helps devices find their public IP behind NAT
TURN: Coturn (separate VPS)
relay when P2P impossible (EA carrier NAT)
Transport: WebRTC in mobile app
actual audio stream between devices
Codec: Opus
adaptive 6kbps (2G) → 64kbps (WiFi)
echo cancellation + noise suppression built in
non-negotiable for EA networks
Jingle Signaling Stanzas
<!-- Step 1: Kibuti initiates call to Juma -->
<iq from="kibuti@nexgate.com/android"
to="juma@nexgate.com"
type="set" id="call-001">
<jingle xmlns="urn:xmpp:jingle:1"
action="session-initiate"
sid="session-abc-123"
initiator="kibuti@nexgate.com/android">
<content name="audio">
<description xmlns="urn:xmpp:jingle:apps:rtp:1"
media="audio">
<payload-type id="111" name="opus" clockrate="48000"/>
</description>
<transport xmlns="urn:xmpp:jingle:transports:ice-udp:1"
ufrag="someUfrag"
pwd="somePassword">
<candidate ... /> <!-- Kibuti's ICE candidates -->
</transport>
</content>
</jingle>
</iq>
<!-- Step 2: Juma accepts -->
<iq from="juma@nexgate.com/android"
to="kibuti@nexgate.com/android"
type="set" id="call-002">
<jingle action="session-accept"
sid="session-abc-123">
<!-- Juma's SDP answer + ICE candidates -->
</jingle>
</iq>
<!-- Step 3: Call ends -->
<iq type="set">
<jingle action="session-terminate"
sid="session-abc-123">
<reason><success/></reason>
</jingle>
</iq>
Full Voice Call Flow
[Kibuti — taps Call]
│
▼ GET /chat/calls/turn-credentials
Spring Boot returns:
{
iceServers: [
{ urls: "stun:chat.nexgate.com:3478" },
{ urls: "turn:turn.nexgate.com:3478",
username: "usr-123:1719446400",
credential: "hmac_token" }
],
ttl: 3600
}
│
▼ Initialize WebRTC PeerConnection
Add audio track (Opus codec)
Gather ICE candidates (STUN discovery)
│
▼ Send Jingle session-initiate via Ejabberd WS
Ejabberd routes to Juma
Ejabberd fires RabbitMQ event: chat.call.initiated
│
Spring Boot:
Create call record:
status: RINGING
started_at: now
If Juma offline → FCM HIGH priority:
{ type: INCOMING_CALL, callId, callerName, callType: VOICE }
│
[Juma's phone rings]
Juma taps Answer
│
▼ Juma sends Jingle session-accept via Ejabberd WS
ICE negotiation begins between devices:
│
├── P2P possible? (good network)
│ Direct connection established ✅
│ No Coturn bandwidth used
│
└── P2P blocked? (EA carrier NAT)
Both connect to Coturn relay
Audio flows: Kibuti → Coturn → Juma
│
Call live 🎉
RTCP monitors quality every 200ms:
Good network → Opus 32-64kbps, clear voice
3G → Opus 16kbps, still good
2G → Opus 8kbps, slightly robotic but connected
Very poor → Opus 6kbps, minimum viable
│
Kibuti taps End
│
▼ Jingle session-terminate via Ejabberd WS
Ejabberd fires: chat.call.ended
Spring Boot:
Update call record:
status: COMPLETED
ended_at: now
duration_seconds: calculated
relay_used: true/false
Call State Machine
IDLE
│ user taps Call
▼
INITIATING
│ getting TURN credentials
│ creating WebRTC offer
▼
RINGING ──────────────────────▶ MISSED (45s timeout)
│ Jume answers
▼
CONNECTING
│ ICE negotiation
│ finding best path
▼
CONNECTED ────────────────────▶ RECONNECTING (network drop)
│ call live │ ICE restart
│ │ 10s timeout → FAILED
│ user ends
▼
ENDING
│ Jingle terminate sent
▼
COMPLETED / MISSED / DECLINED / FAILED
Codec Ladder — Opus Adaptive
Network Bitrate Quality
─────────────────────────────────────────────────
WiFi / 4G strong 64 kbps HD voice
4G standard 32 kbps Clear
3G 16 kbps Good enough
2G / Edge 8 kbps Robotic but connected
Barely alive 6 kbps Minimum viable
─────────────────────────────────────────────────
Opus switches automatically based on RTCP feedback
No configuration needed — adaptive by design
10. Video Calls
Same Architecture as Voice + Camera
Everything from voice call applies
Additional components:
Video codec: H.264 (primary)
Hardware accelerated on Tecno, Infinix, Samsung
Low battery drain — GPU handles encoding
Fallback: VP8 (software, more CPU)
Camera: Front camera default (switchable)
Device detects capability at call start
Resolution ladder (adaptive):
─────────────────────────────────────────────────
WiFi 720p 30fps 1.5 Mbps
4G strong 480p 24fps 800 kbps
3G 360p 15fps 400 kbps
2G 240p 10fps 150 kbps
Very poor VIDEO OFF — audio only (Opus)
─────────────────────────────────────────────────
Degradation order (never drops call):
1. Reduce color depth
2. Reduce resolution (720→480→360→240)
3. Reduce frame rate (30→24→15→10fps)
4. Reduce audio bitrate
5. Disable video entirely → audio only
6. Reduce audio to minimum (6kbps Opus)
Device Tier Detection
App detects device capability at call start:
High-end (Pixel, Samsung S series):
H.264 hardware encoder (GPU)
Start at 720p 30fps
Low battery impact
Mid-range (Samsung A series):
H.264 hardware encoder
Start at 480p 24fps
Medium battery impact
Low-end (Tecno Spark, Infinix Hot):
H.264 software encoder (CPU)
Start at 360p 15fps
High battery impact
Show warning: "Video call may drain battery faster"
Auto-disable video after 10min if battery < 20%
Jingle for Video — Additional Content Block
<!-- Video call adds video content block -->
<jingle action="session-initiate" sid="session-xyz">
<!-- Audio block (same as voice) -->
<content name="audio">
<description media="audio">
<payload-type name="opus" clockrate="48000"/>
</description>
<transport .../>
</content>
<!-- Video block (added for video calls) -->
<content name="video">
<description media="video">
<payload-type id="96" name="H264" clockrate="90000"/>
<payload-type id="97" name="VP8" clockrate="90000"/>
</description>
<transport .../>
</content>
</jingle>
11. Coturn — TURN Relay
Why TURN is Mandatory for EA
Direct P2P (ideal):
Both devices negotiate directly
Audio/video flows device-to-device
Ejabberd not involved in media
No bandwidth cost on your servers
EA reality — P2P often blocked:
Vodacom, Airtel, Tigo use CGNAT
Multiple users share one public IP
P2P connection cannot be established
Without TURN → call fails
TURN relay (fallback):
Both devices connect to Coturn
Coturn relays audio/video between them
Call works regardless of carrier NAT
Bandwidth cost on your server (~50KB/min voice)
Coturn Config Highlights
listening-port=3478
tls-listening-port=5349
relay-ip=YOUR_COTURN_VPS_IP
realm=nexgate.com
lt-cred-mech # time-limited credentials
use-auth-secret
static-auth-secret=${COTURN_SECRET} # from Vault
min-port=49152
max-port=65535
TURN Credentials Generation
Credentials are time-limited HMAC tokens
Generated by Spring Boot per call session
Coturn validates them — prevents abuse
Format:
username: {userId}:{expiry_timestamp}
credential: HMAC-SHA1(secret, username)
ttl: 3600 seconds (1 hour per call)
Only NexGate users can use your TURN server
No credential → Coturn rejects connection
Bandwidth Estimation
Voice call via TURN:
Opus 16kbps × 2 directions = ~4KB/min
1 hour call ≈ 240KB per participant
Video call via TURN:
360p H.264 × 2 directions = ~6MB/min
Force 360p max when on relay to control cost
Coturn VPS sizing:
Hetzner CX11 (€4/month, 1vCPU/2GB)
20TB bandwidth included
Handles ~500 concurrent voice relay calls
Upgrade to CX21 at scale
12. MessagePack Encoding
Why Switch from JSON
JSON message frame:
{"type":"MSG_SEND","conv_id":"conv-123456","sender_id":"usr-789012",
"body":"Habari","timestamp":1719446400000,"temp_id":"abc-def-ghi"}
Size: ~140 bytes
Every key repeated as string on every message
Numbers encoded as ASCII characters
Parsing: character by character
MessagePack same message:
[binary representation]
Size: ~50 bytes
Keys encoded as integers (schema registered)
Numbers encoded as actual bytes (int32 = 4 bytes)
Parsing: read fixed byte positions
Result:
60-65% smaller on wire
3-5x faster to parse
Critical for users on 2G/3G with limited data bundles
Migration Strategy (No Breaking Change)
Both formats supported simultaneously:
Client sends header:
Content-Type: application/msgpack → MessagePack
Content-Type: application/json → JSON (default)
Ejabberd detects Content-Type
Routes to appropriate deserializer
Migration flow:
Old app version → sends JSON → works fine
New app version → sends MessagePack → works fine
No forced update required
Gradual migration over 30-60 days
Remove JSON support after 90%+ adoption
13. Broadcast Channels
What They Are
Creator → unlimited followers
One-directional: creator posts, followers receive
Like Telegram channels
No replies from followers (unless creator enables Q&A)
Use cases:
Shop announcement channel ($techstore updates)
Creator content channel (@kibuti posts)
NexGate system channel (platform announcements)
Fan-out Strategy
Small channel (< 10,000 followers):
Write-on-send — Chat Service pushes to each follower
Same as group chat fan-out
Large channel (10,000+ followers):
Lazy fan-out — store message once
Followers fetch on open (read-time delivery)
No per-follower push for casual followers
FCM push only to followers with notifications enabled
Same celebrity bypass pattern as VP Feed:
Hot channels → read-time merge
Normal channels → write-time fan-out
Ejabberd MUC for Channels
Broadcast channel = MUC room with restrictions:
Only owner/admins can send messages
Members are read-only subscribers
mod_muc handles this with role configuration:
Role: moderator → can send
Role: visitor → read only
This means channels are built on the
same MUC infrastructure as group chats
No separate implementation needed
14. MQTT — Mini Apps Foundation
What MQTT Enables
Ejabberd runs MQTT broker on port 1883
No extra infrastructure — already in Ejabberd
Mini Apps subscribe to topics:
orders/{orderId} → real-time order updates
delivery/{trackingId} → GPS delivery tracking
live/{streamId}/viewers → viewer count updates
jiko/{restaurantId} → JikoXpress kitchen events
Spring Boot publishes events:
Order shipped → publish to orders/{orderId}
Mini App receives instantly
No polling needed
MQTT vs XMPP for Mini Apps
XMPP (chat):
Full protocol, complex stanzas
Designed for human conversation
Bidirectional, stateful sessions
Right tool for chat
MQTT (events):
Lightweight pub/sub protocol
Designed for IoT and event streams
Minimal overhead (2-byte header)
Right tool for Mini App events
Works on very limited connections
Both live inside Ejabberd:
Same server, different protocols
Mobile app uses XMPP for chat
Mini Apps use MQTT for events
Zero additional infrastructure
15. Message Interactions
All message interaction features are handled via standard XMPP XEPs. Ejabberd routes the stanzas automatically — Spring Boot handles persistence and business rules via RabbitMQ events.
Overview — All Four Features
Feature XEP Status Ejabberd
────────────────────────────────────────────────────────
Edit message XEP-0308 Stable ✅ auto routed
Delete for everyone XEP-0424 Stable ✅ auto routed
Reactions XEP-0444 Stable ✅ auto routed
Forwarding XEP-0297 Stable ✅ auto routed
Reply to message XEP-0461 Experimental auto routed
Stable stanza IDs XEP-0359 Stable ✅ auto assigned
────────────────────────────────────────────────────────
All routed by Ejabberd
Spring Boot handles: validation, persistence, rules
XEP-0359 — Stable Stanza IDs (Foundation)
Before the features — this XEP is the foundation all others depend on. Every message gets a stable server-assigned ID used by reactions, edits, retractions, and replies to reference the correct message.
<!-- Ejabberd automatically adds stanza-id to every message -->
<message from="kibuti@nexgate.com"
to="juma@nexgate.com"
id="client-generated-id">
<body>Habari</body>
<stanza-id xmlns="urn:xmpp:sid:0"
id="server-stable-id-abc123"
by="nexgate.com"/>
<!-- server-stable-id-abc123 is what reactions/edits reference -->
</message>
Message Editing — XEP-0308
Who can edit: Original sender only
Time window: 15 minutes after send
What: Text body only
Commerce cards: ❌ BLOCKED — financial records are immutable
System messages: ❌ BLOCKED — never editable
<!-- Kibuti edits his message -->
<message from="kibuti@nexgate.com"
to="juma@nexgate.com"
type="chat"
id="edit-002">
<body>Habari yako Juma, vipi biashara?</body>
<replace xmlns="urn:xmpp:message-correct:0"
id="server-stable-id-abc123"/>
<!-- references original message by stanza-id -->
</message>
Flow:
Kibuti edits → stanza sent via Ejabberd WS
Ejabberd routes to Juma (if online)
Ejabberd fires RabbitMQ: chat.message.edited
│
Spring Boot:
Is sender original author? ✅
Within 15 minute window? ✅
Not a commerce/system message? ✅
Update messages.body = new text
Update messages.edited_at = now
Increment messages.edit_count
│
Juma's app:
Receives edit stanza
Updates message in place (same position in thread)
Shows "Edited" label under message
Group chats:
Same stanza sent to MUC room JID
Ejabberd MUC broadcasts to all members
All see updated message simultaneously
Delete for Everyone — XEP-0424
Two delete modes:
Delete for me:
Local filter only
No Ejabberd stanza needed
Spring Boot records: message_deletions (scope: SELF)
Recipient unaffected
Delete for everyone:
XEP-0424 retraction stanza
Ejabberd routes to all recipients
Time window: 15 minutes
Commerce cards: ❌ BLOCKED
System messages: ❌ BLOCKED
<!-- Delete for everyone — retraction stanza -->
<message from="kibuti@nexgate.com"
to="juma@nexgate.com"
type="chat"
id="retract-003">
<apply-to xmlns="urn:xmpp:fasten:0"
id="server-stable-id-abc123">
<retract xmlns="urn:xmpp:message-retract:1"/>
</apply-to>
</message>
Flow:
Kibuti retracts → stanza via Ejabberd WS
Ejabberd routes to Juma
Ejabberd fires RabbitMQ: chat.message.retracted
│
Spring Boot:
Is sender original author? ✅
Within 15 minute window? ✅
Not blocked message type? ✅
Soft delete:
messages.deleted_at = now
messages.deleted_by = usr-kibuti
messages.delete_scope = EVERYONE
body NOT removed (audit trail kept)
│
Juma's app:
Receives retraction stanza
Replaces message with:
"This message was deleted"
Same position in thread
Nothing is ever hard deleted from PostgreSQL:
Legal compliance (EA regulations)
Dispute resolution (order/payment disputes)
Admin investigation (fraud cases)
Soft delete always — hard delete never
Reactions — XEP-0444
Model: One reaction per user per message
Emoji set: Limited set at launch
❤️ 👍 😂 😮 😢 🙏
Change: Send new emoji → replaces old
Remove: Send empty → removes reaction
Commerce cards: ✅ ALLOWED (reactions don't modify content)
System messages: ❌ BLOCKED
<!-- Kibuti reacts 👍 to message -->
<message from="kibuti@nexgate.com"
to="juma@nexgate.com"
type="chat"
id="reaction-001">
<reactions xmlns="urn:xmpp:reactions:0"
id="server-stable-id-abc123">
<reaction>👍</reaction>
</reactions>
</message>
<!-- Kibuti changes to ❤️ -->
<message ...>
<reactions xmlns="urn:xmpp:reactions:0"
id="server-stable-id-abc123">
<reaction>❤️</reaction>
</reactions>
</message>
<!-- Kibuti removes reaction -->
<message ...>
<reactions xmlns="urn:xmpp:reactions:0"
id="server-stable-id-abc123">
<!-- empty = removed -->
</reactions>
</message>
Flow:
Kibuti taps 👍 → reaction stanza via Ejabberd WS
Ejabberd routes to Juma
Ejabberd fires RabbitMQ: chat.message.reaction
│
Spring Boot:
Upsert in message_reactions:
ON CONFLICT (message_id, user_id)
→ update emoji + timestamp
Empty emoji received → delete reaction record
│
Juma's app:
Receives reaction stanza
Updates reaction display below message:
👍 1
Kibuti's own reaction: highlighted
Group chats:
Stanza sent to MUC room
Ejabberd MUC broadcasts to all members
All screens update simultaneously:
👍 3 ❤️ 2 😂 1
Notification:
Reaction on your message → FCM push
"Juma reacted 👍 to your message"
Level: NORMAL (FCM only — no SMS)
Muted conversations → no reaction notification
Message Forwarding — XEP-0297
What it is:
Client creates NEW message in target conversation
Original message wrapped inside as reference
Server never "moves" anything
Forwarded label shown with original sender name
Forward chain tracking:
chain = 1: "↪ Forwarded from Juma Mwangi"
chain = 2-4: "↪ Forwarded"
chain = 5+: "↪ Forwarded many times" (misinformation warning)
Multi-forward: up to 5 conversations per action
Max chain: no hard limit but UI degrades label
Commerce rules:
Product card: ✅ anyone can forward
Custom price offer: ❌ private deal — blocked
Order confirmation: ❌ private record — blocked
Payment record: ❌ private record — blocked
System messages: ❌ blocked
<!-- Kibuti forwards Juma's message to Amina -->
<message from="kibuti@nexgate.com"
to="amina@nexgate.com"
type="chat"
id="fwd-001">
<body>Angalia hii</body>
<forwarded xmlns="urn:xmpp:forward:0">
<delay xmlns="urn:xmpp:delay"
stamp="2026-07-02T10:32:00Z"/>
<!-- original send time preserved -->
<message from="juma@nexgate.com"
to="kibuti@nexgate.com"
type="chat"
id="msg-original-001">
<body>Habari yako rafiki!</body>
</message>
</forwarded>
<!-- NexGate forward metadata -->
<nexgate-forward xmlns="urn:nexgate:forward">
<original_sender_name>Juma Mwangi</original_sender_name>
<forward_chain>1</forward_chain>
</nexgate-forward>
</message>
Flow:
Kibuti taps Forward on Juma's message
Picks Amina's conversation
App creates new message stanza (not routing original)
Sends via Ejabberd WS to Amina
Ejabberd routes normally as new message
Fires RabbitMQ: chat.message.inbound (same as any message)
│
Spring Boot:
Validates forward is allowed (type check)
Creates new messages record:
is_forwarded: true
original_sender_name: "Juma Mwangi"
forward_chain: 1
media_ref: original fileId (no re-upload)
│
Amina's app:
Receives as new message
Renders with forwarded label:
┌────────────────────────────────┐
│ ↪ Forwarded from Juma Mwangi │
│ │
│ Habari yako rafiki! │
│ 10:45 │
└────────────────────────────────┘
Media forwarding:
References original fileId — no re-upload
10 people forward same image
→ 1 file in MinIO, 10 message records
File Thunder serves same file to all
Message Replies — XEP-0461
Reply to a specific message in thread
Like WhatsApp/Telegram quote-reply
Shows original message above reply
Status: Experimental ⚠️
Not yet stable standard
But widely implemented
(Gajim, Monal, many others use it)
Safe to implement — unlikely to change drastically
<!-- Juma replies to Kibuti's specific message -->
<message from="juma@nexgate.com"
to="kibuti@nexgate.com"
type="chat"
id="reply-001">
<body>Nzuri sana, asante!</body>
<reply xmlns="urn:xmpp:reply:0"
to="kibuti@nexgate.com"
id="server-stable-id-abc123"/>
<!-- id references the message being replied to -->
</message>
UI renders:
┌────────────────────────────────┐
│ ┌──────────────────────────┐ │
│ │ Kibuti │ │ ← quoted original
│ │ Habari yako Juma! │ │
│ └──────────────────────────┘ │
│ │
│ Nzuri sana, asante! │
│ 10:47 │
└────────────────────────────────┘
Tap on quote → scroll to original message
Commerce Messages — Interaction Rules Summary
Message type Edit Delete(all) React Forward
────────────────────────────────────────────────────────
Text message ✅ 15m ✅ 15m ✅ ✅
Voice note ❌ ✅ 15m ✅ ✅
Image / Video ❌ ✅ 15m ✅ ✅
Product card ❌ ❌ ✅ ✅
Custom price offer ❌ ❌ ✅ ❌
Order confirmation ❌ ❌ ✅ ❌
Payment confirmation ❌ ❌ ✅ ❌
System message ❌ ❌ ❌ ❌
────────────────────────────────────────────────────────
Why commerce cards are protected:
Immutable negotiation record
Seller cannot change agreed price after the fact
Buyer cannot claim different price was offered
Full audit trail in thread — legally important
RabbitMQ Events — New in Phase 2 for Interactions
Exchange: nexgate.chat (topic) — additions:
Routing Key Fired When
──────────────────────────────────────────────────────
chat.message.edited XEP-0308 received
chat.message.retracted XEP-0424 received
chat.message.reaction XEP-0444 received
chat.message.forwarded XEP-0297 received
chat.message.delete_self delete for me (REST call)
16. Docker Deployment
docker-compose additions for Phase 2
ejabberd:
image: ghcr.io/processone/ejabberd:latest
container_name: ejabberd
restart: unless-stopped
ports:
- "5222:5222" # XMPP TCP
- "5280:5280" # WebSocket + HTTP
- "1883:1883" # MQTT
- "3478:3478/udp" # STUN
volumes:
- ./ejabberd/ejabberd.yml:/home/ejabberd/conf/ejabberd.yml
- ./ejabberd/data:/home/ejabberd/database
- ./ejabberd/logs:/home/ejabberd/logs
environment:
- EJABBERD_BYPASS_WARNINGS=true
depends_on:
- postgres
- rabbitmq
networks:
- nexgate-internal
# Coturn on separate VPS — not in same compose
# Deployed independently on Hetzner CX11
# Connects back to NexGate via internal network
Traefik — WebSocket Routing
# Ejabberd service labels for Traefik
labels:
- "traefik.enable=true"
# App connects here for chat
- "traefik.http.routers.chat.rule=Host(`chat.nexgate.com`)"
- "traefik.http.routers.chat.tls=true"
- "traefik.http.routers.chat.tls.certresolver=letsencrypt"
- "traefik.http.services.chat.loadbalancer.server.port=5280"
# Sticky sessions — CRITICAL for WebSocket
# Same user must always hit same Ejabberd node
- "traefik.http.services.chat.loadbalancer.sticky.cookie=true"
- "traefik.http.services.chat.loadbalancer.sticky.cookie.name=ejabberd_node"
Why sticky sessions:
User connected to Ejabberd Node 1
Next request hits Node 2
→ connection context lost → disconnected
Sticky cookie ensures:
usr-123 always → Node 1
usr-456 always → Node 2
WS sessions stable across load balancer
Ejabberd Cluster Config
# Second node joins cluster
# On node 2's ejabberd.yml:
hosts:
- "nexgate.com"
# Erlang cookie must match on all nodes
# Set via environment variable
# Both nodes discover each other automatically
# Erlang distributed handles the rest
# Result:
# Message to usr-456 arrives on Node 1
# usr-456 connected to Node 2
# Erlang routes internally — transparent
17. Database Schema
calls (new in Phase 2)
calls
─────────────────────────────────────────────
call_id UUID PK
caller_id UUID
receiver_id UUID
conversation_id UUID FK → conversations
type ENUM VOICE / VIDEO
status ENUM RINGING / CONNECTED / COMPLETED /
MISSED / DECLINED / FAILED
started_at TIMESTAMPTZ
answered_at TIMESTAMPTZ
ended_at TIMESTAMPTZ
duration_seconds INT
relay_used BOOLEAN
end_reason ENUM NORMAL / NETWORK / TIMEOUT / DECLINED
call_quality_logs (new in Phase 2)
call_quality_logs
─────────────────────────────────────────────
log_id UUID PK
call_id UUID FK → calls
timestamp TIMESTAMPTZ
direction ENUM OUTBOUND / INBOUND
bitrate_kbps INT
packet_loss_pct DECIMAL
jitter_ms INT
rtt_ms INT
resolution TEXT "360p" "480p" "720p" or null
codec_audio TEXT "opus"
codec_video TEXT "h264" "vp8" or null
broadcast_channels (new in Phase 2)
broadcast_channels
─────────────────────────────────────────────
channel_id UUID PK
owner_id UUID userId or shopId
owner_type ENUM USER / SHOP
name TEXT
description TEXT
avatar_file_id UUID
subscriber_count INT
type ENUM PERSONAL / SHOP / SYSTEM
created_at TIMESTAMPTZ
message_reactions (new in Phase 2)
message_reactions
─────────────────────────────────────────────
id UUID PK
message_id UUID FK → messages
conversation_id UUID FK → conversations
user_id UUID
emoji TEXT "👍" "❤️" "😂" etc
reacted_at TIMESTAMPTZ
Unique constraint: (message_id, user_id)
→ one reaction per user per message
→ upsert on conflict replaces emoji
message_deletions (new in Phase 2)
message_deletions
─────────────────────────────────────────────
id UUID PK
message_id UUID FK → messages
deleted_by UUID userId
scope ENUM SELF / EVERYONE
deleted_at TIMESTAMPTZ
messages table additions (Phase 2)
New columns added to existing messages table:
edited_at TIMESTAMPTZ when last edited
edit_count INT how many times edited
original_body TEXT body before first edit (audit)
deleted_at TIMESTAMPTZ soft delete timestamp
deleted_by UUID who deleted
delete_scope ENUM SELF / EVERYONE
is_forwarded BOOLEAN was this forwarded
forward_chain INT forwarding depth (1,2,3...)
original_sender_name TEXT display name at forward time
original_message_id UUID source message if forwarded
reply_to_id UUID FK → messages (for replies)
stanza_id TEXT Ejabberd XEP-0359 stable ID
18. Commerce Stanzas & Custom Namespaces
The Extensible Part of XMPP
XMPP was designed to be extended by anyone for anything. The "X" in XMPP = Extensible.
Any application can add custom XML elements inside standard XMPP stanzas using their own namespace. Ejabberd routes the entire stanza as-is — it never parses, validates, or modifies custom elements. Spring Boot reads them on the other side.
Standard stanza:
<message from="a@nexgate.com" to="b@nexgate.com">
<body>Habari</body>
</message>
With NexGate custom element:
<message from="a@nexgate.com" to="b@nexgate.com">
<body>Habari</body>
<nexgate-offer xmlns="urn:nexgate:offer:1">
... your custom data here ...
</nexgate-offer>
</message>
Ejabberd:
Routes whole stanza as-is ✅
Never touches nexgate-offer element ✅
Never validates it ✅
Just delivers it ✅
NexGate Namespace Registry
All custom namespaces NexGate defines:
urn:nexgate:commerce:1 product cards
urn:nexgate:offer:1 price offer sessions
urn:nexgate:groupbuy:1 Bei ya pamoja cards
urn:nexgate:event:1 event cards
urn:nexgate:feed:1 VP Feed post cards
urn:nexgate:live:1 live stream cards
urn:nexgate:audio:1 audio space cards
urn:nexgate:system:1 system messages
urn:nexgate:forward forwarding metadata
urn:nexgate:states recording voice note state
urn:nexgate:meta message metadata
Versioning (:1, :2):
Allows schema evolution
Old app sees :1 → renders fine
New app sees :2 → renders richer UI
Old clients fall back to <body> text
No breaking changes
Product Card Stanza
Sent by: Spring Boot via Ejabberd REST API
When: Buyer taps "Chat with Seller" on product page
<message from="system@nexgate.com"
to="techstore@shops.nexgate.com"
type="chat"
id="card-001">
<!-- Fallback for basic clients -->
<body>Mteja anaomba habari: Samsung A15</body>
<nexgate-commerce xmlns="urn:nexgate:commerce:1">
<type>PRODUCT_CARD</type>
<initiated_by>usr-kibuti</initiated_by>
<conv_id>conv-789</conv_id>
<product>
<id>prod-123</id>
<name>Samsung A15</name>
<public_price>450000</public_price>
<currency>TZS</currency>
<image_url>https://cdn.nexgate.com/img.jpg</image_url>
<stock>12</stock>
<shop_name>TechStore</shop_name>
<shop_id>shop-456</shop_id>
<snapshot_at>2026-07-13T08:30:00Z</snapshot_at>
<!-- price frozen at this moment — never changes -->
</product>
</nexgate-commerce>
</message>
Seller's app renders:
┌─────────────────────────────────────┐
│ 📦 Samsung A15 │
│ TZS 450,000 │
│ Inapatikana: Vipande 12 │
│ TechStore │
│ [Jibu] [Angalia Bidhaa] │
└─────────────────────────────────────┘
Custom Price Offer Stanza
Sent by: Seller's app via Ejabberd WebSocket
When: Seller attaches price offer from shop
(both Flow 1 post-negotiation and Flow 2 direct attach)
<message from="techstore@shops.nexgate.com/amina"
to="kibuti@nexgate.com"
type="chat"
id="offer-002">
<body>Bei yako maalum: TZS 400,000</body>
<nexgate-offer xmlns="urn:nexgate:offer:1">
<offer_id>offer-uuid-abc</offer_id>
<conv_id>conv-789</conv_id>
<valid_minutes>30</valid_minutes>
<initiated_by>SELLER</initiated_by>
<product>
<id>prod-123</id>
<name>Samsung A15</name>
<image_url>https://cdn.nexgate.com/img.jpg</image_url>
<shop_name>TechStore</shop_name>
<shop_id>shop-456</shop_id>
</product>
<pricing>
<public_price>450000</public_price>
<offer_price>400000</offer_price>
<currency>TZS</currency>
<discount_amount>50000</discount_amount>
<discount_pct>11</discount_pct>
</pricing>
<!-- Staff who sent offer — not visible to buyer -->
<!-- Buyer always sees "TechStore" not "Amina" -->
<sent_by_staff>usr-amina</sent_by_staff>
</nexgate-offer>
</message>
Buyer's app renders:
┌─────────────────────────────────────┐
│ 💰 Bei Maalum Kwako │
│ Samsung A15 │
│ ~~TZS 450,000~~ │
│ TZS 400,000 (umepunguziwa 50,000) │
│ Inaisha: dakika 30 │
│ Idadi: [─ 1 +] │
│ [Kataa] [Endelea Kulipa →] │
└─────────────────────────────────────┘
Offer Response Stanzas
<!-- Buyer declines offer -->
<message from="kibuti@nexgate.com"
to="techstore@shops.nexgate.com"
type="chat"
id="resp-003">
<body>Nimekataa bei hii</body>
<nexgate-offer xmlns="urn:nexgate:offer:1">
<offer_id>offer-uuid-abc</offer_id>
<response>DECLINED</response>
</nexgate-offer>
</message>
<!-- System sends expiry notification -->
<message from="system@nexgate.com"
to="conv-789-participants"
type="chat"
id="expire-004">
<nexgate-offer xmlns="urn:nexgate:offer:1">
<offer_id>offer-uuid-abc</offer_id>
<response>EXPIRED</response>
<message_id>offer-002</message_id>
<!-- references offer card message to update its UI -->
</nexgate-offer>
</message>
Order Confirmation Stanza
Sent by: Spring Boot via Ejabberd REST API
When: Buyer completes checkout successfully
<message from="system@nexgate.com"
to="conv-789-participants"
type="chat"
id="confirm-005">
<body>Agizo limefanikiwa!</body>
<nexgate-system xmlns="urn:nexgate:system:1">
<type>ORDER_CONFIRMATION</type>
<order_id>ord-xyz-789</order_id>
<conv_id>conv-789</conv_id>
<offer_id>offer-uuid-abc</offer_id>
<summary>
<product_name>Samsung A15</product_name>
<quantity>1</quantity>
<amount_paid>400000</amount_paid>
<currency>TZS</currency>
<status>CONFIRMED</status>
</summary>
</nexgate-system>
</message>
Both buyer and seller see:
┌─────────────────────────────────────┐
│ ✅ Agizo Limethibitishwa │
│ Ord #ORD-XYZ-789 │
│ Samsung A15 × 1 │
│ TZS 400,000 imelipwa │
│ [Fuatilia Agizo] │
└─────────────────────────────────────┘
Bei ya Pamoja Card Stanza
<message from="kibuti@nexgate.com"
to="juma@nexgate.com"
type="chat"
id="gb-006">
<body>Jiunge na group buy hii!</body>
<nexgate-groupbuy xmlns="urn:nexgate:groupbuy:1">
<group_buy_id>gb-xyz</group_buy_id>
<product_id>prod-123</product_id>
<product_name>Samsung A15</product_name>
<product_image>https://cdn.nexgate.com/img.jpg</product_image>
<group_price>350000</group_price>
<public_price>450000</public_price>
<currency>TZS</currency>
<current_participants>7</current_participants>
<target_participants>10</target_participants>
<expires_at>2026-07-13T18:00:00Z</expires_at>
</nexgate-groupbuy>
</message>
Event Card Stanza
<message from="kibuti@nexgate.com"
to="juma@nexgate.com"
type="chat"
id="evt-007">
<body>Jiunge na event hii!</body>
<nexgate-event xmlns="urn:nexgate:event:1">
<event_id>evt-456</event_id>
<title>Dar Tech Summit 2026</title>
<date>2026-08-15T09:00:00Z</date>
<venue>Julius Nyerere ICC, Dar es Salaam</venue>
<ticket_price>25000</ticket_price>
<currency>TZS</currency>
<cover_image>https://cdn.nexgate.com/evt.jpg</cover_image>
<available_tickets>150</available_tickets>
</nexgate-event>
</message>
VP Feed Post Card Stanza
<message from="kibuti@nexgate.com"
to="juma@nexgate.com"
type="chat"
id="post-008">
<body>Angalia post hii</body>
<nexgate-feed xmlns="urn:nexgate:feed:1">
<post_id>post-789</post_id>
<author_name>Kibuti Mwangi</author_name>
<author_avatar>https://cdn.nexgate.com/av.jpg</author_avatar>
<caption>Bidhaa mpya zimefika! 🔥</caption>
<media_url>https://cdn.nexgate.com/post.jpg</media_url>
<media_type>IMAGE</media_type>
<like_count>245</like_count>
</nexgate-feed>
</message>
Spring Boot — How It Handles Custom Stanzas
All stanzas arrive via RabbitMQ: chat.message.inbound
Spring Boot parses XML and routes by namespace:
Namespace detected Handler
──────────────────────────────────────────────────
urn:nexgate:commerce:1 handleProductCard()
urn:nexgate:offer:1 handleOfferSession()
urn:nexgate:groupbuy:1 handleGroupBuy()
urn:nexgate:event:1 handleEventCard()
urn:nexgate:feed:1 handlePostCard()
urn:nexgate:system:1 handleSystemMessage()
none of the above handleTextMessage()
Offer Session — Spring Boot Processing
CUSTOM_PRICE_OFFER received:
Spring Boot:
Create message record (type: CUSTOM_PRICE_OFFER)
Create commerce_offer_sessions record:
offer_id: from stanza
buyer_id: conversation partner
shop_id: sender shop JID
product snapshot: from stanza
offer_price: from stanza (server authoritative)
expires_at: now + valid_minutes
status: PENDING
Schedule RabbitMQ delayed job:
delay: valid_minutes
payload: { offerId, action: EXPIRE }
Send FCM to buyer:
"TechStore amekutumia bei maalum"
Level: IMPORTANT
Buyer taps "Endelea Kulipa":
POST /checkout/initiate { offerId, quantity }
Spring Boot:
Validate: status=PENDING, not expired, buyer matches
Update status: CHECKOUT
Price from DB — never from client ✅
Return: { checkoutUrl, checkoutToken }
Order completes:
Update status: COMPLETED
order_id: linked
Send ORDER_CONFIRMATION stanza to conversation
Expiry fires (RabbitMQ delayed job):
Status still PENDING? → mark EXPIRED
Status already changed? → do nothing
Send OFFER_EXPIRED stanza to conversation
commerce_offer_sessions Table
commerce_offer_sessions
─────────────────────────────────────────────────────
offer_id UUID PK
conv_id UUID FK → conversations
message_id UUID FK → messages
shop_id UUID
buyer_id UUID
sent_by_staff UUID staff who sent (audit only)
product_id UUID
product_name TEXT
product_image_url TEXT
snapshot_json JSONB full product at offer time
public_price BIGINT TZS
offer_price BIGINT TZS (custom — server auth)
currency TEXT TZS
quantity_min INT
quantity_max INT
discount_amount BIGINT
discount_pct DECIMAL
status ENUM PENDING / ACCEPTED /
DECLINED / EXPIRED /
CHECKOUT / COMPLETED /
CANCELLED / ABANDONED
valid_minutes INT
expires_at TIMESTAMPTZ
initiated_by ENUM BUYER / SELLER
notes TEXT
created_at TIMESTAMPTZ
responded_at TIMESTAMPTZ
checkout_at TIMESTAMPTZ
completed_at TIMESTAMPTZ
order_id UUID FK → orders (after completion)
19. Build Order
NexGate chat is built from scratch — no migration, no Phase 1 to carry forward. This is the recommended sequence:
Week 1-2 — Local Experiments
Ejabberd running in Docker locally
Two containers (node1 + node2) clustered
Auth bridge: Spring Boot validates XMPP tokens
Send first message between two test JIDs
Confirm Erlang dist working between nodes
Confirm RabbitMQ events firing to Spring Boot
Week 3-4 — PostgreSQL Schema + Chat Service
All tables created (messages, conversations,
receipts, calls, offer sessions, reactions etc)
Spring Boot Chat Service:
RabbitMQ consumers for all Ejabberd events
Message persistence
Receipt tracking
Notification routing (FCM + Textfy)
Week 5 — Ejabberd Staging Deployment
Deploy to Hetzner staging VPS
Two node cluster live
Traefik sticky sessions configured
Auth bridge connected to Chat Service
Send first real message through staging Ejabberd
Week 6-7 — Mobile SDK + Basic Chat
NexGate Chat SDK (Android + iOS)
Smack / XMPPFramework wrapper
Clean send/receive API
Auto-reconnect + Stream Management
Text messages working end-to-end
Typing indicators
Delivery + read ticks
Presence (online/offline)
Week 8 — Message Interactions
Reactions (XEP-0444)
Edit messages (XEP-0308)
Delete for everyone (XEP-0424)
Forwarding (XEP-0297)
Replies (XEP-0461)
Week 9 — Voice Calls
TURN credentials endpoint in Spring Boot
Coturn deployed (Hetzner CX11)
Jingle signaling through Ejabberd
WebRTC on Android/iOS
Opus audio confirmed on 2G test
Coturn relay confirmed on EA network
Week 10 — Video Calls
H.264 video track added
Adaptive resolution ladder
Resolution ladder tested on 3G
Week 11 — Commerce DMs
Custom namespace stanzas:
PRODUCT_CARD
CUSTOM_PRICE_OFFER
OFFER_DECLINED / OFFER_EXPIRED
ORDER_CONFIRMATION
Offer session lifecycle
Both commerce flows (buyer initiates + seller attaches)
Checkout redirect flow
Shop inbox isolation + access control
Week 12 — Group Chats + Broadcast
MUC rooms (Ejabberd XEP-0045)
Group message reactions
Group typing indicators
Broadcast channels (read-only MUC)
Week 13 — Offline + Notifications
RabbitMQ offline queue
FCM HIGH priority integration
Textfy SMS escalation
Notification levels (NORMAL/IMPORTANT/CRITICAL)
Catch-up banner on reconnect
Week 14 — MessagePack
MessagePack encoding in SDK
Content-Type header detection in Ejabberd
Both JSON + MessagePack supported simultaneously
EA bandwidth savings confirmed
Week 15 — Testing + EA Network Testing
Test on actual Vodacom/Airtel SIM cards
Test on Tecno/Infinix devices
Test on 2G/3G networks
Call quality on Coturn relay confirmed
Commerce flow end-to-end confirmed
Week 16 — Ship 🚀
Production deployment
Two Ejabberd nodes live
All features confirmed
NexGate chat is live
Summary
NexGate chat is built directly on Phase 2 architecture from scratch. No migration. No legacy code. Greenfield build on carrier-grade infrastructure from day one.
Ejabberd Cluster (two Docker containers, same Hetzner VPS at launch) handles all WebSocket connections, XMPP stanza routing, presence, MUC group chats, Jingle voice/video signaling, and 25+ chat features via standard XEPs — all at zero custom code cost. Erlang Distribution connects the two nodes directly, routing messages between them in microseconds without Redis pub/sub.
Spring Boot Chat Service owns all business logic — message persistence, commerce context, offer session lifecycle, shop inbox access control, notification routing, and call records. It communicates with Ejabberd asynchronously via RabbitMQ for all events except auth, which is synchronous HTTP because Ejabberd needs an immediate allow/deny decision.
Custom XMPP Namespaces extend the protocol for NexGate's commerce features. Product cards, custom price offers, offer session responses, Bei ya pamoja cards, event cards, and post cards all travel as custom XML elements inside standard XMPP stanzas. Ejabberd routes them as-is — Spring Boot parses and handles them. Commerce messages are server-authoritative and immutable: offer prices come from the database, not the client. Public product prices are never touched.
WebRTC + Coturn handles voice and video calls. Jingle stanzas through Ejabberd coordinate setup. Opus adapts audio from 64kbps on WiFi to 6kbps on 2G. H.264 hardware acceleration keeps video calls battery-friendly on EA phones. Coturn relay ensures calls work behind EA carrier NAT on Vodacom, Airtel, and Tigo.
MessagePack reduces message frame size 60-65% — real saving for EA users on limited data bundles. Both JSON and MessagePack supported simultaneously during SDK rollout.
The build is 16 weeks from local experiments to production. WhatsApp-class infrastructure. Commerce-aware from day one. EA network optimized throughout.
NexGate Chat Platform — Phase 2: Production Architecture v1.0 QBIT SPARK | Ejabberd · Coturn · WebRTC · Commerce Stanzas · Edit · Delete · React · Forward
NexGate — Private Chat & Calls Flow
VP Live & VP Audio Spaces
Live Streaming Architecture
NexGate / QBIT SPARK | Version 1.0 SRS · HLS · LiveKit · VP Live Video · VP Audio Radio · VP Audio Spaces
Table of Contents
- Overview
- VP Live vs VP Audio — Key Differences
- How Live Streaming Works
- VP Live — Video Streaming
- VP Audio Radio — One Broadcaster Many Listeners
- VP Audio Spaces — Multi Speaker Rooms
- Live Chat — Ejabberd MUC
- Stream Key System
- File Thunder Integration — VOD After Stream
- Codecs & EA Network Strategy
- Docker Deployment
- Database Schema
- Scale Path
1. Overview
VP Live and VP Audio Spaces live under VP Feed — the social pillar of NexGate. They are not separate products. They are the live expression layer of the social platform — where creators, merchants, and communities connect with their audiences in real time.
VP Feed
┌───────────────────────────────────────────────────┐
│ │
│ Social Posts Stories Reels Live │
│ │
│ ┌─────────────────┐ │
│ │ VP Live │ │
│ │ Video Stream │ │
│ ├─────────────────┤ │
│ │ VP Audio │ │
│ │ Radio │ │
│ ├─────────────────┤ │
│ │ VP Audio │ │
│ │ Spaces │ │
│ └─────────────────┘ │
└───────────────────────────────────────────────────┘
All three modes share the same infrastructure foundation: SRS for ingest and transcoding, Cloudflare CDN for delivery, Ejabberd MUC for live chat, File Thunder for VOD processing, and Spring Boot for stream management and business logic.
2. VP Live vs VP Audio — Key Differences
VP Live VP Audio Radio VP Audio Spaces
(Video) (Radio/Podcast) (Twitter Spaces)
──────────────────────────────────────────────────────────────────────
Broadcasters 1 1 Multiple (up to 30)
Viewers Unlimited Unlimited Unlimited listeners
Direction One way One way Multi-speaker
Broadcaster RTMP RTMP audio WebRTC (LiveKit)
transport (video+audio) (audio only)
Listener HLS video HLS audio HLS audio
transport (adaptive) (adaptive) (listeners)
WebRTC (speakers)
Latency 6-15 seconds 6-15 seconds Speakers: <200ms
Listeners: 6-15s
Bandwidth High Very low Low (speakers)
broadcaster (2-4 Mbps) (128 kbps) Very low (listeners)
Bandwidth Medium Very low Very low
listener (300kbps-2Mbps) (32-128 kbps) (32-128 kbps)
Works on 2G? ❌ No ✅ Yes ✅ Listeners yes
Live chat Ejabberd MUC Ejabberd MUC Ejabberd MUC
Raise hand ❌ ❌ ✅
VOD after ✅ File Thunder ✅ File Thunder ✅ File Thunder
New infra SRS SRS SRS + LiveKit
3. How Live Streaming Works
The Core Pattern — RTMP → HLS → CDN
Broadcasting (sending):
Broadcaster's phone records camera + mic
App encodes: H.264 video + AAC audio
App streams via RTMP protocol to SRS server
One stream upload from broadcaster
Processing (server):
SRS receives RTMP stream
FFmpeg transcodes to multiple quality variants
Packages into HLS format (2-second chunks)
Writes chunks to MinIO storage every 2 seconds
Delivery (viewing):
Cloudflare CDN pulls chunks from MinIO
Caches chunks at edge nodes globally
Viewers request HLS playlist → adaptive player picks quality
10,000 viewers = 10,000 CDN requests, NOT 10,000 SRS requests
SRS barely notices the viewer count
Why HLS and not WebRTC for viewers:
WebRTC to viewers: broadcaster uploads N streams (one per viewer)
HLS via CDN: broadcaster uploads 1 stream → CDN serves all
At 10,000 viewers: WebRTC = impossible, HLS = trivial
HLS — What It Actually Is
HLS (HTTP Live Streaming) — Apple's open standard
SRS generates:
master.m3u8 → playlist of all quality variants
360p/playlist.m3u8 → playlist for 360p variant
360p/seg_000.ts → 2-second video chunk
360p/seg_001.ts → next 2-second chunk
720p/playlist.m3u8
720p/seg_000.ts
...
master.m3u8 looks like:
#EXTM3U
#EXT-X-STREAM-INF:BANDWIDTH=400000,RESOLUTION=640x360
360p/playlist.m3u8
#EXT-X-STREAM-INF:BANDWIDTH=1500000,RESOLUTION=1280x720
720p/playlist.m3u8
Player (ExoPlayer / AVPlayer):
Downloads master.m3u8 first
Measures current network speed
Picks 360p if on 3G → plays seg_000.ts → seg_001.ts → ...
Switches to 720p if network improves → seamless
All automatic — zero app code needed for quality switching
4. VP Live — Video Streaming
Full Architecture
[Broadcaster Phone]
│
│ RTMP stream
│ rtmp://stream.nexgate.com/live/{streamKey}
│ H.264 video + AAC audio
│ ~2-4 Mbps upload
▼
[SRS Media Server]
│
├── Validates stream key:
│ POST /internal/stream/validate
│ { streamKey: "abc123" }
│ Spring Boot: ✅ allow or ❌ reject
│
├── Receives raw RTMP stream
│
├── FFmpeg transcoding (real-time):
│ 1080p H.264 → 3 Mbps (WiFi viewers)
│ 720p H.264 → 1.5 Mbps (4G viewers)
│ 480p H.264 → 600 kbps (3G viewers)
│ 360p H.264 → 300 kbps (2G viewers)
│
├── Package as HLS:
│ Segment every 2 seconds
│ live/{streamKey}/master.m3u8
│ live/{streamKey}/360p/seg_NNN.ts
│ live/{streamKey}/720p/seg_NNN.ts
│
└── Write to MinIO: nexgate-live bucket
New segments every 2 seconds
│
▼
[Cloudflare CDN]
│ Pulls from MinIO automatically
│ Caches at edge (Nairobi edge closest to EA)
│ Short TTL: 10 seconds (live content)
│
▼
[Viewers — ExoPlayer (Android) / AVPlayer (iOS)]
Requests master.m3u8
Player picks quality based on network
Downloads .ts segments every 2 seconds
Seamless adaptive quality switching
Stream Key Validation Flow
Broadcaster taps "Go Live" in app
│
▼ POST /live/start
Spring Boot:
Generate unique stream key
Store in DB:
stream_key: "abc123"
user_id: usr-kibuti
status: PENDING
created_at: now
Return stream key to app
│
App connects RTMP:
rtmp://stream.nexgate.com/live/abc123
│
SRS receives connection
│
▼ POST /internal/stream/validate (SRS webhook)
Spring Boot checks:
Key exists? ✅
User account active? ✅
User has live permission? ✅
No other active stream for this user? ✅
→ 200 OK → SRS allows stream
→ Update DB: status: LIVE, started_at: now
→ Notify followers via FCM:
"Kibuti anastreamu sasa! Tazama live"
→ Create Ejabberd MUC room:
live-abc123@conference.nexgate.com
Broadcaster App — What Mobile Dev Implements
Android library: rtmp-rtsp-stream-client-java
iOS library: HaishinKit (Swift)
Steps for broadcaster app:
1. GET /live/start → receive stream key
2. Initialize camera + microphone
3. Connect RTMP to stream.nexgate.com/live/{key}
4. Start streaming — library handles everything:
H.264 encoding (hardware)
AAC audio encoding
RTMP packet framing
Network reconnection on drop
5. Show: viewer count (from Redis via REST poll)
live comments (from Ejabberd MUC via WS)
duration timer
6. Tap End → POST /live/end → cleanup
Adaptive upload bitrate:
Library monitors upload speed
Reduces video quality if upload struggles
Broadcaster's bad network → lower quality for viewers
Never drops stream if avoidable
Viewer App — What Mobile Dev Implements
Android: ExoPlayer (Google's official video player)
iOS: AVPlayer (built into iOS, zero setup)
Steps for viewer app:
1. GET /live/{streamId}/url
Response: { masterUrl, viewerCount, startedAt }
2. Feed masterUrl to ExoPlayer/AVPlayer
3. Player handles everything automatically:
Downloads master.m3u8
Picks quality based on network
Downloads segments every 2s
Switches quality up/down seamlessly
4. Join Ejabberd MUC room → show live comments
5. Player shows: loading → buffering → playing
That is genuinely all the viewer needs to implement.
HLS + ExoPlayer/AVPlayer is the easiest viewer experience
to build in all of mobile development.
5. VP Audio Radio — One Broadcaster Many Listeners
Why Audio Radio Matters for EA
VP Live video:
Broadcaster needs: 2-4 Mbps upload
Viewer needs: 300kbps minimum
Data cost viewer: ~900MB per hour at 360p
Works on: 4G and strong 3G only
VP Audio Radio:
Broadcaster needs: 64-128 kbps upload
Listener needs: 32 kbps minimum
Data cost listener: ~15MB per hour at 32kbps
Works on: 2G, Edge, any connection
For a farmer in rural Tanzania with 2G:
VP Live video → impossible, too expensive
VP Audio Radio → accessible, affordable
Use cases:
Live podcast / commentary
Religious broadcasts (huge in EA)
Political discussions
Community announcements
Sports commentary
Language learning sessions
Business webinars (audio only)
Architecture — Same SRS, Audio Only
[Broadcaster Phone]
│
│ RTMP audio only (no video track)
│ AAC codec, 128 kbps
│ rtmp://stream.nexgate.com/audio/{streamKey}
▼
[SRS Media Server]
│
├── Same validation flow as VP Live
│
├── FFmpeg transcoding (audio only):
│ AAC 128 kbps → good network listeners
│ AAC 64 kbps → 3G listeners
│ AAC 32 kbps → 2G listeners
│
├── Package as HLS audio:
│ audio/{streamKey}/master.m3u8
│ audio/{streamKey}/128k/seg_NNN.aac
│ audio/{streamKey}/32k/seg_NNN.aac
│
└── Write to MinIO: nexgate-live bucket
│
▼
[Cloudflare CDN]
│
▼
[Listeners — ExoPlayer / AVPlayer]
HLS audio playlist
Adaptive bitrate: 128k → 32k automatically
Same player, same code — just no video surface
Codec Choice — AAC Not Opus
Why AAC for HLS audio radio (not Opus):
Opus is better quality at low bitrates — true
But HLS has a compatibility requirement:
Apple mandates AAC for HLS audio
AVPlayer on iOS does not support Opus in HLS
Using Opus → iOS listeners cannot play
AAC → works on every device, every OS
Opus is used for:
Voice calls (WebRTC — different transport)
Voice notes (file-based, not streaming)
AAC is used for:
VP Live audio track (in video stream)
VP Audio Radio (HLS streaming)
VP Audio Spaces listener HLS output
AAC at 32kbps for EA:
Acceptable speech quality
~15MB per hour
Works on any 2G connection
Universal device support
6. VP Audio Spaces — Multi Speaker Rooms
The Concept
Not one broadcaster → many listeners
Multiple people in a shared audio room
Some speak, many listen
Listeners can raise their hand to speak
Host controls who gets the mic
Like Twitter Spaces, Clubhouse, Discord Stage Channels
Key insight:
Speakers need LOW LATENCY (<200ms)
to have a natural conversation
HLS (6-15s delay) is too slow for speakers
Listeners just need to HEAR clearly
HLS delay is fine — they're not responding
HLS scales to millions via CDN
Solution: TWO transport layers in one room
Speakers → WebRTC (LiveKit SFU) → <200ms
Listeners → HLS via CDN → 6-15s delay → millions scale
LiveKit SFU — What It Is
SFU = Selective Forwarding Unit
Traditional conference (MCU):
Server mixes ALL audio into one stream
Sends mixed stream to everyone
High CPU (server does all mixing)
Simple client
LiveKit SFU approach:
Each speaker sends audio once to LiveKit
LiveKit forwards each speaker's stream
to all other speakers
Speakers' apps mix locally (device CPU)
Much lower server CPU
Lower latency
Better quality (no mixing artifacts)
For listeners:
LiveKit outputs a mixed HLS stream
Goes through SRS → Cloudflare CDN
Listeners get one mixed audio stream
Same HLS pattern as Audio Radio
Who built LiveKit:
The same team that built Twitter Spaces
Then open sourced it
Actively maintained, Docker ready
Official Android + iOS SDKs available
Full Architecture
[Speaker A phone] ──WebRTC──▶┐
[Speaker B phone] ──WebRTC──▶│
[Speaker C phone] ──WebRTC──▶│
▼
[LiveKit SFU]
│
┌──────────┼──────────────┐
│ │ │
WebRTC fwd HLS output Room events
to speakers (mixed audio) to Spring Boot
│ │
[Speakers [SRS receives
hear each HLS from LiveKit]
other live] │
▼
[Cloudflare CDN]
│
▼
[Thousands of listeners
via HLS audio player]
ExoPlayer / AVPlayer
(same as Audio Radio)
Room events (raise hand, join, leave):
LiveKit → Spring Boot via webhook
Spring Boot → Ejabberd MUC → all participants
Ejabberd MUC → Listeners also see events
(who joined as speaker etc)
Raise Hand Flow
Listener wants to speak:
│ taps "Raise Hand" 🖐
│ sends via Ejabberd WS to MUC room:
│ { type: RAISE_HAND, roomId: "space-abc" }
│
▼
Spring Boot:
Records raise hand request
Notifies host via Ejabberd WS:
{ type: HAND_RAISED, userId, displayName }
Host sees list of raised hands in UI
│
Host taps "Allow to speak" on a listener:
│
▼
Spring Boot:
Calls LiveKit API:
Update participant permissions:
canPublish: true ← now allowed to send audio
Generate new LiveKit token for this user
(speaker token, not listener token)
Send token to user via Ejabberd WS:
{ type: SPEAKER_PROMOTED, livekitToken: "..." }
│
Former listener's app:
Receives promotion event
Stops HLS player (was listening at 15s delay)
Connects WebRTC to LiveKit with speaker token
Starts sending audio
Now hears speakers at <200ms latency
Other speakers hear them immediately
│
Host can also:
Lower someone's hand (dismiss)
Mute a specific speaker
Remove speaker (back to listener)
End the space entirely
Speaker vs Listener — Connection Types
┌──────────────────────────────────────────────────────┐
│ Audio Space Room │
│ │
│ Speakers (up to ~20-30): │
│ Connected via WebRTC to LiveKit │
│ Send and receive audio streams │
│ Latency: <200ms (real conversation) │
│ Connection: persistent WebRTC │
│ │
│ Listeners (unlimited): │
│ Connected via HLS to Cloudflare CDN │
│ Receive mixed audio only │
│ Latency: 6-15 seconds (fine — just listening) │
│ Connection: HTTP requests every 2s │
│ Scale: millions — CDN handles it │
│ │
│ All participants: │
│ Connected to Ejabberd MUC room │
│ Text chat, reactions, raise hand events │
│ Room membership awareness │
└──────────────────────────────────────────────────────┘
LiveKit Token System
Spring Boot manages all LiveKit tokens
(LiveKit has official Java SDK)
Host token:
canPublish: true
canSubscribe: true
roomAdmin: true
→ full control, can speak, manage
Speaker token:
canPublish: true
canSubscribe: true
roomAdmin: false
→ can speak, cannot manage room
Listener token:
canPublish: false ← cannot send audio
canSubscribe: true ← can hear speakers
roomAdmin: false
→ receive only
Token generation:
GET /audio-spaces/{spaceId}/join
Spring Boot checks:
Is user the host? → host token
Is user an approved speaker? → speaker token
Otherwise → listener token (gets HLS URL instead)
LiveKit Docker Config
livekit:
image: livekit/livekit-server:latest
container_name: livekit
restart: unless-stopped
ports:
- "7880:7880" # HTTP API (Spring Boot calls here)
- "7881:7881" # WebRTC TCP
- "7882:7882/udp" # WebRTC UDP (primary)
- "50000-60000:50000-60000/udp" # ICE relay ports
volumes:
- ./livekit/livekit.yaml:/etc/livekit.yaml
command: --config /etc/livekit.yaml
# livekit.yaml
port: 7880
rtc:
tcp_port: 7881
udp_port: 7882
use_external_ip: true
redis:
address: redis:6379 # reuses existing Redis ✅
turn:
enabled: true
domain: turn.nexgate.com
tls_port: 5349
credential: "${COTURN_SECRET}" # reuses existing Coturn ✅
room:
max_participants: 10000
empty_timeout: 300
LiveKit reuses:
Redis → already deployed ✅
Coturn → already deployed for calls ✅
No new infrastructure beyond LiveKit container itself
7. Live Chat — Ejabberd MUC
All three live modes (VP Live, Audio Radio, Audio Spaces) use Ejabberd MUC rooms for real-time text interaction.
Room Lifecycle
Stream / space starts:
│
Spring Boot → Ejabberd REST API:
POST /api/create_room
{
name: "live-{streamId}",
service: "conference.nexgate.com"
}
Room created: live-abc@conference.nexgate.com
│
Broadcaster / host auto-joined as moderator
│
Viewers / listeners join room as participants:
App connects Ejabberd WS
Sends MUC join stanza:
<presence to="live-abc@conference.nexgate.com/Kibuti">
<x xmlns="http://jabber.org/protocol/muc"/>
</presence>
│
Comments sent as MUC messages:
<message to="live-abc@conference.nexgate.com"
type="groupchat">
<body>Mzuri sana! 🔥</body>
</message>
│
All room members receive instantly
No delay — Ejabberd MUC is real-time
│
Stream / space ends:
Spring Boot → Ejabberd REST API:
POST /api/destroy_room
{ name: "live-abc", service: "conference.nexgate.com" }
Room destroyed, members disconnected
Special Events in Live Chat
Beyond text comments, the MUC room carries:
Reactions (emoji bursts):
{ type: REACTION, emoji: "🔥", userId, displayName }
Client renders floating emoji animation
Gifts:
{ type: GIFT, giftId, giftName, amount, userId, displayName }
Client renders gift animation
Spring Boot processes payment separately
Raise hand (Audio Spaces only):
{ type: RAISE_HAND, userId, displayName }
Host sees in management panel
Speaker promoted (Audio Spaces only):
{ type: SPEAKER_PROMOTED, userId, displayName }
All participants see "Amina joined as speaker"
Viewer count updates:
Broadcast every 30 seconds from Spring Boot
{ type: VIEWER_COUNT, count: 12453 }
Product card dropped by broadcaster:
{ type: PRODUCT_CARD, productId, name, price }
Viewers tap → go to VP Shop product page
Commerce during live ✅
Viewer / Listener Count
Two sources of truth:
1. Ejabberd MUC occupant count:
GET ejabberd REST /api/get_room_occupants_count
{ room: "live-abc", host: "conference.nexgate.com" }
→ exact WebSocket-connected count
2. Redis counter (includes HLS-only listeners):
INCR live:{streamId}:viewers → on HLS playlist request
DECR → on playlist stop / timeout
More accurate for Audio Radio/Spaces
where many listeners never connect WS
Display count = Redis counter (higher, more accurate)
Spring Boot broadcasts to MUC every 30 seconds
8. Stream Key System
Stream Key Design
Stream key = single-use authentication token
Broadcaster uses it to connect RTMP to SRS
SRS validates with Spring Boot before accepting stream
Format: random 32-character alphanumeric string
Example: nx_live_a1b2c3d4e5f6g7h8i9j0k1l2m3n4
Lifecycle:
PENDING → generated, not yet used
LIVE → broadcaster connected, stream active
ENDED → stream finished normally
EXPIRED → generated but never used (24h TTL)
REVOKED → manually stopped by admin
One active stream per user at a time
Attempting second stream → rejected by Spring Boot validation
SRS Webhooks to Spring Boot
SRS fires these events to Spring Boot:
on_publish → broadcaster connected RTMP
Spring Boot: validate key, update status LIVE,
notify followers FCM,
create Ejabberd MUC room,
create LiveKit room (if audio space)
on_unpublish → broadcaster disconnected
Spring Boot: update status ENDED,
trigger File Thunder for VOD,
destroy Ejabberd MUC room,
log stream duration + peak viewers
on_play → viewer started watching HLS
Spring Boot: increment Redis viewer counter
on_stop → viewer stopped watching
Spring Boot: decrement Redis viewer counter
9. File Thunder Integration — VOD After Stream
What Happens After Stream Ends
Stream ends (broadcaster taps End / disconnects)
│
SRS fires on_unpublish webhook
│
Spring Boot:
Update stream record: status ENDED
Trigger File Thunder for VOD processing
SRS has saved full recording as .mp4
│
▼
Spring Boot → File Thunder:
POST /api/v1/upload/request (HMAC signed)
{
ownerId: broadcasterId,
domain: POSTS,
context: LIVE_RECORDING,
filename: "stream_{streamId}.mp4",
mimeType: "video/mp4"
}
Returns: presigned MinIO PUT URL
│
Spring Boot pulls recording from SRS
Uploads to MinIO via presigned URL
POST /api/v1/confirm { fileId }
│
▼
File Thunder VideoWheel processes:
HLS transcoding (all quality variants)
Thumbnail extraction (best frame detection)
Watermark: "@{broadcasterUsername}"
NO outro — live recordings are long
NO shortClip — full stream only
Store in nexgate-public bucket
│
▼
File Thunder fires webhook: media ready
Spring Boot:
Creates VOD post on broadcaster's profile
"Watch replay" button appears
Appears in VP Feed for followers
Stream record linked to VOD fileId
New File Thunder Contexts for Live
Existing contexts (unchanged):
SOCIAL_VIDEO regular video posts
DM_ATTACHMENT files sent in DMs
DIGITAL_PRODUCT digital goods in VP Shop
...
New contexts added for live:
LIVE_RECORDING full stream VOD
VideoWheel — no outro, no shortClip
always HLS, always long
AUDIO_RECORDING audio space / radio recording
AudioWheel processes
outputs: .m4a (AAC)
podcast episode on profile
waveform extracted (like voice notes)
nexgate-live MinIO Bucket
Existing buckets:
nexgate-raw temp uploads
nexgate-public social content
nexgate-private DMs and private files
nexgate-digital VP Shop digital products
New bucket:
nexgate-live live stream segments only
Why separate:
SRS writes directly here (not via File Thunder)
Short TTL segments — deleted after stream ends + VOD ready
Different CDN caching rules (10s TTL vs 1 year for VOD)
Different access pattern (SRS writes, CDN reads)
Easy to monitor storage growth separately
Lifecycle:
Stream starts → SRS creates live/{streamKey}/ folder
During stream → .ts segments written every 2 seconds
Stream ends → Spring Boot schedules cleanup job
VOD confirmed → delete nexgate-live/{streamKey}/ folder
Total life: stream duration + ~1 hour buffer
10. Codecs & EA Network Strategy
VP Live Video Codecs
Broadcaster encoding (phone → SRS):
Video: H.264 (hardware encoder — mandatory)
Software H.264 too slow for real-time on phones
H.264 hardware support: every phone since 2013
Audio: AAC 128kbps (RTMP standard)
Container: RTMP (streaming protocol)
SRS transcoding (server-side):
Receives H.264 + AAC
Transcodes to HLS quality ladder:
Quality Video bitrate Audio Resolution EA target
─────────────────────────────────────────────────────────
1080p 3 Mbps 128k 1920×1080 WiFi only
720p 1.5 Mbps 128k 1280×720 4G
480p 600 kbps 64k 854×480 3G
360p 300 kbps 48k 640×360 2G minimum
─────────────────────────────────────────────────────────
ExoPlayer/AVPlayer auto-selects based on network
VP Audio Codecs
Audio Radio (broadcaster → SRS):
Codec: AAC 128kbps
Container: RTMP audio only
Audio Radio (SRS → HLS):
128kbps → WiFi/4G listeners
64kbps → 3G listeners
32kbps → 2G listeners (15MB/hour — affordable)
Audio Spaces (speaker → LiveKit):
Codec: Opus (WebRTC standard)
Adaptive: 32-64kbps per speaker
Echo cancellation: mandatory (multiple people)
Noise suppression: mandatory (EA background noise)
Audio Spaces (LiveKit → HLS for listeners):
LiveKit mixes speaker streams
Outputs mixed audio → SRS → HLS
Same AAC ladder as Audio Radio
Listeners hear all speakers in one stream
Adaptive Streaming — EA Principle
The player always knows the network speed
because it measures how fast segments download
Segment download faster than playback → upgrade quality
Segment download slower than playback → downgrade quality
For a viewer in Dodoma on shaky 3G:
Opens stream → starts at 360p (safe default)
Network good → player tries 480p
Stays stable → tries 720p
Network drops → immediately back to 360p
No rebuffering if switch is fast enough
Buffer strategy:
Player buffers 3-4 segments ahead (6-8 seconds)
Gives time to switch quality before buffer empties
Viewer may notice brief quality dip — never a freeze
NexGate player config recommendation:
Min buffer: 6 seconds
Max buffer: 30 seconds
Quality switch: aggressive downgrade, conservative upgrade
→ Prioritize uninterrupted playback over quality
→ EA networks fluctuate — better to be at 360p than buffering
11. Docker Deployment
Full docker-compose for Live Features
# SRS Media Server
srs:
image: ossrs/srs:5
container_name: srs
restart: unless-stopped
ports:
- "1935:1935" # RTMP ingest (broadcaster connects here)
- "8080:8080" # HTTP API + HLS output
- "1985:1985" # SRS management API
volumes:
- ./srs/srs.conf:/usr/local/srs/conf/srs.conf
- ./srs/logs:/usr/local/srs/logs
- ./srs/recordings:/usr/local/srs/objs/recordings
depends_on:
- chat-service
networks:
- nexgate-internal
# LiveKit SFU (Audio Spaces)
livekit:
image: livekit/livekit-server:latest
container_name: livekit
restart: unless-stopped
ports:
- "7880:7880"
- "7881:7881"
- "7882:7882/udp"
- "50000-60000:50000-60000/udp"
volumes:
- ./livekit/livekit.yaml:/etc/livekit.yaml
command: --config /etc/livekit.yaml
depends_on:
- redis
networks:
- nexgate-internal
SRS Config Highlights
listen 1935; # RTMP port
max_connections 1000;
vhost __defaultVhost__ {
# Validate stream key with Spring Boot
http_hooks {
enabled on;
on_publish http://chat-service:8082/internal/stream/validate;
on_unpublish http://chat-service:8082/internal/stream/ended;
on_play http://chat-service:8082/internal/stream/viewer-join;
on_stop http://chat-service:8082/internal/stream/viewer-leave;
}
# HLS output for viewers
hls {
enabled on;
hls_path ./objs/nginx/html;
hls_fragment 2; # 2 second chunks
hls_window 10; # keep last 10 chunks in playlist
}
# FFmpeg transcoding to multiple qualities
transcode {
enabled on;
ffmpeg /usr/bin/ffmpeg;
engine 360p {
enabled on;
vcodec libx264;
vbitrate 300;
vfps 15;
vwidth 640;
vheight 360;
acodec aac;
abitrate 48;
output rtmp://localhost:1935/live360p/{stream};
}
engine 720p {
enabled on;
vcodec libx264;
vbitrate 1500;
vfps 30;
vwidth 1280;
vheight 720;
acodec aac;
abitrate 128;
output rtmp://localhost:1935/live720p/{stream};
}
}
}
Traefik — RTMP Does Not Go Through Traefik
Important: RTMP is TCP port 1935
Traefik handles HTTP/HTTPS only
RTMP port 1935 exposed directly on VPS
What Traefik does handle:
stream.nexgate.com → SRS port 8080 (HLS output)
TLS termination for HLS delivery
RTMP broadcaster connects:
rtmp://stream.nexgate.com:1935/live/{key}
No TLS on RTMP (RTMPS is complex, not needed for launch)
HLS viewers connect via Cloudflare CDN:
https://cdn.nexgate.com/live/{key}/master.m3u8
Cloudflare pulls from SRS port 8080
Traefik handles TLS for this path
12. Database Schema
live_streams
live_streams
─────────────────────────────────────────────
stream_id UUID PK
broadcaster_id UUID FK → users
type ENUM VIDEO / AUDIO_RADIO / AUDIO_SPACE
title TEXT
description TEXT
cover_file_id UUID File Thunder fileId (stream thumbnail)
stream_key TEXT UNIQUE, used for RTMP auth
status ENUM PENDING / LIVE / ENDED / EXPIRED / REVOKED
started_at TIMESTAMPTZ
ended_at TIMESTAMPTZ
duration_seconds INT
peak_viewers INT
total_viewers INT
muc_room_id TEXT Ejabberd MUC room name
vod_file_id UUID File Thunder fileId after processing
created_at TIMESTAMPTZ
audio_spaces
audio_spaces
─────────────────────────────────────────────
space_id UUID PK
stream_id UUID FK → live_streams
livekit_room_id TEXT LiveKit room name
host_id UUID FK → users
title TEXT
status ENUM SCHEDULED / LIVE / ENDED
max_speakers INT default 30
started_at TIMESTAMPTZ
ended_at TIMESTAMPTZ
audio_space_participants
audio_space_participants
─────────────────────────────────────────────
space_id UUID FK → audio_spaces
user_id UUID
role ENUM HOST / SPEAKER / LISTENER
joined_at TIMESTAMPTZ
left_at TIMESTAMPTZ
hand_raised_at TIMESTAMPTZ
promoted_at TIMESTAMPTZ when promoted from listener to speaker
promoted_by UUID host who approved
stream_viewer_stats
stream_viewer_stats
─────────────────────────────────────────────
stat_id UUID PK
stream_id UUID FK → live_streams
timestamp TIMESTAMPTZ
viewer_count INT
quality_360p_pct DECIMAL % of viewers on 360p
quality_720p_pct DECIMAL % of viewers on 720p
avg_watch_seconds INT
13. Scale Path
Current Architecture Limits
Single SRS node (Hetzner CPX31 — €19/month):
Concurrent streams: ~200 (with transcoding)
Concurrent viewers: ~50,000 (before CDN helps)
Bandwidth: 20TB/month included
With Cloudflare CDN:
Concurrent viewers: Unlimited (CDN absorbs it)
SRS only serves cache misses
99%+ cache hit rate → SRS barely loaded
LiveKit single node:
Concurrent spaces: ~500
Speakers per space: up to 30
Listeners per space: Unlimited (HLS via CDN)
This is enough for NexGate launch and
strong early growth — tens of thousands of users
Growth Stage — SRS Horizontal Scale
When 200 concurrent streams is not enough:
SRS Origin node:
Receives RTMP from all broadcasters
Passes stream to Transcode Farm
Transcode Farm (2-3 nodes):
Each node handles FFmpeg transcoding
Horizontal — add nodes as streams grow
CPU-bound work distributed
SRS Edge nodes:
Serve HLS to viewers
Pull from Origin
Multiple edges → load distributed
┌──────────────────────────────────────────┐
│ Broadcaster → SRS Origin │
│ │ │
│ Transcode Farm │
│ (3 nodes, FFmpeg) │
│ │ │
│ ┌──────────┴──────────┐ │
│ SRS Edge 1 SRS Edge 2 │
│ │ │ │
│ Cloudflare CDN ────────────┘ │
│ │ │
│ All viewers (millions) │
└──────────────────────────────────────────┘
WeChat EA Scale — Infrastructure
Broadcaster latency problem:
Current: Broadcaster in Dar → stream goes to Hetzner Germany
150-300ms upload latency
Acceptable but not ideal
At scale: SRS nodes in EA region
Google Cloud Johannesburg
OR AWS Cape Town
Broadcaster → nearby SRS → low latency upload
Better broadcaster experience
Storage cost at scale:
Current: MinIO on Hetzner
At scale: Cloudflare R2
Zero egress cost (unlike AWS S3 which charges per GB)
S3 compatible → zero code change to migrate
At millions of viewer-hours: massive cost saving
Transcoding cost:
CPU-heavy work
At scale: GPU-accelerated FFmpeg nodes
NVIDIA hardware encoding
5-10x faster than CPU
Lower cost per stream transcoded
Summary
VP Live and VP Audio Spaces are the live social layer of NexGate — living under VP Feed alongside regular posts, stories, and reels.
All three modes (VP Live video, VP Audio Radio, VP Audio Spaces) share the same infrastructure foundation. SRS handles all RTMP ingest and transcoding. Cloudflare CDN distributes HLS to unlimited viewers and listeners. Ejabberd MUC powers live chat and room events for all modes. File Thunder processes every stream into a VOD after it ends. Spring Boot manages stream keys, webhooks, room lifecycle, and all business logic.
VP Audio Spaces adds LiveKit SFU for the multi-speaker experience — speakers connect via WebRTC for real-time conversation while listeners receive the same HLS audio stream that Audio Radio uses, scaled to millions via Cloudflare CDN.
The EA network strategy is woven into every decision: HLS adaptive streaming down to 32kbps means VP Audio Radio works on 2G in rural Tanzania. Video quality ladders from 1080p to 360p ensure VP Live is accessible on 3G. The entire viewer and listener experience requires only ExoPlayer or AVPlayer — the simplest possible mobile integration.
For VOD, File Thunder's VideoWheel and new AudioWheel process every recording automatically after the stream ends — creating replay content with thumbnails, watermarks, and adaptive variants, stored in nexgate-public for CDN delivery. The live platform generates permanent content with zero extra work.
NexGate VP Live & VP Audio Spaces — Architecture v1.0 QBIT SPARK | SRS · LiveKit · HLS · Ejabberd MUC · File Thunder VOD
NexGate — Private Chat & Calls Flow
NexGate Chat — Phase 1
Foundation & Local Experiments
NexGate / QBIT SPARK | Version 1.0 Spring Boot WebSocket · Commerce DMs · Offline Delivery · Local Experiments
Table of Contents
- Phase 1 Goals
- What Ships in Phase 1
- Architecture Overview
- The Four Wheels
- Service Design
- Data Flows
- Commerce DM Flows
- Offline Delivery & Notification Escalation
- Message Status System
- Database Schema
- Inbox Model Implementation
- Local Experiments
1. Phase 1 Goals
Phase 1 is about shipping fast and learning.
The goal is not to build the perfect infrastructure from day one. The goal is to get NexGate chat into users' hands as quickly as possible — while running local experiments in parallel that prepare for Phase 2.
Two tracks running simultaneously:
Track A — Production (ship it): Track B — Experiments (learn it):
Spring Boot WebSocket Ejabberd local Docker setup
Text chat + voice notes Spring Boot ↔ Ejabberd auth bridge
Commerce DMs (both flows) WebRTC voice call on emulators
Offline delivery + notifications Coturn TURN relay local test
Message receipts MessagePack encoding test
Isolated shop inbox
Phase 1 architecture is intentionally simpler than Phase 2. Everything designed here carries forward — the schema, the commerce logic, the notification system, the inbox model. Phase 2 only swaps the transport layer.
2. What Ships in Phase 1
Messaging Features
1:1 Personal DMs text, voice notes, media cards
Group chats up to 500 members
Broadcast channels creator → fans (one-way)
Voice notes Opus recorded, waveform rendered
Rich content cards product, custom price, event,
Bei ya pamoja, post, stream link
Commerce DM Features
Buyer initiates chat from product page → product card auto-appears
Seller attaches product WhatsApp-style from inside any conversation
Custom price offer private to buyer, public price unchanged
Proceed to checkout button in chat → redirects to checkout
Order updates in thread confirmation, shipping, delivery
Inbox Features
Isolated shop inbox separate tab per shop
Personal inbox always private, owner only
Shop inbox shared with authorized staff (Pro tier)
Message receipts sent / delivered / read ticks
Typing indicators ephemeral, Redis TTL based
Online presence shown in conversation header
Notification Features
FCM push (Android + iOS) HIGH priority, bypasses Doze mode
Textfy SMS escalation CRITICAL and IMPORTANT messages
Offline queue RabbitMQ holds messages until delivery
Catch-up banner summary on reconnect after offline
3. Architecture Overview
┌─────────────────────────────────────────────────┐
│ NexGate Mobile App │
└──────────────┬──────────────────────────────────┘
│
│ WebSocket (JSON over WS)
│ wss://chat.nexgate.com
│
┌──────────────▼──────────────────────────────────┐
│ Spring Boot Chat Gateway │
│ │
│ Manages WebSocket connections │
│ Validates JWT on connect │
│ Routes incoming frames via Redis pub/sub │
│ Pushes outgoing frames to clients │
│ Registers presence in Redis │
│ Thin — zero business logic │
└──────────────┬──────────────────────────────────┘
│
│ Redis pub/sub
│
┌──────────────▼──────────────────────────────────┐
│ Spring Boot Chat Service │
│ │
│ Message validation + persistence │
│ Conversation + inbox management │
│ Commerce context handling │
│ Receipt tracking │
│ Notification routing │
│ Offline escalation │
│ Shop inbox role enforcement │
└──────┬──────────┬──────────┬────────────────────┘
│ │ │
▼ ▼ ▼
┌──────────┐ ┌────────┐ ┌───────────────────────┐
│PostgreSQL│ │ Redis │ │ RabbitMQ │
│ │ │ │ │ │
│messages │ │presence│ │ chat.offline.delivery │
│convs │ │hot msgs│ │ chat.notify.push │
│receipts │ │typing │ │ chat.notify.escalation│
│calls │ │pub/sub │ │ chat.commerce.events │
└──────────┘ └────────┘ └───────────┬───────────┘
│
┌────────────┼────────────┐
▼ ▼ ▼
┌─────┐ ┌───────┐ ┌────────┐
│ FCM │ │Textfy │ │ Main │
│push │ │ SMS │ │Backend │
└─────┘ └───────┘ └────────┘
Key Rule
Gateway never touches business logic
Chat Service never manages WS connections
Both communicate only via Redis pub/sub
Gateway → Redis pub/sub → Chat Service (inbound)
Chat Service → Redis pub/sub → Gateway (outbound)
4. The Four Wheels
Just as File Thunder has four processing wheels, the Phase 1 chat engine has four foundational components:
┌─────────────────────────────────────────────────┐
│ NEXGATE CHAT ENGINE │
│ │
│ ┌──────────┐ ┌──────────┐ ┌────────────┐ │
│ │ Wheel 1 │ │ Wheel 2 │ │ Wheel 3 │ │
│ │ │ │ │ │ │ │
│ │ Netty │ │ Redis │ │ RabbitMQ │ │
│ │ WS │ │ State │ │ Queue │ │
│ └──────────┘ └──────────┘ └────────────┘ │
│ │
│ ┌────────────┐ │
│ │ Wheel 4 │ │
│ │ │ │
│ │ Textfy │ │
│ │ SMS │ │
│ └────────────┘ │
└─────────────────────────────────────────────────┘
Wheel 1 — Netty WebSocket Spring Boot uses Netty under the hood for WebSocket connections. Handles connect, disconnect, heartbeat, and frame routing. Scales to 50k+ concurrent connections per pod.
Wheel 2 — Redis State Tracks everything ephemeral and hot: online presence per user, typing indicators (5s TTL), last 50 messages per conversation (capped list), unread counts, cross-pod pub/sub routing, notification escalation timers.
Wheel 3 — RabbitMQ Queue Handles everything async: offline message delivery queue, delayed SMS escalation jobs, commerce event publishing to Main Backend, receipt acknowledgment processing.
Wheel 4 — Textfy SMS Critical and important message fallback. NexGate's own SMS platform — zero third-party cost. Swahili templates per message type. Deep links back to specific conversation. Full delivery audit log.
5. Service Design
Chat Gateway Responsibilities
On WebSocket connect:
Validate JWT token
Register user presence in Redis:
presence:{userId} → TTL 30s (refreshed by heartbeat)
Drain offline queue via RabbitMQ trigger
On frame received (inbound):
Validate session
Publish to Redis: chat:inbound
ACK client immediately with temp_id
On Redis pub/sub message (outbound):
Find client connection for recipient
Push WS frame to device
On WebSocket disconnect:
Remove presence from Redis
Update last_seen_at via RabbitMQ event
Chat Service Responsibilities
On inbound message event (from Redis):
Validate sender is conversation member
Check conversation not blocked/archived
Resolve message level (NORMAL / IMPORTANT / CRITICAL)
Write to PostgreSQL
Write to Redis hot cache
Fan-out to recipients:
Online → Redis pub/sub → Gateway → WS push
Offline → RabbitMQ queue + FCM + escalation timer
On commerce message:
Attach product snapshot (frozen at send time)
Emit commerce event to Main Backend via RabbitMQ
On receipt ack:
Update message_receipts in PostgreSQL
Notify sender (tick update) via Redis pub/sub
On presence event (user online):
Drain RabbitMQ offline queue for user
Send catch-up summary if messages missed
RabbitMQ Exchange Design
Exchange: nexgate.chat (topic)
Routing Key Flow
──────────────────────────────────────────────────────
chat.message.inbound Gateway → Chat Service
chat.message.outbound Chat Service → Gateway
chat.notify.push Chat Service → FCM Worker
chat.notify.escalation Chat Service → SMS Worker
chat.receipts.delivered Gateway → Chat Service
chat.receipts.read App → Chat Service
chat.commerce.initiated Chat Service → Main Backend
chat.commerce.price.attached Chat Service → Main Backend
chat.presence.online Gateway → Chat Service
chat.presence.offline Gateway → Chat Service
6. Data Flows
Text Message — Full Sequence
[Client A] [Gateway] [Chat Service] [Client B]
type "Habari"
tap Send
show pending ⏳
│
│ WS frame:
│ { type: MSG_SEND
│ temp_id: "abc"
│ conv_id: "conv-123"
│ body: "Habari" }
│──────────────────▶│
│ publish Redis:
│ chat:inbound
│ ACK: { temp_id: "abc" }
│◀──────────────────────
│ single tick ✓ ◀───│
│ ...................▶ │ consume Redis
│ validate sender
│ write PostgreSQL
│ write Redis cache
│ check B online? ✅
│ publish outbound
│◀ .................. │
│ push WS to B
│─────────────────────────────────────▶│
│ │ show message
│◀ ......................... DELIVERED ack ──│
│ write receipt
│ notify A:
│ double tick ✓✓ ◀──│◀ ................. │
.
│ [B reads conv] .
│◀─────── READ ack ────────────────────│
│ write receipt
│ blue tick ✓✓ ◀────│◀ ................. │
─── solid line = WebSocket (real-time)
... dotted line = Redis pub/sub (async, cross-pod)
Voice Note — Send Flow
User records audio (Opus codec, 16kHz)
│
▼ Upload to File Thunder
GET /media/upload-request (DM_ATTACHMENT context)
→ presigned MinIO URL returned
Upload .ogg file directly to MinIO
POST /media/confirm { fileId }
│
▼ File Thunder processes:
ClamAV scan
Waveform extraction via FFmpeg
→ amplitude array (50 values for UI bars)
→ waveform.webp thumbnail
Store in nexgate-private/messages/{convId}/{fileId}/
│
▼ File Thunder returns: { fileId, waveformData[], durationSeconds }
│
App sends WS frame:
{ type: MSG_SEND
content_type: VOICE_NOTE
media_ref: fileId
duration_seconds: 15
waveform: [0.2, 0.8, 0.6, ...] } ← embedded for instant UI
│
▼ Same path as text message
Recipient receives message with waveform data
Waveform bars render instantly (no extra request)
Tap play → GET /chat/media/{fileId}/url
→ signed URL (5 min TTL)
→ stream audio progressively
Offline Delivery Flow
Message arrives for offline user B
│
Chat Service:
Check Redis: B online? ❌
│
├──▶ RabbitMQ: chat.offline.delivery
│ { messageId, recipientId, level }
│
├──▶ FCM HIGH priority push
│ { type: NEW_MESSAGE
│ convId, senderName, preview }
│
└──▶ RabbitMQ: chat.notify.escalation
delay: CRITICAL=0min, IMPORTANT=10min
.
. (time passes)
.
Escalation consumer wakes:
Check Redis: B online now? ✅ → cancel, done
❌ → send Textfy SMS
│
Textfy SMS:
"NexGate: Ujumbe mpya kutoka Juma.
nexgate.app/chat/conv-123"
│
User turns on WiFi / opens app:
WS reconnects → Gateway registers presence
Chat Service drains RabbitMQ queue (priority order)
CRITICAL first → IMPORTANT → NORMAL
Show catch-up banner:
"Umekosa: 2 maagizo, 5 ujumbe"
DELIVERED receipts fire for all drained messages
7. Commerce DM Flows
Flow 1 — Buyer Initiates from Product Page
Buyer on product page
│ taps "Chat with Seller"
▼
POST /chat/commerce/initiate
{ productId, shopId }
│
Chat Service:
Find or create DM conversation
conversation.type = COMMERCE
conversation.owner_type = SHOP
conversation.owner_id = shopId
│
Fetch product snapshot from Main Backend:
{ name, price, images[0], stock, shopName }
Snapshot frozen at this exact moment ✅
Public price change later → does not affect this card
│
Create first message automatically:
type: PRODUCT_CARD
context_type: PRODUCT
context_ref_id: productId
snapshot_json: { frozen product data }
│
▼ Seller receives in shop inbox tab
┌─────────────────────────────────┐
│ 📦 Samsung A15 │
│ TZS 450,000 │
│ In stock: 12 units │
│ TechStore │
│ │
│ [Reply] [View Product] │
└─────────────────────────────────┘
│
Negotiation happens in thread
│ Agreement reached
▼
Seller attaches custom price offer:
┌─────────────────────────────────┐
│ 💰 Special Price Offer │
│ Samsung A15 │
│ TZS 400,000 (was 450,000) │
│ Valid for you only │
│ │
│ Quantity: [─ 1 +] │
│ [Proceed to Checkout →] │
└─────────────────────────────────┘
│
Buyer taps Proceed → redirected to checkout
Checkout outside inbox — at negotiated price
Public product price: TZS 450,000 unchanged ✅
│
Order placed → confirmation back in thread:
┌─────────────────────────────────┐
│ ✅ Order Confirmed │
│ Order #ORD-789 │
│ Samsung A15 × 1 │
│ TZS 400,000 paid │
└─────────────────────────────────┘
Flow 2 — Seller Attaches from Inside Chat
Seller inside any conversation
│ taps attach (+)
▼
┌─────────────────────────────────┐
│ Attach │
│ │
│ 📷 Image │
│ 🎵 Voice Note │
│ 📄 File │
│ 🏪 From My Shop ◀──── this one │
│ 📅 Event │
│ 👥 Group Purchase │
└─────────────────────────────────┘
│ taps "From My Shop"
▼
Seller browses their shop products
Picks product
Sets custom price for this buyer (optional)
│
▼ Sends as card in chat
┌─────────────────────────────────┐
│ 🏪 TechStore Offer │
│ Samsung A15 │
│ TZS 400,000 │
│ │
│ Quantity: [─ 1 +] │
│ [Proceed to Checkout →] │
└─────────────────────────────────┘
│
Buyer taps Proceed → checkout outside inbox
Commerce Message Types
message.type values for commerce:
PRODUCT_CARD product shared in chat
CUSTOM_PRICE_OFFER seller's private price for this buyer
EVENT_CARD event shared in chat
GROUP_PURCHASE_CARD Bei ya pamoja shared in chat
POST_CARD VP Feed post shared in chat
ORDER_CONFIRMATION system message after order placed
ORDER_STATUS_UPDATE system message for shipping/delivery
PAYMENT_CONFIRMATION system message after payment
8. Offline Delivery & Notification Escalation
Notification Levels
Message level resolved by Chat Service before fan-out:
CRITICAL:
Order placed / payment received / payment failed
Order status changed / delivery update
→ FCM HIGH + Textfy SMS simultaneously (no waiting)
IMPORTANT:
Commerce DM from buyer
Custom price offer received
Bei ya pamoja threshold reached
→ FCM HIGH immediately
→ Textfy SMS after 10 minutes if no delivery ack
NORMAL:
Regular DMs, group messages
Social cards, post shares
→ FCM HIGH only
→ No SMS escalation
Textfy SMS Templates
Order notification (Swahili):
"NexGate: Agizo jipya kutoka [Buyer]!
Kiasi: TZS [amount]. Kagua: nexgate.app/orders/[id]"
Payment received:
"NexGate: Malipo ya TZS [amount]
yamepokelewa kutoka [Buyer].
nexgate.app/wallet"
Commerce DM:
"NexGate: [Buyer] anakuuliza kuhusu
[Product]. Jibu: nexgate.app/chat/[convId]"
Bei ya pamoja:
"NexGate: Watu [n]/[target] wamejiunga!
nexgate.app/group-buy/[id]"
Notification Delivery Log
All notifications tracked for audit — critical for order disputes:
notification_log
─────────────────────────────────────
id UUID
user_id UUID
message_id UUID
level ENUM (NORMAL/IMPORTANT/CRITICAL)
fcm_status ENUM (SENT/DELIVERED/FAILED)
sms_status ENUM (SENT/DELIVERED/FAILED/SKIPPED)
sms_provider TEXT
sent_at TIMESTAMPTZ
delivered_at TIMESTAMPTZ
opened_at TIMESTAMPTZ
9. Message Status System
Client shows status via tick indicators:
⏳ Pending message on device, not yet sent
(no connection)
✓ Sent server received and persisted
Gateway ACK returned with temp_id
✓✓ Delivered recipient device received
WS delivery ack received by Chat Service
✓✓ Read recipient opened the conversation
READ event sent by recipient app
(blue ticks)
Flow:
[Send] → pending ⏳
[Gateway ACK] → sent ✓
[Recipient WS ack] → delivered ✓✓
[Recipient opens conv] → read ✓✓ (blue)
Typing Indicators
Ephemeral — never persisted to PostgreSQL
User starts typing:
App sends: { type: TYPING_START, convId }
Chat Service sets Redis key:
typing:{convId}:{userId} → TTL 5 seconds
Redis pub/sub notifies conversation members
Recipients see "Juma anaandika..."
User stops typing (or TTL expires):
Key auto-expires after 5 seconds
Chat Service notifies: typing stopped
Indicator disappears
10. Database Schema
conversations
conversations
─────────────────────────────────────────────
id UUID PK
type ENUM DM / GROUP / BROADCAST / COMMERCE
owner_type ENUM USER / SHOP
owner_id UUID userId or shopId
title TEXT for groups and broadcast channels
avatar_file_id UUID File Thunder fileId
status ENUM ACTIVE / ARCHIVED / BLOCKED
created_by UUID userId
created_at TIMESTAMPTZ
conversation_members
conversation_members
─────────────────────────────────────────────
conversation_id UUID FK → conversations
user_id UUID
role ENUM MEMBER / ADMIN / OWNER
joined_at TIMESTAMPTZ
last_read_at TIMESTAMPTZ
last_read_seq BIGINT sequence ID (for gap detection)
is_muted BOOLEAN
muted_until TIMESTAMPTZ
messages
messages
─────────────────────────────────────────────
id UUID PK
conversation_id UUID FK → conversations
sender_id UUID
seq BIGINT monotonic per conversation
type ENUM TEXT / IMAGE / VIDEO / VOICE_NOTE /
FILE / PRODUCT_CARD / CUSTOM_PRICE_OFFER /
EVENT_CARD / GROUP_PURCHASE_CARD /
POST_CARD / ORDER_CONFIRMATION /
ORDER_STATUS_UPDATE / PAYMENT_CONFIRMATION /
SYSTEM
body TEXT
media_ref UUID File Thunder fileId
context_type ENUM PRODUCT / ORDER / PAYMENT / EVENT / GROUP_PURCHASE
context_ref_id UUID ref to relevant entity
snapshot_json JSONB frozen context data at send time
reply_to_id UUID FK → messages (thread replies)
status ENUM SENT / DELIVERED / READ / FAILED
level ENUM NORMAL / IMPORTANT / CRITICAL
created_at TIMESTAMPTZ
edited_at TIMESTAMPTZ
deleted_at TIMESTAMPTZ
message_receipts
message_receipts
─────────────────────────────────────────────
message_id UUID FK → messages
user_id UUID
status ENUM DELIVERED / READ
device_id TEXT
timestamp TIMESTAMPTZ
calls
calls
─────────────────────────────────────────────
call_id UUID PK
caller_id UUID
receiver_id UUID
conversation_id UUID FK → conversations
type ENUM VOICE / VIDEO
status ENUM RINGING / CONNECTED / COMPLETED /
MISSED / DECLINED / FAILED
started_at TIMESTAMPTZ
answered_at TIMESTAMPTZ
ended_at TIMESTAMPTZ
duration_seconds INT
relay_used BOOLEAN was TURN relay used?
end_reason ENUM NORMAL / NETWORK / TIMEOUT / DECLINED
shop_conversation_access
shop_conversation_access
─────────────────────────────────────────────
shop_id UUID
user_id UUID staff member
role ENUM MANAGER / SUPPORT_AGENT / READ_ONLY
granted_by UUID owner userId
granted_at TIMESTAMPTZ
revoked_at TIMESTAMPTZ
11. Inbox Model Implementation
Conversation Ownership
Personal DM:
owner_type = USER
owner_id = usr-kibuti
Access: only usr-kibuti
Shop DM:
owner_type = SHOP
owner_id = shop-techstore
Access: anyone with role in shop_conversation_access
for shop-techstore
Tab Resolution (App Side)
App fetches inbox tabs on load:
GET /chat/inbox/tabs
Response:
[
{ type: PERSONAL, label: "Personal", unread: 3 },
{ type: SHOP, shopId: "shop-techstore",
label: "TechStore", unread: 12 },
{ type: SHOP, shopId: "shop-clothinghub",
label: "ClothingHub", unread: 0 }
]
Chat Service resolves tabs by:
1. User's own personal conversations
2. All shops where user has role in
shop_conversation_access
Access Control Check
Every request to open a shop conversation:
Does requesting user own the shop?
YES → allow
NO → check shop_conversation_access:
user_id = requester
shop_id = conversation.owner_id
revoked_at IS NULL
Found? → allow with their role
Not found? → 403 Forbidden
12. Local Experiments
These run in parallel with Phase 1 production work. All throwaway code — not NexGate quality. Goal is understanding, not production output.
Experiment 1 — Ejabberd Local Docker
Goal: get Ejabberd running, send one message
Steps:
docker run -d --name ejabberd \
-p 5222:5222 \
-p 5280:5280 \
-p 5285:5285 \
ghcr.io/processone/ejabberd
Open: http://localhost:5280/admin
Create two test users: alice, bob
Install Conversations app on Android
Connect to localhost:5222 as alice
Send message to bob
See it arrive
What you learn:
How Ejabberd config works
What XMPP stanzas look like in logs
How the dashboard shows connections
What errors look like and how to fix them
Success: message delivered between two test users
XMPP Stanzas — What They Look Like
When Alice sends "Habari" to Bob, this travels over the wire:
<!-- Alice sends message to Bob -->
<message from="alice@localhost"
to="bob@localhost"
type="chat"
id="msg-001">
<body>Habari</body>
<active xmlns="http://jabber.org/protocol/chatstates"/>
</message>
<!-- Bob's typing indicator back -->
<message from="bob@localhost"
to="alice@localhost"
type="chat">
<composing xmlns="http://jabber.org/protocol/chatstates"/>
</message>
<!-- Presence — Alice comes online -->
<presence from="alice@localhost/android">
<show>available</show>
<status>Ninafanya kazi</status>
</presence>
<!-- Message delivery receipt (XEP-0184) -->
<message from="bob@localhost" to="alice@localhost">
<received xmlns="urn:xmpp:receipts"
id="msg-001"/>
</message>
These stanzas are what Ejabberd routes. In Phase 2, NexGate wraps its own data inside custom XMPP stanzas.
Experiment 2 — Spring Boot Auth Bridge
Goal: Ejabberd calls Spring Boot to validate users
Setup:
Simple Spring Boot app (H2 in-memory DB)
One endpoint: POST /internal/ejabberd/auth
receives: { username, token }
returns: 200 (allow) or 401 (deny)
ejabberd.yml:
auth_method: http
auth_opts:
url: "http://host.docker.internal:8080/internal/ejabberd/auth"
Test:
Connect Conversations app to Ejabberd
Ejabberd calls Spring Boot for auth
Spring Boot validates → returns 200
Connection allowed
What you learn:
Auth flow between Ejabberd and Spring Boot
How fast Spring Boot must respond (< 200ms)
What happens when auth fails
How to structure the internal endpoint
Success: Ejabberd rejects unknown users,
allows users Spring Boot approves
Experiment 3 — WebRTC Voice Call on Emulators
Goal: voice call between two Android emulators
Setup:
Two Android Studio emulators running
Simple Android app — just two buttons:
[Call] and [Answer]
WebRTC library: io.getstream:stream-webrtc-android
No UI — just logcat output
What to test:
Create PeerConnection on both
Exchange SDP offer/answer manually (copy-paste between logs)
Exchange ICE candidates
Hear audio between emulators
What you learn:
How WebRTC PeerConnection works in practice
What SDP looks like
What ICE candidates look like
How long negotiation actually takes
What errors appear and how to fix them
Success: audio heard between two emulators
SDP offer example (what WebRTC generates):
v=0
o=- 123456 2 IN IP4 127.0.0.1
s=-
t=0 0
a=group:BUNDLE 0
m=audio 9 UDP/TLS/RTP/SAVPF 111
c=IN IP4 0.0.0.0
a=rtcp:9 IN IP4 0.0.0.0
a=ice-ufrag:someRandomString
a=ice-pwd:anotherRandomString
a=fingerprint:sha-256 AA:BB:CC:...
a=rtpmap:111 opus/48000/2 ← Opus codec negotiated here
a=fmtp:111 minptime=10;useinbandfec=1
Experiment 4 — Coturn TURN Relay
Goal: force audio through TURN relay, confirm it works
Setup:
docker run -d --network=host coturn/coturn \
-n --log-file=stdout \
--min-port=49152 --max-port=65535 \
--lt-cred-mech \
--user=test:password123 \
--realm=nexgate.com
Test:
Disable P2P in WebRTC config (force TURN only)
Run voice call experiment
Confirm audio still flows through relay
What you learn:
How Coturn logs relay connections
Bandwidth used per call (check with iftop)
How to generate HMAC credentials (not plain text)
What happens when Coturn is unavailable
Success: audio heard between emulators via TURN relay
Coturn logs show relay traffic
Experiment 5 — MessagePack Encoding
Goal: compare JSON vs MessagePack on same message
Simple Spring Boot test:
Serialize same chat message object
Once as JSON
Once as MessagePack
Measure:
Byte size comparison
Serialization speed
Deserialization speed
Expected result:
JSON: ~180-200 bytes per message
MessagePack: ~60-70 bytes per message
~65% size reduction confirmed
Success: numbers prove EA bandwidth saving is real
Experiment Success Criteria Summary
Experiment 1 — Ejabberd local:
✅ Message delivered between two users
✅ Dashboard shows live connections
Experiment 2 — Auth bridge:
✅ Ejabberd rejects unknown tokens
✅ Ejabberd allows Spring Boot approved users
Experiment 3 — WebRTC emulators:
✅ Audio heard between two emulators
✅ SDP and ICE flow understood
Experiment 4 — Coturn relay:
✅ Audio heard via forced TURN relay
✅ Bandwidth per call measured
Experiment 5 — MessagePack:
✅ Size reduction confirmed
✅ Serialization speed measured
All five done → ready to write Phase 2 doc
→ ready to build NexGate chat Phase 2
Summary
Phase 1 is the foundation. It ships real features — text chat, voice notes, commerce DMs, offline delivery, the isolated shop inbox — using a clean Spring Boot WebSocket architecture that runs on existing infrastructure.
Everything designed here carries directly into Phase 2. The schema stays. The commerce logic stays. The notification system stays. The inbox model stays. Only the transport layer swaps — Spring Boot WS gateway out, Ejabberd in.
The five local experiments running in parallel are not wasted time. They are the insurance policy that makes Phase 2 a confident execution rather than a risky exploration. Every surprise Ejabberd, WebRTC, and Coturn have in store — you want to find them in a throwaway experiment, not in your production chat system.
NexGate Chat Platform — Phase 1: Foundation & Local Experiments v1.0 QBIT SPARK | Spring Boot WebSocket · Commerce DMs · Offline Delivery
NexGate — Private Chat & Calls Flow
NexGate Messaging — Product Requirements & Feature Flows
NexGate / QBIT SPARK | Version 1.0 What the messaging platform does — rules, flows, permissions, scenarios
Table of Contents
- Document Purpose
- User Identity & Discovery
- Contact Sync
- Inbox Model
- Messaging Permissions
- Call Permissions
- 1:1 Messaging Flows
- Group Chat Flows
- Commerce DM Flows
- Offer Session Flows
- Shareable Content
- Message Interactions
- Voice & Video Call Flows
- Group Call Flows
- Offline & Notification Flows
- Privacy & Safety
- Shop Inbox & Staff Access
- Notification Settings
- Edge Cases & Scenarios
- Follow System — Users & Shops
- Group Join Model
- Shop Chat Initiation
- Broadcast Channels — Why Not Needed
- Commerce Attach — 1:1 vs Group
1. Document Purpose
This document defines WHAT NexGate messaging does — the rules, user flows, permissions, and scenarios for every messaging feature.
It is the product reference document. It does not describe HOW features are built technically. For technical architecture see:
- Doc 3: Phase 2 Production Architecture
- Doc 5: Private Chat & Calls Deep Dive
Audience: Product, mobile developers, QA, design.
2. User Identity & Discovery
Identity on NexGate
Every NexGate user has:
@username unique, chosen at registration
shown publicly on VP Feed
used for messaging: @kibuti
Display name full name shown in conversations
can differ from username
Phone number used for registration + auth
NOT shown publicly by default
discoverable only with user permission
NexGate ID internal UUID
never shown to users
How Users Find Each Other
Discovery mechanism Requires contact sync?
──────────────────────────────────────────────────────
VP Feed follow ❌ No
VP Shop (buyer/seller) ❌ No
Username search (@kibuti) ❌ No
QR code scan ❌ No
Shareable profile link ❌ No
Group chat membership ❌ No
Phone number search ❌ No (type manually)
Contact sync (phonebook) ✅ Yes (optional)
NexGate has multiple discovery paths. Contact sync is one option — not the only one.
Discoverability Settings
Two independent settings:
"Sync my contacts"
Find NexGate users in my phonebook
Default: OFF (user must opt in)
"Let others find me via phone number"
If someone has my number → can find me
Default: ON (can turn off)
These are INDEPENDENT:
I can sync contacts without being findable
I can be findable without syncing contacts
3. Contact Sync
Rules
✅ Optional — never mandatory
✅ Can be enabled/disabled anytime
✅ Can delete synced data anytime from settings
✅ Only finds users who have discoverability ON
❌ Not required to use messaging
❌ Not required for commerce DMs
❌ Contacts data never shared with third parties
❌ Non-NexGate contacts never stored on server
Privacy-Preserving Implementation
How it works when user accepts:
Step 1: Phone numbers hashed ON DEVICE
sha256(+255712345678) → hash
Actual numbers never leave the device
Step 2: Hashes uploaded to NexGate server
Not real phone numbers
Server cannot reverse hashes
Step 3: Server matches hashes
Against registered users who have
"Let others find me via phone number" = ON
Returns: which hashes are NexGate users
Step 4: App shows matched users
"3 of your contacts are on NexGate"
[Connect] buttons shown
Non-NexGate contacts:
Their hashes deleted from server immediately
Never stored
Never tracked
First Launch Prompt
Shown once at first launch after onboarding:
┌──────────────────────────────────────────────┐
│ 📱 Pata Marafiki Wako NexGate │
│ │
│ Ruhusu NexGate kutumia contacts zako │
│ kupata marafiki wanaotumia NexGate tayari │
│ │
│ ✓ Contacts zako hazitashirikiwa na mtu │
│ ✓ Unaweza kufuta ruhusa wakati wowote │
│ ✓ Nambari za simu zinabadilishwa kuwa │
│ msimbo kabla ya kupakiwa │
│ │
│ [Ruhusu Contacts] [Sasa Hivi Sio] │
└──────────────────────────────────────────────┘
If declined:
App works fully ✅
Remind once after 7 days
Never prompt again after second decline
Always accessible in Settings
Sync Scenarios
Scenario 1 — User accepts sync:
Hashes uploaded
Matched users shown
User can message/follow matched users
Sync runs again when contacts change
Scenario 2 — User declines sync:
App works fully
Can still find people via username/QR/VP Feed
No contacts uploaded
Scenario 3 — User accepts then disables:
Settings → Privacy → Contact Sync → OFF
All hash data deleted from server
Matched contacts remain as connections
(connections not removed — only future sync stops)
Scenario 4 — User wants to delete contact data:
Settings → Privacy → Delete Contacts Data
All hashes deleted from server immediately
Confirmation shown: "Contacts data imefutwa"
Scenario 5 — Contact not on NexGate:
Their hash deleted from server
User sees option: "Invite Juma to NexGate"
Tap → share NexGate invite link via SMS/WhatsApp
4. Inbox Model
Two Separate Inboxes
Every NexGate user has TWO completely separate inboxes:
Personal Inbox:
1:1 DMs with other users (personal)
Group chats (personal)
Friend conversations
Private — only the account owner sees this
NEVER accessible to shop staff
Shop Inbox (one per shop owned):
All customer conversations for that shop
Commerce DMs with buyers
Shared with authorized shop staff
Completely separate from personal inbox
Customer sees shop identity — not personal name
Inbox Tabs
User with 2 shops sees:
┌─────────────────────────────────────────────┐
│ 💬 Inbox │
│ │
│ [Personal ●3] [TechStore ●12] [ClothingHub]│
│ │
│ Personal tab: │
│ Regular DMs + group chats │
│ Private — owner only │
│ │
│ TechStore tab: │
│ Customer commerce conversations │
│ Shared with assigned staff │
│ │
│ ClothingHub tab: │
│ Separate shop conversations │
│ Different staff assigned │
└─────────────────────────────────────────────┘
Staff member (Amina assigned to TechStore):
┌─────────────────────────────────────────────┐
│ [Personal] [TechStore ●12] │
│ │
│ ClothingHub: NOT visible (no access) │
│ Owner personal: NEVER visible │
└─────────────────────────────────────────────┘
Conversation Types
Type Owner Participants
────────────────────────────────────────────────────
DM USER 2 users
GROUP USER 3-500 users
COMMERCE SHOP 1 shop + 1 buyer
BROADCAST USER or SHOP 1 sender + N followers
5. Messaging Permissions
Who Can Message Who
Relationship Can message? How
────────────────────────────────────────────────────────────
Saved contacts (phone) ✅ Always Direct
Mutual followers (VP Feed) ✅ Always Direct
Commerce relationship ✅ Always Direct (shop inbox)
Group chat members ✅ Always Via group only
Strangers (no relationship) ⚠️ Request Message request first
Blocked users ❌ Never Blocked
Message Request System
Stranger (no relationship) wants to message you:
They can send ONE message request:
┌─────────────────────────────────────────┐
│ 📩 Maombi ya Ujumbe │
│ │
│ Juma Mwangi anataka kukutumia ujumbe │
│ @juma_mwangi · 245 wafuasi │
│ │
│ "Habari, nilikuona kwenye VP Shop..." │
│ │
│ [Kubali] [Kataa] [Zuia] │
└─────────────────────────────────────────┘
If Kubali (Accept):
Full conversation opens
Juma can now message freely
Can call after conversation established
If Kataa (Decline):
Juma cannot send another request
One attempt only per account
Juma sees: "Ujumbe haukukubaliwa"
If Zuia (Block):
Juma blocked immediately
Cannot contact again in any way
Juma sees: nothing (as if you don't exist)
If Ignored (no response):
Request stays in request folder
Juma cannot send more messages
No call attempt allowed
Request expires after 30 days
Message Request Limits
Per account per day:
Max 10 message requests sent
Prevents spam
Per account to same person:
1 request only (ever)
If declined → cannot request again
Request folder:
Separate from main inbox
User checks it voluntarily
No notification for low-priority requesters
(unless they have mutual connections)
6. Call Permissions
Permission Tiers
Tier Voice call Video call
────────────────────────────────────────────────────────
Saved contacts ✅ ✅
Mutual followers ✅ ✅
Commerce relationship ✅ (shop) ✅ (shop)
Active conversation partner ✅ ✅
(message request accepted)
Group chat member ✅ (group) ✅ (group)
Strangers ❌ ❌
Blocked users ❌ ❌
Commerce Call Rules
Buyer can call:
The SHOP JID — not the owner personally
techstore@shops.nexgate.com
Any available staff answers as TechStore
Seller (shop) can call:
Buyers they have active commerce DM with
Call goes from shop JID to buyer
Buyer sees "TechStore" calling — not a personal name
Protecting personal contact:
Owner's personal phone/JID never revealed
Commerce calls always through shop identity
Staff cannot reveal personal number to customers
Anti-Harassment Call Rules
Call rate limiting:
Max 3 unanswered calls to same person in 24 hours
4th attempt → blocked for 24 hours automatically
Resets daily
Silence unknown callers:
Setting: "Pumzisha simu kutoka kwa wasio contacts"
Calls from non-contacts ring silently
Notification appears after — not during
First call warning:
First EVER call from this person:
┌───────────────────────────────────────┐
│ 📞 Simu ya kwanza kutoka │
│ Juma Mwangi (@juma_mwangi) │
│ │
│ [Jibu] [Kataa] [Zuia] │
└───────────────────────────────────────┘
Quick block during call:
Report button visible during call
One tap → ends call + reports + blocks
Privacy Settings for Calls
Who can call me:
○ Mutual followers + contacts (default)
○ My contacts only
○ Nobody
User can change anytime in Settings → Privacy → Calls
Call Permission Check Flow
Kibuti tries to call Juma:
│
Spring Boot permission check:
Is Juma blocked by Kibuti? ❌ → reject immediately
Is Kibuti blocked by Juma? ❌ → reject immediately
Are they saved contacts? ✅ → allow
Are they mutual followers? ✅ → allow
Active commerce relationship? ✅ → allow
Active accepted conversation? ✅ → allow
Does Juma's privacy setting allow? ✅ → allow
None of above → ❌ reject
│
If rejected:
App shows: "Huwezi kupiga simu mtu huyu"
No Jingle stanza sent
No TURN credentials issued
If allowed:
TURN credentials generated
Jingle session-initiate sent via Ejabberd
Juma's device rings
7. 1:1 Messaging Flows
Send Text Message
Happy path:
User types message
Taps send
Message shows: pending ⏳
Server receives → single tick ✓
Recipient receives → double tick ✓✓
Recipient reads → blue tick ✓✓
Offline path:
User taps send
Single tick ✓ (server received)
Recipient offline → queued
FCM push fires to recipient device
Recipient comes online → message delivered
Double tick ✓✓ appears
Send Voice Note
User holds mic button:
"Kibuti anarekodia..." shown to recipient (if online)
Recording in progress
Timer shown: 0:01, 0:02...
User releases:
Voice note sent
Waveform shown in thread
Duration shown: 0:15
User swipes up while holding:
Lock mode — records without holding
Release sends
User swipes left while holding:
Cancel recording — nothing sent
Limits:
Basic users: max 2 minutes
Pro users: max 15 minutes
File size: max 16MB
Send Media
Images:
Max 10 images per message
Formats: JPG, PNG, WebP, HEIC
Max 16MB per image
Auto-compressed for EA networks
Videos:
Max 1 video per message
Max 2 minutes (short video)
Max 64MB
Auto-transcoded to HLS by File Thunder
Files/Documents:
Max 100MB per file
Formats: PDF, DOCX, XLSX, ZIP, etc
ClamAV scanned before delivery
Available for 30 days then archived
8. Group Chat Flows
Create Group
User creates group:
Tap new group
Add members (min 2, max 500)
Set group name (required)
Set group photo (optional)
Tap create
Group created:
Members notified: "Kibuti amekuongeza kwenye kikundi"
All members see welcome message
Creator is automatically OWNER
Group Roles & Permissions
Role Add members Remove Send msg Delete msg Change info
──────────────────────────────────────────────────────────────────
OWNER ✅ ✅ ✅ Any msg ✅ ✅
ADMIN ✅ ✅ ✅ Any msg ✅ ✅
MEMBER ❌ ❌ ✅ Own only ✅ ❌
Group Size Limits
Free group: max 50 members
Pro group: max 500 members
At 500 members:
Consider using a Public Group instead
Public group + announcement mode = large audience
Group = two way community discussion
Leave / Remove
Member leaves:
"Juma ameacha kikundi" shown in thread
No longer receives messages
Admin removes member:
"Amina ameondolewa na Kibuti" shown
Member notified: "Umeondolewa"
Deleted group (owner only):
All members removed
All messages deleted (for everyone)
Cannot be undone
9. Commerce DM Flows
Flow 1 — Buyer Initiates from Product Page
Step 1: Buyer on product page
Sees: [Chat na Muuzaji] button
Tap button
Step 2: Conversation opens
New DM opens OR existing conversation if already had one
Conversation type: COMMERCE
Conversation owner: SHOP (not personal)
Product card auto-appears:
┌──────────────────────────────────────┐
│ 📦 Samsung A15 │
│ TZS 450,000 │
│ Inapatikana: Vipande 12 │
│ TechStore │
│ [Jibu] [Angalia Bidhaa] │
└──────────────────────────────────────┘
NOTE: Price in card is frozen at this moment
Public price changes later → does NOT affect this card
Step 3: Negotiation
Buyer and seller exchange text messages
Normal conversation — no restrictions on text
Step 4: Agreement reached
Seller taps attach (+) → From My Shop
Selects product
Sets custom price for THIS buyer ONLY
Public product price: unchanged ✅
Sends price offer card
Step 5: Buyer receives offer
┌──────────────────────────────────────┐
│ 💰 Bei Maalum Kwako │
│ Samsung A15 │
│ ~~TZS 450,000~~ │
│ TZS 400,000 (-TZS 50,000) │
│ Inaisha: dakika 28 │
│ Idadi: [─ 1 +] │
│ [Kataa] [Endelea Kulipa →] │
└──────────────────────────────────────┘
Step 6: Buyer taps "Endelea Kulipa"
Redirected to checkout flow (OUTSIDE inbox)
Checkout at custom offer price
Quantity confirmed in checkout
Step 7: Order placed
Confirmation message appears in same thread:
┌──────────────────────────────────────┐
│ ✅ Agizo Limethibitishwa │
│ Ord #ORD-XYZ-789 │
│ Samsung A15 × 1 │
│ TZS 400,000 imelipwa │
│ [Fuatilia Agizo] │
└──────────────────────────────────────┘
Flow 2 — Seller Attaches from Inside Any Chat
Step 1: Seller inside any DM conversation
Taps attach (+) button
Menu appears:
📷 Picha
🎵 Sauti
📄 Faili
🏪 Kutoka Dukani ← this one
📅 Tukio
👥 Ununuzi wa Pamoja
Step 2: Seller taps "Kutoka Dukani"
Their shop product list opens
Seller browses and selects product
Sets custom price (optional)
Sets quantity limit (optional)
Adds note (optional): "Bei hii ni leo tu"
Taps Send
Step 3: Buyer receives offer card
Same offer card UI as Flow 1
Same checkout redirect
Same order confirmation in thread
Commerce DM Rules
✅ Buyer can initiate from any product page
✅ Seller can attach from inside any conversation
✅ Custom price is private to this buyer only
✅ Public product price never changes
✅ Checkout always happens outside inbox
✅ Order confirmation appears in thread
✅ Multiple offers allowed in one conversation
❌ Buyer cannot request specific price (only negotiate via text)
❌ Offer cannot be edited after sending (send new one instead)
❌ Offer cannot be forwarded to other conversations
❌ Order confirmation cannot be deleted or edited
10. Offer Session Flows
Offer Lifecycle
PENDING → offer sent, waiting for buyer response
│
├──▶ DECLINED buyer tapped "Kataa"
│ seller notified
│ seller can send new offer
│
├──▶ EXPIRED timer ran out
│ both parties notified in thread
│ seller can send new offer
│
▼
ACCEPTED → buyer tapped "Endelea Kulipa"
│ buyer enters checkout
▼
CHECKOUT → buyer in payment flow
│
├──▶ ABANDONED buyer left checkout without paying
│ offer returns to PENDING? NO
│ offer marked ABANDONED
│ seller must send new offer
│
▼
COMPLETED → order placed successfully
│ cannot be reversed here (order system handles)
▼
CANCELLED → order cancelled (handled by order system)
offer marked CANCELLED
Multiple Offers — Same Conversation
Seller can send multiple offers:
Each is independent session
Previous offers remain in thread with their status
Thread shows history:
┌──────────────────────────────────────┐
│ 💰 Bei Maalum: TZS 430,000 │
│ ❌ Ilikataliwa │
└──────────────────────────────────────┘
┌──────────────────────────────────────┐
│ 💰 Bei Maalum: TZS 410,000 │
│ ⏰ Imeisha muda │
└──────────────────────────────────────┘
┌──────────────────────────────────────┐
│ 💰 Bei Maalum: TZS 400,000 │
│ ✅ Imekubaliwa │
└──────────────────────────────────────┘
Offer Expiry Scenarios
Scenario 1 — Expires while buyer is reading:
Offer card shows live countdown timer
Timer hits 0:00
Card updates: "Imeisha muda"
Proceed button disabled automatically
Buyer sees: [Omba Bei Mpya] button
Scenario 2 — Expires while buyer in checkout:
Buyer was on checkout page when offer expired
Checkout validates offer at payment time
If expired: payment rejected
Buyer returned to conversation
"Bei yako imeisha muda. Omba bei mpya"
Scenario 3 — Seller sends same price again:
Allowed — new offer session created
New 30-minute timer
Both offers visible in thread (old + new)
Scenario 4 — Buyer tries to use expired offer link:
Deep link from notification opens expired offer
Shows: "Bei hii imeisha muda"
[Rudi kwa Mazungumzo] button
Price Security Rules
✅ Offer price stored server-side only
✅ Checkout validates price from server (not client)
✅ Client cannot manipulate price
✅ Offer is single-use (cannot complete twice)
✅ Offer belongs to specific buyer (others cannot use)
❌ Seller cannot change price after offer sent
(must send new offer)
❌ Buyer cannot change price
❌ Public product price never affected
11. Shareable Content
What Can Be Shared Into Any DM or Group
Content Who can share Restrictions
────────────────────────────────────────────────────────────
Text message Anyone None
Voice note Anyone Max 2 min (basic)
Image/Video Anyone Size limits
Product card Anyone Shows public price
Custom price offer Seller only Private — NOT forwardable
Event card Anyone None
Bei ya pamoja card Anyone None
VP Feed post Anyone None
VP Live stream Anyone None
Audio Space Anyone None
File/Document Anyone Max 100MB
Forwarding Rules
Forward chain tracking:
chain 1: "↪ Imetumwa kutoka Juma Mwangi"
chain 2-4: "↪ Imetumwa"
chain 5+: "↪ Imetumwa mara nyingi"
(misinformation warning — different icon)
Multi-forward limit:
Max 5 conversations per forward action
Prevents spam broadcasting
Cannot forward:
Custom price offers (private deal)
Order confirmations (private record)
Payment confirmations (private record)
System messages
12. Message Interactions
Editing Messages
Rules:
✅ Only original sender can edit
✅ Text messages only (not media captions yet)
✅ Within 15 minutes of sending
✅ Shows "Imehaririwa" label after edit
✅ Original send time stays the same
✅ Works in group chats (sender edits their own)
❌ Commerce cards cannot be edited (immutable)
❌ System messages cannot be edited
❌ Voice notes cannot be edited
❌ After 15 minutes: edit option disappears
UI:
Long press message → [Hariri] option (within 15 min)
After edit: message updates in place
"Imehaririwa" label appears below message
Position in thread unchanged (no jump to bottom)
Deleting Messages
Delete for Me:
✅ Any message, any time, no limit
✅ Only removed from your view
✅ Recipient still sees it
✅ Works on any message type
Delete for Everyone:
✅ Only original sender
✅ Within 15 minutes of sending
✅ Removed from all screens
✅ Shows: "Ujumbe huu umefutwa"
✅ Works in group chats
❌ Commerce cards: NOT allowed
❌ System messages: NOT allowed
❌ After 15 minutes: option disappears
Nothing is ever permanently deleted:
Soft delete only
For legal compliance + dispute resolution
Admin can view deleted messages for support cases
Users cannot recover deleted messages
Reactions
Rules:
✅ Any message can be reacted to
✅ One reaction per user per message
✅ Change reaction: send new emoji (replaces old)
✅ Remove reaction: tap same emoji again
✅ Works on commerce cards (reactions don't modify content)
❌ System messages: no reactions
Available emojis at launch:
❤️ 👍 😂 😮 😢 🙏
Expand to full emoji keyboard: Phase 3
Group reaction display:
1 type: "👍 3"
2 types: "👍 3 ❤️ 2"
3+ types: "👍 3 ❤️ 2 +2 zaidi"
Tap reaction to see who reacted:
Bottom sheet opens
List of names per emoji
Replying to Messages
Rules:
✅ Anyone in conversation can reply to any message
✅ Reply shows quoted original above new message
✅ Tap quote → scrolls to original
✅ Works in group chats
✅ Works on any message type
❌ Cannot reply to deleted messages
(shows: "Ujumbe umefutwa")
UI:
Swipe right on message → reply mode
OR long press → [Jibu] option
Quote appears in input field
Send normally
Forwarding
Rules:
✅ Anyone can forward allowed content
✅ Max 5 conversations per forward action
✅ Forward chain tracked and shown
✅ Original sender name shown (chain 1 only)
✅ Media: references original file (no re-upload)
❌ Custom price offers: not forwardable
❌ Order/payment records: not forwardable
❌ System messages: not forwardable
UI:
Long press message → [Tuma] option
Conversation picker opens
Select up to 5 conversations
Tap send
13. Voice & Video Call Flows
Initiating a Call
Kibuti taps call button on Juma's profile or conversation:
│
Permission check:
Relationship exists? ✅
Juma's privacy allows? ✅
Kibuti not blocked? ✅
│
TURN credentials generated (server)
│
Juma's phone rings:
┌──────────────────────────────────────┐
│ │
│ 📞 Simu Inayoingia │
│ │
│ [Kibuti Mwangi] │
│ @kibuti │
│ │
│ [❌ Kataa] [✅ Jibu] │
│ │
└──────────────────────────────────────┘
Works even if:
App is closed (FCM HIGH wakes it)
Screen is locked (full screen notification)
App is in background
During a Call — Controls
Voice call controls:
🔇 Mute/unmute microphone
🔊 Speaker on/off
📷 Enable camera (upgrade to video)
❌ End call
Video call controls:
🔇 Mute/unmute microphone
📷 Camera on/off
🔄 Switch camera (front/rear)
🖥️ Share screen
🔊 Speaker on/off
❌ End call
Switching Audio ↔ Video
Voice → Video:
Kibuti taps camera button during voice call
Juma sees: "Kibuti anataka kuongeza video"
Auto-accepted (based on Juma's settings)
OR Juma taps Accept
Video starts — same call session continues
Audio uninterrupted during upgrade
Video → Audio (manual):
Kibuti taps camera OFF
Video stops immediately for both
Audio continues
No renegotiation needed
Video → Audio (automatic):
Network degrades below video threshold
Video disabled automatically
Banner shown: "Video imezimwa — mtandao dhaifu"
Audio continues
Video resumes when network improves
Screen Sharing
Start screen share:
Tap screen share icon during video call
System permission dialog appears (Android/iOS)
"Ruhusu NexGate kunasa skrini yako?"
User accepts
Screen share starts
Other party sees your screen
During screen share:
Your camera: small PiP (picture in picture)
Their view: your screen (large) + your face (small)
Your view: normal call view + "Unaonyesha skrini" banner
Stop screen share:
Tap stop button
Returns to normal video call
Call Quality Indicators
Signal bars shown during call:
████ Excellent (WiFi / 4G strong)
███░ Good (4G)
██░░ Fair (3G) — may show quality banner
█░░░ Poor (2G) — video disabled, audio only
░░░░ Very poor — "Mtandao dhaifu sana" banner
Quality banner examples:
"Ubora wa sauti umepungua kwa sababu ya mtandao"
"Video imezimwa — data ndogo"
"Unaunganika tena..."
Call End Scenarios
Normal end:
One party taps end
Other party sees call ended
Duration shown in conversation: "Simu ya dakika 4:32"
Declined:
Juma taps Kataa
Kibuti sees: "Simu ilikataliwa"
Missed call notification NOT sent (was declined)
No answer (timeout 45 seconds):
Kibuti sees: "Hakujibu"
Juma sees: "Simu iliyokosekana kutoka Kibuti" notification
Network failure:
Both lose connection
App attempts reconnect (10 seconds)
If reconnect fails:
Call marked as FAILED
"Simu ilikatizwa" shown to both
Kibuti can redial
14. Group Call Flows
Starting a Group Call
From a group chat:
Tap call icon in group header
Choose: Voice only OR Video
All group members receive incoming call notification
Members who join → enter call
Members who don't → miss it (missed call shown)
From a 1:1 conversation:
Not supported directly
Must create group first
OR use "Add person" button during active 1:1 call
Add person to active 1:1 call:
During call → tap "Ongeza Mtu"
Pick from allowed contacts
They receive group call invitation
They join → 1:1 becomes group call
Group Call Limits
Voice only group call:
Max 8 participants (comfortable for 3G)
Up to 12 possible but discouraged on EA networks
Video group call:
Max 4 video feeds shown simultaneously
5th person onwards: audio only tile shown
Active speaker highlighted (larger tile)
Tap any tile to pin/feature them
Layout options:
Grid view: all tiles equal size
Speaker view: active speaker large, others small
Auto: switches based on who is talking
Group Call Permission Rules
Within a group chat:
Any member can start group call ✅
Any member can join ✅
No extra permission check needed
(group membership = call permission)
Adding someone outside the group:
Same call permission check as 1:1
Must have relationship (contact/follower/commerce)
If no relationship → cannot add
15. Offline & Notification Flows
Message Notification Levels
Level When FCM SMS after
──────────────────────────────────────────────────────────────
CRITICAL Order placed/paid/failed ✅ 0 min
Delivery update
Payment confirmation
IMPORTANT Commerce DM from buyer ✅ 10 min
Custom price offer received
Bei ya pamoja threshold reached
Call missed (commerce context)
NORMAL Regular DM ✅ Never
Group message
Reaction
Follow notification
Offline Message Delivery
User offline — message sent to them:
│
├── FCM HIGH priority push fired immediately
│ (all message levels)
│
├── CRITICAL: Textfy SMS sent simultaneously
│
├── IMPORTANT: Textfy SMS after 10 min
│ (if FCM not acknowledged)
│
└── Message queued in RabbitMQ
User comes back online:
│
├── WS connects → presence registered
├── Queued messages drained (priority order)
│ CRITICAL first → IMPORTANT → NORMAL
│
└── Catch-up banner shown:
"Umekosa: maagizo 2, ujumbe 8"
[Angalia Maagizo] [Ona Ujumbe]
Textfy SMS Templates
Order placed (to seller):
"NexGate: Agizo jipya kutoka [Buyer]!
TZS [amount]. Kagua: nexgate.app/orders/[id]"
Payment received:
"NexGate: Malipo ya TZS [amount]
yamepokelewa kutoka [Buyer].
nexgate.app/wallet"
Commerce DM (to seller):
"NexGate: [Buyer] anakuuliza kuhusu
[Product]. Jibu: nexgate.app/chat/[id]"
Bei ya pamoja:
"NexGate: Watu [n]/[target] wamejiunga!
nexgate.app/group-buy/[id]"
All SMS:
Swahili first ✅
Deep link included ✅
Shop name shown (not staff name) ✅
16. Privacy & Safety
Privacy Settings Matrix
Setting Options Default
────────────────────────────────────────────────────────────────
Who can message me Everyone / Followers Followers
+ Contacts / Contacts + Contacts
/ Nobody
Who can call me Followers+Contacts / Contacts
Contacts / Nobody only
Last seen Everyone / Contacts / Contacts
Nobody
Profile picture Everyone / Contacts / Everyone
Nobody
Be found by phone number On / Off On
Contact sync On / Off Off
Read receipts (blue ticks) On / Off On
(off = others see delivered but not read)
Block System
User A blocks User B:
User B cannot:
❌ See User A's profile
❌ Send messages to User A
❌ Call User A
❌ See User A in search
❌ See User A's VP Feed posts
❌ See User A's online status
User B sees:
Profile: "Mtumiaji huyu hayupo"
Messages: appear sent but never delivered
User A can:
✅ Unblock anytime from Settings
✅ See User B's profile still (A blocked B — not reverse)
Group chats:
If in same group: messages visible but
cannot DM or call each other directly
Admin can remove either from group
Report System
Report options:
Spam
Harassment / Vitisho
Inappropriate content
Fake account
Scam / Udanganyifu
Other
Report flow:
Long press message → [Ripoti]
OR profile → [...] → [Ripoti]
Choose reason
Optional: add description
Submit
After report:
User not notified they were reported
Report goes to NexGate moderation queue
Automatic temporary restrictions may apply
for high-volume reporters
Block on report:
"Ripoti na Zuia" option available
Blocks immediately + sends report
Safety for Women — Specific Features
This is important for EA platform trust:
Silence unknown callers:
ON by default for new accounts
Calls from non-contacts ring silently
User sees missed call — no disruption
Call rate limiting:
3 unanswered calls per day to same person
Automatic block after 4th attempt
Quick block during call:
One tap visible during any call
Ends + reports + blocks in single action
Message request system:
Strangers cannot freely DM
Must send request first
User controls who enters their inbox
Default privacy settings:
New accounts: strict defaults
Users open up if they choose
Better to protect by default
17. Shop Inbox & Staff Access
The Golden Rule — No Staff Name Ever
This is the most important rule in this section:
Customer NEVER sees staff name
In ANY communication channel:
❌ Not in DM replies
❌ Not in commerce messages
❌ Not in offer cards
❌ Not in order updates
❌ Not in system messages
❌ Not in call screen ("TechStore calling" not "Amina calling")
❌ Not in read receipts
❌ Not in typing indicators ("TechStore anaandika..." not "Amina...")
❌ Not in voice/video calls
❌ Not in notifications
Customer always sees:
✅ Shop name only: "TechStore"
✅ Shop avatar only
✅ Shop JID only
Why this rule exists:
Customer relationship is with THE SHOP
Not with individual staff members
Staff privacy protected
Staff turnover invisible to customer
Brand consistency always maintained
Cannot be broken by any staff action
System enforces this — not just a guideline
How Others Handle Staff Access
Facebook Pages:
Staff have OWN Facebook accounts
Owner assigns role via Page Settings
Staff switches to page context
Posts/replies as page — not personally
✅ No password sharing
WhatsApp Business API:
Connected to CRM (Zendesk, Freshdesk)
Each agent logs into CRM with own account
CRM sends via WhatsApp API as business
Customer sees business name only
✅ No password sharing
Shopify:
Owner invites staff via email
Staff creates own Shopify login
Access scoped to their role
Customer always sees store name
✅ No password sharing
NexGate follows same pattern:
Staff have own NexGate accounts
Owner invites staff to shop
Staff accesses shop context
Customer sees shop name only
✅ No password sharing ever
Shop Tiers
Basic shop (free):
Owner manages inbox alone
No staff assignment
Standard inbox features
Shop name shown to customers ✅
Pro shop (paid):
Staff roles unlocked
Multiple staff share shop inbox
Advanced analytics
Full audit logs
Priority support
Staff management dashboard
Shop name shown to customers ✅ (same rule)
Staff Roles & Permissions
Role Inbox Send msg Products Analytics Settings Staff mgmt
──────────────────────────────────────────────────────────────────────────────
OWNER ✅ ✅ ✅ ✅ ✅ ✅
MANAGER ✅ ✅ ✅ ✅ ❌ ❌
SUPPORT_AGENT ✅ ✅ ❌ ❌ ❌ ❌
READ_ONLY 👁️ only ❌ ❌ ❌ ❌ ❌
Notes:
OWNER: full control — only they can invite/remove staff
only they can change shop tier
only they can delete shop
MANAGER: day-to-day shop management
can send price offers
can view product catalog
cannot change settings or manage staff
SUPPORT_AGENT: inbox only
can reply to customers as shop
can send price offers
cannot see/edit products
READ_ONLY: view conversations only
cannot reply
useful for supervisors/auditors
Staff Invitation Flow
Step 1 — Owner sends invitation:
Shop Settings → Staff → Alika Mfanyakazi
┌──────────────────────────────────────────┐
│ ➕ Alika Mfanyakazi │
│ │
│ Barua pepe au nambari ya simu: │
│ [amina@gmail.com ] │
│ │
│ Jukumu: │
│ ○ Msimamizi (Manager) │
│ ● Wakala wa Msaada (Support Agent) │
│ ○ Soma Tu (Read Only) │
│ │
│ [Tuma Mwaliko] │
└──────────────────────────────────────────┘
System:
Generate secure invitation token
Token expires: 48 hours
Send to Amina via SMS + email:
"Kibuti amekualika kuwa mfanyakazi
wa TechStore kwenye NexGate.
Bonyeza hapa: nexgate.app/invite/TOKEN
Mwaliko unaisha baada ya masaa 48."
Step 2 — Staff receives invitation:
Case A — Amina already has NexGate account:
Taps invitation link
Logs in with OWN credentials
Sees invitation screen:
┌──────────────────────────────────────────┐
│ 🏪 Mwaliko wa TechStore │
│ │
│ Kibuti Mwangi anakualika kujiunga na │
│ TechStore kama: │
│ Wakala wa Msaada │
│ │
│ Utaweza: │
│ ✓ Kujibu ujumbe wa wateja │
│ ✓ Kutuma ofa za bei │
│ ✓ Kuona mazungumzo yote ya duka │
│ │
│ Hutaweza: │
│ ✗ Kuona mazungumzo ya kibinafsi ya owner│
│ ✗ Kubadilisha mipangilio ya duka │
│ │
│ [Kubali] [Kataa] │
└──────────────────────────────────────────┘
Taps Kubali → linked to TechStore immediately
Case B — Amina has no NexGate account:
Taps invitation link
Registration page opens
Registers with own phone number + PIN
Invitation auto-accepted after registration
Linked to TechStore as Support Agent
Case C — Token expired (>48 hours):
"Mwaliko huu umeisha muda"
Owner must send new invitation
Step 3 — Staff accesses shop:
Amina logs into NexGate normally:
Her own phone number
Her own PIN
Her own account entirely
Her inbox shows tabs:
[Personal ●3] [TechStore ●12]
Personal tab:
Her own DMs and groups
Completely private
TechStore cannot see this
TechStore tab:
All TechStore customer conversations
Shared with all TechStore staff
She replies as "TechStore" ✅
Step 4 — Staff replies to customer:
Amina opens TechStore conversation
Context indicator clearly shown:
┌──────────────────────────────────────────┐
│ 🏪 Unajibu kama: TechStore │
│ (Wakala wa Msaada) │
│──────────────────────────────────────────│
│ Customer: "Je, Samsung A15 ipo?" │
│ │
│ [Andika ujumbe kama TechStore...] │
│ │
│ [📷] [🎤] [📎] [➤ Tuma] │
└──────────────────────────────────────────┘
Customer receives:
TechStore: "Ndio, Samsung A15 ipo!"
No "Amina" anywhere ✅
No mention of staff ✅
Just TechStore brand ✅
What Customer Sees vs What Owner Sees
Customer view of conversation:
10:30 [Customer]: "Je, A15 ipo?"
10:32 [TechStore]: "Ndio, ipo! Bei TZS 450,000"
10:35 [Customer]: "Naweza kupata punguzo?"
10:36 [TechStore]: "Nitaona ninachoweza kufanya"
10:40 [TechStore]: 💰 Bei Maalum: TZS 400,000
[Offer card]
Customer never knows:
Who replied at 10:32
Who replied at 10:36
Whether same person replied
How many staff exist
Whether it's the owner or staff
Owner/Manager audit view:
10:30 Customer: "Je, A15 ipo?"
10:32 Amina (Support Agent): "Ndio, ipo! Bei TZS 450,000"
10:35 Customer: "Naweza kupata punguzo?"
10:36 Amina (Support Agent): "Nitaona ninachoweza kufanya"
10:40 John (Manager): Bei Maalum TZS 400,000 imetumwa
Staff Identity Rules — Complete List
Staff name NEVER appears in:
❌ DM message content ("TechStore" always)
❌ Offer cards (shop name only)
❌ Order confirmation messages
❌ Typing indicator ("TechStore anaandika...")
❌ Read receipts (customer sees shop read it)
❌ Call screen ("TechStore" calling / answering)
❌ Missed call notification ("Simu kutoka TechStore")
❌ Voice note sender name
❌ Reaction attribution (customer sees shop reacted)
❌ System messages
Staff identity ONLY appears in:
✅ Internal audit log (owner + manager view)
✅ Staff management dashboard
✅ Internal analytics (who handled most conversations)
✅ Staff's own inbox context indicator
"Unajibu kama: TechStore (Wakala wa Msaada)"
This rule is enforced by the system:
Staff JID: techstore@shops.nexgate.com/amina
Customer sees: techstore@shops.nexgate.com
Resource (/amina) stripped before sending to customer
Cannot be overridden by any staff action
Typing Indicator — Shop Name Not Staff
Amina is typing reply:
Customer sees: "TechStore anaandika..."
NOT: "Amina anaandika..."
NOT: "Mfanyakazi anaandika..."
Two staff typing simultaneously:
Customer sees: "TechStore anaandika..."
(same — no way to know it's two people)
This is correct behavior
Ejabberd handles this via shop JID:
Composing stanza from: techstore@shops.nexgate.com
Resource stripped before routing to customer
Voice/Video Call — Staff Anonymity
Customer calls TechStore:
Whoever answers (any available staff)
Call screen shows to customer:
"TechStore" (shop name)
Shop avatar
NOT staff name
Staff receives call:
Their screen shows:
Customer name (from conversation)
"Simu kwa TechStore" indicator
They answer as TechStore
Staff initiates call to customer:
Customer's incoming call screen shows:
"TechStore" calling
NOT "Amina" calling
NOT a personal number
Call log in customer's conversation:
"Simu na TechStore — dakika 4:32"
NOT "Simu na Amina"
Owner Managing Staff
Staff management dashboard (owner only):
┌──────────────────────────────────────────────────────┐
│ 👥 Wafanyakazi wa TechStore │
│ │
│ Amina Hassan │
│ Support Agent · Imeunganishwa: Jan 15, 2026 │
│ Imetumika mara ya mwisho: Leo 14:30 │
│ Mazungumzo 47 wiki hii │
│ [Badilisha Jukumu] [Simamisha] [Ondoa] │
│ │
│ John Doe │
│ Manager · Imeunganishwa: Feb 1, 2026 │
│ Imetumika mara ya mwisho: Jana 09:15 │
│ Mazungumzo 23 wiki hii │
│ [Badilisha Jukumu] [Simamisha] [Ondoa] │
│ │
│ [+ Alika Mfanyakazi Mpya] │
└──────────────────────────────────────────────────────┘
Owner actions:
✅ Invite new staff
✅ Change role (Manager ↔ Support Agent ↔ Read Only)
✅ Suspend temporarily (keeps access frozen, not removed)
✅ Remove permanently (instant access revocation)
✅ View activity stats per staff
✅ View full audit log
Remove staff — what happens:
Access revoked IMMEDIATELY
TechStore tab disappears from their inbox
All pending conversations stay in shop inbox
(conversations not lost — just staff can't see them)
No notification sent to customers
Historical messages remain attributed in audit log
Multi-Shop Staff
Amina works for two shops:
Shop A: TechStore (Support Agent)
Shop B: ClothingHub (Manager)
— Different owner (different business)
Her inbox shows:
[Personal] [TechStore] [ClothingHub]
Rules:
TechStore conversations: visible ✅
ClothingHub conversations: visible ✅
Each shop completely isolated from other ✅
TechStore cannot see ClothingHub data ✅
ClothingHub cannot see TechStore data ✅
Her personal inbox: only she sees ✅
Security Rules
✅ Each staff has own NexGate account
✅ Owner never shares password
✅ Invitation token secure + expires 48 hours
✅ Role-based access strictly enforced
✅ Customer always sees shop name (system enforced)
✅ Audit log tracks every action
✅ Owner can revoke access instantly
✅ Staff cannot access unassigned shops
✅ Staff cannot access owner personal inbox
✅ Staff cannot change their own role
✅ Staff cannot invite other staff (Manager+ only)
✅ Removed staff cannot export conversation history
❌ No shared passwords ever
❌ No staff name to customers ever
❌ No personal contact revealed to customers
❌ No cross-shop data access
❌ No personal inbox access by staff
Database Schema
shop_staff_invitations
─────────────────────────────────────────────
invitation_id UUID
shop_id UUID
invited_by UUID owner userId
invitee_email TEXT
invitee_phone TEXT
role ENUM MANAGER / SUPPORT_AGENT / READ_ONLY
token_hash TEXT hashed secure token
status ENUM PENDING / ACCEPTED / DECLINED / EXPIRED
expires_at TIMESTAMPTZ 48 hours from creation
created_at TIMESTAMPTZ
responded_at TIMESTAMPTZ
shop_staff_members
─────────────────────────────────────────────
id UUID
shop_id UUID
user_id UUID staff NexGate userId
role ENUM OWNER / MANAGER / SUPPORT_AGENT / READ_ONLY
invited_by UUID
joined_at TIMESTAMPTZ
last_active_at TIMESTAMPTZ
status ENUM ACTIVE / SUSPENDED / REMOVED
suspended_at TIMESTAMPTZ
removed_at TIMESTAMPTZ
removed_by UUID
shop_staff_audit_log
─────────────────────────────────────────────
log_id UUID
shop_id UUID
staff_user_id UUID
action ENUM MESSAGE_SENT / OFFER_SENT /
OFFER_CREATED / CALL_ANSWERED /
PRODUCT_VIEWED / MEMBER_ADDED /
MEMBER_REMOVED / ROLE_CHANGED
conversation_id UUID
message_id UUID
timestamp TIMESTAMPTZ
metadata JSONB action-specific details
18. Notification Settings
Per-Conversation Settings
Each conversation has:
Notifications:
○ All messages
○ Mentions only (groups)
○ Muted (until: 8 hours / 1 week / forever)
Media auto-download:
○ WiFi only
○ WiFi + Mobile data
○ Never
Global Notification Settings
Message notifications: On / Off
Call notifications: On / Off
Reaction notifications: On / Off
Group notifications: On / Off
Commerce notifications: On / Off (cannot turn off CRITICAL)
Sound: Default / Custom / Silent
Vibration: On / Off
In-app preview: Show / Hide content
CRITICAL Notifications — Always On
These cannot be turned off by user:
Order placed (seller receives)
Payment confirmed (buyer receives)
Payment failed (buyer receives)
Order cancelled (both receive)
Reason:
Financial events
User could miss critical money information
Platform liability without guaranteed delivery
Textfy SMS fallback ensures they always arrive
19. Edge Cases & Scenarios
Messaging Edge Cases
User deletes their account:
Their messages remain visible in conversations
Name shows: "Mtumiaji aliyefuta akaunti"
Profile picture: default avatar
Cannot be messaged or called
User changes username:
All conversations update automatically
Old username links still work (redirect)
No broken references
Very long message (>4000 chars):
Truncated in thread: first 200 chars + "...Soma zaidi"
Tap to expand full message
Not split into multiple messages
Same message sent twice (duplicate):
Detected by temp_id
Only one stored in DB
User sees single message (not duplicate)
Message sent to blocked user:
Appears sent (single tick)
Never delivered (stays at single tick)
User not informed of block
Commerce Edge Cases
Product goes out of stock while offer pending:
Offer still valid (was frozen at send time)
Checkout validates stock at payment time
If out of stock at payment:
Payment rejected
"Bidhaa hii imeisha"
Buyer returned to conversation
Seller notified to resend offer or cancel
Product price changes while offer pending:
Offer price locked at creation ✅
Public price change does NOT affect offer
Buyer pays the offer price always
Seller deletes product while offer pending:
Offer still valid (snapshot frozen)
Checkout validates product existence
If product deleted at payment:
Payment rejected
"Bidhaa hii haipatikani tena"
Two buyers get offers for last item:
Both offers exist simultaneously
First to complete checkout gets the item
Second buyer's payment rejected:
"Bidhaa hii imeisha"
Standard e-commerce race condition handling
Seller sends offer to wrong person:
Cannot retract offer
Offer expires naturally
Seller can contact NexGate support if urgent
Buyer in checkout when offer expires:
Payment attempt fails
"Bei yako imeisha muda"
Must request new offer from seller
Call Edge Cases
Call drops mid-conversation:
WebRTC detects loss
Auto-reconnect attempted (10 seconds)
If reconnect success: call continues
If reconnect fails: call ended
Both see: "Simu ilikatizwa"
Kibuti can redial immediately
Both call each other simultaneously:
Race condition — both see "Simu inayoingia"
System picks one (first to reach server)
Other cancelled automatically
One call established
Call during active call:
Second call: goes to missed calls
Busy signal not sent (no UX for this)
After first call: notification of missed call
Phone runs out of battery during call:
WebRTC detects disconnect
Same as network drop
Other party sees: "Simu ilikatizwa"
Someone calls during Do Not Disturb (DND):
DND on device: depends on device DND rules
NexGate mute: call goes to missed calls silently
CRITICAL commerce calls: bypass mute (configurable)
Group Chat Edge Cases
Owner leaves group:
Must transfer ownership first
OR system auto-assigns to oldest admin
OR if no admins: oldest member becomes owner
Last person leaves group:
Group archived automatically
Messages preserved for 30 days
Then permanently deleted
Adding member who blocked you:
Cannot add blocked users to groups
System rejects silently
No error shown (privacy)
Member blocked in group:
They stay in group (cannot remove via block)
Cannot DM each other
Can both still see group messages
Admin can remove either from group
20. Follow System — Users & Shops
Two Entity Types — Two Sigils
NexGate has two distinct entity types:
User: @kibuti (a person)
Shop: $techstore (a business)
The sigil makes it immediately clear
what type of entity you are interacting with
@ = social relationship
$ = commercial relationship
Follow Button — Users
On @kibuti profile page:
[+ Follow]
What following a user gives you:
Their VP Feed posts appear in your feed
Their VP Live streams appear in your feed
Their events appear in your feed
You can message them (mutual relationship)
They can message you back
Mutual follow:
Both @kibuti and @juma follow each other
= stronger relationship
= can call each other
= message requests not needed
One-way follow:
@kibuti follows @juma
@juma does NOT follow @kibuti
Kibuti sees Juma's content
Juma does not see Kibuti's content
Kibuti can message Juma (follower relationship)
Juma sees message request (not mutual)
Follow Button — Shops
On $techstore profile page:
[+ Follow Shop]
NOT "Subscribe" because:
Subscribe implies payment (Netflix, Spotify)
EA users might think it costs money
Follow is free and familiar ✅
"Follow Shop" label sets clear expectation
NOT just "Follow" because:
"Follow Shop" makes clear it's a business
Sets expectation: commercial content
Not personal/social content
What following a shop gives you:
Shop products appear in your VP Feed
Shop promotions appear in feed
Shop VP Live streams appear in feed
Shop events appear in feed
FCM notification: "TechStore posted new products"
Commerce DM permission (can initiate chat)
Can be invited to shop customer groups
Can receive Bei ya pamoja from shop
Shop Following Rules
Shops do NOT follow anyone:
No "Following" count on shop profile
Shops are followed — they don't follow
Like a Facebook Page
Like a YouTube channel
Shop profile shows:
Followers: 3,420 ← people following the shop
No "Following" count
User profile shows:
Followers: 1,240
Following: 856
Both counts shown (social graph)
Unfollow Behavior
User unfollows @kibuti:
Kibuti NOT notified ✅ (standard)
Kibuti's content leaves your feed ✅
Can still message Kibuti ✅
(if previously connected)
Relationship weakened but not broken
User unfollows $techstore:
Shop NOT notified ✅
Shop content leaves your feed ✅
Can still initiate commerce DM ✅
(unfollow ≠ block)
Cannot be added to shop groups ✅
(lost follower relationship)
Existing order conversations remain ✅
Follow Count Display
@kibuti profile:
┌────────────────────────────────┐
│ @kibuti │
│ Kibuti Mwangi │
│ │
│ 1,240 856 │
│ Followers Following │
│ │
│ [+ Follow] [Message] │
└────────────────────────────────┘
$techstore profile:
┌────────────────────────────────┐
│ $techstore │
│ TechStore │
│ Electronics · Dar es Salaam │
│ │
│ 3,420 │
│ Followers │
│ │
│ [+ Follow Shop] [Message] │
└────────────────────────────────┘
21. Group Join Model
Two Group Types
PRIVATE GROUP:
Closed — controlled membership
Not discoverable in search
Default when creating a group
Like WhatsApp groups
PUBLIC GROUP:
Open — anyone can join
Discoverable in NexGate search
Explicit choice by creator
Like Telegram public groups
How People Join Groups — The Philosophy
NexGate principle:
Nobody ends up in a group
without choosing to be there
No WhatsApp-style direct add
(added before you know it)
Two mechanisms instead:
1. Consent DM invitation (proactive)
2. Invite link (self-service)
Both require the person to
actively choose to join
Mechanism 1 — Consent DM Invitation
Admin selects people from:
Their contacts ✅
Their followers ✅
Their commerce relationships ✅
NOT random strangers ❌
Each selected person receives a DM:
┌──────────────────────────────────────────┐
│ 📨 Group Invitation │
│ │
│ Kibuti Mwangi invited you to join: │
│ │
│ 🏘️ Business Friends │
│ 47 members · Private Group │
│ "Discussion for Dar founders" │
│ │
│ [Accept & Join] [Decline] │
└──────────────────────────────────────────┘
If Accept: member immediately ✅
If Decline: not added ✅
admin NOT notified (privacy)
If Ignored: auto-declined after 48 hours ✅
Why this model:
✅ Familiar to EA users (like WhatsApp add)
✅ But with consent (unlike WhatsApp)
✅ Commerce groups work (seller invites customers)
✅ Event groups work (organizer invites attendees)
✅ Family/friends groups easy to start
✅ User always in control
Mechanism 2 — Invite Link
Admin generates invite link:
nexgate.app/join/abc-xyz-def
Person taps link → sees group preview:
PRIVATE GROUP link:
┌─────────────────────────────────────────┐
│ 🔒 Private Group │
│ Business Friends │
│ 47 members │
│ Created by Kibuti Mwangi │
│ │
│ This group requires admin approval │
│ │
│ [Request to Join] │
└─────────────────────────────────────────┘
Admin sees request:
Name, username, mutual connections
[Approve] [Decline]
If approved → member ✅
If declined → person not notified (privacy)
PUBLIC GROUP link:
┌─────────────────────────────────────────┐
│ 🌍 Public Group │
│ Dar Tech Community │
│ 1,247 members │
│ Created by @kibuti │
│ "Discussion for Dar tech founders" │
│ │
│ [Join Group] │
└─────────────────────────────────────────┘
Tap [Join Group] → member immediately ✅
No approval needed
Invite Link Settings
Admin controls (Group Settings → Invite Link):
┌──────────────────────────────────────────┐
│ Invite Link │
│ nexgate.app/join/abc-xyz │
│ [Copy] [Share] [Revoke] │
│ │
│ Expiry: │
│ ● Never │
│ ○ 24 hours │
│ ○ 7 days │
│ ○ 30 days │
│ │
│ Max joins: [Unlimited ▾] │
│ Options: 10 / 25 / 50 / 100 / Unlimited │
└──────────────────────────────────────────┘
Revoke link:
Old link immediately dead
"This invite link is no longer valid"
New link auto-generated
Invitation Limits (Anti-Spam)
Per group per day:
Admin can send max 50 consent DM invitations
Prevents mass-invite spam
Per user per day:
User can receive max 10 group invitations
11th → goes to group requests folder
User reviews when ready
Rate limiting:
If 80%+ of your invitations declined
System flags account
Temporary invite restriction applied
Group Admin System
OWNER (1 per group):
Created the group OR ownership transferred
Cannot be removed by anyone
Full control over everything
Can delete the group
Can transfer ownership
ADMIN (multiple):
Appointed by OWNER
Can add/remove members
Can remove any message
Can pin messages
Can change group info
Cannot remove OWNER
Cannot remove other ADMINS
MEMBER (everyone else):
Can send messages
Can react, reply, forward
Can delete own messages only
Cannot manage others
Group Admin Permissions Matrix
Action Owner Admin Member
────────────────────────────────────────────────────
Send messages ✅ ✅ ✅
Delete own messages ✅ ✅ ✅
Delete any message ✅ ✅ ❌
Send consent DM invite ✅ ✅ ❌
Remove members ✅ ✅ ❌
Make someone admin ✅ ❌ ❌
Remove admin ✅ ❌ ❌
Change group name/photo ✅ ✅ ❌
Generate invite link ✅ ✅ ❌
Revoke invite link ✅ ✅ ❌
Pin messages ✅ ✅ ❌
Announcement mode ✅ ❌ ❌
Transfer ownership ✅ ❌ ❌
Delete group ✅ ❌ ❌
Leave group ✅* ✅ ✅
*must transfer ownership first
Group Settings Panel
Admin opens Group Settings:
┌──────────────────────────────────────────┐
│ ⚙️ Group Settings │
│ │
│ Group Name │
│ [Business Friends ] │
│ │
│ Description │
│ [Discussion for Dar founders ] │
│ │
│ Group Type │
│ ● 🔒 Private (invite + approval) │
│ ○ 🌍 Public (anyone can join) │
│ │
│ Who can send messages? │
│ ● All members │
│ ○ Admins only (announcement mode) │
│ │
│ Who can send invitations? │
│ ● Admins only │
│ ○ All members │
│ │
│ Invite Link ──────────────────────── │
│ nexgate.app/join/abc-xyz │
│ [Copy] [Share] [Revoke] │
│ Expiry: [Never ▾] │
│ Max joins: [Unlimited ▾] │
│ │
│ ─────────────────────────────────────── │
│ 🗑️ Delete Group │
└──────────────────────────────────────────┘
NexGate Natural Group Contexts
These groups form naturally without strangers:
VP Events:
Organizer creates event group
Ticket buyers receive consent DM automatically
"Dar Tech Summit invited you to Attendees group"
All attendees = commerce relationship ✅
VP Shop customer group:
Seller creates customer community group
Past buyers receive consent DM invitation
"TechStore invited you to VIP Customers group"
All = previous buyers ✅
VP Feed creator community:
Creator makes PUBLIC group
Shares link on VP Feed
Followers join themselves
All = followers (relationship exists) ✅
Friends/family:
Admin sends consent DM to close contacts
Small group bootstrapped easily
All = contacts/followers ✅
22. Shop Chat Initiation
The Core Principle
Customer relationship = permission to initiate
No relationship = cannot initiate
This applies to shops messaging customers
Same rule as personal messaging
Consistent across all of NexGate
When Shop Can Initiate
ALWAYS ALLOWED (transactional):
Order placed → shop sends confirmation
Order shipped → shop sends update
Order issue → shop contacts buyer
Payment problem → shop contacts buyer
→ Goes directly to customer commerce inbox
→ Customer expects this ✅
→ Cannot be turned off
ALLOWED with relationship:
Customer bought from shop before ✅
Customer sent message to shop before ✅
Customer follows the shop ($techstore) ✅
→ Shop can initiate from order/customer list
→ Goes to existing commerce thread
→ OR new message request if no thread yet
NEVER ALLOWED:
Cold message to random NexGate users ❌
Mass promotional outreach ❌
Message to users who never interacted ❌
→ System blocks this
→ "You can only message customers
who have interacted with your shop"
How Shop Staff Initiates
From existing order:
Shop → Orders → Find Kibuti's order
Tap [Message Customer]
→ Opens existing commerce thread
→ Send message as TechStore ✅
From customer list:
Shop → Customers → Find Kibuti
Tap [Send Message]
→ Opens existing thread OR
→ New message request to Kibuti
→ Kibuti sees: "TechStore wants to send you a message"
From active conversation:
Staff sees conversation in shop inbox
Replies as TechStore
Normal response flow ✅
From VP Feed (not messaging — different):
Shop posts on VP Feed
Followers see it in feed
Interested followers DM the shop
Shop replies ✅
Transactional Messages — Auto System
These fire automatically from Spring Boot
No manual staff action needed:
Order placed:
→ System message in commerce thread:
"Your order ORD-789 has been confirmed
Samsung A15 × 1 — TZS 400,000
[Track Order]"
Order shipped:
→ "Your order has been shipped
Expected: 2-3 days
[Track Delivery]"
Order delivered:
→ "Your order has been delivered!
How was your experience?
[Leave Review]"
Payment failed:
→ "Payment issue with your order
Please update your payment method
[Fix Payment]"
All go directly to commerce inbox ✅
Customer always expects these ✅
Cannot be disabled by customer ✅
Customer Controls
Settings → Privacy → Shop Messages:
Who can send me shop messages?
● Shops I have bought from (default)
○ Shops I follow + bought from
○ Nobody (all go to requests)
Allow shops to send me promotions?
○ Yes
● No (default)
Allow order updates from shops?
● Always (cannot turn off)
(transactional — always needed)
23. Broadcast Channels — Why Not Needed
The Question
Should NexGate have broadcast channels?
Like Telegram channels or WhatsApp channels?
One-way: creator/shop → followers
What VP Feed Already Covers
VP Feed does everything a broadcast channel does:
Public announcements:
Shop posts new product → VP Feed ✅
Creator posts update → VP Feed ✅
All followers see it ✅
Urgent alerts:
FCM HIGH priority notification on post ✅
"TechStore posted: Flash sale today!"
Same urgency as channel message ✅
Exclusive content:
VP Feed close friends feature ✅
Post visible to selected followers only
Covers "exclusive subscriber" use case ✅
Shop promotions:
Shop posts on VP Feed ✅
Product tags + sale tags ✅
Followers see in feed ✅
The question becomes:
What does a broadcast channel add
that VP Feed with notifications doesn't?
Honestly — very little
Why NOT to Build Broadcast Channels
Duplication:
Two places creator manages content
Two places follower checks for updates
"Should I post this to feed or channel?"
Confusing for both creators and followers
WeChat lesson:
WeChat has both feed (Moments)
AND official accounts (channels)
Users find it confusing what goes where
Even WeChat admits this overlap
NexGate should be cleaner
Simpler is better:
One place for content → VP Feed
One place for conversation → Messaging
No hybrid in between
Clean product with clear purpose
Inbox stays clean:
Personal DMs
Group chats
Shop commerce DMs
That's it
No broadcast section cluttering inbox
What Covers Each Use Case Instead
Use case Solution (no broadcast needed)
──────────────────────────────────────────────────────────
Public announcements VP Feed post ✅
Urgent alerts VP Feed + FCM notification ✅
Flash sales VP Feed with sale tag ✅
Exclusive content VP Feed close friends ✅
Private community Group chat ✅
Personal deals 1:1 Commerce DM ✅
Group deals Group + Bei ya pamoja ✅
Customer updates Transactional DMs (auto) ✅
Order updates Commerce thread (auto) ✅
Decision
Broadcast channels: NOT BUILT ✅
VP Feed is NexGate's content distribution layer
Messaging is NexGate's conversation layer
They serve different purposes
They stay separate
No overlap needed
24. Commerce Attach — 1:1 vs Group
The Clean Separation
Custom price offer:
1:1 DMs ONLY
Private negotiation between one seller and one buyer
Makes no sense in a group
(why negotiate privately in front of everyone?)
Bei ya pamoja (group purchase):
Group chats (primary home)
Also shareable in 1:1 DMs
Group power buying
Dynamic pricing
Perfect for group context
Attach Menu — 1:1 DM
Inside a 1:1 conversation:
Tap attach (+)
📷 Image
🎵 Voice Note
📄 File
🏪 From My Shop → custom price offer
(private deal — this buyer only)
📅 Event Card → share any event
👥 Bei ya pamoja → share a group buy
(recipient can join or share further)
Attach Menu — Group Chat
Inside a group conversation:
Tap attach (+)
📷 Image
🎵 Voice Note
📄 File
🏪 From My Shop → product card only
(public price shown)
(NO custom price in group)
📅 Event Card → share any event
👥 Bei ya pamoja → start or share group buy
THIS is group commerce ✅
Why Custom Price NOT in Groups
Custom price = private negotiation
"I'll give you a special deal"
Said to ONE person in private
Makes sense in 1:1 ✅
Custom price in group = awkward
"I'll give everyone TZS 400,000"
In front of 50 group members
Why is this price special?
How is it different from a sale?
It's NOT private anymore
Loses its meaning ❌
Bei ya pamoja = group power
"If we get 10 people together we save"
Collective action in a group
Group chat is the PERFECT home for this ✅
Dynamic pricing makes sense collectively ✅
Bei ya Pamoja in Group — Flow
Someone shares Bei ya pamoja in group:
┌──────────────────────────────────────────┐
│ 👥 Group Purchase │
│ Samsung A15 │
│ │
│ Public price: TZS 450,000 │
│ Group price: TZS 350,000 (10 people) │
│ │
│ Progress: ████████░░ 8 / 10 │
│ 2 more people needed │
│ │
│ Expires: 23 hours 45 minutes │
│ │
│ [Join Group Buy] │
└──────────────────────────────────────────┘
Group members:
See card in group thread ✅
Anyone can tap [Join Group Buy] ✅
Progress updates in real time ✅
When 10 people join → all checkout ✅
Card updates: "Target reached! Proceeding..." ✅
Summary Table
Feature 1:1 DM Group Chat
──────────────────────────────────────────────────
Text message ✅ ✅
Voice note ✅ ✅
Media (image/video) ✅ ✅
File/document ✅ ✅
Product card ✅ (public) ✅ (public)
Custom price offer ✅ seller only ❌ not available
Bei ya pamoja ✅ shareable ✅ primary home
Event card ✅ ✅
Post card ✅ ✅
Stream card ✅ ✅
Summary Updates
NexGate's messaging platform has been refined with five additional decisions:
Follow System: Users are followed with [@Follow], shops are followed with [Follow Shop]. The $ sigil already signals commercial relationship. Shops have followers but do not follow anyone — like a Facebook Page. "Subscribe" is avoided because it implies payment to EA users.
Group Join Model: Nobody enters a group without choosing to. Two mechanisms: consent DM invitation (admin proactively invites their network — each person accepts or declines) and invite link (self-service joining). Private groups require admin approval on link join. Public groups allow instant join. No WhatsApp-style forced adding.
Shop Chat Initiation: Shops can only initiate chat with customers who have an existing relationship (previous buyer, follower, or active conversation). Transactional messages (order updates) fire automatically. Cold outreach to strangers is blocked at system level. Customer controls what types of shop messages they receive.
Broadcast Channels: Not built. VP Feed already covers all content distribution use cases. Adding channels would duplicate VP Feed and confuse users about where to post content. The inbox stays clean: personal DMs, group chats, and shop commerce DMs only.
Commerce Attach Separation: Custom price offers live in 1:1 DMs only — private negotiation has no place in a group context. Group commerce is Bei ya pamoja — dynamic group buying where price drops as more members join. Each feature has one clear home and one clear purpose.
NexGate Messaging — Product Requirements & Feature Flows v1.0 QBIT SPARK | Rules · Flows · Permissions · Scenarios · Edge Cases
NexGate — Private Chat & Calls Flow
Private Chat & Calls (DEEP)
Phase 2 Deep Dive
NexGate / QBIT SPARK | Version 1.0 1:1 DMs · Group Chats · Voice Calls · Video Calls · Ejabberd · WebRTC
Table of Contents
- Scope
- Architecture Overview
- XMPP & Ejabberd Fundamentals
- Ejabberd Cluster — Two Nodes
- Connection Lifecycle
- 1:1 Private DMs
- Group Chats
- Chat States — Typing & Recording
- Message Receipts
- Message Interactions
- Presence System
- Voice Calls — Deep Dive
- Video Calls — Deep Dive
- Audio ↔ Video Switching & Screen Share
- Group Calls
- Offline Handling
- Multi Device
- Shop Inbox in Phase 2
- Security
- Database Schema
1. Scope
This document covers only private communication features in Phase 2:
IN SCOPE:
1:1 private DMs (personal + shop commerce DMs)
Group chats — private + public (up to 500 members)
Group join model (consent DM + invite link)
Voice calls (1:1 + group)
Video calls (1:1 + group)
Audio ↔ video switching during calls
Screen sharing
Chat states (typing, recording voice note)
Message interactions (edit, delete, react, forward, reply)
Message receipts (sent, delivered, read)
Presence (online, offline, last seen)
Multi-device support
Offline delivery
OUT OF SCOPE:
Broadcast channels → NOT built (VP Feed covers this — see Doc 6)
VP Live streaming → covered in VP Live doc
VP Audio Spaces → covered in VP Live doc
File Thunder → covered in File Thunder docs
2. Architecture Overview
┌──────────────────────────────────────────────────────┐
│ NexGate Mobile App │
│ │
│ Personal Inbox Shop Inbox Call Screen │
└────────┬──────────────────┬──────────────┬───────────┘
│ │ │
WebSocket WebSocket WebRTC
XMPP stanzas XMPP stanzas (calls only)
MessagePack MessagePack
│ │ │
└──────────────────┼──────────────┘
│
▼
┌───────────────────────────────┐
│ Ejabberd Cluster │
│ │
│ Node 1 Node 2 │
│ ┌──────────┐ ┌──────────┐ │
│ │ Erlang │◀▶│ Erlang │ │
│ │ dist │ │ dist │ │
│ └──────────┘ └──────────┘ │
│ │
│ Handles: │
│ · All WS connections │
│ · XMPP stanza routing │
│ · Presence protocol │
│ · Chat states (XEP-0085) │
│ · Message receipts (XEP-0184)│
│ · MUC group chats (XEP-0045) │
│ · Jingle call signaling │
│ (XEP-0166) │
│ · Stream management │
│ (XEP-0198) │
└──────────────┬────────────────┘
│
┌───────────┼───────────┐
│ │ │
HTTP auth RabbitMQ REST API
(sync) (async) (Spring Boot
→ Ejabberd)
│ │
▼ ▼
┌────────────────────────────────────────┐
│ Spring Boot Chat Service │
│ │
│ · Message persistence │
│ · Conversation management │
│ · Commerce context │
│ · Receipt tracking │
│ · Notification routing │
│ · Call records │
│ · Shop inbox access control │
│ · Offline escalation │
└──────────┬─────────────┬──────────────┘
│ │
▼ ▼
┌──────────┐ ┌──────────────────────┐
│PostgreSQL│ │ Redis │
│ │ │ presence cache │
│messages │ │ hot message cache │
│convs │ │ unread counts │
│receipts │ │ typing indicators │
│calls │ │ auth token cache │
└──────────┘ └──────────────────────┘
│
▼
┌──────────────────────────────┐
│ RabbitMQ │
│ offline delivery queue │
│ SMS escalation jobs │
│ commerce events │
│ call event logging │
└──────────────────────────────┘
│
┌──────┴──────┐
▼ ▼
┌───────┐ ┌─────────┐
│ FCM │ │ Textfy │
│ push │ │ SMS │
└───────┘ └─────────┘
Also:
Coturn TURN server (separate VPS)
→ relay for voice/video calls
→ when EA carrier NAT blocks P2P
3. XMPP & Ejabberd Fundamentals
JID — Every Entity Has an Address
In XMPP every connected entity has a JID (Jabber ID)
Works like an email address for messaging
Personal user (full JID):
kibuti@nexgate.com/android
│ │ │
user domain resource (device)
Personal user (bare JID):
kibuti@nexgate.com
(without device — used for addressing)
Shop identity:
techstore@shops.nexgate.com
(the shop — not the person behind it)
System bot:
system@nexgate.com
(order updates, notifications)
Group chat room:
group-abc@conference.nexgate.com
Multi-device — same user, multiple resources:
kibuti@nexgate.com/android ← phone
kibuti@nexgate.com/tablet ← tablet
Both receive messages simultaneously
READ on one → Ejabberd notifies other to clear notification
XEPs — XMPP Extension Protocols
XMPP base protocol = just message/presence/iq stanzas
XEPs add specific capabilities on top
XEPs enabled for NexGate private chat:
XEP-0045 Multi-User Chat (MUC)
→ group chats up to 500 members
XEP-0085 Chat State Notifications
→ typing indicators, recording indicators
XEP-0184 Message Delivery Receipts
→ sent / delivered ticks
XEP-0198 Stream Management
→ reliable delivery on bad networks
→ reconnect without losing messages
→ ACK at stanza level
XEP-0166 Jingle
→ voice and video call signaling
XEP-0357 Push Notifications
→ FCM/APNs bridge when user offline
XEP-0333 Chat Markers
→ read receipts (blue ticks)
XEP-0280 Message Carbons
→ sync messages across multiple devices
Three Stanza Types — Everything Is One of These
<!-- 1. Message — send content -->
<message from="kibuti@nexgate.com"
to="juma@nexgate.com"
type="chat"
id="msg-001">
<body>Habari yako!</body>
</message>
<!-- 2. Presence — announce availability -->
<presence from="kibuti@nexgate.com/android">
<show>available</show>
<status>I am here</status>
</presence>
<!-- 3. IQ (Info/Query) — request/response -->
<iq type="get" id="req-001">
<query xmlns="jabber:iq:roster"/>
<!-- asking for contact list -->
</iq>
4. Ejabberd Cluster — Two Nodes
Why Two Nodes Over One
Running a single Ejabberd node works technically. But one node means one point of failure. If that container crashes or the VPS reboots during a deployment — every connected user loses their session, every active call drops, every in-flight message is lost.
Two nodes change the picture completely:
Single node:
Node 1 crashes
→ 100% of users disconnected
→ all active calls dropped
→ messages in-flight lost
→ users notice immediately
Two nodes:
Node 1 crashes
→ 50% of users reconnect to Node 2 (seconds)
→ Node 2 was already running — no cold start
→ active calls on Node 2 unaffected
→ Ejabberd cluster detects Node 1 gone
→ routes everything to Node 2 automatically
→ most users experience a brief reconnect
not a full outage
Two nodes also doubles the connection capacity:
One node → ~500k-1M concurrent connections
Two nodes → ~1M-2M concurrent connections
Same cost increase as one extra container
Launch Plan — Same VPS, Two Containers
For NexGate launch, both nodes run on the same Hetzner VPS. This is the right starting point:
✅ Cheaper — one VPS bill not two
✅ Simpler — same Docker network, zero latency between nodes
✅ Enough — two containers on one VPS still gives redundancy
against container crashes and restarts
✅ Learning — operate cluster on familiar single VPS first
⚠️ VPS hardware failure → both nodes gone
(acceptable risk at launch stage)
When to move to two VPS:
NexGate has paying users depending on uptime
VPS hardware failure = real revenue loss
At that point: Option B (two VPS) is worth the cost
What is Erlang Dist?
This is the mechanism that makes the two containers feel like one system.
Erlang was designed in 1986 for telecom — specifically for telephone switches that could never go down even when individual machines failed. The solution Ericsson built was Erlang Distribution: multiple Erlang nodes connected over a network, sharing a process registry, able to send messages between processes on different machines as if they were local.
Normal programming:
Process on Machine A cannot directly talk to
process on Machine B
Need: HTTP, gRPC, message queue, shared DB
Always an extra hop
Erlang distribution:
Process on Node 1 sends message to process on Node 2
Directly — like calling a local function
No extra infrastructure
No Redis, no RabbitMQ for this
Just: node1_process ! { message_to, node2_process }
Erlang runtime handles delivery across the network
Applied to Ejabberd:
Every connected user = one Erlang process (~2KB RAM)
Kibuti connected to Node 1 = process on Node 1
Juma connected to Node 2 = process on Node 2
Kibuti sends "Habari" to Juma:
Node 1 (Erlang):
"Find Juma's process"
Check local process registry → not here
Check Node 2 via Erlang dist → FOUND
Send message directly to Juma's process on Node 2
Node 2 delivers to Juma's WebSocket
No Redis pub/sub
No RabbitMQ for this routing
No extra network hops
Microsecond latency between nodes
This is why Ejabberd routes at 2M concurrent
where Spring Boot WS needs Redis pub/sub
Erlang Cookie — The Cluster Password
Before two Erlang nodes trust each other
they must prove they belong to the same cluster
The shared secret = Erlang Cookie
Node 1 starts → "my cookie is: nexgate_erlang_cookie_xyz"
Node 2 starts → "my cookie is: nexgate_erlang_cookie_xyz"
Same cookie → they trust each other → cluster formed
Unknown node attempts to join:
"my cookie is: wrong_cookie"
→ rejected → cannot join cluster
Rules:
Same cookie on ALL nodes — mandatory
Long random string — not a simple word
Stored in HashiCorp Vault → injected as env variable
Never committed to git
Rotate periodically like any secret
How Nodes Discover and Join Each Other
Step 1 — EPMD (Erlang Port Mapper Daemon):
Each Erlang node registers with EPMD on port 4369
EPMD is like a local DNS for Erlang nodes
"I am ejabberd@ejabberd-node1, listening on port X"
Step 2 — Node 2 finds Node 1:
Node 2 asks EPMD on Node 1's host:
"Where is ejabberd@ejabberd-node1?"
EPMD responds with port number
Node 2 connects directly
Step 3 — Cookie handshake:
Node 2: "here is my cookie hash"
Node 1: validates → matches → accept
Erlang dist connection established
Step 4 — Join cluster:
ejabberdctl join_cluster ejabberd@ejabberd-node1
Nodes sync:
MUC room state
User session registry
Mnesia tables (Ejabberd internal DB)
Cluster ready ✅
In Docker — hostname is critical:
Container hostname must match Erlang node name
ejabberd@ejabberd-node1 → container hostname: ejabberd-node1
Mismatch = nodes cannot find each other
Docker Compose — Two Nodes on Same VPS
ejabberd-node1:
image: ghcr.io/processone/ejabberd:latest
container_name: ejabberd-node1
hostname: ejabberd-node1 # must match ERLANG_NODE
restart: unless-stopped
environment:
- ERLANG_NODE=ejabberd@ejabberd-node1
- ERLANG_COOKIE=${EJABBERD_ERLANG_COOKIE} # from Vault
ports:
- "5222:5222" # XMPP TCP
- "5280:5280" # WebSocket
- "5285:5285" # REST API (internal)
- "1883:1883" # MQTT
- "4369:4369" # EPMD (Erlang port mapper)
volumes:
- ./ejabberd/ejabberd.yml:/home/ejabberd/conf/ejabberd.yml
- ./ejabberd/node1/data:/home/ejabberd/database
- ./ejabberd/node1/logs:/home/ejabberd/logs
networks:
- nexgate-internal
ejabberd-node2:
image: ghcr.io/processone/ejabberd:latest
container_name: ejabberd-node2
hostname: ejabberd-node2 # different hostname
restart: unless-stopped
environment:
- ERLANG_NODE=ejabberd@ejabberd-node2
- ERLANG_COOKIE=${EJABBERD_ERLANG_COOKIE} # same cookie
ports:
- "5223:5222" # different host ports
- "5281:5280"
- "5286:5285"
- "4370:4369"
volumes:
- ./ejabberd/ejabberd.yml:/home/ejabberd/conf/ejabberd.yml
- ./ejabberd/node2/data:/home/ejabberd/database
- ./ejabberd/node2/logs:/home/ejabberd/logs
depends_on:
- ejabberd-node1
networks:
- nexgate-internal
How Traefik Load Balances Between Nodes
Traefik sits in front of both nodes:
chat.nexgate.com → Traefik → Node 1 or Node 2
Critical: WebSocket needs sticky sessions
Once a user connects to Node 1 — they must
always go to Node 1 for that session
(the WS connection lives on that node)
Traefik labels:
sticky.cookie: true
sticky.cookie.name: "ejabberd_node"
First connection:
User hits chat.nexgate.com
Traefik picks Node 1 (round robin)
Sets cookie: ejabberd_node=node1
User connects WebSocket to Node 1
Subsequent requests same session:
Cookie present: ejabberd_node=node1
Traefik always routes to Node 1
WebSocket session stable ✅
Node 1 crashes:
Cookie points to dead node
Traefik detects Node 1 unhealthy
Routes to Node 2
User reconnects (brief disconnect)
Node 2 was already running → fast reconnect
What Happens When One Node Goes Down
Scenario: ejabberd-node1 container crashes
Immediately:
~50% of users lose WebSocket connection
Their apps detect disconnect
Exponential backoff reconnect starts
Within seconds:
Apps reconnect to chat.nexgate.com
Traefik detects Node 1 unhealthy
Routes all new connections to Node 2
Users reconnect to Node 2
Stream Management (XEP-0198):
Short disconnects (< 5 min): session resumable
Users reconnect → Ejabberd resends missed stanzas
No messages lost
Longer outage:
RabbitMQ offline queue holds messages
FCM push notifications already fired
When user reconnects → queue drains
Messages delivered
Calls during crash:
WebRTC audio/video continues flowing
(P2P or Coturn — not through Ejabberd)
Signaling channel dropped
Active calls: audio continues but
call management (mute, end) needs reconnect
Node 2 (other 50%):
Completely unaffected
No interruption for their users
Their calls continue perfectly
Cluster Architecture — Visual
Same VPS (Launch):
┌──────────────────────────────────────────────────────┐
│ Hetzner VPS │
│ │
│ ┌────────────────────────────────────────────┐ │
│ │ Traefik │ │
│ │ chat.nexgate.com (wss://) │ │
│ │ sticky sessions enabled │ │
│ └───────────────┬─────────────┬──────────────┘ │
│ │ │ │
│ 50% │ │ 50% │
│ ▼ ▼ │
│ ┌─────────────────┐ ┌─────────────────┐ │
│ │ ejabberd-node1 │ │ ejabberd-node2 │ │
│ │ │◀─▶│ │ │
│ │ ~500k users │ │ ~500k users │ │
│ │ Kibuti here │ │ Juma here │ │
│ │ │ │ │ │
│ └─────────────────┘ └─────────────────┘ │
│ Erlang dist (Docker internal network) │
│ microsecond message routing │
│ │
│ ┌─────────────────────────────────────────────┐ │
│ │ Spring Boot · Redis · RabbitMQ · PostgreSQL│ │
│ │ MinIO · File Thunder · FCM · Textfy │ │
│ └─────────────────────────────────────────────┘ │
└──────────────────────────────────────────────────────┘
Two VPS (Growth stage):
┌──────────────────────┐ ┌──────────────────────┐
│ Hetzner VPS 1 │ │ Hetzner VPS 2 │
│ │ │ │
│ ┌────────────────┐ │ │ ┌────────────────┐ │
│ │ ejabberd-node1 │◀─┼────┼─▶│ ejabberd-node2 │ │
│ │ │ │ │ │ │ │
│ │ ~1M users │ │ │ │ ~1M users │ │
│ └────────────────┘ │ │ └────────────────┘ │
│ │ │ │
└──────────────────────┘ └──────────────────────┘
Erlang dist via Hetzner private network
(free bandwidth between Hetzner VPS)
low latency — same data center region
Mnesia — Ejabberd's Internal Database
Ejabberd uses Mnesia (Erlang's built-in DB)
for internal operational data:
What Mnesia stores:
Active user sessions (who is connected where)
MUC room membership + state
Presence subscriptions (roster)
Offline message buffer (short-term)
In a cluster:
Mnesia replicates across nodes automatically
Node 1 and Node 2 share same Mnesia data
Node 1 updates session table → Node 2 sees it
This is how Node 2 knows Juma is on Node 2
and can route Kibuti's message correctly
Mnesia is NOT:
A replacement for PostgreSQL
Where NexGate messages are stored
Where conversation history lives
(that is all PostgreSQL via Spring Boot)
Mnesia is purely Ejabberd internal
NexGate Spring Boot never touches Mnesia
Scale Path Summary
Launch:
1 VPS
2 Docker containers (node1 + node2)
Erlang dist over Docker internal network
Traefik sticky sessions
~1M concurrent capacity
Zero redundancy against VPS hardware failure
✅ Right for launch
Growth:
2 VPS (Hetzner private network)
1 container per VPS
Erlang dist over private network (low latency)
True hardware redundancy
~2M concurrent capacity
VPS failure → other VPS serves all users
✅ Right when uptime = revenue
WeChat EA scale:
3-5 VPS nodes
Each node handles ~500k-1M users
Erlang cluster routes everything
Geographic distribution possible
✅ Right when NexGate is EA infrastructure
5. Connection Lifecycle
Full Connect → Reconnect → Disconnect Flow
App launches / user logs in:
│
▼ POST /auth/login (Main Backend)
Receive two tokens:
REST JWT → HTTP API calls (7 days)
XMPP Token → Ejabberd connection (24 hours)
│
▼ Connect WebSocket
wss://chat.nexgate.com/ws
Header: Authorization: Bearer {XMPP_TOKEN}
│
▼ XMPP stream opened
<stream:stream to="nexgate.com"
version="1.0">
│
▼ Ejabberd → Spring Boot (sync HTTP auth)
POST /internal/ejabberd/auth
{ username: "usr-kibuti", token: "XMPP_TOKEN" }
Spring Boot:
Check Redis cache first (< 5ms if cached)
Validate JWT signature
Check user not suspended
Return 200 or 401
│
▼ Auth success — Ejabberd sends features
<stream:features>
<sm xmlns="urn:ietf:params:xml:ns:xmpp-session"/>
<!-- Stream Management XEP-0198 -->
</stream:features>
│
▼ Client enables Stream Management
<enable xmlns="urn:ietf:params:xml:ns:xmpp-sm:3"
resume="true"/>
Ejabberd: <enabled id="session-abc" resume="true"/>
│
▼ Client sends presence (I am online)
<presence/>
│
Ejabberd:
Registers kibuti@nexgate.com/android as ONLINE
Publishes RabbitMQ: chat.presence.online
Spring Boot: drain offline queue for kibuti
│
Connection established ✅
App shows conversations, unread counts
Stream Management — Why It Matters for EA
Problem without Stream Management:
Network drops (very common on EA mobile)
TCP connection breaks
In-flight messages LOST
User reconnects — no idea what was missed
XEP-0198 Stream Management solution:
Every stanza gets a sequence number
Client ACKs received stanzas:
<a xmlns="..." h="5"/> ← "I received up to stanza 5"
Server ACKs received stanzas the same way
On reconnect:
Client sends: <resume id="session-abc" h="5"/>
Ejabberd knows: client got up to stanza 5
Ejabberd resends: stanzas 6, 7, 8 (unacknowledged)
Zero message loss
For EA mobile networks:
Connection drops constantly (3G → 2G → WiFi)
Stream Management means users never miss messages
Even on unstable connections
Critical for NexGate commerce DMs
(missing an order negotiation message = lost sale)
Reconnection Strategy
Connection drops detected:
│
App: exponential backoff reconnect
Attempt 1: wait 1 second
Attempt 2: wait 2 seconds
Attempt 3: wait 4 seconds
Attempt 4: wait 8 seconds
Max wait: 30 seconds
│
On reconnect:
If session resumable (< 5 minutes offline):
Resume: <resume id="session-abc" h="last_ack"/>
Ejabberd resends missed stanzas
No message loss ✅
If session expired (> 5 minutes offline):
Full re-auth with XMPP token
Fetch conversation list from REST API
Missed messages come from MAM (message archive)
or from RabbitMQ offline queue drain
6. 1:1 Private DMs
Sending a Text Message
[Kibuti types "Habari" — taps Send]
│
▼ App sends XMPP message stanza:
<message from="kibuti@nexgate.com/android"
to="juma@nexgate.com"
type="chat"
id="msg-abc-123">
<body>Habari</body>
<request xmlns="urn:xmpp:receipts"/>
<!-- Request delivery receipt XEP-0184 -->
<nexgate xmlns="urn:nexgate:meta">
<conv_id>conv-789</conv_id>
<temp_id>local-xyz</temp_id>
<level>NORMAL</level>
</nexgate>
<!-- NexGate custom namespace for app metadata -->
</message>
│
App shows message as: pending ⏳
│
▼
[Ejabberd Node 1 — Kibuti's node]
Receives stanza
ACKs via Stream Management:
<a h="6"/> ← "I got your stanza"
App: pending → sent ✓
│
Is Juma online?
YES → Juma on Node 2:
Erlang distributed message → Node 2
Node 2 delivers to Juma's WS
NO → Store in offline queue
XEP-0160 offline storage
OR RabbitMQ (NexGate custom)
│
Fire RabbitMQ: chat.message.inbound
(async, does not block delivery)
│
▼
[Spring Boot Chat Service — async]
Write to PostgreSQL
Write to Redis hot cache
Resolve offline escalation if needed
Receiving a Message
[Juma's app — connected on Node 2]
│
Ejabberd Node 2 pushes stanza:
<message from="kibuti@nexgate.com"
to="juma@nexgate.com"
type="chat"
id="msg-abc-123">
<body>Habari</body>
<request xmlns="urn:xmpp:receipts"/>
</message>
│
Juma's app:
Displays message in conversation
Automatically sends delivery receipt:
<message from="juma@nexgate.com"
to="kibuti@nexgate.com">
<received xmlns="urn:xmpp:receipts"
id="msg-abc-123"/>
</message>
│
Ejabberd routes receipt to Kibuti
Kibuti's app: sent ✓ → delivered ✓✓
│
Juma opens conversation:
<message from="juma@nexgate.com"
to="kibuti@nexgate.com">
<displayed xmlns="urn:xmpp:chat-markers:0"
id="msg-abc-123"/>
</message>
│
Kibuti's app: delivered ✓✓ → read ✓✓ (blue)
Rich Content Cards in DMs
NexGate extends XMPP with custom namespaces
for rich content (product cards, events etc)
Product card message:
<message from="kibuti@nexgate.com"
to="juma@nexgate.com"
type="chat"
id="msg-card-001">
<body>Angalia bidhaa hii</body>
<nexgate-card xmlns="urn:nexgate:cards">
<type>PRODUCT</type>
<ref_id>prod-123</ref_id>
<snapshot>
<name>Samsung A15</name>
<price>450000</price>
<currency>TZS</currency>
<image_url>...</image_url>
<shop_name>TechStore</shop_name>
</snapshot>
</nexgate-card>
</message>
Receiving app:
Detects nexgate-card element
Renders rich card UI instead of plain text
Tappable → deep links to product page
7. Group Chats
MUC — Multi User Chat
Group chats in Ejabberd use XEP-0045 (MUC)
Each group = a MUC room with its own JID:
group-abc@conference.nexgate.com
Two group types in NexGate:
PRIVATE: closed, controlled membership
not discoverable in search
default when creating a group
PUBLIC: open, anyone can join via link
discoverable in NexGate search
explicit choice by creator
Creating a Group — Technical Flow
User creates group in app:
│
▼ POST /chat/groups/create (Spring Boot)
{
name: "Business Friends",
type: "PRIVATE", ← or PUBLIC
description: "Dar founders discussion"
}
│
Spring Boot:
Create conversation record (type: GROUP)
Call Ejabberd REST API:
POST /api/create_room
{
name: "group-abc",
service: "conference.nexgate.com",
options: {
persistent: true,
public: false, ← PRIVATE
members_only: true,
allow_private_messages: false
}
}
Creator auto-joined as OWNER
Generate invite link token
Return: { groupId, inviteLink }
Group Join Model — Two Mechanisms
NexGate principle:
Nobody ends up in a group
without actively choosing to join
No forced direct add
Two consent-based mechanisms only
Mechanism 1 — Consent DM Invitation
Admin selects people from contacts/followers/
commerce relationships (NOT strangers)
Each selected person receives a DM:
Spring Boot → Ejabberd REST API:
POST /api/send_message
{
from: "system@nexgate.com",
to: "juma@nexgate.com",
extra: {
type: "GROUP_INVITATION",
group_id: "group-abc",
group_name: "Business Friends",
group_type: "PRIVATE",
member_count: 47,
description: "Dar founders discussion",
invited_by: "Kibuti Mwangi",
expires_in: "48h"
}
}
XMPP stanza (NexGate custom namespace):
<message from="system@nexgate.com"
to="juma@nexgate.com"
type="chat">
<body>You have been invited to join a group</body>
<nexgate-group-invite xmlns="urn:nexgate:group:1">
<group_id>group-abc</group_id>
<group_name>Business Friends</group_name>
<group_type>PRIVATE</group_type>
<member_count>47</member_count>
<invited_by>Kibuti Mwangi</invited_by>
<expires_at>2026-07-15T10:00:00Z</expires_at>
</nexgate-group-invite>
</message>
Recipient app renders:
┌──────────────────────────────────────────┐
│ 📨 Group Invitation │
│ │
│ Kibuti Mwangi invited you to join: │
│ 🏘️ Business Friends │
│ 47 members · Private Group │
│ "Dar founders discussion" │
│ │
│ [Accept & Join] [Decline] │
└──────────────────────────────────────────┘
If Accept:
App sends: POST /chat/groups/group-abc/join
Spring Boot: ejabberdctl add_member group-abc juma
Juma is now a group member ✅
If Decline:
POST /chat/groups/group-abc/decline
Not added ✅
Kibuti NOT notified (privacy)
If Ignored (48h passes):
Invitation auto-expired
Auto-declined silently ✅
Mechanism 2 — Invite Link
Admin generates link:
nexgate.app/join/abc-xyz-def
Person taps link → sees group preview:
PRIVATE GROUP:
App calls: GET /chat/groups/preview/abc-xyz-def
Shows preview + [Request to Join] button
Spring Boot creates join request
Admin sees request in group management
Admin approves → member ✅
Admin declines → person not notified
PUBLIC GROUP:
App calls: GET /chat/groups/preview/abc-xyz-def
Shows preview + [Join Group] button
Tap → instant member ✅
No approval needed
Link settings (stored in DB):
expires_at: nullable (never if null)
max_joins: nullable (unlimited if null)
revoked_at: nullable (active if null)
On revoke:
Old token deleted
New token generated
Old link: "This invite link is no longer valid"
Sending a Group Message
<!-- Member sends to group -->
<message to="group-abc@conference.nexgate.com"
type="groupchat"
id="gmsg-001">
<body>Hello everyone!</body>
</message>
<!-- Ejabberd MUC reflects back with room JID -->
<message from="group-abc@conference.nexgate.com/Kibuti"
type="groupchat"
id="gmsg-001">
<body>Hello everyone!</body>
</message>
Ejabberd MUC fan-out:
Receives from Kibuti
Broadcasts to ALL room members simultaneously
Each member's WS gets the stanza
Erlang handles fan-out natively
No Redis pub/sub needed
Spring Boot persists via RabbitMQ event
Group Roles — Ejabberd MUC Mapping
Ejabberd MUC role NexGate role Permissions
─────────────────────────────────────────────────────────
owner OWNER everything
delete group
transfer ownership
admin ADMIN manage members
delete any message
pin messages
change group info
moderator MODERATOR mute members
remove members
participant MEMBER send messages
delete own messages
visitor READ_ONLY view only
(announcement mode)
Fan-out Strategy
Groups up to 500 members:
Ejabberd MUC native fan-out
All members get stanza in real time
Erlang handles it — no extra logic
Groups approaching 500:
Recommend switching to PUBLIC group
with announcement mode (admins only post)
Better for large audiences
No broadcast channels needed
(VP Feed covers mass content distribution)
Group Invite Link DB Schema
group_invite_links
─────────────────────────────────────────────
link_id UUID
group_id UUID
token TEXT unique random token
created_by UUID admin userId
expires_at TIMESTAMPTZ nullable
max_joins INT nullable
join_count INT default 0
revoked_at TIMESTAMPTZ nullable
created_at TIMESTAMPTZ
8. Chat States — Typing & Recording
XEP-0085 — Built Into Ejabberd
No custom backend code needed
Ejabberd routes chat state stanzas automatically
Spring Boot never sees them (not persisted)
Pure real-time ephemeral signals
All States and When App Sends Them
State App sends when Recipient sees
──────────────────────────────────────────────────────────────
composing user starts typing "Kibuti is typing..."
paused user stopped typing indicator disappears
(3s no keystroke)
active user opened conversation no indicator
but not typing
inactive user left conversation no indicator
screen (10s elapsed)
gone user closed conversation no indicator
recording user holding mic button "Kibuti is recording..."
Stanzas
<!-- User starts typing -->
<message from="kibuti@nexgate.com/android"
to="juma@nexgate.com"
type="chat">
<composing xmlns="http://jabber.org/protocol/chatstates"/>
</message>
<!-- User paused typing -->
<message from="kibuti@nexgate.com/android"
to="juma@nexgate.com"
type="chat">
<paused xmlns="http://jabber.org/protocol/chatstates"/>
</message>
<!-- User is recording voice note -->
<message from="kibuti@nexgate.com/android"
to="juma@nexgate.com"
type="chat">
<composing xmlns="http://jabber.org/protocol/chatstates"/>
<recording xmlns="urn:nexgate:states"/>
<!-- NexGate custom namespace for recording state -->
</message>
<!-- User stopped recording (sent or cancelled) -->
<message from="kibuti@nexgate.com/android"
to="juma@nexgate.com"
type="chat">
<paused xmlns="http://jabber.org/protocol/chatstates"/>
</message>
Throttling — Don't Spam the Network
Wrong (naive) approach:
Send composing stanza on every single keystroke
100 keystrokes = 100 stanzas
Wastes bandwidth — bad for EA data bundles
Correct approach:
User starts typing → send composing once
Keep typing → resend composing every 3 seconds
User stops → wait 3 seconds → send paused
Total: ~1 stanza per 3 seconds while typing
Much more efficient
Mobile dev implements this with a timer:
startTypingTimer() → fires composing once
resetTimer() on each keystroke
onTimerExpire() → send paused
Group Chat States
In group chats:
Same stanzas — sent to room JID instead of personal JID
Ejabberd MUC broadcasts to all room members
UI handling when multiple people type:
1 person: "Juma is typing..."
2 people: "Juma and Amina are typing..."
3+ people: "3 people are typing..."
App collects composing events from room
Tracks: Set<userId> currentlyTyping
Renders string based on set size
9. Message Receipts
Three Tick States
✓ Sent Server received + stored stanza
Stream Management ACK received
✓✓ Delivered Recipient device received stanza
XEP-0184 receipt returned
✓✓ Read Recipient opened conversation
(blue) XEP-0333 chat marker returned
XEP-0184 — Delivery Receipt
<!-- Kibuti requests receipt in original message -->
<message id="msg-001" ...>
<body>Habari</body>
<request xmlns="urn:xmpp:receipts"/>
</message>
<!-- Juma's app automatically responds on delivery -->
<message from="juma@nexgate.com"
to="kibuti@nexgate.com">
<received xmlns="urn:xmpp:receipts"
id="msg-001"/>
<!-- id references the original message -->
</message>
Kibuti's app receives this:
→ updates message msg-001 status: DELIVERED
→ shows ✓✓
XEP-0333 — Chat Markers (Read Receipt)
<!-- Juma opens conversation — app sends displayed marker -->
<message from="juma@nexgate.com"
to="kibuti@nexgate.com"
type="chat">
<displayed xmlns="urn:xmpp:chat-markers:0"
id="msg-001"/>
<!-- marks msg-001 and all previous as read -->
</message>
Kibuti's app receives this:
→ updates msg-001 and all before: READ
→ shows ✓✓ blue
Group Message Receipts
In groups: receipts work per member
Message sent to group of 5:
Each member's delivery → individual receipt
All 5 delivered → show ✓✓
Read receipts in groups:
Show count: "Read by 3"
Tap to see who read it
(WhatsApp same pattern)
Spring Boot aggregates:
Stores each receipt in message_receipts table
Computes: delivered_count, read_count
Returns to sender on request
11. Presence System
How Presence Works in XMPP
Presence is built into XMPP protocol
No custom implementation needed
Ejabberd handles all presence routing
User connects:
Sends: <presence/>
Ejabberd broadcasts to all contacts
who have presence subscription
User disconnects:
Ejabberd auto-sends: <presence type="unavailable"/>
All subscribed contacts notified
This is fully automatic
Spring Boot only needs to:
Listen to RabbitMQ presence events
Update last_seen_at in PostgreSQL
Cache presence in Redis (for fast lookup)
Presence States
<!-- Available (online) -->
<presence from="kibuti@nexgate.com/android"/>
<!-- Away (phone locked / app backgrounded) -->
<presence from="kibuti@nexgate.com/android">
<show>away</show>
</presence>
<!-- Do Not Disturb -->
<presence from="kibuti@nexgate.com/android">
<show>dnd</show>
<status>In a meeting</status>
</presence>
<!-- Offline -->
<presence from="kibuti@nexgate.com/android"
type="unavailable"/>
Last Seen
When user goes offline:
Ejabberd fires: chat.presence.offline (RabbitMQ)
Spring Boot:
Update users.last_seen_at = now
Remove presence:{userId} from Redis
When contact opens chat with offline user:
App requests: GET /chat/users/{userId}/presence
Spring Boot returns:
{ status: "offline", lastSeenAt: "2026-07-02T08:30:00Z" }
App shows: "Mwisho kuonekana leo saa 2:30"
Privacy settings (Spring Boot enforces):
EVERYONE → anyone can see last seen
CONTACTS → only conversation partners
NOBODY → hide last seen from all
Online Indicator in Conversation
How app shows "online" in DM header:
Option 1 — Subscribe to presence (XMPP native):
App sends presence subscription to contact
Contact auto-notified when they come online
Ejabberd handles real-time push
Option 2 — Poll on conversation open:
GET /chat/users/{userId}/presence
Check Redis: presence:{userId} exists? → online
Simple, no subscription management
NexGate recommendation: Option 2
Simpler to implement
No subscription state to manage
Polling on conversation open is fine
(user only cares when they're IN the conversation)
12. Voice Calls — Deep Dive
Complete Component Map
┌──────────────────────────────────────────────────────┐
│ Voice Call │
│ │
│ Signaling: Ejabberd Jingle (XEP-0166) │
│ "who calls who, exchange network info" │
│ │
│ Discovery: STUN (built into Ejabberd) │
│ "find your public IP behind NAT" │
│ │
│ Relay: Coturn TURN server │
│ "relay audio when P2P impossible" │
│ EA carrier NAT blocks most P2P │
│ │
│ Transport: WebRTC PeerConnection │
│ "actual audio stream between devices" │
│ │
│ Codec: Opus │
│ "compress audio for EA networks" │
│ "adaptive 6kbps (2G) → 64kbps (WiFi)" │
│ │
│ Encryption: SRTP (built into WebRTC) │
│ "all audio encrypted end to end" │
└──────────────────────────────────────────────────────┘
ICE — How Devices Find Each Other
ICE = Interactive Connectivity Establishment
The algorithm that finds the best path between devices
Step 1 — Gather candidates (both devices do this):
Local candidate:
192.168.1.5:54321 ← local network IP
STUN candidate:
41.188.xxx.xxx:54321 ← public IP (Vodacom/Airtel IP)
Found by asking STUN server: "What is my public IP?"
TURN candidate:
turn.nexgate.com:3478 ← relay fallback
Step 2 — Exchange candidates:
Both share their candidate lists
Via Ejabberd Jingle stanzas
Step 3 — Try connections (priority order):
1. Direct local network (same WiFi) → fastest
2. Direct P2P via public IPs → good
3. TURN relay → always works, higher latency
Step 4 — Use best working path:
Call starts on winning candidate
Can switch mid-call if network changes
EA reality:
Direct local → rarely (different networks)
Direct P2P → sometimes (depends on carrier)
TURN relay → most common on Vodacom/Airtel/Tigo
Full Voice Call Sequence
[Kibuti taps "Call Juma"]
│
▼ App: GET /chat/calls/turn-credentials
Spring Boot generates HMAC TURN credentials:
{
iceServers: [
{ urls: "stun:chat.nexgate.com:3478" },
{ urls: "turn:turn.nexgate.com:3478",
username: "usr-kibuti:1751500000",
credential: "hmac_sha1_token" }
]
}
│
▼ App initializes WebRTC PeerConnection
Config: iceServers from above
Add audio track:
Opus codec
echoCancellation: true
noiseSuppression: true
autoGainControl: true
│
▼ App creates SDP offer
WebRTC generates offer describing:
Codecs supported (Opus preferred)
Audio capabilities
Security parameters (DTLS)
│
▼ App sends Jingle session-initiate
<iq from="kibuti@nexgate.com/android"
to="juma@nexgate.com"
type="set"
id="call-001">
<jingle xmlns="urn:xmpp:jingle:1"
action="session-initiate"
sid="sid-abc-123"
initiator="kibuti@nexgate.com/android">
<content name="audio" creator="initiator">
<description xmlns="urn:xmpp:jingle:apps:rtp:1"
media="audio">
<payload-type id="111"
name="opus"
clockrate="48000"
channels="2"/>
</description>
<transport xmlns="urn:xmpp:jingle:transports:ice-udp:1"
ufrag="abc"
pwd="password123">
<candidate component="1"
foundation="1"
type="host"
ip="192.168.1.5"
port="54321"
priority="2130706431"/>
<candidate component="1"
foundation="2"
type="srflx"
ip="41.188.xxx.xxx"
port="54321"
priority="1694498815"/>
</transport>
</content>
</jingle>
</iq>
│
▼ Ejabberd routes to Juma
Ejabberd fires RabbitMQ: chat.call.initiated
Spring Boot:
Creates call record (status: RINGING)
If Juma offline → FCM HIGH priority:
{ type: INCOMING_CALL, callId: "sid-abc-123",
callerName: "Kibuti", callType: VOICE }
│
[Juma's phone rings — incoming call screen]
Juma taps Answer
│
▼ Juma: get TURN credentials
Initialize PeerConnection (same config)
Set remote description (Kibuti's SDP)
Create SDP answer
Gather own ICE candidates
│
▼ Juma sends Jingle session-accept
<iq from="juma@nexgate.com/android"
to="kibuti@nexgate.com/android"
type="set">
<jingle action="session-accept"
sid="sid-abc-123">
<!-- Juma's SDP answer + ICE candidates -->
</jingle>
</iq>
│
▼ Ejabberd routes to Kibuti
Kibuti's app:
Sets remote description (Juma's SDP)
ICE negotiation completes
Best path selected (likely TURN on EA networks)
│
CALL LIVE 🎉
Opus audio flowing between devices
│
RTCP monitors quality every 200ms:
Reports: packet loss, jitter, RTT, bandwidth
Opus adapts bitrate automatically:
64kbps → 32kbps → 16kbps → 8kbps → 6kbps
Never drops — always degrades gracefully
│
Kibuti taps End
│
▼ Jingle session-terminate
<iq type="set">
<jingle action="session-terminate"
sid="sid-abc-123">
<reason><success/></reason>
</jingle>
</iq>
│
Ejabberd fires RabbitMQ: chat.call.ended
Spring Boot:
Update call record:
status: COMPLETED
ended_at: now
duration_seconds: 247
relay_used: true
end_reason: NORMAL
Call State Machine
IDLE
│ user taps Call
▼
INITIATING ─────────────────────────────▶ FAILED
│ TURN credentials fetched (network error)
│ PeerConnection created
│ Jingle initiate sent
▼
RINGING ────────────────────────────────▶ MISSED
│ waiting for answer (45s timeout)
│ DECLINED
▼ (Juma rejects)
CONNECTING
│ Jingle accepted
│ ICE negotiation in progress
▼
CONNECTED ──────────────────────────────▶ RECONNECTING
│ audio flowing │ network drop
│ │ ICE restart
│ │ 10s → FAILED
│ user ends
▼
ENDING
│ Jingle terminate sent
▼
COMPLETED
Opus Codec Ladder
Network Bitrate What it sounds like
──────────────────────────────────────────────────────
WiFi / 4G strong 64 kbps HD voice, crystal clear
4G normal 32 kbps Clear, natural voice
3G 16 kbps Good, slight compression
2G / Edge 8 kbps Robotic but intelligible
Barely alive 6 kbps Minimum — still connected
──────────────────────────────────────────────────────
Opus switches between these automatically
based on RTCP feedback every 200ms
Mobile dev configures nothing — it just works
Key Opus features for EA:
inbandfec: true → Forward Error Correction
recovers from packet loss
without retransmit
usedtx: true → Discontinuous Transmission
silence = no packets sent
saves bandwidth during pauses
stereo: false → Mono only for calls
half the bitrate vs stereo
13. Video Calls — Deep Dive
Additional Components vs Voice
Voice call +
Video codec (H.264 primary)
Camera capture (front/rear switchable)
Video rendering (remote + local preview)
Higher bandwidth requirement
Higher CPU on device
More Coturn relay bandwidth if P2P fails
H.264 — Why for EA
H.264 (AVC) chosen because:
Hardware acceleration:
Every phone since 2013 has H.264 hardware encoder
Including Tecno Spark, Infinix Hot (dominant in EA)
Hardware encoder = GPU does the work
Battery impact: LOW
CPU: barely used
Software encoding (VP8, VP9, AV1):
CPU does all encoding work
On low-end EA phones: hot, slow, battery drain
10 minutes of video = significant battery cost
Users notice and complain
H.264 at low bitrates:
360p @ 400kbps → works on 3G
240p @ 150kbps → works on 2G
Quality acceptable for face-to-face conversation
Video Resolution Ladder
Device tier + network → resolution selected:
Device Network Resolution FPS Bitrate
────────────────────────────────────────────────────────
Any WiFi 720p 30 1.5 Mbps
Any 4G strong 480p 24 800 kbps
Any 3G 360p 15 400 kbps
Any 2G 240p 10 150 kbps
Any Very poor AUDIO ONLY — Opus only
────────────────────────────────────────────────────────
Degradation order (call never drops):
1. Reduce color depth
2. Reduce resolution (720→480→360→240)
3. Reduce frame rate (30→24→15→10)
4. Reduce audio bitrate
5. Disable video completely → audio only
6. Audio minimum (6kbps Opus)
Upgrade is conservative:
Wait 5 seconds of stable improved bandwidth
Then upgrade one step (e.g. 360p → 480p)
Prevents quality flapping on unstable networks
Jingle for Video — Two Content Blocks
<jingle action="session-initiate" sid="vid-001">
<!-- Audio block — always present -->
<content name="audio" creator="initiator">
<description media="audio">
<payload-type id="111" name="opus"
clockrate="48000"/>
</description>
<transport ...>
<candidate .../> <!-- ICE candidates -->
</transport>
</content>
<!-- Video block — added for video calls -->
<content name="video" creator="initiator">
<description media="video">
<!-- H.264 preferred -->
<payload-type id="96" name="H264"
clockrate="90000"/>
<!-- VP8 as fallback -->
<payload-type id="97" name="VP8"
clockrate="90000"/>
</description>
<transport ...>
<candidate .../>
</transport>
</content>
</jingle>
Camera UI Features
Mobile dev implements:
Local preview (Picture-in-Picture):
Small corner window showing your own camera
Standard in all video call UIs
Switch camera:
Front → Rear → Front toggle
WebRTC: videoCapturer.switchCamera()
Camera off (privacy):
videoTrack.setEnabled(false)
Remote sees: black screen or avatar
Audio continues
Auto-disable video on battery:
Monitor: battery < 20% AND on Coturn relay
Show warning: "Battery low — switching to audio only"
Disable video track
Continue audio call
10. Message Interactions
All message interactions use standard XMPP XEPs. Ejabberd routes stanzas automatically. Spring Boot validates rules and persists via RabbitMQ.
XEP Overview
Feature XEP Status Ejabberd
─────────────────────────────────────────────────────
Edit message XEP-0308 Stable ✅ auto routed
Delete message XEP-0424 Stable ✅ auto routed
Reactions XEP-0444 Stable ✅ auto routed
Forwarding XEP-0297 Stable ✅ auto routed
Reply/Quote XEP-0461 Exp ⚠️ auto routed
Stable IDs XEP-0359 Stable ✅ auto assigned
Edit — XEP-0308
<!-- Kibuti edits his sent message -->
<message from="kibuti@nexgate.com"
to="juma@nexgate.com"
type="chat"
id="edit-002">
<body>Hello Juma, how is business today?</body>
<replace xmlns="urn:xmpp:message-correct:0"
id="server-stable-id-abc"/>
<!-- references original by stanza-id (XEP-0359) -->
</message>
Rules:
Only original sender can edit ✅
Text messages only ✅
Within 15 minutes of sending ✅
Shows "Edited" label after ✅
Commerce cards: NOT editable ❌
System messages: NOT editable ❌
Spring Boot on receiving edit event (RabbitMQ):
Validate author + time window
Update messages.body
Update messages.edited_at
Increment messages.edit_count
Delete — XEP-0424
<!-- Delete for everyone -->
<message from="kibuti@nexgate.com"
to="juma@nexgate.com"
type="chat"
id="retract-003">
<apply-to xmlns="urn:xmpp:fasten:0"
id="server-stable-id-abc">
<retract xmlns="urn:xmpp:message-retract:1"/>
</apply-to>
</message>
Delete for me:
No stanza needed
Local REST call only
POST /chat/messages/{id}/delete { scope: SELF }
Recipient unaffected
Delete for everyone:
XEP-0424 retraction stanza
Within 15 minutes only
Commerce cards: NOT deletable ❌
System messages: NOT deletable ❌
Recipient sees: "This message was deleted"
Nothing hard-deleted from PostgreSQL (audit trail)
Reactions — XEP-0444
<!-- Add reaction -->
<message from="kibuti@nexgate.com"
to="juma@nexgate.com"
type="chat">
<reactions xmlns="urn:xmpp:reactions:0"
id="server-stable-id-abc">
<reaction>👍</reaction>
</reactions>
</message>
<!-- Remove reaction (empty = removed) -->
<message from="kibuti@nexgate.com"
to="juma@nexgate.com"
type="chat">
<reactions xmlns="urn:xmpp:reactions:0"
id="server-stable-id-abc">
</reactions>
</message>
Rules:
One reaction per user per message ✅
Change: send new emoji (replaces) ✅
Remove: send empty reactions element ✅
Commerce cards: reactions ALLOWED ✅
System messages: reactions NOT allowed ❌
Launch emoji set: ❤️ 👍 😂 😮 😢 🙏
Forwarding — XEP-0297
<!-- Kibuti forwards Juma's message to Alice -->
<message from="kibuti@nexgate.com"
to="alice@nexgate.com"
type="chat"
id="fwd-001">
<body>Check this out</body>
<forwarded xmlns="urn:xmpp:forward:0">
<delay xmlns="urn:xmpp:delay"
stamp="2026-07-13T10:32:00Z"/>
<message from="juma@nexgate.com"
to="kibuti@nexgate.com"
type="chat">
<body>Hello everyone!</body>
</message>
</forwarded>
<nexgate-forward xmlns="urn:nexgate:forward">
<original_sender_name>Juma Mwangi</original_sender_name>
<forward_chain>1</forward_chain>
</nexgate-forward>
</message>
Rules:
Max 5 conversations per forward action ✅
Chain 1: "Forwarded from Juma Mwangi"
Chain 2-4: "Forwarded"
Chain 5+: "Forwarded many times" (warning)
Media: references original fileId — no re-upload ✅
Custom price offers: NOT forwardable ❌
Order/payment records: NOT forwardable ❌
Reply — XEP-0461
<!-- Reply to a specific message -->
<message from="juma@nexgate.com"
to="kibuti@nexgate.com"
type="chat"
id="reply-001">
<body>Thanks, appreciate it!</body>
<reply xmlns="urn:xmpp:reply:0"
to="kibuti@nexgate.com"
id="server-stable-id-abc"/>
</message>
Renders as:
┌────────────────────────────────┐
│ ┌──────────────────────────┐ │
│ │ Kibuti │ │ ← quoted
│ │ Hello Juma! │ │
│ └──────────────────────────┘ │
│ Thanks, appreciate it! │
└────────────────────────────────┘
Tap quote → scrolls to original message
14. Audio ↔ Video Switching & Screen Share
Switch Audio → Video During Call
Call starts as voice only
User taps camera button during call
No hang up needed — same WebRTC session
Kibuti enables camera:
Creates video track (H.264)
Adds to existing PeerConnection
Sends Jingle content-add stanza:
<iq from="kibuti@nexgate.com/android"
to="juma@nexgate.com"
type="set">
<jingle xmlns="urn:xmpp:jingle:1"
action="content-add"
sid="sid-abc-123">
<!-- sid = SAME session as voice call -->
<content name="video" creator="initiator">
<description media="video">
<payload-type id="96" name="H264"
clockrate="90000"/>
</description>
<transport .../>
</content>
</jingle>
</iq>
Juma accepts:
Jingle action="content-accept"
Video starts flowing — same TURN relay ✅
Audio uninterrupted during upgrade ✅
Switch back (video → audio):
Jingle action="content-remove"
Removes video content block
Audio continues
Auto-downgrade (network-triggered):
RTCP detects bandwidth too low
App sends content-remove automatically
Banner: "Video disabled — poor network"
Resumes when network improves
Screen Sharing
Screen share = special video track
Instead of camera → captures device screen
Same H.264 encoding
Lower frame rate (5-15fps — screen changes slowly)
Android: MediaProjection API
iOS: ReplayKit broadcast extension
Start screen share:
User taps screen share icon during call
System permission dialog appears:
"Allow NexGate to capture your screen?"
User accepts
Screen capture starts
Jingle stanza (adds screen content block):
<jingle action="content-add"
sid="sid-abc-123">
<content name="screen" creator="initiator">
<description media="video">
<payload-type id="96" name="H264"
clockrate="90000"/>
</description>
<transport .../>
</content>
</jingle>
During screen share:
Remote side sees: screen (large) + face (PiP)
Local side sees: "Sharing screen" banner
Camera optional: can keep or disable
Stop screen share:
Jingle content-remove (screen)
Returns to normal video/audio call
EA network consideration:
Screen content is mostly static
H.264 compresses static content very well
720p screen at ~300kbps (vs 720p camera at 1.5Mbps)
Works on 3G for text/document sharing ✅
15. Group Calls
Why LiveKit for Group Calls
1:1 call:
P2P or Coturn relay
Two devices, one path
No server media processing
Group call (3+ people):
Cannot P2P to everyone simultaneously
Kibuti uploads 1 stream to LiveKit
LiveKit forwards to all other participants
Each participant uploads once → downloads N-1
SFU = Selective Forwarding Unit
LiveKit already deployed for Audio Spaces
Same Docker container
Same Coturn relay reused
Zero new infrastructure ✅
Group Call Flow
Kibuti starts group call from group chat:
│
▼ POST /chat/calls/group/start
Spring Boot:
Create LiveKit room: group-call-{callId}
Generate token per participant:
canPublish: true
canSubscribe: true
Return tokens + LiveKit WS URL
│
Jingle session-initiate sent to all group members:
<message from="system@nexgate.com"
to="juma@nexgate.com"
type="chat">
<nexgate-call xmlns="urn:nexgate:call:1">
<type>GROUP_CALL_JOIN_INFO</type>
<call_id>call-xyz</call_id>
<call_type>VIDEO</call_type>
<livekit_url>wss://livekit.nexgate.com</livekit_url>
<livekit_token>eyJ...</livekit_token>
<room_id>group-call-xyz</room_id>
<expires_in>300</expires_in>
</nexgate-call>
</message>
Each member receives → phone rings
Members who join → connect WebRTC to LiveKit
LiveKit SFU forwards all streams ✅
[LiveKit SFU]
Kibuti stream ──▶ forwarded to Juma + Alice
Juma stream ──▶ forwarded to Kibuti + Alice
Alice stream ──▶ forwarded to Kibuti + Juma
EA Network Limits for Group Calls
Group voice (audio only, Opus):
3 people: each downloads 64kbps → works on 3G ✅
5 people: each downloads 128kbps → works on 3G ✅
8 people: each downloads 224kbps → needs 4G ⚠️
Group video (H.264 + Opus):
3 people: each downloads 800kbps → needs 4G ⚠️
4 people: each downloads 1.2Mbps → needs strong 4G ⚠️
5+ people: reduce to active speaker only ✅
Max participants shown:
Voice: up to 8 (3G compatible)
Video: up to 4 feeds simultaneously
5th+ person: audio tile only (no video feed)
Active speaker highlighted (larger tile)
Simulcast — EA Network Diversity
Each participant uploads 3 quality versions:
Low: 180p + Opus 16kbps
Medium: 360p + Opus 32kbps
High: 720p + Opus 64kbps
LiveKit delivers appropriate quality per receiver:
Receiver on 2G → low quality streams
Receiver on WiFi → high quality streams
Each receiver gets quality their network allows
Independently per stream
Result:
Good network user sees HD video
Poor network user sees low quality
Everyone stays in the call ✅
No one's bad network drops everyone else
16. Offline Handling
Three Layers of Offline Delivery
Layer 1 — Ejabberd XEP-0160 (offline storage):
User disconnects mid-session
Ejabberd stores pending stanzas
On reconnect: delivers immediately
Covers: short disconnections (seconds to minutes)
Layer 2 — RabbitMQ queue:
User has been offline longer
Spring Boot queues messages
On reconnect: Chat Service drains queue
Priority order: CRITICAL → IMPORTANT → NORMAL
Covers: hours to days offline
Layer 3 — FCM + Textfy (notifications):
Wakes device even when completely offline
User sees notification → opens app
Triggers Layer 1 + 2 delivery
Covers: device asleep, app killed
FCM for Calls (Special Case)
If Juma is offline when Kibuti calls:
Spring Boot sends FCM HIGH priority:
{
type: "INCOMING_CALL",
callId: "sid-abc-123",
callerName: "Kibuti Mwangi",
callerAvatar: "https://...",
callType: "VOICE",
turnCredentials: { ... } ← included for fast answer
}
│
FCM wakes Juma's phone
App shows full-screen incoming call UI
(even if app was completely killed)
│
Juma taps Answer:
App already has TURN credentials
Immediately creates PeerConnection
Sends Jingle session-accept
No extra round trip to get credentials
Faster answer time ✅
Call ringing timeout: 45 seconds
After 45s → Spring Boot marks: MISSED
→ Juma sees missed call notification
Catch-Up on Reconnect
User was offline — comes back online:
│
▼ WS connects → Ejabberd → auth success
Spring Boot receives: chat.presence.online (RabbitMQ)
│
Spring Boot:
Drain RabbitMQ offline queue for user
Check MAM (Message Archive) for any gaps
Build catch-up summary
│
App receives catch-up payload:
Missed messages pushed via WS
App shows banner:
"Umekosa ujumbe 12, maagizo 2"
[Angalia] button
Message Archive (MAM — XEP-0313):
Ejabberd stores last N days of messages
Client can query: "give me messages since X"
Covers edge cases where queue was lost
17. Multi Device
How Multiple Devices Work
Kibuti logged into:
kibuti@nexgate.com/android ← phone
kibuti@nexgate.com/tablet ← tablet
Message arrives:
Ejabberd delivers to BOTH devices
Both show the message
Both show notification
Kibuti reads on phone:
Phone sends: <displayed id="msg-001"/>
Ejabberd: sees kibuti read the message
XEP-0280 Message Carbons:
Tablet automatically receives the read marker
Tablet clears notification and marks read
Without user doing anything on tablet
This is how WhatsApp multi-device works
Ejabberd handles it natively via XEP-0280
Device Priority
If Kibuti active on phone + tablet:
Both receive messages (carbons)
If only one device active:
That device receives normally
Presence priority:
Each resource has a priority number
Higher priority = preferred delivery target
Phone: priority 10 (main device)
Tablet: priority 5 (secondary)
When both online: phone gets delivery first
Tablet gets carbon copy
Set in presence stanza:
<presence>
<priority>10</priority>
</presence>
18. Shop Inbox in Phase 2
Shop JID — The Shop as XMPP Entity
Each NexGate shop has its own JID:
techstore@shops.nexgate.com
This is NOT Kibuti's personal JID
This is the SHOP's identity
When customer messages TechStore:
Customer sends to: techstore@shops.nexgate.com
Any authorized staff member sees it
All staff respond AS techstore@shops.nexgate.com
Customer sees "TechStore" — not individual names
Staff authentication to shop JID:
Staff logs in with own account
Switches to shop context in app
Spring Boot issues shop XMPP sub-token:
{ jid: "techstore@shops.nexgate.com",
staffId: "usr-amina",
role: "SUPPORT_AGENT" }
Ejabberd allows staff to auth as shop JID
All messages from staff appear as TechStore
Multiple Staff — Shared Inbox
TechStore has 3 staff:
Kibuti (owner — Manager role)
Amina (Support Agent)
John (Support Agent)
Customer sends message to TechStore:
Message arrives at techstore@shops.nexgate.com
Ejabberd delivers to ALL connected TechStore staff
(All three see the incoming message simultaneously)
Amina responds:
Response appears as "TechStore" to customer
Spring Boot audit log:
{ messageId, respondedBy: "usr-amina",
shopId: "shop-techstore", timestamp }
Kibuti and John see Amina's response in their inbox too
(full shared inbox — everyone sees everything)
Benefits:
No missed customer messages
Any staff can pick up any conversation
Owner can monitor all conversations
Customer always talks to "TechStore"
19. Security
Transport Security
WebSocket:
wss:// (WebSocket Secure)
TLS 1.3 termination at Traefik
All chat traffic encrypted in transit
TURN relay:
SRTP (Secure Real-time Transport Protocol)
Voice/video encrypted even through Coturn
Coturn relays encrypted packets
Coturn cannot decrypt audio/video
XMPP tokens:
Short-lived (24 hours)
Signed with RS256 (asymmetric)
Separate from REST JWT
Stored in Vault
Internal Service Security
Ejabberd → Spring Boot:
X-Internal-Secret header
Secret stored in Vault
Only Ejabberd knows this secret
Spring Boot rejects any request without it
Spring Boot → Ejabberd:
Admin token (Ejabberd API key)
Stored in Vault
Port 5285 bound to 127.0.0.1 only
Not exposed to public internet
All inter-service secrets:
Stored in HashiCorp Vault ✅
Rotatable without restart
Never in environment files
Never in Docker Compose plain text
Message Privacy
Server-side:
Messages stored in PostgreSQL (encrypted at rest)
Media stored in MinIO (server-side encryption)
Shop conversations isolated from personal inbox
Staff cannot access personal DMs of owner
In transit:
WSS for all WebSocket traffic
SRTP for all call media
Future (E2E encryption):
Signal Protocol integration possible
Would use OMEMO (XEP-0384) on top of XMPP
Ejabberd supports OMEMO natively
Messages encrypted on device
Server stores ciphertext only
Not in Phase 2 scope — plan for Phase 3
20. Database Schema
conversations
conversations
─────────────────────────────────────────────
id UUID
type ENUM DM / GROUP / COMMERCE
owner_type ENUM USER / SHOP
owner_id UUID userId or shopId
title TEXT groups only
avatar_file_id UUID
status ENUM ACTIVE / ARCHIVED / BLOCKED
created_by UUID
created_at TIMESTAMPTZ
last_message_at TIMESTAMPTZ
last_message_preview TEXT
conversation_members
conversation_members
─────────────────────────────────────────────
conversation_id UUID
user_id UUID
role ENUM OWNER / ADMIN / MODERATOR / MEMBER
joined_at TIMESTAMPTZ
last_read_at TIMESTAMPTZ
last_read_seq BIGINT
is_muted BOOLEAN
muted_until TIMESTAMPTZ
notifications ENUM ALL / MENTIONS / NONE
messages
messages
─────────────────────────────────────────────
id UUID
conversation_id UUID
sender_id UUID
seq BIGINT monotonic per conversation
type ENUM TEXT / IMAGE / VIDEO /
VOICE_NOTE / FILE /
PRODUCT_CARD / CUSTOM_PRICE_OFFER /
EVENT_CARD / GROUP_PURCHASE_CARD /
POST_CARD / ORDER_CONFIRMATION /
ORDER_STATUS_UPDATE /
PAYMENT_CONFIRMATION / SYSTEM
body TEXT
media_ref UUID File Thunder fileId
context_type ENUM PRODUCT / ORDER / PAYMENT /
EVENT / GROUP_PURCHASE
context_ref_id UUID
snapshot_json JSONB frozen context at send time
reply_to_id UUID
status ENUM SENT / DELIVERED / READ / FAILED
level ENUM NORMAL / IMPORTANT / CRITICAL
edited_at TIMESTAMPTZ
deleted_at TIMESTAMPTZ
created_at TIMESTAMPTZ
message_receipts
message_receipts
─────────────────────────────────────────────
message_id UUID
user_id UUID
status ENUM DELIVERED / READ
device_id TEXT
timestamp TIMESTAMPTZ
calls
calls
─────────────────────────────────────────────
call_id UUID
caller_id UUID
receiver_id UUID
conversation_id UUID
type ENUM VOICE / VIDEO
status ENUM RINGING / CONNECTED /
COMPLETED / MISSED /
DECLINED / FAILED
started_at TIMESTAMPTZ
answered_at TIMESTAMPTZ
ended_at TIMESTAMPTZ
duration_seconds INT
relay_used BOOLEAN
end_reason ENUM NORMAL / NETWORK /
TIMEOUT / DECLINED
call_quality_logs
call_quality_logs
─────────────────────────────────────────────
log_id UUID
call_id UUID
timestamp TIMESTAMPTZ
bitrate_kbps INT
packet_loss_pct DECIMAL
jitter_ms INT
rtt_ms INT
resolution TEXT null for voice calls
codec_audio TEXT "opus"
codec_video TEXT "h264" "vp8" null
shop_conversation_access
shop_conversation_access
─────────────────────────────────────────────
shop_id UUID
user_id UUID
role ENUM MANAGER / SUPPORT_AGENT / READ_ONLY
granted_by UUID
granted_at TIMESTAMPTZ
revoked_at TIMESTAMPTZ
notification_log
notification_log
─────────────────────────────────────────────
id UUID
user_id UUID
message_id UUID
level ENUM NORMAL / IMPORTANT / CRITICAL
fcm_status ENUM SENT / DELIVERED / FAILED
sms_status ENUM SENT / DELIVERED / FAILED / SKIPPED
sms_provider TEXT
sent_at TIMESTAMPTZ
delivered_at TIMESTAMPTZ
opened_at TIMESTAMPTZ
Summary
Private chat and calls in NexGate Phase 2 are built on four pillars:
Ejabberd Cluster runs as two Docker containers on the same Hetzner VPS at launch. Erlang Distribution connects them directly — messages between nodes route in microseconds without Redis pub/sub. Traefik sticky sessions keep each user's WebSocket on one node. If one node crashes the other keeps serving. At growth stage two separate Hetzner VPS give true hardware redundancy.
Ejabberd handles everything real-time — WebSocket connections, XMPP stanza routing, presence, chat states, message receipts, MUC group chats, and Jingle call signaling. All message interactions (edit XEP-0308, delete XEP-0424, reactions XEP-0444, forwarding XEP-0297, replies XEP-0461) are routed automatically — Spring Boot only handles persistence and rule validation.
Group chats use a consent-based join model. Nobody enters a group without actively choosing. Two mechanisms: consent DM invitation (admin handpicks from their network, each person accepts or declines) and invite link (private groups require admin approval, public groups allow instant join). Both private and public group types supported. No forced adding — better than WhatsApp.
WebRTC handles all calls. 1:1 calls use P2P or Coturn relay via Jingle signaling. Group calls use LiveKit SFU (already deployed for Audio Spaces) — zero new infrastructure. Audio↔video switching uses Jingle content-add/remove without ending the session. Screen sharing uses MediaProjection (Android) and ReplayKit (iOS) as a special video track. Opus adapts from 64kbps to 6kbps. H.264 hardware acceleration keeps battery impact low on EA phones.
Spring Boot Chat Service handles all business logic — message persistence, commerce context, offer sessions, shop inbox access control, notification routing, and call records. Auth with Ejabberd is synchronous HTTP (needs immediate allow/deny). Everything else is async via RabbitMQ.
The shop inbox is isolated from personal DMs at the JID level — the shop has its own Ejabberd identity, multiple staff share it, and customers always see the shop brand, never individual staff names.
NexGate Private Chat & Calls — Phase 2 Deep Dive v1.0 QBIT SPARK | XMPP · Ejabberd · WebRTC · Jingle · Coturn · Opus · H.264 · Group Calls · Screen Share
NexGate — Private Chat & Calls Flow
Live Streaming Architecture
NexGate / QBIT SPARK | Version 1.0 SRS · HLS · LiveKit · VP Live Video · VP Audio Radio · VP Audio Spaces
Table of Contents
- Overview
- VP Live vs VP Audio — Key Differences
- How Live Streaming Works
- VP Live — Video Streaming
- VP Audio Radio — One Broadcaster Many Listeners
- VP Audio Spaces — Multi Speaker Rooms
- Live Chat — Ejabberd MUC
- Stream Key System
- File Thunder Integration — VOD After Stream
- Codecs & EA Network Strategy
- Docker Deployment
- Database Schema
- Scale Path
1. Overview
VP Live and VP Audio Spaces live under VP Feed — the social pillar of NexGate. They are not separate products. They are the live expression layer of the social platform — where creators, merchants, and communities connect with their audiences in real time.
VP Feed
┌───────────────────────────────────────────────────┐
│ │
│ Social Posts Stories Reels Live │
│ │
│ ┌─────────────────┐ │
│ │ VP Live │ │
│ │ Video Stream │ │
│ ├─────────────────┤ │
│ │ VP Audio │ │
│ │ Radio │ │
│ ├─────────────────┤ │
│ │ VP Audio │ │
│ │ Spaces │ │
│ └─────────────────┘ │
└───────────────────────────────────────────────────┘
All three modes share the same infrastructure foundation: SRS for ingest and transcoding, Cloudflare CDN for delivery, Ejabberd MUC for live chat, File Thunder for VOD processing, and Spring Boot for stream management and business logic.
2. VP Live vs VP Audio — Key Differences
VP Live VP Audio Radio VP Audio Spaces
(Video) (Radio/Podcast) (Twitter Spaces)
──────────────────────────────────────────────────────────────────────
Broadcasters 1 1 Multiple (up to 30)
Viewers Unlimited Unlimited Unlimited listeners
Direction One way One way Multi-speaker
Broadcaster RTMP RTMP audio WebRTC (LiveKit)
transport (video+audio) (audio only)
Listener HLS video HLS audio HLS audio
transport (adaptive) (adaptive) (listeners)
WebRTC (speakers)
Latency 6-15 seconds 6-15 seconds Speakers: <200ms
Listeners: 6-15s
Bandwidth High Very low Low (speakers)
broadcaster (2-4 Mbps) (128 kbps) Very low (listeners)
Bandwidth Medium Very low Very low
listener (300kbps-2Mbps) (32-128 kbps) (32-128 kbps)
Works on 2G? ❌ No ✅ Yes ✅ Listeners yes
Live chat Ejabberd MUC Ejabberd MUC Ejabberd MUC
Raise hand ❌ ❌ ✅
VOD after ✅ File Thunder ✅ File Thunder ✅ File Thunder
New infra SRS SRS SRS + LiveKit
3. How Live Streaming Works
The Core Pattern — RTMP → HLS → CDN
Broadcasting (sending):
Broadcaster's phone records camera + mic
App encodes: H.264 video + AAC audio
App streams via RTMP protocol to SRS server
One stream upload from broadcaster
Processing (server):
SRS receives RTMP stream
FFmpeg transcodes to multiple quality variants
Packages into HLS format (2-second chunks)
Writes chunks to MinIO storage every 2 seconds
Delivery (viewing):
Cloudflare CDN pulls chunks from MinIO
Caches chunks at edge nodes globally
Viewers request HLS playlist → adaptive player picks quality
10,000 viewers = 10,000 CDN requests, NOT 10,000 SRS requests
SRS barely notices the viewer count
Why HLS and not WebRTC for viewers:
WebRTC to viewers: broadcaster uploads N streams (one per viewer)
HLS via CDN: broadcaster uploads 1 stream → CDN serves all
At 10,000 viewers: WebRTC = impossible, HLS = trivial
HLS — What It Actually Is
HLS (HTTP Live Streaming) — Apple's open standard
SRS generates:
master.m3u8 → playlist of all quality variants
360p/playlist.m3u8 → playlist for 360p variant
360p/seg_000.ts → 2-second video chunk
360p/seg_001.ts → next 2-second chunk
720p/playlist.m3u8
720p/seg_000.ts
...
master.m3u8 looks like:
#EXTM3U
#EXT-X-STREAM-INF:BANDWIDTH=400000,RESOLUTION=640x360
360p/playlist.m3u8
#EXT-X-STREAM-INF:BANDWIDTH=1500000,RESOLUTION=1280x720
720p/playlist.m3u8
Player (ExoPlayer / AVPlayer):
Downloads master.m3u8 first
Measures current network speed
Picks 360p if on 3G → plays seg_000.ts → seg_001.ts → ...
Switches to 720p if network improves → seamless
All automatic — zero app code needed for quality switching
4. VP Live — Video Streaming
Full Architecture
[Broadcaster Phone]
│
│ RTMP stream
│ rtmp://stream.nexgate.com/live/{streamKey}
│ H.264 video + AAC audio
│ ~2-4 Mbps upload
▼
[SRS Media Server]
│
├── Validates stream key:
│ POST /internal/stream/validate
│ { streamKey: "abc123" }
│ Spring Boot: ✅ allow or ❌ reject
│
├── Receives raw RTMP stream
│
├── FFmpeg transcoding (real-time):
│ 1080p H.264 → 3 Mbps (WiFi viewers)
│ 720p H.264 → 1.5 Mbps (4G viewers)
│ 480p H.264 → 600 kbps (3G viewers)
│ 360p H.264 → 300 kbps (2G viewers)
│
├── Package as HLS:
│ Segment every 2 seconds
│ live/{streamKey}/master.m3u8
│ live/{streamKey}/360p/seg_NNN.ts
│ live/{streamKey}/720p/seg_NNN.ts
│
└── Write to MinIO: nexgate-live bucket
New segments every 2 seconds
│
▼
[Cloudflare CDN]
│ Pulls from MinIO automatically
│ Caches at edge (Nairobi edge closest to EA)
│ Short TTL: 10 seconds (live content)
│
▼
[Viewers — ExoPlayer (Android) / AVPlayer (iOS)]
Requests master.m3u8
Player picks quality based on network
Downloads .ts segments every 2 seconds
Seamless adaptive quality switching
Stream Key Validation Flow
Broadcaster taps "Go Live" in app
│
▼ POST /live/start
Spring Boot:
Generate unique stream key
Store in DB:
stream_key: "abc123"
user_id: usr-kibuti
status: PENDING
created_at: now
Return stream key to app
│
App connects RTMP:
rtmp://stream.nexgate.com/live/abc123
│
SRS receives connection
│
▼ POST /internal/stream/validate (SRS webhook)
Spring Boot checks:
Key exists? ✅
User account active? ✅
User has live permission? ✅
No other active stream for this user? ✅
→ 200 OK → SRS allows stream
→ Update DB: status: LIVE, started_at: now
→ Notify followers via FCM:
"Kibuti is live now! Watch here"
→ Create Ejabberd MUC room:
live-abc123@conference.nexgate.com
Broadcaster App — What Mobile Dev Implements
Android library: rtmp-rtsp-stream-client-java
iOS library: HaishinKit (Swift)
Steps for broadcaster app:
1. GET /live/start → receive stream key
2. Initialize camera + microphone
3. Connect RTMP to stream.nexgate.com/live/{key}
4. Start streaming — library handles everything:
H.264 encoding (hardware)
AAC audio encoding
RTMP packet framing
Network reconnection on drop
5. Show: viewer count (from Redis via REST poll)
live comments (from Ejabberd MUC via WS)
duration timer
6. Tap End → POST /live/end → cleanup
Adaptive upload bitrate:
Library monitors upload speed
Reduces video quality if upload struggles
Broadcaster's bad network → lower quality for viewers
Never drops stream if avoidable
Viewer App — What Mobile Dev Implements
Android: ExoPlayer (Google's official video player)
iOS: AVPlayer (built into iOS, zero setup)
Steps for viewer app:
1. GET /live/{streamId}/url
Response: { masterUrl, viewerCount, startedAt }
2. Feed masterUrl to ExoPlayer/AVPlayer
3. Player handles everything automatically:
Downloads master.m3u8
Picks quality based on network
Downloads segments every 2s
Switches quality up/down seamlessly
4. Join Ejabberd MUC room → show live comments
5. Player shows: loading → buffering → playing
That is genuinely all the viewer needs to implement.
HLS + ExoPlayer/AVPlayer is the easiest viewer experience
to build in all of mobile development.
5. VP Audio Radio — One Broadcaster Many Listeners
Why Audio Radio Matters for EA
VP Live video:
Broadcaster needs: 2-4 Mbps upload
Viewer needs: 300kbps minimum
Data cost viewer: ~900MB per hour at 360p
Works on: 4G and strong 3G only
VP Audio Radio:
Broadcaster needs: 64-128 kbps upload
Listener needs: 32 kbps minimum
Data cost listener: ~15MB per hour at 32kbps
Works on: 2G, Edge, any connection
For a farmer in rural Tanzania with 2G:
VP Live video → impossible, too expensive
VP Audio Radio → accessible, affordable
Use cases:
Live podcast / commentary
Religious broadcasts (huge in EA)
Political discussions
Community announcements
Sports commentary
Language learning sessions
Business webinars (audio only)
Architecture — Same SRS, Audio Only
[Broadcaster Phone]
│
│ RTMP audio only (no video track)
│ AAC codec, 128 kbps
│ rtmp://stream.nexgate.com/audio/{streamKey}
▼
[SRS Media Server]
│
├── Same validation flow as VP Live
│
├── FFmpeg transcoding (audio only):
│ AAC 128 kbps → good network listeners
│ AAC 64 kbps → 3G listeners
│ AAC 32 kbps → 2G listeners
│
├── Package as HLS audio:
│ audio/{streamKey}/master.m3u8
│ audio/{streamKey}/128k/seg_NNN.aac
│ audio/{streamKey}/32k/seg_NNN.aac
│
└── Write to MinIO: nexgate-live bucket
│
▼
[Cloudflare CDN]
│
▼
[Listeners — ExoPlayer / AVPlayer]
HLS audio playlist
Adaptive bitrate: 128k → 32k automatically
Same player, same code — just no video surface
Codec Choice — AAC Not Opus
Why AAC for HLS audio radio (not Opus):
Opus is better quality at low bitrates — true
But HLS has a compatibility requirement:
Apple mandates AAC for HLS audio
AVPlayer on iOS does not support Opus in HLS
Using Opus → iOS listeners cannot play
AAC → works on every device, every OS
Opus is used for:
Voice calls (WebRTC — different transport)
Voice notes (file-based, not streaming)
AAC is used for:
VP Live audio track (in video stream)
VP Audio Radio (HLS streaming)
VP Audio Spaces listener HLS output
AAC at 32kbps for EA:
Acceptable speech quality
~15MB per hour
Works on any 2G connection
Universal device support
6. VP Audio Spaces — Multi Speaker Rooms
The Concept
Not one broadcaster → many listeners
Multiple people in a shared audio room
Some speak, many listen
Listeners can raise their hand to speak
Host controls who gets the mic
Like Twitter Spaces, Clubhouse, Discord Stage Channels
Key insight:
Speakers need LOW LATENCY (<200ms)
to have a natural conversation
HLS (6-15s delay) is too slow for speakers
Listeners just need to HEAR clearly
HLS delay is fine — they're not responding
HLS scales to millions via CDN
Solution: TWO transport layers in one room
Speakers → WebRTC (LiveKit SFU) → <200ms
Listeners → HLS via CDN → 6-15s delay → millions scale
LiveKit SFU — What It Is
SFU = Selective Forwarding Unit
Traditional conference (MCU):
Server mixes ALL audio into one stream
Sends mixed stream to everyone
High CPU (server does all mixing)
Simple client
LiveKit SFU approach:
Each speaker sends audio once to LiveKit
LiveKit forwards each speaker's stream
to all other speakers
Speakers' apps mix locally (device CPU)
Much lower server CPU
Lower latency
Better quality (no mixing artifacts)
For listeners:
LiveKit outputs a mixed HLS stream
Goes through SRS → Cloudflare CDN
Listeners get one mixed audio stream
Same HLS pattern as Audio Radio
Who built LiveKit:
The same team that built Twitter Spaces
Then open sourced it
Actively maintained, Docker ready
Official Android + iOS SDKs available
LiveKit serves TWO purposes in NexGate:
1. VP Audio Spaces (multi-speaker rooms)
2. Group voice + video calls (Phase 2)
Same Docker container
Same Coturn relay reused
Zero extra infrastructure for group calls
Full Architecture
[Speaker A phone] ──WebRTC──▶┐
[Speaker B phone] ──WebRTC──▶│
[Speaker C phone] ──WebRTC──▶│
▼
[LiveKit SFU]
│
┌──────────┼──────────────┐
│ │ │
WebRTC fwd HLS output Room events
to speakers (mixed audio) to Spring Boot
│ │
[Speakers [SRS receives
hear each HLS from LiveKit]
other live] │
▼
[Cloudflare CDN]
│
▼
[Thousands of listeners
via HLS audio player]
ExoPlayer / AVPlayer
(same as Audio Radio)
Room events (raise hand, join, leave):
LiveKit → Spring Boot via webhook
Spring Boot → Ejabberd MUC → all participants
Ejabberd MUC → Listeners also see events
(who joined as speaker etc)
Raise Hand Flow
Listener wants to speak:
│ taps "Raise Hand" 🖐
│ sends via Ejabberd WS to MUC room:
│ { type: RAISE_HAND, roomId: "space-abc" }
│
▼
Spring Boot:
Records raise hand request
Notifies host via Ejabberd WS:
{ type: HAND_RAISED, userId, displayName }
Host sees list of raised hands in UI
│
Host taps "Allow to speak" on a listener:
│
▼
Spring Boot:
Calls LiveKit API:
Update participant permissions:
canPublish: true ← now allowed to send audio
Generate new LiveKit token for this user
(speaker token, not listener token)
Send token to user via Ejabberd WS:
{ type: SPEAKER_PROMOTED, livekitToken: "..." }
│
Former listener's app:
Receives promotion event
Stops HLS player (was listening at 15s delay)
Connects WebRTC to LiveKit with speaker token
Starts sending audio
Now hears speakers at <200ms latency
Other speakers hear them immediately
│
Host can also:
Lower someone's hand (dismiss)
Mute a specific speaker
Remove speaker (back to listener)
End the space entirely
Speaker vs Listener — Connection Types
┌──────────────────────────────────────────────────────┐
│ Audio Space Room │
│ │
│ Speakers (up to ~20-30): │
│ Connected via WebRTC to LiveKit │
│ Send and receive audio streams │
│ Latency: <200ms (real conversation) │
│ Connection: persistent WebRTC │
│ │
│ Listeners (unlimited): │
│ Connected via HLS to Cloudflare CDN │
│ Receive mixed audio only │
│ Latency: 6-15 seconds (fine — just listening) │
│ Connection: HTTP requests every 2s │
│ Scale: millions — CDN handles it │
│ │
│ All participants: │
│ Connected to Ejabberd MUC room │
│ Text chat, reactions, raise hand events │
│ Room membership awareness │
└──────────────────────────────────────────────────────┘
LiveKit Token System
Spring Boot manages all LiveKit tokens
(LiveKit has official Java SDK)
Host token:
canPublish: true
canSubscribe: true
roomAdmin: true
→ full control, can speak, manage
Speaker token:
canPublish: true
canSubscribe: true
roomAdmin: false
→ can speak, cannot manage room
Listener token:
canPublish: false ← cannot send audio
canSubscribe: true ← can hear speakers
roomAdmin: false
→ receive only
Token generation:
GET /audio-spaces/{spaceId}/join
Spring Boot checks:
Is user the host? → host token
Is user an approved speaker? → speaker token
Otherwise → listener token (gets HLS URL instead)
LiveKit Docker Config
livekit:
image: livekit/livekit-server:latest
container_name: livekit
restart: unless-stopped
ports:
- "7880:7880" # HTTP API (Spring Boot calls here)
- "7881:7881" # WebRTC TCP
- "7882:7882/udp" # WebRTC UDP (primary)
- "50000-60000:50000-60000/udp" # ICE relay ports
volumes:
- ./livekit/livekit.yaml:/etc/livekit.yaml
command: --config /etc/livekit.yaml
# livekit.yaml
port: 7880
rtc:
tcp_port: 7881
udp_port: 7882
use_external_ip: true
redis:
address: redis:6379 # reuses existing Redis ✅
turn:
enabled: true
domain: turn.nexgate.com
tls_port: 5349
credential: "${COTURN_SECRET}" # reuses existing Coturn ✅
room:
max_participants: 10000
empty_timeout: 300
LiveKit reuses:
Redis → already deployed ✅
Coturn → already deployed for calls ✅
No new infrastructure beyond LiveKit container itself
7. Live Chat — Ejabberd MUC
All three live modes (VP Live, Audio Radio, Audio Spaces) use Ejabberd MUC rooms for real-time text interaction.
Room Lifecycle
Stream / space starts:
│
Spring Boot → Ejabberd REST API:
POST /api/create_room
{
name: "live-{streamId}",
service: "conference.nexgate.com"
}
Room created: live-abc@conference.nexgate.com
│
Broadcaster / host auto-joined as moderator
│
Viewers / listeners join room as participants:
App connects Ejabberd WS
Sends MUC join stanza:
<presence to="live-abc@conference.nexgate.com/Kibuti">
<x xmlns="http://jabber.org/protocol/muc"/>
</presence>
│
Comments sent as MUC messages:
<message to="live-abc@conference.nexgate.com"
type="groupchat">
<body>Looking great! 🔥</body>
</message>
│
All room members receive instantly
No delay — Ejabberd MUC is real-time
│
Stream / space ends:
Spring Boot → Ejabberd REST API:
POST /api/destroy_room
{ name: "live-abc", service: "conference.nexgate.com" }
Room destroyed, members disconnected
Special Events in Live Chat
Beyond text comments, the MUC room carries:
Reactions (emoji bursts):
{ type: REACTION, emoji: "🔥", userId, displayName }
Client renders floating emoji animation
Gifts:
{ type: GIFT, giftId, giftName, amount, userId, displayName }
Client renders gift animation
Spring Boot processes payment separately
Raise hand (Audio Spaces only):
{ type: RAISE_HAND, userId, displayName }
Host sees in management panel
Speaker promoted (Audio Spaces only):
{ type: SPEAKER_PROMOTED, userId, displayName }
All participants see "Amina joined as speaker"
Viewer count updates:
Broadcast every 30 seconds from Spring Boot
{ type: VIEWER_COUNT, count: 12453 }
Product card dropped by broadcaster:
{ type: PRODUCT_CARD, productId, name, price }
Viewers tap → go to VP Shop product page
Commerce during live ✅
Viewer / Listener Count
Two sources of truth:
1. Ejabberd MUC occupant count:
GET ejabberd REST /api/get_room_occupants_count
{ room: "live-abc", host: "conference.nexgate.com" }
→ exact WebSocket-connected count
2. Redis counter (includes HLS-only listeners):
INCR live:{streamId}:viewers → on HLS playlist request
DECR → on playlist stop / timeout
More accurate for Audio Radio/Spaces
where many listeners never connect WS
Display count = Redis counter (higher, more accurate)
Spring Boot broadcasts to MUC every 30 seconds
8. Stream Key System
Stream Key Design
Stream key = single-use authentication token
Broadcaster uses it to connect RTMP to SRS
SRS validates with Spring Boot before accepting stream
Format: random 32-character alphanumeric string
Example: nx_live_a1b2c3d4e5f6g7h8i9j0k1l2m3n4
Lifecycle:
PENDING → generated, not yet used
LIVE → broadcaster connected, stream active
ENDED → stream finished normally
EXPIRED → generated but never used (24h TTL)
REVOKED → manually stopped by admin
One active stream per user at a time
Attempting second stream → rejected by Spring Boot validation
SRS Webhooks to Spring Boot
SRS fires these events to Spring Boot:
on_publish → broadcaster connected RTMP
Spring Boot: validate key, update status LIVE,
notify followers FCM,
create Ejabberd MUC room,
create LiveKit room (if audio space)
on_unpublish → broadcaster disconnected
Spring Boot: update status ENDED,
trigger File Thunder for VOD,
destroy Ejabberd MUC room,
log stream duration + peak viewers
on_play → viewer started watching HLS
Spring Boot: increment Redis viewer counter
on_stop → viewer stopped watching
Spring Boot: decrement Redis viewer counter
9. File Thunder Integration — VOD After Stream
What Happens After Stream Ends
Stream ends (broadcaster taps End / disconnects)
│
SRS fires on_unpublish webhook
│
Spring Boot:
Update stream record: status ENDED
Trigger File Thunder for VOD processing
SRS has saved full recording as .mp4
│
▼
Spring Boot → File Thunder:
POST /api/v1/upload/request (HMAC signed)
{
ownerId: broadcasterId,
domain: POSTS,
context: LIVE_RECORDING,
filename: "stream_{streamId}.mp4",
mimeType: "video/mp4"
}
Returns: presigned MinIO PUT URL
│
Spring Boot pulls recording from SRS
Uploads to MinIO via presigned URL
POST /api/v1/confirm { fileId }
│
▼
File Thunder VideoWheel processes:
HLS transcoding (all quality variants)
Thumbnail extraction (best frame detection)
Watermark: "@{broadcasterUsername}"
NO outro — live recordings are long
NO shortClip — full stream only
Store in nexgate-public bucket
│
▼
File Thunder fires webhook: media ready
Spring Boot:
Creates VOD post on broadcaster's profile
"Watch replay" button appears
Appears in VP Feed for followers
Stream record linked to VOD fileId
New File Thunder Contexts for Live
Existing contexts (unchanged):
SOCIAL_VIDEO regular video posts
DM_ATTACHMENT files sent in DMs
DIGITAL_PRODUCT digital goods in VP Shop
...
New contexts added for live:
LIVE_RECORDING full stream VOD
VideoWheel — no outro, no shortClip
always HLS, always long
AUDIO_RECORDING audio space / radio recording
AudioWheel processes
outputs: .m4a (AAC)
podcast episode on profile
waveform extracted (like voice notes)
nexgate-live MinIO Bucket
Existing buckets:
nexgate-raw temp uploads
nexgate-public social content
nexgate-private DMs and private files
nexgate-digital VP Shop digital products
New bucket:
nexgate-live live stream segments only
Why separate:
SRS writes directly here (not via File Thunder)
Short TTL segments — deleted after stream ends + VOD ready
Different CDN caching rules (10s TTL vs 1 year for VOD)
Different access pattern (SRS writes, CDN reads)
Easy to monitor storage growth separately
Lifecycle:
Stream starts → SRS creates live/{streamKey}/ folder
During stream → .ts segments written every 2 seconds
Stream ends → Spring Boot schedules cleanup job
VOD confirmed → delete nexgate-live/{streamKey}/ folder
Total life: stream duration + ~1 hour buffer
10. Codecs & EA Network Strategy
VP Live Video Codecs
Broadcaster encoding (phone → SRS):
Video: H.264 (hardware encoder — mandatory)
Software H.264 too slow for real-time on phones
H.264 hardware support: every phone since 2013
Audio: AAC 128kbps (RTMP standard)
Container: RTMP (streaming protocol)
SRS transcoding (server-side):
Receives H.264 + AAC
Transcodes to HLS quality ladder:
Quality Video bitrate Audio Resolution EA target
─────────────────────────────────────────────────────────
1080p 3 Mbps 128k 1920×1080 WiFi only
720p 1.5 Mbps 128k 1280×720 4G
480p 600 kbps 64k 854×480 3G
360p 300 kbps 48k 640×360 2G minimum
─────────────────────────────────────────────────────────
ExoPlayer/AVPlayer auto-selects based on network
VP Audio Codecs
Audio Radio (broadcaster → SRS):
Codec: AAC 128kbps
Container: RTMP audio only
Audio Radio (SRS → HLS):
128kbps → WiFi/4G listeners
64kbps → 3G listeners
32kbps → 2G listeners (15MB/hour — affordable)
Audio Spaces (speaker → LiveKit):
Codec: Opus (WebRTC standard)
Adaptive: 32-64kbps per speaker
Echo cancellation: mandatory (multiple people)
Noise suppression: mandatory (EA background noise)
Audio Spaces (LiveKit → HLS for listeners):
LiveKit mixes speaker streams
Outputs mixed audio → SRS → HLS
Same AAC ladder as Audio Radio
Listeners hear all speakers in one stream
Adaptive Streaming — EA Principle
The player always knows the network speed
because it measures how fast segments download
Segment download faster than playback → upgrade quality
Segment download slower than playback → downgrade quality
For a viewer in Dodoma on shaky 3G:
Opens stream → starts at 360p (safe default)
Network good → player tries 480p
Stays stable → tries 720p
Network drops → immediately back to 360p
No rebuffering if switch is fast enough
Buffer strategy:
Player buffers 3-4 segments ahead (6-8 seconds)
Gives time to switch quality before buffer empties
Viewer may notice brief quality dip — never a freeze
NexGate player config recommendation:
Min buffer: 6 seconds
Max buffer: 30 seconds
Quality switch: aggressive downgrade, conservative upgrade
→ Prioritize uninterrupted playback over quality
→ EA networks fluctuate — better to be at 360p than buffering
11. Docker Deployment
Full docker-compose for Live Features
# SRS Media Server
srs:
image: ossrs/srs:5
container_name: srs
restart: unless-stopped
ports:
- "1935:1935" # RTMP ingest (broadcaster connects here)
- "8080:8080" # HTTP API + HLS output
- "1985:1985" # SRS management API
volumes:
- ./srs/srs.conf:/usr/local/srs/conf/srs.conf
- ./srs/logs:/usr/local/srs/logs
- ./srs/recordings:/usr/local/srs/objs/recordings
depends_on:
- chat-service
networks:
- nexgate-internal
# LiveKit SFU (Audio Spaces)
livekit:
image: livekit/livekit-server:latest
container_name: livekit
restart: unless-stopped
ports:
- "7880:7880"
- "7881:7881"
- "7882:7882/udp"
- "50000-60000:50000-60000/udp"
volumes:
- ./livekit/livekit.yaml:/etc/livekit.yaml
command: --config /etc/livekit.yaml
depends_on:
- redis
networks:
- nexgate-internal
SRS Config Highlights
listen 1935; # RTMP port
max_connections 1000;
vhost __defaultVhost__ {
# Validate stream key with Spring Boot
http_hooks {
enabled on;
on_publish http://chat-service:8082/internal/stream/validate;
on_unpublish http://chat-service:8082/internal/stream/ended;
on_play http://chat-service:8082/internal/stream/viewer-join;
on_stop http://chat-service:8082/internal/stream/viewer-leave;
}
# HLS output for viewers
hls {
enabled on;
hls_path ./objs/nginx/html;
hls_fragment 2; # 2 second chunks
hls_window 10; # keep last 10 chunks in playlist
}
# FFmpeg transcoding to multiple qualities
transcode {
enabled on;
ffmpeg /usr/bin/ffmpeg;
engine 360p {
enabled on;
vcodec libx264;
vbitrate 300;
vfps 15;
vwidth 640;
vheight 360;
acodec aac;
abitrate 48;
output rtmp://localhost:1935/live360p/{stream};
}
engine 720p {
enabled on;
vcodec libx264;
vbitrate 1500;
vfps 30;
vwidth 1280;
vheight 720;
acodec aac;
abitrate 128;
output rtmp://localhost:1935/live720p/{stream};
}
}
}
Traefik — RTMP Does Not Go Through Traefik
Important: RTMP is TCP port 1935
Traefik handles HTTP/HTTPS only
RTMP port 1935 exposed directly on VPS
What Traefik does handle:
stream.nexgate.com → SRS port 8080 (HLS output)
TLS termination for HLS delivery
RTMP broadcaster connects:
rtmp://stream.nexgate.com:1935/live/{key}
No TLS on RTMP (RTMPS is complex, not needed for launch)
HLS viewers connect via Cloudflare CDN:
https://cdn.nexgate.com/live/{key}/master.m3u8
Cloudflare pulls from SRS port 8080
Traefik handles TLS for this path
12. Database Schema
live_streams
live_streams
─────────────────────────────────────────────
stream_id UUID PK
broadcaster_id UUID FK → users
type ENUM VIDEO / AUDIO_RADIO / AUDIO_SPACE
title TEXT
description TEXT
cover_file_id UUID File Thunder fileId (stream thumbnail)
stream_key TEXT UNIQUE, used for RTMP auth
status ENUM PENDING / LIVE / ENDED / EXPIRED / REVOKED
started_at TIMESTAMPTZ
ended_at TIMESTAMPTZ
duration_seconds INT
peak_viewers INT
total_viewers INT
muc_room_id TEXT Ejabberd MUC room name
vod_file_id UUID File Thunder fileId after processing
created_at TIMESTAMPTZ
audio_spaces
audio_spaces
─────────────────────────────────────────────
space_id UUID PK
stream_id UUID FK → live_streams
livekit_room_id TEXT LiveKit room name
host_id UUID FK → users
title TEXT
status ENUM SCHEDULED / LIVE / ENDED
max_speakers INT default 30
started_at TIMESTAMPTZ
ended_at TIMESTAMPTZ
audio_space_participants
audio_space_participants
─────────────────────────────────────────────
space_id UUID FK → audio_spaces
user_id UUID
role ENUM HOST / SPEAKER / LISTENER
joined_at TIMESTAMPTZ
left_at TIMESTAMPTZ
hand_raised_at TIMESTAMPTZ
promoted_at TIMESTAMPTZ when promoted from listener to speaker
promoted_by UUID host who approved
stream_viewer_stats
stream_viewer_stats
─────────────────────────────────────────────
stat_id UUID PK
stream_id UUID FK → live_streams
timestamp TIMESTAMPTZ
viewer_count INT
quality_360p_pct DECIMAL % of viewers on 360p
quality_720p_pct DECIMAL % of viewers on 720p
avg_watch_seconds INT
13. Scale Path
Current Architecture Limits
Single SRS node (Hetzner CPX31 — €19/month):
Concurrent streams: ~200 (with transcoding)
Concurrent viewers: ~50,000 (before CDN helps)
Bandwidth: 20TB/month included
With Cloudflare CDN:
Concurrent viewers: Unlimited (CDN absorbs it)
SRS only serves cache misses
99%+ cache hit rate → SRS barely loaded
LiveKit single node:
Concurrent spaces: ~500
Speakers per space: up to 30
Listeners per space: Unlimited (HLS via CDN)
This is enough for NexGate launch and
strong early growth — tens of thousands of users
Growth Stage — SRS Horizontal Scale
When 200 concurrent streams is not enough:
SRS Origin node:
Receives RTMP from all broadcasters
Passes stream to Transcode Farm
Transcode Farm (2-3 nodes):
Each node handles FFmpeg transcoding
Horizontal — add nodes as streams grow
CPU-bound work distributed
SRS Edge nodes:
Serve HLS to viewers
Pull from Origin
Multiple edges → load distributed
┌──────────────────────────────────────────┐
│ Broadcaster → SRS Origin │
│ │ │
│ Transcode Farm │
│ (3 nodes, FFmpeg) │
│ │ │
│ ┌──────────┴──────────┐ │
│ SRS Edge 1 SRS Edge 2 │
│ │ │ │
│ Cloudflare CDN ────────────┘ │
│ │ │
│ All viewers (millions) │
└──────────────────────────────────────────┘
WeChat EA Scale — Infrastructure
Broadcaster latency problem:
Current: Broadcaster in Dar → stream goes to Hetzner Germany
150-300ms upload latency
Acceptable but not ideal
At scale: SRS nodes in EA region
Google Cloud Johannesburg
OR AWS Cape Town
Broadcaster → nearby SRS → low latency upload
Better broadcaster experience
Storage cost at scale:
Current: MinIO on Hetzner
At scale: Cloudflare R2
Zero egress cost (unlike AWS S3 which charges per GB)
S3 compatible → zero code change to migrate
At millions of viewer-hours: massive cost saving
Transcoding cost:
CPU-heavy work
At scale: GPU-accelerated FFmpeg nodes
NVIDIA hardware encoding
5-10x faster than CPU
Lower cost per stream transcoded
Summary
VP Live and VP Audio Spaces are the live social layer of NexGate — living under VP Feed alongside regular posts, stories, and reels.
All three modes (VP Live video, VP Audio Radio, VP Audio Spaces) share the same infrastructure foundation. SRS handles all RTMP ingest and transcoding. Cloudflare CDN distributes HLS to unlimited viewers and listeners. Ejabberd MUC powers live chat and room events for all modes. File Thunder processes every stream into a VOD after it ends. Spring Boot manages stream keys, webhooks, room lifecycle, and all business logic.
VP Audio Spaces adds LiveKit SFU for the multi-speaker experience — speakers connect via WebRTC for real-time conversation while listeners receive the same HLS audio stream that Audio Radio uses, scaled to millions via Cloudflare CDN. LiveKit also serves double duty as the infrastructure for group voice and video calls — the same Docker container, the same Coturn relay, zero additional infrastructure needed.
The EA network strategy is woven into every decision: HLS adaptive streaming down to 32kbps means VP Audio Radio works on 2G in rural Tanzania. Video quality ladders from 1080p to 360p ensure VP Live is accessible on 3G. The entire viewer and listener experience requires only ExoPlayer or AVPlayer — the simplest possible mobile integration.
For VOD, File Thunder's VideoWheel and new AudioWheel process every recording automatically after the stream ends — creating replay content with thumbnails, watermarks, and adaptive variants, stored in nexgate-public for CDN delivery. The live platform generates permanent content with zero extra work.
NexGate VP Live & VP Audio Spaces — Architecture v1.0 QBIT SPARK | SRS · LiveKit · HLS · Ejabberd MUC · File Thunder VOD · Group Calls
NexGate — Private Chat & Calls Flow
XMPP From Zero to Production Book
A Complete Developer's Guide
Build real-time messaging that survives 2G — from your first stanza to a clustered Ejabberd deployment.
Running example throughout: MaasaiChat, a chat app for Maasai communities across rural Tanzania and Kenya — 2G/3G links, low-end Android phones, connections that drop constantly. Every technical decision in this book is shaped by that reality.
First Edition · 2026 Written for backend engineers, mobile developers, and architects who need to own their messaging stack.
No prior XMPP knowledge assumed. By the final chapter you will have deployed a production Ejabberd server, implemented every XEP MaasaiChat needs on Android and iOS, and hardened it for real traffic.
How to use this book
- Part 1 teaches the protocol from absolute zero. Read it in order.
- Part 2 is a reference — one chapter per XEP, all built on the same six-part template. Read the Core and Messaging chapters, then dip into the rest as you need them.
- Part 3 is hands-on Ejabberd. Keep a terminal open.
- Part 4 is the mobile SDK guide (Android Smack, iOS XMPPFramework).
- Part 5 is production: security, database, scaling, network tuning, pitfalls.
Every XEP chapter answers the same six questions in the same order:
1. What problem does it solve?
2. The XML stanzas, explained line by line
3. A real MaasaiChat example
4. How Ejabberd handles it
5. What the mobile dev implements
6. What the backend dev implements
Every chapter ends with a What you learned box and a checkpoint before the next one.
Conventions: terminal commands are shown in fenced blocks, XML stanzas are complete and copy-pasteable, and the four MaasaiChat users appear in every example:
ole@maasaichat.com Ole Saitoti — elder
naserian@maasaichat.com Naserian — young warrior
enkiama@maasaichat.com Enkiama — village chief
nkeri@maasaichat.com Nkeri — cattle trader
Table of Contents
Part 1 — Foundation
1 What is XMPP?
2 The XMPP Vocabulary — Key Terms & Definitions
3 XML Basics — Streams, Stanzas & Nonzas
4 The JID Address System
5 The Three Stanzas
6 Namespaces & XEPs Explained
7 The XMPP Stream Lifecycle
Part 2 — XEPs Complete Reference
Core
8 XEP-0030 Service Discovery
9 XEP-0115 Entity Capabilities
10 XEP-0199 XMPP Ping
11 XEP-0198 Stream Management (critical for 2G/3G)
12 XEP-0280 Message Carbons
13 XEP-0313 Message Archive Management (MAM)
Messaging
14 XEP-0085 Chat State Notifications
15 XEP-0184 Message Delivery Receipts
16 XEP-0333 Chat Markers
17 XEP-0308 Last Message Correction
18 XEP-0424 Message Retraction
19 XEP-0444 Message Reactions
20 XEP-0297 Stanza Forwarding
21 XEP-0461 Message Replies
22 XEP-0359 Stable & Unique Stanza IDs
23 XEP-0334 Message Processing Hints
Group Chat
24 XEP-0045 Multi-User Chat (MUC)
25 XEP-0249 Direct MUC Invitations
26 XEP-0317 Hats
27 XEP-0425 Message Moderation
28 XEP-0490 Message Displayed Synchronization
File & Media
29 XEP-0363 HTTP File Upload (the one you'll actually use)
30 XEP-0065 SOCKS5 Bytestreams
31 XEP-0234 Jingle File Transfer
32 XEP-0264 Jingle Content Thumbnails
Calls
33 XEP-0166 Jingle
34 XEP-0167 Jingle RTP Sessions
35 XEP-0176 Jingle ICE-UDP Transport
36 XEP-0177 Jingle Raw UDP Transport
37 XEP-0215 External Service Discovery (TURN/STUN credentials)
38 XEP-0320 DTLS-SRTP in Jingle
Push & Notifications
39 XEP-0357 Push Notifications (FCM on Android)
Security & Encryption
40 XEP-0384 OMEMO Encryption
41 XEP-0388 Extensible SASL Profile
42 XEP-0440 SASL Channel Binding
Presence & Roster
43 XEP-0054 vcard-temp
44 XEP-0153 vCard-Based Avatars
45 XEP-0292 vCard4 Over XMPP
46 XEP-0083 Nested Roster Groups
47 XEP-0144 Roster Item Exchange
PubSub
48 XEP-0060 Publish-Subscribe
49 XEP-0163 Personal Eventing Protocol (PEP)
History & Archive
50 XEP-0059 Result Set Management
51 XEP-0313 MAM in depth (querying, paging)
52 XEP-0430 Inbox
Enterprise
53 XEP-0050 Ad-Hoc Commands
54 XEP-0004 Data Forms
55 XEP-0055 Jabber Search
56 XEP-0077 In-Band Registration
57 XEP-0133 Service Administration
Federation
58 XEP-0220 Server Dialback
59 XEP-0288 Bidirectional Server-to-Server
Part 3 — Ejabberd in Practice
60 Architecture & the Erlang/BEAM Foundation
61 Docker Setup, Step by Step
62 ejabberd.yml — The Complete Guide
63 ejabberdctl — Every Command You'll Use
64 REST API — Complete Reference
65 OAuth Authentication
66 The Auth Bridge (HTTP auth → your backend as gatekeeper)
67 Clustering Two Nodes
68 MUC Administration
69 Monitoring & Logging
Part 4 — Mobile SDK Guide
70 Android with Smack — connect, auth, send/receive, all XEPs, custom stanzas
71 iOS with XMPPFramework — same coverage
Part 5 — Production
72 Security Hardening
73 PostgreSQL Backend
74 Scaling Path
75 East African Network Optimization
76 Common Pitfalls
77 Monitoring Setup
Part 1 — Foundation
Chapter 1 — What is XMPP?
1.1 The one-sentence answer
XMPP is an open protocol for sending small pieces of XML from one address to another, in real time, over a long-lived connection.
That's it. Everything else in this book is detail on top of that sentence. Each word was chosen deliberately:
- Open protocol — nobody owns it. It's an IETF standard (RFC 6120 for the core, RFC 6121 for instant messaging and presence). You don't ask permission or pay a license. Contrast this with WhatsApp's protocol, which is closed — you cannot legally build a server that speaks it.
- Small pieces of XML — the unit of communication is a stanza, a short XML fragment. A single chat message is a few hundred bytes.
- From one address to another — every user and service has an address called a JID (Jabber ID). Routing is built into the protocol.
- In real time — messages arrive the instant they're sent, not when the client next polls.
- Over a long-lived connection — the phone opens one TCP connection to the server and keeps it open for hours. Messages flow both ways over that single pipe.
XMPP originally stood for eXtensible Messaging and Presence Protocol. It started in 1999 under the name Jabber, created by Jeremie Miller. You'll still see "Jabber" everywhere — in the JID name, in library names, in old docs. Treat "Jabber" and "XMPP" as the same thing.
1.2 Why a long-lived connection matters (and why HTTP doesn't fit)
To understand XMPP, understand the problem it was built to avoid.
The web runs on HTTP, which is request/response. The client asks, the server answers, the connection closes. Perfect for loading a web page. Terrible for chat, because chat is server-initiated: the server needs to push a message to you the moment someone sends it, and it has no idea when that will be.
There are three ways to force chat onto HTTP, and all three are bad on a rural network:
Approach How it works Problem on 2G/3G
--------------------------------------------------------------------------
Short polling Ask "any messages?" every 3s Wastes battery + data,
message lag up to 3s
Long polling Ask, server holds request open Constant reconnects,
until a message arrives, repeat each carries TCP+TLS cost
WebSocket over One socket, but you build the You reinvent routing,
raw HTTP entire chat protocol yourself presence, offline, MUC...
XMPP solves this at the protocol level. The phone opens one connection, authenticates once, then both sides send stanzas whenever they want. No repeated handshakes. No polling. On a 2G link where a new TCP+TLS handshake costs several round-trips over 300–800 ms latency, "connect once and stay connected" is the difference between a usable app and an unusable one.
MaasaiChat phone maasaichat.com server
(Naserian's Tecno) (Ejabberd)
| |
|------ open TCP connection -------------->|
|<----- keep it open, both directions -----|
| |
| message to ole@maasaichat.com ------->| routes it
| |
|<------ message from enkiama@... ---------| pushed instantly
| |
| (connection stays open |
| for hours) |
One pipe. Messages flow both directions. That's the core idea.
1.3 What actually travels down the pipe
Once connected, the phone and server exchange stanzas. There are exactly three kinds (Chapter 4 goes deep). For now, just see them.
A chat message from Naserian to the elder Ole:
<message from='naserian@maasaichat.com/tecno'
to='ole@maasaichat.com'
type='chat'>
<body>Elder, the cattle are safe at the river.</body>
</message>
Naserian telling the server she is online:
<presence from='naserian@maasaichat.com/tecno'>
<show>chat</show>
<status>Herding near Ngorongoro</status>
</presence>
Naserian asking the server a question and expecting an answer:
<iq from='naserian@maasaichat.com/tecno'
type='get'
id='disco1'>
<query xmlns='http://jabber.org/protocol/disco#info'/>
</iq>
Three stanza types — message, presence, iq — and everything XMPP does is one of those three, possibly with extra XML tucked inside. That extra XML is what a XEP is, and it's why XMPP can grow reactions, calls, and file upload without ever changing the core (Chapter 5).
1.4 XMPP is the rulebook, Ejabberd is the builder
Here is the single most common confusion for newcomers, and clearing it up now makes the rest of the book effortless: XMPP and Ejabberd are not the same thing, and one is not "inside" the other.
XMPP is a specification — a set of documents written by the XMPP Standards Foundation. It is pure guidance. It says things like "a chat message must have a <body> element," "addresses look like user@domain/resource," "a typing indicator uses this namespace." That's all it is. Rules on paper. The XMPP spec, by itself, cannot route a single message — in the same way a rulebook cannot play the game.
Ejabberd is a builder that read those rules and wrote the code. The team at ProcessOne read every relevant document and implemented it in Erlang, producing an actual running server that obeys the rules. Ejabberd is the thing that accepts connections, routes stanzas, and stores offline messages.
The cleanest way to hold this:
Building code (the rules) The builder (follows the rules)
------------------------- ------------------------------
"walls must be 30cm thick" --> reads the code,
"doors must be 2m high" actually builds the house,
"foundation must be concrete" following every instruction
The building code is just paper. The house is real and you
It builds nothing by itself. can live in it.
XMPP = the building code Ejabberd = the builder
Developers already know this pattern from tools they use every day:
The language / rules The thing that speaks it
-------------------- ------------------------
HTTP --> Nginx, Apache, Tomcat
SQL --> PostgreSQL, MySQL
XMPP --> Ejabberd, Prosody, Openfire
Nobody says "HTTP is inside Nginx." HTTP is the rulebook; Nginx is a program that follows it. XMPP and Ejabberd relate the exact same way.
The chain: who writes the rules, who builds
Because XMPP is open guidance, many different teams read the same documents and each build their own piece — and because they all follow the same rules, all the pieces interoperate:
XMPP Standards Foundation writes the guidance (the XEPs)
│ "delivery receipts work like THIS"
▼
ProcessOne reads it, writes Erlang → Ejabberd (server)
Gajim team reads it, writes Python → Gajim (desktop client)
Smack team reads it, writes Java → Smack (Android library)
sendxmpp author reads it, writes Perl → sendxmpp (CLI tool)
Different code. Same guidance. They all understand each other. ✅
This is why the Gajim desktop app on Ole's laptop, the Smack-powered MaasaiChat app on Naserian's Tecno, and a sendxmpp script on a server can all exchange messages through Ejabberd without anyone coordinating — they are all following the same rulebook.
And it's the reason you write almost no protocol code yourself. Ejabberd is already built. Smack is already built. You read the XEPs to understand, then use those implementations, and only write custom code for MaasaiChat's own extensions (its own namespaces for things the standard doesn't cover). The 25-year-old rulebook, and the battle-tested builders who followed it, do the rest.
You (MaasaiChat) read XEPs to understand the rules
use Ejabberd (server, already built)
use Smack (Android, already built)
write custom code ONLY for your own
app-specific extensions ✅
Catch it in one line: XMPP is the language, Ejabberd speaks it, Gajim and Smack also speak it — everyone understands each other because everyone follows the same rulebook.
1.5 Who uses XMPP today
XMPP is not a museum piece. It quietly runs a large chunk of the messaging world:
- WhatsApp was built on a heavily modified fork of Ejabberd — the exact server this book deploys. The protocol you're learning is the ancestor of the app two billion people use.
- Nintendo Switch uses XMPP/Ejabberd for online chat and presence.
- Zoom acquired the XMPP-based team-chat product now sold as Zoom Team Chat.
- Google Talk was XMPP for its entire life and even federated with outside XMPP servers.
- Jitsi, HCL Sametime, and many carrier, ISP, and government messaging systems run on XMPP today.
The common thread: when an organization needs self-hosted, standards-based, massively scalable real-time messaging that they fully control, XMPP keeps being the answer. That is exactly MaasaiChat's position — you cannot depend on someone else's servers for a community tool in rural Tanzania and Kenya, so you run your own.
1.6 The messaging landscape — XMPP and its alternatives
XMPP is not the only way to build a chat app. Before committing, you should know what else exists, who runs on it, and the honest trade-offs. There are six real families of choice.
1. XMPP (this book)
Open IETF standard. Servers: Ejabberd (what we use), Prosody, Openfire. Clients everywhere.
Pros Cons
--------------------------------------------------------------
Open standard, no vendor lock-in Learning curve — it's a real
Self-hosted, you own the data protocol with real depth
Routing, presence, offline, MUC, XML is verbose vs binary
archive, push are built in (mitigated by compression)
Federation across servers Some XEPs are optional/uneven
Proven to millions of connections across servers
Free (Ejabberd Community) You assemble the client stack
Who uses it: WhatsApp (originally), Nintendo Switch, Google Talk, Jitsi.
2. Matrix
The main modern open-standard rival. Server: Synapse (also Dendrite, Conduit). Client: Element. Instead of XMPP's live XML stream, Matrix syncs a replicated JSON event graph over HTTP — every message is an event, and history is a shared, eventually-consistent room state.
Pros Cons
--------------------------------------------------------------
Open standard, federated Heavier — Synapse is resource-
Strong built-in E2E encryption hungry vs Ejabberd
JSON over HTTP — familiar to devs Sync model uses more bandwidth,
Great for team/community chat worse fit for strict 2G budgets
Rich ecosystem, bridges galore Younger, protocol still evolving
Who uses it: the French government (Tchap), the German armed forces (BwMessenger), Mozilla, KDE, and many privacy-focused communities. It's the serious open alternative — but its "replay the room's event history" model is more bandwidth-hungry than XMPP's lean stanza stream, which matters when your users pay per megabyte on 2G. That single fact is a large part of why MaasaiChat chooses XMPP over Matrix.
3. MQTT
A lightweight publish/subscribe protocol from the IoT world. Broker: Mosquitto, EMQX, HiveMQ.
Pros Cons
--------------------------------------------------------------
Extremely lightweight wire format Not a chat protocol — no roster,
Tiny overhead, ideal for low presence, offline history, MUC
bandwidth / battery You build ALL chat semantics
Great pub/sub fan-out on top yourself
Simple to reason about No federation, no identity model
Who uses it: sensors, cars, smart devices — and famously Facebook Messenger, which used MQTT for years to get fast, low-overhead delivery on poor mobile networks. MQTT is a fantastic transport, but it gives you a pipe, not a chat system. You'd rebuild everything XMPP already provides. (Note: Ejabberd itself speaks MQTT natively, so you can even use both.)
4. Proprietary binary protocols
The big consumer apps mostly rolled their own closed protocols:
- WhatsApp — started on Ejabberd/XMPP, later moved to a custom binary protocol using the Noise Protocol Framework for its handshake, with the Signal Protocol for end-to-end encryption.
- Signal — the Signal Protocol over its own service; the gold standard for E2E encryption.
- Telegram — its own MTProto protocol.
- Discord — a custom WebSocket gateway protocol, backed by Elixir/Erlang (the same platform Ejabberd runs on).
Pros Cons
--------------------------------------------------------------
Fully optimized for one app You must design + maintain the
Smallest possible wire format entire protocol yourself
Total control No standard, no federation
Years of engineering
Wrong choice unless you're at
massive scale with a big team
Who uses it: WhatsApp, Signal, Telegram, Discord. Great if you're a well-funded platform. Not a starting point for a community app.
5. Hosted / Backend-as-a-Service
Buy chat as an API. Firebase (Firestore + Cloud Messaging), Stream, Sendbird, PubNub, Twilio Conversations.
Pros Cons
--------------------------------------------------------------
Fastest to ship Pay per user / per message forever
No servers to run You don't own the data
Handles scale for you Vendor can change pricing or
Nice SDKs cut you off
Data lives outside your country
Costs balloon as you grow
Who uses it: startups that want chat live this week. Wrong fit for a self-reliant community tool where every user is cost-sensitive and independence is the point — a pricing change in San Francisco should never be able to shut down messaging in Ngorongoro.
6. Raw WebSocket + your own protocol
Open a WebSocket, invent your own message format, build the rest by hand.
Pros Cons
--------------------------------------------------------------
Total freedom You reimplement routing, presence,
Simple to start ("just a socket") offline, groups, receipts, archive,
Familiar to web devs reconnection — for years
No standard, no interoperability
Every bug is yours to discover
Who uses it: Slack and Discord built custom protocols on top of WebSocket — but with large engineering teams. For a solo or small team, this is the "reinvent XMPP, badly" path.
The verdict for MaasaiChat
Own data Low 2G Built-in Free / Effort to
& control bandwidth chat feats cheap ship
--------------------------------------------------------------------------
XMPP/Ejabberd YES Excellent YES YES Medium
Matrix YES Fair YES Cheap-ish Medium
MQTT YES Excellent NO YES High
Proprietary YES Best NO No Very high
Hosted (SaaS) NO Varies YES No Low
Raw WebSocket YES Good NO YES Very high
XMPP is the only row that is yes on ownership, excellent on bandwidth, yes on built-in chat features, and free — at merely medium effort. For a self-hosted community app on 2G in East Africa, no other option matches on all four. That is why the rest of this book is XMPP.
1.7 Why XMPP fits rural East Africa specifically
Every decision in this book is shaped by one reality: MaasaiChat users are on 2G/3G, low-end Android, with connections that drop constantly. XMPP earns its place for concrete reasons.
Tiny messages. A stanza is a few hundred bytes. On a metered 2G plan where users pay per megabyte, that matters. No fat envelopes, no HTTP headers per message.
Built for connections that drop. Stream Management (XEP-0198, Chapter 10) is designed for exactly this. When Naserian rides out of coverage near Ngorongoro and back ten minutes later, her session resumes — messages that arrived while she was gone are delivered, and messages she sent that didn't quite make it are re-sent, with no full reconnect and re-authentication. This single extension is worth the whole protocol on a rural network.
Offline delivery is standard. If Ole's phone is off when Enkiama messages him, the server holds the message and delivers it when Ole reconnects. You don't build this.
One server, huge capacity. A single Ejabberd node has handled two million concurrent connections in production. MaasaiChat across two countries won't come close to stressing it — so it runs on modest, affordable infrastructure.
You own it. Self-hosted, open source, no per-message fees, no vendor who can cut you off.
Rural constraint XMPP answer
---------------------------------------------------------------
Expensive metered data --> Tiny stanzas, no polling
Connection drops constantly --> Stream Management (resume, XEP-0198)
Phone often off/asleep --> Server-side offline storage + push
Low-end hardware --> Lightweight client, one connection
Must be self-run --> Open protocol, free Ejabberd
1.8 The mental model to carry forward
Before the next chapter, lock in this picture:
- Each user has an address (JID):
naserian@maasaichat.com. - Each device opens one long-lived connection to
maasaichat.com. - Over it flow stanzas — small XML fragments.
- There are exactly three stanza types:
message,presence,iq. - New features are added as extra XML inside stanzas, defined by XEPs, never by changing the core.
- The server (Ejabberd) handles routing, offline storage, presence, groups, and archive so you don't have to.
Everything from here builds on those six facts.
✅ What you learned in this chapter
- XMPP is an open, standardized protocol for exchanging small XML stanzas in real time over one long-lived connection.
- It exists because HTTP request/response can't push server-initiated messages efficiently — fatal on metered, high-latency rural links.
- The three stanza types are
message,presence,iq; features are added via XEPs without changing the core.- XMPP is the rulebook; Ejabberd is a builder that followed it — like HTTP↔Nginx or SQL↔PostgreSQL. Gajim, Smack, and sendxmpp are other builders following the same rules, which is why they all interoperate.
- Real, current users include WhatsApp (built on Ejabberd), Nintendo Switch, Zoom, and Google Talk.
- The alternatives are Matrix, MQTT, proprietary binary protocols, hosted SaaS, and raw WebSocket — each with real trade-offs; XMPP uniquely wins on ownership + bandwidth + built-in features + cost for a self-hosted 2G community app.
- XMPP fits rural East Africa: tiny messages, Stream Management for dropping connections, standard offline delivery, one high-capacity self-hosted server, and full ownership.
- MaasaiChat runs its own Ejabberd on
maasaichat.com— no dependence on outside messaging providers.
Ready for next chapter? (Chapter 2 — The XMPP Vocabulary: every key term you'll meet in this book — stanza, JID, resource, stream, namespace, roster, presence, MUC, MAM, SASL, and the rest — each defined plainly with a MaasaiChat example, so no word is ever a mystery in the chapters ahead.)
Chapter 2 — The XMPP Vocabulary
Key Terms & Definitions
XMPP has a lot of vocabulary, and the deep-dive chapters ahead assume you know it. So this chapter is a dictionary you read once and refer back to forever. Every term gets a plain definition, an analogy, and — where it helps — a MaasaiChat example. You'll meet each of these again in depth later; the goal here is that no word is ever a mystery.
The terms are grouped the way they actually relate, not alphabetically:
A. The absolute core XMPP, stanza, stream, JID, resource
B. Addressing details bare/full JID, message types
C. Connecting & login TLS, SASL, bind, stream features
D. Staying connected Stream Management, ping, keepalive
E. Messaging features receipts, markers, chat states, IDs, carbons
F. Presence & contacts presence, roster, subscription
G. Group chat MUC, affiliation vs role, occupant
H. Storage & history offline messages, MAM, RSM, inbox
I. Discovery & extensions namespace, XEP, disco, caps, PubSub, PEP
J. Media, calls, push, E2E file upload, Jingle, push, OMEMO
K. Federation & transports s2s, dialback, BOSH, WebSocket
L. Ejabberd-specific mod_, ejabberdctl, Mnesia, vhost, ACL,
Erlang cookie, Erlang distribution
A one-page Quick Reference Card sits at the end of the chapter — tear it out (metaphorically) and keep it beside you.
A. The absolute core
XMPP
The rulebook. Extensible Messaging and Presence Protocol, an open IETF standard (RFC 6120/6121) created in 1999 as "Jabber." It defines how chat works: the shape of messages, the address format, how you log in, how features extend the core. It runs nothing by itself — a server like Ejabberd implements it. (See Chapter 1.)
Stanza
The basic unit of communication — one small XML fragment, like one sentence in a conversation. Everything you send or receive is a stanza. There are exactly three types: <message>, <presence>, <iq>.
<message to='ole@maasaichat.com' type='chat'>
<body>The cattle arrived safely at the river.</body>
</message>
<!-- that whole thing = one stanza -->
Stream
The single long-lived connection between client and server. A tunnel that stays open; every stanza flows through it. Opening it is like starting a phone call; closing it is hanging up.
<!-- Naserian's phone opens the stream -->
<stream:stream to='maasaichat.com' xmlns='jabber:client' ...>
... all stanzas flow here for hours ...
</stream:stream> <!-- closed when she leaves the app -->
JID (Jabber ID)
The address of every entity in XMPP — users, servers, group rooms. Like an email address, but for real-time chat. Format: user@domain/resource.
ole@maasaichat.com/android a specific device
ole@maasaichat.com the person (any device)
maasaichat.com the server itself
warriors@conference.maasaichat.com a group room
Resource
The device-identifier part of a JID, after the slash. It exists because one person logs in from several devices at once, and the server must tell them apart.
ole@maasaichat.com/phone Ole's Tecno
ole@maasaichat.com/tablet Ole's tablet
Send to the bare JID (ole@maasaichat.com) and Ejabberd chooses the best device; send to a full JID (.../phone) and it goes to that one device only.
B. Addressing details
Bare JID vs Full JID
- Bare JID —
ole@maasaichat.com. Identifies the person. Used for offline messages, roster entries, MUC membership. - Full JID —
ole@maasaichat.com/android. Identifies a specific connected device/session. Used to reach exactly one device, e.g. during a call.
Rule of thumb: person → bare, device → full.
Message types
The type attribute on a <message> tells the server and client how to treat it:
chat one-to-one conversation (Naserian → Ole)
groupchat a MUC room message (to warriors@conference...)
normal a single message, no ongoing chat (system notices)
headline broadcast/alert, never stored offline (announcements)
error something failed; carries an <error> child
Using the wrong type causes real bugs — e.g. a headline won't be saved for an offline user, so MaasaiChat uses chat/groupchat for anything that must survive a dropped connection.
C. Connecting & login
TLS / STARTTLS
Encryption of the stream. Before any password is sent, the client upgrades the plain TCP connection to an encrypted one (STARTTLS), or connects to an already-encrypted port (Direct TLS, 5223). No TLS = passwords and messages travel in the clear. MaasaiChat requires TLS always.
SASL
Simple Authentication and Security Layer — the login system, i.e. how you prove who you are. It supports several mechanisms:
PLAIN sends the password directly (only safe inside TLS)
SCRAM-SHA-1 secure challenge/response, no password on the wire
SCRAM-SHA-256 stronger
SCRAM-SHA-512 strongest classic option (what Gajim used)
X-OAUTH2 log in with an OAuth token instead of a password
With SCRAM, Ole's phone never sends his password — it solves a cryptographic challenge that proves it knows the password. (See also §J, channel binding.)
Stream features
Right after the stream opens, the server sends a <stream:features> list — "here's what's available/required next": STARTTLS, which SASL mechanisms, resource binding, Stream Management, and so on. The client walks through them in order. It's the server announcing the login menu.
Bind (resource binding)
After SASL proves who you are, binding assigns which session — it hands you your full JID by attaching a resource.
<iq type='set'><bind xmlns='urn:ietf:params:xml:ns:xmpp-bind'>
<resource>android</resource>
</bind></iq>
<!-- server replies -->
<iq type='result'><bind>
<jid>ole@maasaichat.com/android</jid>
</bind></iq>
Now Ole's full JID exists and messages can be routed to this exact session.
D. Staying connected (vital on 2G/3G)
Stream Management (SM) — XEP-0198
Makes delivery reliable on bad networks with an acknowledgement counter, and lets a dropped session resume instead of fully reconnecting.
<r/> "please confirm how many of my stanzas you got"
<a h='32'/> "confirmed — I have received up to stanza 32"
<resume/> "I dropped and reconnected — resend from where we left off"
MaasaiChat example: Ole sends message #30 on 2G near the boma, the signal drops, he reconnects, the server sees he only acked #29, and re-sends #30. Zero loss. This is the single most important extension for rural networks.
Ping — XEP-0199
A tiny "are you still there?" iq. The client or server pings periodically to detect a silently-dead connection (common when a mobile network drops without closing the socket).
<iq type='get'><ping xmlns='urn:xmpp:ping'/></iq>
<iq type='result'/> <!-- still alive -->
Whitespace keepalive
Even cheaper than a ping: the client sends a single space character down the stream now and then, just to keep NATs and carrier gateways from closing an "idle" connection. Common on mobile.
E. Messaging features
Delivery Receipt — XEP-0184
Proof a message reached the recipient's device — the single grey/blue tick "delivered."
<message to='naserian@maasaichat.com'>
<body>Are the warriors ready?</body>
<request xmlns='urn:xmpp:receipts'/> <!-- please confirm delivery -->
</message>
Chat Markers — XEP-0333
Proof a message was received and read — the "read" tick. Distinct from a delivery receipt: delivery = it arrived on the device; marker (displayed) = the human actually saw it.
received landed on the device
displayed shown to the user (the "read" tick)
acknowledged app-level handled
Chat State Notifications — XEP-0085
The "typing…" experience. Tiny signals about what the other side is doing:
active looking at the chat
composing typing right now ("Naserian is typing…")
paused stopped typing but still there
inactive tab/chat idle
gone left the conversation
Stable & Unique Stanza IDs — XEP-0359
Gives every message two dependable IDs so all devices agree on identity: an origin-id set by the sender and a stanza-id assigned by the server/room. Essential for edits, reactions, replies, and de-duplication when the same message arrives via several paths.
Last Message Correction — XEP-0308
Editing a sent message. The new stanza points at the old one's ID with <replace/>, and clients swap the display in place (the "edited" label).
Message Retraction — XEP-0424
"Delete for everyone." A stanza that tells clients to remove a previously-sent message, referenced by its ID.
Message Reactions — XEP-0444
Emoji reactions attached to a message ID (👍 on Ole's cattle update), rather than a new separate message.
Message Carbons — XEP-0280
Multi-device sync for one-to-one chats. When Naserian messages Ole, a copy is delivered to every one of Ole's connected devices, and copies of what Ole sends also appear on his other devices — so phone and tablet stay identical (like WhatsApp Web mirroring your phone).
F. Presence & contacts
Presence
An announcement of availability, broadcast to those allowed to see it.
available online (the default)
away stepped away
xa extended away (gone a while)
dnd do not disturb
unavailable offline
<presence><show>chat</show><status>Herding near Ngorongoro</status></presence>
When Ole closes the app, the server sends unavailable on his behalf, and his contacts see him go offline.
Presence priority
When Ole is online on several devices, each presence carries a numeric <priority>. Messages to his bare JID go to the highest-priority device. Negative priority means "never auto-deliver here."
Roster
Your contact list — stored on the server, synced to every device. Switch phones, log in, and all contacts reappear instantly. Each entry holds a JID, a display name, subscription state, and groups.
Ole's roster:
naserian@maasaichat.com (Warriors)
enkiama@maasaichat.com (Elders)
nkeri@maasaichat.com (Traders)
Presence Subscription
Permission to see someone's presence — like a mutual follow. You must ask and be approved.
subscribe "I want to see your status"
subscribed "granted — you may see mine"
unsubscribe "stop seeing my status"
unsubscribed "denied / revoked"
Ole sends subscribe to Naserian; she returns subscribed; now Ole sees when she's online.
Roster push
When your roster changes (you add Nkeri), the server pushes the update to all your logged-in devices automatically, so they stay in sync without asking.
G. Group chat
MUC (Multi-User Chat) — XEP-0045
Group chat rooms. Each room is itself a JID on the conference service:
warriors@conference.maasaichat.com
^room ^MUC service domain
Send with type='groupchat' and every occupant receives it.
Occupant & nickname
Inside a room you appear under a nickname, and your in-room address is room@service/nickname (a full JID whose "resource" is your nick). Your real JID may be hidden depending on room settings.
Affiliation vs Role (the classic MUC confusion)
Two different permission systems in every MUC:
AFFILIATION long-term membership status (persists across visits)
owner created the room, full control
admin can manage members/admins
member allowed into a members-only room
outcast banned
none no special standing
ROLE what you can do RIGHT NOW, this visit
moderator can kick, grant voice, moderate
participant can speak
visitor can only read (no voice)
none not in the room
Affiliation is who you are to the room over time; role is what you may do in this session. Enkiama the chief might be an owner (affiliation) who is currently acting as moderator (role).
Message Moderation — XEP-0425
Lets a room moderator retract/hide someone else's message in the room (spam control), distinct from a user deleting their own.
H. Storage & history
Offline Messages — XEP-0160
If the recipient is offline, the server stores the message and delivers it on reconnect — voicemail for chat. Enkiama is out of signal in the bush; Ole's message waits on the server and lands when Enkiama's phone finds a tower.
MAM (Message Archive Management) — XEP-0313
The server keeps a searchable archive of conversations; clients fetch history on demand. Buy a new phone, log in, ask for "the last 7 days," and your full history appears. Without MAM, a new device starts empty.
RSM (Result Set Management) — XEP-0059
Paging for large result sets — "give me 20 messages before this point," then the next 20. MAM uses RSM so a phone on 2G loads history in small chunks instead of one huge download.
Inbox — XEP-0430
A server-built list of your conversations with the latest message and unread count for each — the "chat list" screen, computed server-side so it's instant and consistent across devices.
IQ (Info/Query)
The request/response stanza — XMPP's version of an HTTP GET/POST. Always in pairs, always with a matching id:
get ask for information result success (may carry data)
set do or change something error it failed
<iq type='get' id='r1'><query xmlns='jabber:iq:roster'/></iq>
<iq type='result' id='r1'><query> ...contacts... </query></iq>
Roster fetch, ping, disco, bind, MAM queries — all are IQs.
I. Discovery & extensions
Namespace (xmlns)
A unique string that says which extension an element belongs to. Without it, <request> is meaningless; with xmlns='urn:xmpp:receipts' everyone knows it's a delivery receipt. Think of it as the department stamp on a memo.
See xmlns="urn:xmpp:something" → it's a XEP.
Search that string → you find the exact XEP instantly.
URN
Uniform Resource Name — the format most XMPP namespaces use. A permanent identifier that names what something is, as opposed to a URL, which locates where something is. urn:xmpp:receipts is never fetched from a server; it's simply an agreed string meaning "delivery receipts." Trailing digits are versions (urn:xmpp:sm:3), and :0 marks an experimental XEP. (Full treatment in §3.5.)
XEP
XMPP Extension Protocol — a document that adds a feature on top of core XMPP, each with a number: XEP-0045 (group chat), XEP-0184 (receipts), XEP-0166 (calls). Base XMPP is a basic phone; XEPs bolt on the camera, caller-ID, and voicemail. (Full treatment in Chapter 6.)
Service Discovery (disco) — XEP-0030
How one entity asks another "what are you, and what can you do?" A client discos the server to learn which features and services (MUC, file upload, push) exist.
<iq type='get' to='maasaichat.com'>
<query xmlns='http://jabber.org/protocol/disco#info'/>
</iq>
Entity Capabilities (caps) — XEP-0115
An optimization on top of disco: each client advertises a short hash of its feature set in presence, so others cache "a client with hash X supports these XEPs" instead of re-asking every time. Saves bandwidth — good on 2G.
PubSub (Publish-Subscribe) — XEP-0060
A general publish/subscribe system on the server: publishers post items to a node, subscribers get them. The backbone for many features (avatars, bookmarks, presence-like data).
PEP (Personal Eventing Protocol) — XEP-0163
A simplified PubSub attached to a user's own account — "my nodes." Used for things like your current avatar, mood, or bookmarks, auto-broadcast to contacts who care.
J. Media, calls, push, encryption
HTTP File Upload — XEP-0363
How you send photos/files. The client asks the server for an upload slot (a PUT URL + a GET URL), uploads the file over HTTPS, then sends the GET URL in a normal message. This is the file-transfer method MaasaiChat actually uses — it works fine over flaky mobile links, unlike peer-to-peer transfer.
Jingle — XEP-0166 (+0167/0176/0215/0320)
The signaling framework for voice/video calls. Jingle negotiates the call (who, what codec, network path) inside XMPP, while the actual audio/video flows over WebRTC. Related XEPs handle RTP media (0167), ICE network traversal (0176), TURN/STUN discovery (0215), and encryption (0320).
Push Notifications — XEP-0357
Wakes a sleeping phone. When a message arrives for an app that's backgrounded/disconnected, the server triggers a push (via FCM on Android) so the user gets notified without holding a socket open — essential for battery on low-end phones.
OMEMO — XEP-0384
Modern end-to-end encryption for XMPP (built on the Signal Protocol's ideas). Messages are encrypted per-device so that not even the server can read them. Optional; a design decision for later MaasaiChat phases.
SASL Channel Binding — XEP-0440 / Extensible SASL — XEP-0388
Hardening for login: channel binding ties the SASL authentication to the exact TLS connection, blocking a class of man-in-the-middle attacks. XEP-0388 modernizes how SASL mechanisms are negotiated.
K. Federation & transports
c2s and s2s
Two connection kinds Ejabberd listens for: c2s (client-to-server, port 5222 — phones connecting) and s2s (server-to-server, port 5269 — other XMPP servers connecting for federation).
Federation
Different XMPP servers talking to each other, so naserian@maasaichat.com could message someone@another-server.org — the same way email crosses providers. Enabled by s2s.
Server Dialback — XEP-0220
A verification handshake that lets a receiving server confirm a connecting server really owns the domain it claims, preventing spoofed federation. (Bidirectional s2s, XEP-0288, lets one connection carry traffic both ways.)
BOSH and WebSocket
Alternative transports for when a raw TCP stream isn't possible (e.g. a browser). BOSH tunnels XMPP over HTTP long-polling; WebSocket carries the XML stream over a WebSocket. Native mobile apps use raw TCP; web clients use WebSocket.
L. Ejabberd-specific terms
mod_ (module)
A plugin that adds a feature to Ejabberd, switched on in ejabberd.yml — like browser extensions.
mod_muc group chat mod_mam message archive
mod_offline offline storage mod_http_api REST API
mod_ping keepalive pings mod_mqtt MQTT protocol
mod_push push notifications mod_admin_extra extra CLI commands
No mod_muc → no group chat. Add it → group chat works.
ejabberdctl
Ejabberd's command-line control tool, run inside the container — the psql/redis-cli equivalent for Ejabberd.
docker exec ejabberd ejabberdctl status
docker exec ejabberd ejabberdctl registered_users maasaichat.com
docker exec ejabberd ejabberdctl send_message chat ole@maasaichat.com \
naserian@maasaichat.com "" "Meeting at the manyatta tonight"
vhost (virtual host)
One Ejabberd process can serve several domains at once (maasaichat.com, staging.maasaichat.com), each with its own users and settings — like Nginx server blocks.
ACL & Access Rules
Ejabberd's permission system: an ACL defines who (e.g. "admins = admin@maasaichat.com"), and an access rule grants those groups rights (e.g. who may use the REST API). Misconfigured ACLs are the classic cause of Account does not have the right to perform the operation when calling the API.
Mnesia
Erlang's built-in database, bundled with Ejabberd, used for its internal live state — who's connected where, room state, roster, short-term offline spool. It does not hold MaasaiChat's business data; that lives in PostgreSQL.
Erlang Cookie
Erlang Distribution (dist)
Erlang's native node-to-node networking. In a cluster, if Ole is on Node 1 and Naserian is on Node 2, Node 1 hands the stanza straight to Node 2 over Erlang distribution — no Redis pub/sub needed. This is why an Ejabberd cluster is simpler than a hand-built socket cluster.
Spool
The queue where offline messages wait for a disconnected user. When Enkiama reconnects, Ejabberd flushes his spool to his device.
Quick Reference Card
Term Simple definition
────────────────────────────────────────────────────────────────
XMPP The language/rules of chat
Stanza One unit of communication (message/presence/iq)
Stream The single open connection (tunnel)
JID Address: user@domain/resource
Resource Device identifier (/phone /tablet)
Bare / Full JID Person (bare) vs specific device/session (full)
Message type chat / groupchat / normal / headline / error
TLS / STARTTLS Encryption of the stream
SASL Authentication system (SCRAM, PLAIN, OAuth)
Stream features Server's "menu" of what to negotiate next
Bind Claiming your resource → full JID
Stream Management Reliable delivery + resume on bad networks (0198)
Ping "Are you alive?" heartbeat (0199)
Keepalive Whitespace to stop NAT/carrier timeouts
Delivery Receipt "Delivered" tick (0184)
Chat Markers "Read/displayed" tick (0333)
Chat States typing / paused indicators (0085)
Stanza IDs Stable message identity (0359)
Correction Edit a sent message (0308)
Retraction Delete for everyone (0424)
Reactions Emoji on a message (0444)
Carbons Multi-device 1:1 sync (0280)
Presence Online/offline status
Priority Which device gets bare-JID messages
Roster Contact list (stored on server)
Subscription Permission to see presence (mutual)
MUC Group chat room (0045)
Occupant / nick Your identity inside a room
Affiliation Long-term membership (owner/admin/member/outcast)
Role Current-session powers (moderator/participant/visitor)
Offline Messages Stored when recipient offline (0160)
MAM Server message archive/history (0313)
RSM Paging for large results (0059)
Inbox Server-built conversation list (0430)
IQ Request/response stanza (get/set/result/error)
Namespace (xmlns) Which XEP an element belongs to
URN Permanent NAME (urn:xmpp:...), not a fetchable URL
XEP A document adding a feature to XMPP
disco Service discovery: "what can you do?" (0030)
caps Cached capability hashes (0115)
PubSub / PEP Publish-subscribe / personal eventing (0060/0163)
HTTP File Upload Send files via upload slot (0363)
Jingle Voice/video call signaling (0166)
Push Wake a sleeping phone via FCM (0357)
OMEMO End-to-end encryption (0384)
c2s / s2s Client-to-server / server-to-server
Federation Cross-server messaging (like email)
Dialback Verifies a federating server (0220)
BOSH / WebSocket XMPP over HTTP / over WebSocket (for browsers)
mod_ Ejabberd plugin/module
ejabberdctl Ejabberd CLI management tool
vhost One server hosting multiple domains
ACL / Access Rule Ejabberd permission system
Mnesia Ejabberd's internal live-state database
Erlang Cookie Cluster membership password
Erlang dist Direct node-to-node communication
Spool Queue of waiting offline messages
✅ What you learned in this chapter
- Stanza, stream, JID, resource are the four words the whole protocol stands on — one XML unit, one open connection, one address, one device tag.
- Addressing splits into bare (person) vs full (device) JIDs, and every message carries a type (chat/groupchat/normal/headline/error) that decides how it's handled.
- Login is a pipeline: TLS → SASL → bind → stream features, and staying connected on 2G relies on Stream Management, ping, and keepalives.
- Messaging polish is a stack of small XEPs — receipts (0184), markers (0333), chat states (0085), stable IDs (0359), carbons (0280) — plus edit/retract/react.
- Presence + roster + subscription run contacts and status; MUC runs groups, where affiliation (long-term) and role (this session) are different things.
- History is offline messages → MAM → RSM → inbox; discovery is disco + caps; extensions are identified by namespace and documented as XEPs.
- Ejabberd-specific: mod_ modules, ejabberdctl, vhosts, ACLs, Mnesia, the Erlang cookie, and Erlang distribution — the operational vocabulary for Part 3.
Ready for next chapter? (Chapter 3 — XML Basics — Streams, Stanzas & Nonzas: now that you know the words, we read the XML itself — the three layers, the container-vs-contents distinction, one complete annotated chat session from TCP handshake to stream close, what order is fixed and what is free, and the tools to type raw stanzas yourself.)
Chapter 3 — XML Basics: Streams, Stanzas & Nonzas
You know the vocabulary. Now we read the actual XML — slowly, once, all the way through a real session. By the end of this chapter you will be able to look at any raw XMPP exchange and know exactly what layer you're in, what's happening, and what should come next.
3.1 Is <stream:stream> a stanza?
No. This trips up almost everyone, so let's settle it immediately.
stream:stream = the CONTAINER
stanzas = what goes INSIDE the container
Like:
stream = the envelope
stanzas = the letters inside it
Open the envelope once (stream opens)
Put many letters inside (stanzas, repeated)
Close it when done (stream closes)
The stream is opened once per session and closed once per session. Stanzas flow inside it, many thousands of times.
Notice something odd about the stream tag: it is a single XML document whose root element stays open for hours. XMPP is not "send a document, get a document" — the entire session is one enormous XML document, streamed a piece at a time, and each stanza is a child element of its root. That's the trick that makes XMPP real-time.
3.2 The three layers
Always know which layer you're looking at:
┌──────────────────────────────────────────────────┐
│ LAYER 3 — STANZAS (repeat forever) │
│ <message> <presence> <iq> │
├──────────────────────────────────────────────────┤
│ LAYER 2 — XMPP STREAM (once per session) │
│ <stream:stream> ......... </stream:stream> │
├──────────────────────────────────────────────────┤
│ LAYER 1 — TCP + TLS (the wire) │
│ raw bytes, port 5222, encrypted │
└──────────────────────────────────────────────────┘
Layer 1 is plumbing. Layer 2 is the envelope. Layer 3 is the conversation.
3.3 The third thing: nonzas
Here's a distinction that will make the rest of this book click. Inside the stream, not everything is a stanza.
Only three elements are stanzas: <message>, <presence>, <iq>. Everything else that flows inside the stream — <stream:features>, <starttls/>, <auth/>, <challenge/>, <success/>, <enable/>, <r/>, <a/> — is officially called a nonza (literally: "not a stanza").
STANZAS NONZAS
------------------------ ------------------------------------
<message> <stream:features> <starttls/>
<presence> <auth/> <challenge/> <response/>
<iq> <success/> <failure/>
<enable/> <enabled/> <r/> <a/>
Routable — they have Not routable — they are between
'to' and 'from', the THIS client and THIS server only.
server delivers them Pure connection machinery.
anywhere on the network.
The practical rule: stanzas travel; nonzas negotiate. A message can cross the planet to another server. An <auth/> never goes anywhere — it's a private word between your phone and Ejabberd.
This is why the SASL exchange and Stream Management acks look "different" from chat traffic. They're not stanzas at all.
3.4 Reading XMPP's XML: the four things to look for
You don't need to be an XML expert. You need four things:
1. Element — the tag name. It tells you the kind of thing: <message>, <iq>, <body>.
2. Attributes — the key facts on the tag itself: who, to whom, what kind, which id.
<message from='ole@maasaichat.com/android'
to='naserian@maasaichat.com'
type='chat'
id='msg-001'>
3. Children — nested elements carrying the payload: <body>, <show>, <query>.
4. xmlns (the namespace) — the most important attribute in XMPP. It tells you which extension a child element belongs to. <request/> alone is meaningless; <request xmlns='urn:xmpp:receipts'/> is unambiguously a delivery-receipt request.
See an xmlns you don't recognize?
→ search that exact string
→ you land on the XEP that defines it
That single habit makes XMPP self-documenting.
3.5 Anatomy of a namespace: what urn: actually means
You just met xmlns='urn:xmpp:receipts'. Two questions always follow: what is a URN, and why do some namespaces look like web addresses instead?
URN = Uniform Resource Name
Uniform consistent, standard format
Resource anything that can be named
Name a permanent identifier — NOT a location
The contrast with a URL is the whole point:
URL = Uniform Resource LOCATOR URN = Uniform Resource NAME
Tells you WHERE something is Tells you WHAT something is
Breaks if the server moves Never changes, never breaks
Must be fetched to be useful Nothing to fetch — it's just a name
https://xmpp.org/extensions/ urn:xmpp:receipts
xep-0184.html permanent, forever
(can 404 tomorrow)
Like a person versus their address: "Ole Saitoti" is a name that never changes; "the manyatta by the river" is a location that can. A URN is the name.
The format
urn : namespace : specific-name : version
↑ ↑ ↑ ↑
always who owns the thing optional, and
"urn" this space being named very common in XMPP
urn : xmpp : receipts delivery receipts
urn : xmpp : sm : 3 stream management, version 3
urn : xmpp : sid : 0 stable stanza IDs, version 0
urn : maasaichat : cattle : 1 OUR extension, version 1
That trailing number matters. urn:xmpp:sm:3 is not decoration — it's version 3 of Stream Management, and a client that speaks sm:2 cannot assume it understands sm:3. When you see :0 (as in urn:xmpp:sid:0), the XEP is still experimental and the namespace is expected to change when it stabilizes. That single digit tells you how much to trust the feature.
Why XMPP chose URNs
XMPP needed identifiers that could never break or collide across two decades and thousands of implementations.
If namespaces were URLs: Because they're URNs:
xmpp.org restructures → broken nothing to break
server down → ambiguity no server involved
someone hijacks the domain nobody can hijack a string
A namespace is not a resource to fetch. Nothing ever requests urn:xmpp:receipts over the network. It's a shared agreement: when you see this exact string, you know it means delivery receipts. It lives in developers' heads and in the if statements of every XMPP library on earth. That's all it needs to do.
So why do some namespaces look like URLs?
Because XMPP is 25 years old and has three generations of naming, all still in use:
Style Era Example
------------------------------------ -------- ----------------------------
jabber:xxx 1999- jabber:client
the original short names jabber:iq:roster
http://jabber.org/protocol/xxx ~2000s http://jabber.org/protocol/
older XEPs, URL-SHAPED but still chatstates
just a name — never fetched! http://jabber.org/protocol/
disco#info
urn:xmpp:xxx modern urn:xmpp:receipts
what all new XEPs use urn:xmpp:sm:3
urn:ietf:params:xml:ns:xxx IETF core urn:ietf:params:xml:ns:
the base protocol itself (RFC 6120) xmpp-sasl / xmpp-bind
The crucial point: http://jabber.org/protocol/chatstates is not a URL that gets loaded. It looks like one, but it is used exactly the same way as a URN — as an opaque, permanent identifier string. Nobody's phone has ever made an HTTP request to it. Early XMPP borrowed the URL shape (a common XML convention for guaranteeing uniqueness via a domain you own), then the community moved to urn:xmpp:* because the URL shape misled people into thinking it meant something fetchable.
So when a stanza mixes both styles — and real ones constantly do — nothing is inconsistent. You're just seeing XEPs from different decades:
<message to='naserian@maasaichat.com' type='chat' id='msg-001'>
<body>The cattle arrived safely.</body>
<active xmlns='http://jabber.org/protocol/chatstates'/> <!-- 2000s XEP -->
<origin-id xmlns='urn:xmpp:sid:0' id='msg-001'/> <!-- modern XEP -->
<request xmlns='urn:xmpp:receipts'/> <!-- modern XEP -->
</message>
Naming your own extensions
When MaasaiChat needs something the standard doesn't cover — say, attaching a cattle-market listing to a message — you mint your own namespace. Follow the URN convention and version it from day one:
Good: urn:maasaichat:cattle:1 a name, permanent, versioned
Bad: https://maasaichat.com/xmpp/cattle
→ looks fetchable, breaks the day the domain changes
<message to='cattle-market@conference.maasaichat.com' type='groupchat'>
<body>12 head of cattle, Ngorongoro, ready Friday</body>
<listing xmlns='urn:maasaichat:cattle:1' count='12' ready='2026-07-17'/>
</message>
Any client that doesn't know urn:maasaichat:cattle:1 simply ignores the <listing/> child and still shows the <body> text — which is exactly why XMPP extends so gracefully. And that version digit means when the listing format changes, you bump to :2 and old clients keep working instead of misreading new data.
Catch it in one line: A namespace is a permanent name, never a place — even the ones shaped like URLs are never fetched.
3.6 One complete 1:1 chat session, annotated
This is the whole thing — Ole opens MaasaiChat on his phone, chats with Naserian, and closes the app. Every byte in order. Read it once now; it will make sense in pieces as the book continues.
<!-- ============================================================ -->
<!-- LAYER 1: TCP connection to maasaichat.com:5222 -->
<!-- (no XML yet — just the TCP handshake) -->
<!-- ============================================================ -->
<!-- ============================================================ -->
<!-- LAYER 2: STREAM OPENS -->
<!-- ============================================================ -->
<!-- Ole's phone opens the stream -->
<?xml version='1.0'?>
<stream:stream
to='maasaichat.com'
version='1.0'
xmlns='jabber:client'
xmlns:stream='http://etherx.jabber.org/streams'>
<!-- Server opens ITS stream back (two streams: one each direction) -->
<?xml version='1.0'?>
<stream:stream
from='maasaichat.com'
id='session-abc-123'
version='1.0'
xmlns='jabber:client'
xmlns:stream='http://etherx.jabber.org/streams'>
<!-- ---- STEP 1: SERVER OFFERS FEATURES (nonza) ---- -->
<stream:features>
<starttls xmlns='urn:ietf:params:xml:ns:xmpp-tls'>
<required/> <!-- MaasaiChat demands encryption -->
</starttls>
</stream:features>
<!-- ---- STEP 2: ENCRYPT FIRST (nonzas) ---- -->
<starttls xmlns='urn:ietf:params:xml:ns:xmpp-tls'/>
<proceed xmlns='urn:ietf:params:xml:ns:xmpp-tls'/>
<!-- TLS handshake happens now. Everything below is encrypted.
The stream RESTARTS from scratch after TLS. -->
<?xml version='1.0'?>
<stream:stream to='maasaichat.com' version='1.0'
xmlns='jabber:client'
xmlns:stream='http://etherx.jabber.org/streams'>
<!-- Now the server offers login options -->
<stream:features>
<mechanisms xmlns='urn:ietf:params:xml:ns:xmpp-sasl'>
<mechanism>SCRAM-SHA-512</mechanism>
<mechanism>SCRAM-SHA-256</mechanism>
<mechanism>PLAIN</mechanism>
</mechanisms>
</stream:features>
<!-- ---- STEP 3: AUTHENTICATION (nonzas — NOT iq!) ---- -->
<auth xmlns='urn:ietf:params:xml:ns:xmpp-sasl'
mechanism='SCRAM-SHA-512'>BASE64_INITIAL</auth>
<challenge xmlns='urn:ietf:params:xml:ns:xmpp-sasl'>BASE64_CHALLENGE</challenge>
<response xmlns='urn:ietf:params:xml:ns:xmpp-sasl'>BASE64_RESPONSE</response>
<success xmlns='urn:ietf:params:xml:ns:xmpp-sasl'>BASE64_VERIFY</success>
<!-- Ole's password never crossed the wire — only proofs of knowing it -->
<!-- ---- STEP 4: STREAM REOPENS AFTER AUTH (required by spec) ---- -->
<?xml version='1.0'?>
<stream:stream to='maasaichat.com' version='1.0'
xmlns='jabber:client'
xmlns:stream='http://etherx.jabber.org/streams'>
<!-- Post-auth features -->
<stream:features>
<bind xmlns='urn:ietf:params:xml:ns:xmpp-bind'/>
<sm xmlns='urn:xmpp:sm:3'/>
</stream:features>
<!-- ============================================================ -->
<!-- LAYER 3: STANZAS BEGIN -->
<!-- ============================================================ -->
<!-- ---- STEP 5: RESOURCE BINDING (a real iq stanza) ---- -->
<iq type='set' id='bind-001'>
<bind xmlns='urn:ietf:params:xml:ns:xmpp-bind'>
<resource>android</resource>
</bind>
</iq>
<iq type='result' id='bind-001'>
<bind xmlns='urn:ietf:params:xml:ns:xmpp-bind'>
<jid>ole@maasaichat.com/android</jid> <!-- full JID exists now -->
</bind>
</iq>
<!-- ---- STEP 6: ENABLE STREAM MANAGEMENT (nonzas) ---- -->
<enable xmlns='urn:xmpp:sm:3' resume='true'/>
<enabled xmlns='urn:xmpp:sm:3' id='sm-session-xyz' resume='true'/>
<!-- Now Ole can survive the 2G drops near Ngorongoro -->
<!-- ---- STEP 7: FETCH ROSTER (iq) ---- -->
<iq type='get' id='roster-001'>
<query xmlns='jabber:iq:roster'/>
</iq>
<iq type='result' id='roster-001'>
<query xmlns='jabber:iq:roster'>
<item jid='naserian@maasaichat.com' name='Naserian' subscription='both'/>
<item jid='enkiama@maasaichat.com' name='Enkiama' subscription='both'/>
<item jid='nkeri@maasaichat.com' name='Nkeri' subscription='both'/>
</query>
</iq>
<!-- ---- STEP 8: GO ONLINE (presence) ---- -->
<!-- NOTE: no type attribute = available. There is no <show>available</show> -->
<presence>
<c xmlns='http://jabber.org/protocol/caps'
hash='sha-1' node='https://maasaichat.com' ver='q07IKJEyjvHSyhy//CH0CxmKi8w='/>
<!-- caps: "here's a hash of which XEPs I support" — saves re-asking -->
</presence>
<!-- ---- STEP 9: CONTACTS' PRESENCE ARRIVES ---- -->
<presence from='naserian@maasaichat.com/phone'>
<show>chat</show>
<status>Herding near Ngorongoro</status>
</presence>
<!-- ---- STEP 10: OLE SENDS THE MESSAGE ---- -->
<message from='ole@maasaichat.com/android'
to='naserian@maasaichat.com'
type='chat'
id='msg-001'>
<body>The cattle arrived safely at the river.</body>
<origin-id xmlns='urn:xmpp:sid:0' id='msg-001'/> <!-- XEP-0359 -->
<request xmlns='urn:xmpp:receipts'/> <!-- XEP-0184: confirm delivery -->
<markable xmlns='urn:xmpp:chat-markers:0'/> <!-- XEP-0333: allow read tick -->
</message>
<!-- Ole's client asks "did you get everything?" (nonza) -->
<r xmlns='urn:xmpp:sm:3'/>
<a xmlns='urn:xmpp:sm:3' h='5'/> <!-- server: "I have 5 stanzas from you" -->
<!-- ---- STEP 11: DELIVERY RECEIPT (Naserian's app, automatic) ---- -->
<message from='naserian@maasaichat.com/phone' to='ole@maasaichat.com'>
<received xmlns='urn:xmpp:receipts' id='msg-001'/>
</message>
<!-- Ole's UI: ✓ becomes ✓✓ -->
<!-- ---- STEP 12: NASERIAN IS TYPING ---- -->
<message from='naserian@maasaichat.com/phone'
to='ole@maasaichat.com' type='chat'>
<composing xmlns='http://jabber.org/protocol/chatstates'/>
<!-- no <body> — this is a pure signal, costs ~100 bytes -->
</message>
<!-- ---- STEP 13: NASERIAN REPLIES ---- -->
<message from='naserian@maasaichat.com/phone'
to='ole@maasaichat.com' type='chat' id='msg-002'>
<body>Great news! How many?</body>
<active xmlns='http://jabber.org/protocol/chatstates'/>
<origin-id xmlns='urn:xmpp:sid:0' id='msg-002'/>
<request xmlns='urn:xmpp:receipts'/>
<markable xmlns='urn:xmpp:chat-markers:0'/>
</message>
<!-- ---- STEP 14: OLE READS IT ---- -->
<message from='ole@maasaichat.com/android'
to='naserian@maasaichat.com' type='chat'>
<displayed xmlns='urn:xmpp:chat-markers:0' id='msg-002'/>
</message>
<!-- Naserian's UI: ✓✓ becomes the blue "read" tick -->
<!-- ---- STEP 15: KEEPALIVE (iq ping) ---- -->
<iq from='ole@maasaichat.com/android' to='maasaichat.com'
type='get' id='ping-001'>
<ping xmlns='urn:xmpp:ping'/>
</iq>
<iq from='maasaichat.com' to='ole@maasaichat.com/android'
type='result' id='ping-001'/>
<!-- ---- STEP 16: OLE CLOSES THE APP ---- -->
<presence type='unavailable'/>
<!-- ---- STREAM CLOSES ---- -->
</stream:stream>
<!-- TCP connection closes -->
That's a complete XMPP session. Every chapter in Part 2 is a deep-dive into one line of what you just read.
Two corrections worth burning in
Two mistakes appear constantly in blog posts and even in some tutorials:
<show>available</show>does not exist. The legal<show>values are onlyaway,chat,dnd,xa. "Available" is the default — signalled by a<presence/>with notypeattribute at all. Offline is<presence type='unavailable'/>.- SASL auth and Stream Management are not
iq.<auth/>,<success/>,<enable/>,<r/>,<a/>are nonzas. If you go looking for an<iq>wrapper around them you'll be confused forever.
3.7 What order is fixed, and what is free
FIXED — the setup handshake, always this order:
1. TCP connect
2. Stream open
3. Features → STARTTLS → (stream restarts)
4. Features → SASL auth → (stream restarts)
5. Resource bind → full JID assigned
6. (optional) Enable Stream Management
FREE — once bound, anything, any time, both directions:
Presence, roster fetch, messages, iq requests, pings...
Convention (not law) after binding: enable SM → fetch roster → send initial presence. Clients do it in that order because you want reliability on before traffic, and contacts loaded before you announce yourself.
After that, XMPP is fully asynchronous and bidirectional. Ole can send three messages without waiting for any reply; the server can push Naserian's presence in the middle of them. The one hard rule is that every iq you send with an id will get exactly one result or error back carrying that same id — that's how you match responses to requests on a pipe where anything can arrive at any moment.
Stream setup = a strict staircase, step by step
After binding = a busy two-way street
3.8 Tools — type stanzas yourself
Reading XML teaches you some of it. Typing raw stanzas at a live server and watching it answer teaches you all of it. Four tools, easiest first:
1. Gajim's XML Console (start here)
The best tool, and you likely already have it. Gajim → Accounts → Advanced → XML Console. It shows every stanza in and out and lets you type raw XML and send it.
Try this first — a ping:
<iq type='get' id='test-001' to='maasaichat.com'>
<ping xmlns='urn:xmpp:ping'/>
</iq>
Watch <iq type='result' id='test-001'/> come back. You just spoke XMPP by hand.
2. Psi
sudo apt install psi
Another desktop client with a cleaner, more technical XML console. Some people prefer it purely for learning.
3. websocat — raw stream from the terminal
Talk to Ejabberd's WebSocket endpoint with nothing in between:
websocat --protocol xmpp ws://localhost:5280/ws
Then paste the stream opener and watch the server's raw reply:
<open xmlns='urn:ietf:params:xml:ns:xmpp-framing'
to='maasaichat.com' version='1.0'/>
(The WebSocket binding uses <open/> instead of <stream:stream> — same idea, framed differently. Note the --protocol xmpp flag; Ejabberd requires that subprotocol header.)
4. A throwaway Python script
For scripted sequences, nothing beats a socket:
import socket, ssl
sock = socket.create_connection(("localhost", 5222))
sock.send(b"""<?xml version='1.0'?>
<stream:stream to='maasaichat.com' version='1.0'
xmlns='jabber:client' xmlns:stream='http://etherx.jabber.org/streams'>""")
print(sock.recv(4096).decode()) # server's stream + features
You'll get the stream header and <stream:features> back — proof you're speaking the protocol with 6 lines of code. (To go further than STARTTLS you'd wrap the socket with ssl; easier to let a library handle it — see Part 4.)
5. Server-side: send a stanza as the admin
Ejabberd can inject a stanza for you from the CLI — useful for testing what a client receives:
docker exec ejabberd ejabberdctl send_stanza \
'maasaichat.com' 'ole@maasaichat.com' \
'<message type="chat"><body>Test from the server</body></message>'
Best for visual learning: Gajim XML console
Best for seeing the truth: websocat / raw socket
Best for automation: ejabberdctl send_stanza
3.9 A practice loop that actually works
Four short sessions will give you real fluency:
1. ECHO Open Gajim's console. Just watch. Send a message
in the UI and read the XML it produced.
2. HAND-SEND Type a ping by hand. Then a message. Then a
<composing/> chat state. Watch the other client react.
3. FROM SCRATCH Pick a goal ("mark msg-002 as read"), write the
stanza with no reference, send it, see if it works.
4. DECODE Grab any unknown stanza from the console and answer:
which type? which xmlns/XEP? what does it do?
what response should come back?
Step 4 is the one that matters. When you can look at a stanza, spot the xmlns, and say "that's XEP-0333, it's a read marker, no response expected" — you've stopped memorizing and started reading XMPP.
✅ What you learned in this chapter
<stream:stream>is not a stanza — it's the container. The whole session is one long XML document whose root stays open for hours; stanzas are its children.- There are three layers: TCP+TLS (wire) → stream (envelope) → stanzas (conversation).
- Only
<message>,<presence>,<iq>are stanzas. Everything else inside the stream — features, STARTTLS, SASL,<enable/>,<r/>,<a/>— is a nonza. Stanzas travel; nonzas negotiate.- Reading XMPP means reading four things: element, attributes, children, and
xmlns— and thexmlnsis the pointer to the XEP that explains everything else.- A namespace is a URN — a permanent name, never a location. Nothing is ever fetched from it; it's a shared agreement. The trailing digit is a version (
urn:xmpp:sm:3), and:0means the XEP is still experimental.- Namespaces come in three generations —
jabber:*,http://jabber.org/protocol/*, and modernurn:xmpp:*— all used the same way. The URL-shaped ones are still never fetched.- You walked one complete session, TCP handshake → STARTTLS → SASL → bind → SM → roster → presence → message → receipt → typing → read marker → ping → unavailable → close.
- Setup order is fixed (connect → TLS → auth → bind); everything after binding is free-form and asynchronous — with the one rule that every
iqgets exactly one matchingresult/errorbyid.- Two myths corrected:
<show>available</show>isn't a thing (available = presence with no type), and SASL/SM are notiq.- Tools to speak XMPP by hand: Gajim's XML console, Psi, websocat (
--protocol xmpp), a raw Python socket, andejabberdctl send_stanza.
Ready for next chapter? (Chapter 4 — The JID Address System: bare vs full JIDs, how Ejabberd decides which of Ole's devices gets a message, priorities, resource conflicts, and the addressing rules that quietly break apps that get them wrong.)
NexGate — Private Chat & Calls Flow
NexGate Chat System — Development Architecture Guide
QBIT SPARK | NexGate Platform The complete reference for building the chat_system module inside nexgate_backend
Table of Contents
- Overview & Philosophy
- The Four Tunnels — How chat_system Works
- Inbox Model — Three Separate Inboxes
- Cards — Rich Content Type
- Content Types & User Boxes
- Where Chat Lives in the Codebase
- The Three Communication Channels
- Full System Diagram
- Authentication Flow
- Message Flow — 1:1 Chat
- Message Flow — Group Chat
- Commerce DM Flow
- Call Flow — Voice & Video
- Secret Chat — End-to-End Encryption
- NexGate Stanza Standard
- NexGate Custom Namespaces
- RabbitMQ Event Pipeline
- Shop Inbox & Staff Access
- Offline & Push Notification Flow
- Multi-Device Support
- Redis — What to Cache & Why
- Best Practices & Anti-Patterns
- Package Structure
- Infrastructure Stack
- Build Order
1. Overview & Philosophy
What NexGate Chat Is
NexGate chat is NOT a standalone service.
It is a package inside nexgate_backend.
Three pillars of NexGate:
VP Social → social content
VP Shop → commerce
VP Events → event management
Chat → the connective tissue between all three
Chat is deeply integrated with commerce.
Every conversation can become a transaction.
Every transaction has a conversation behind it.
The Key Principle
Ejabberd = the transport layer (moves stanzas)
Spring Boot = the business logic layer (decides what happens)
PostgreSQL = the persistence layer (stores everything)
RabbitMQ = the event bus (connects Ejabberd to Spring Boot)
Ejabberd knows NOTHING about:
NexGate users
Orders
Shops
Offers
Commerce
Spring Boot knows EVERYTHING about:
Who can talk to whom
What messages mean
Commerce context
Offer lifecycle
Notifications
Why Not a Separate Microservice
Chat is INSIDE nexgate_backend because:
Shares user data constantly
(display names, avatars, shop memberships)
Shares order data
(order confirmations, shipping updates)
Shares product data
(product cards, offer sessions)
No distributed transactions needed
Same team, same codebase
Simple joins instead of API calls
Extract chat service later IF:
Chat traffic overwhelms platform
Separate team manages chat
DB connections exhausted by chat
NOT before — premature optimization
2. The Four Tunnels — How chat_system Works
Inspiration
Inspired by the Cu Chi tunnel system in Vietnam —
250km of specialized tunnels, each built for a
specific purpose, each carrying specific content,
all invisible to those above.
chat_system works the same way:
Four specialized tunnels
Each carries specific content type
Each powered by the right engine
All meet at the Central Station (Ejabberd)
All invisible to the network above
The Four Tunnels
┌─────────────────────────────────────────────────────────┐
│ chat_system │
│ │
│ ┌─────────────┐ ┌─────────────┐ ┌─────────────┐ │
│ │ TEXT │ │ MEDIA │ │ VOICE │ │
│ │ TUNNEL │ │ TUNNEL │ │ TUNNEL │ │
│ │ │ │ │ │ │ │
│ │ Messages │ │ Images │ │ 1:1 Audio │ │
│ │ Cards │ │ Videos │ │ Group Audio│ │
│ │ Typing │ │ Voice note │ │ │ │
│ │ Receipts │ │ Documents │ │ │ │
│ │ Secret🔒 │ │ GIFs │ │ │ │
│ │ │ │ Stickers │ │ │ │
│ └──────┬──────┘ └──────┬──────┘ └──────┬──────┘ │
│ │ │ │ │
│ ┌──────▼──────────────────────────────────────────┐ │
│ │ EJABBERD (Central Station) │ │
│ │ Routes ALL tunnels to destination │ │
│ └──────────────────────────────────────────────────┘ │
│ │ │ │ │
│ ┌──────▼──────┐ │ ┌──────▼──────┐ │
│ │ VIDEO │ │ │ COTURN │ │
│ │ TUNNEL │ │ │ LIVEKIT │ │
│ │ │ │ │ RELAY │ │
│ │ 1:1 Video │ │ │ ENGINES │ │
│ │ Group Video│ │ │ │ │
│ │ Screen share│ │ │ │ │
│ └─────────────┘ │ └─────────────┘ │
│ FILE THUNDER │
│ processes media │
│ before delivery │
└─────────────────────────────────────────────────────────┘
Tunnel 1 — Text Tunnel
What it carries:
Plain text messages
Cards (all 6 categories)
Typing indicators
Delivery receipts (✓✓)
Read receipts (🔵)
Reactions (👍❤️😂)
Message edits + deletes
Presence (online/offline)
Commerce stanzas
Group invitations
System notifications
Secret Chat (encrypted) 🔒
Engine: Ejabberd (XMPP)
Protocol: XML stanzas over WebSocket/TCP
Port: 5222 with TLS
Security levels inside Text Tunnel:
Level 1 — Normal: stored in DB (readable)
Level 2 — Commerce: auditable, legal record
Level 3 — Secret Chat: OMEMO encrypted
nobody can read, not even NexGate
Tunnel 2 — Media Tunnel
What it carries:
Images (JPEG, PNG, WebP)
Videos (MP4, HLS)
Voice notes (OGG/Opus)
Documents (PDF, DOC etc)
GIFs (MP4 loop)
Stickers (WebP)
Files (any type)
Engines:
File Thunder → processes + stores + CDN
Ejabberd → delivers the stanza with CDN URL
Flow:
Sender uploads → File Thunder processes
File Thunder → CDN URL
Ejabberd stanza → delivers URL to recipient
Recipient → downloads from CDN directly
Ejabberd never touches the file itself
Only delivers the URL stanza ✅
Tunnel 3 — Voice Tunnel
What it carries:
1:1 voice calls
Group voice calls (3+ people)
Engines:
Ejabberd → Jingle signaling (call setup)
WebRTC → audio capture + encoding (Opus)
Coturn → relay when P2P blocked (EA NAT)
LiveKit → group calls (3+ people only)
1:1 voice:
Try P2P direct first
If blocked (carrier NAT) → Coturn relay
Engine: WebRTC + Coturn
Group voice (3+):
Always LiveKit SFU
No P2P attempt
Each person uploads 1 stream
LiveKit forwards to all others
Switching 1:1 → group mid-call:
Spring Boot creates LiveKit room
Sends join stanzas to all
Old WebRTC terminates
All join LiveKit seamlessly ✅
Tunnel 4 — Video Tunnel
What it carries:
1:1 video calls
Group video calls (3+ people)
Screen sharing
Engines:
Ejabberd → Jingle signaling
WebRTC → video capture + H.264 encoding
Coturn → relay when P2P blocked
LiveKit → group calls (3+ people only)
1:1 video:
Try WebRTC P2P first
If blocked → Coturn relay
Engine: WebRTC + Coturn
Group video (3+):
Always LiveKit SFU
Simulcast: 3 quality levels per stream
Low (180p): 2G networks
Medium (360p): 3G networks
High (720p): 4G/WiFi
Each receiver gets quality their network allows
Screen share:
Android: MediaProjection API
iOS: ReplayKit broadcast extension
H.264 encoding, 5-15fps
Works in both 1:1 and group calls
Switching 1:1 → group mid-call:
Same as Voice Tunnel upgrade flow ✅
Why Not LiveKit for Everything?
Question: why not use LiveKit for 1:1 calls too?
WebRTC P2P (1:1):
Alice → directly → Bob (no server)
Zero extra latency ✅
Zero server cost ✅
Privacy: nobody in the middle ✅
Faster connection ✅
LiveKit SFU (group):
Alice → LiveKit server → Bob
Extra server hop = more latency ❌
Server cost per call ❌
Server sees media ❌
Rule: P2P when 2 people (always faster + cheaper)
SFU when 3+ people (P2P impossible on mobile)
Why P2P fails for groups:
5 people P2P mesh:
Each uploads 4 streams
Each downloads 4 streams
= 8 streams simultaneously
= 8Mbps needed per person ❌
EA 4G average = 5Mbps ❌
Impossible on EA networks ❌
5 people via LiveKit SFU:
Each uploads 1 stream only ✅
LiveKit forwards to others
Feasible on EA 3G/4G ✅
The Engine Registry
Engine Tunnel(s) Role
──────────────────────────────────────────────────────────
Ejabberd ALL (signaling) Central Station
Routes everything
XMPP stanzas
File Thunder Media Tunnel Process + store
Virus scan
CDN delivery
WebRTC Voice + Video Audio/video capture
Encode/decode
On-device processing
Coturn Voice + Video Relay station
Bypasses EA NAT
TURN protocol
LiveKit SFU Voice + Video Group calls (3+)
(group only) Simulcast engine
One→many forwarding
OMEMO Text Tunnel Encryption engine
(Secret Chat) Keys on device only
Level 3 security
Tunnel Security Levels
Inspired by Cu Chi tunnel depth levels:
Level 1 — Surface (Normal Chat):
Regular conversations
Stored in DB (Spring Boot can read)
Used for: casual chat, group chat
Engine: Ejabberd + PostgreSQL
Level 2 — Mid-depth (Commerce DMs):
Business conversations
Stored + auditable
Legal record for disputes
Staff audit log
Used for: shop DMs, order updates
Engine: Ejabberd + PostgreSQL (audit)
Level 3 — Deep (Secret Chat):
End-to-end encrypted (OMEMO)
Encrypted blob stored in DB
NexGate CANNOT read content ✅
Keys NEVER leave device ✅
Used for: private personal conversations
Engine: OMEMO + Ejabberd (transport only)
3. Inbox Model — Three Separate Inboxes
The Three Inboxes
NexGate has THREE completely separate chat inboxes:
Inbox 1 — Personal:
1:1 DMs with other users (@username)
Group chats (friends, community)
Accessed via: Chat → Personal tab
Inbox 2 — Commerce:
DMs with shops ($tag)
Order updates from shops
Offer sessions
Accessed via: Chat → Commerce tab
Inbox 3 — Secret 🔒:
End-to-end encrypted 1:1 conversations
Completely separate from personal chats
Handled like Telegram Secret Chat model:
Long press contact → Start Secret Chat
Opens parallel secret conversation
Same contact can have both normal + secret
Accessed via: Chat → Secret tab
OR: lock icon 🔒 entry point from contact profile
Why three separate?
Personal: social conversations (friends, groups)
Commerce: transactional (shops, orders)
Secret: private E2EE (fully isolated)
User never loses personal chats in orders ✅
User never accidentally sends to wrong chat ✅
Clean mental model ✅
Telegram model for Secret = familiar ✅
Conversation Types
DIRECT:
Between two @users
Normal 1:1 conversation
Inbox: Personal
Security: Level 1
GROUP:
Multiple @users
Created explicitly
Inbox: Personal
Security: Level 1
DIRECT_SECRET:
Between two @users
End-to-end encrypted (OMEMO)
Parallel to normal DIRECT (same contact)
Inbox: Secret 🔒
Security: Level 3
Telegram model: contact has both DIRECT + DIRECT_SECRET
COMMERCE_DM:
Between @user and $shop
Initiated by either side
Contains: product cards, offers, order updates
Inbox: Commerce
Security: Level 2 (auditable)
SYSTEM:
Auto-generated per order
No human on shop side (automated)
Order confirmation, shipping updates
Inbox: Commerce
Security: Level 2 (immutable)
NOTE: Bei Ya Pamoja is NOT a conversation type
It is a CARD (GROUP_BUY) that can be shared in:
DIRECT conversations ✅
GROUP conversations ✅
COMMERCE_DM conversations ✅
NOT in DIRECT_SECRET ❌
How Conversations Are Created
DIRECT — user taps on @username → start chat:
POST /chat/conversations/direct
{ targetUserId }
Spring Boot: find existing OR create new
type = DIRECT
GROUP — user creates a group:
POST /chat/groups/create
{ name, type: PRIVATE|PUBLIC }
Spring Boot: create + Ejabberd room
type = GROUP
DIRECT_SECRET — long press contact → Start Secret Chat:
POST /chat/conversations/secret
{ targetUserId }
Spring Boot: find existing OR create new secret thread
type = DIRECT_SECRET
App: initializes OMEMO session with target's public keys
COMMERCE_DM — buyer taps [Chat with Shop]:
POST /chat/conversations/commerce
{ shopId }
Spring Boot: find existing OR create new
Ejabberd: uses shop JID (shop-456@nexgate.com)
type = COMMERCE_DM
COMMERCE_DM — order placed (auto):
Spring Boot order service triggers:
CommerceDmService.getOrCreateCommerceThread(userId, shopId)
First message auto-sent: "Your order ORD-789 confirmed"
type = COMMERCE_DM
SYSTEM — one per order (auto):
Spring Boot: createSystemConversation(orderId)
Only system messages — user cannot reply
type = SYSTEM
Inbox API — What Mobile Calls
Personal inbox:
GET /chat/conversations?type=personal
Returns: DIRECT + GROUP conversations
Sorted by: last_message DESC
Commerce inbox:
GET /chat/conversations?type=commerce
Returns: COMMERCE_DM + SYSTEM conversations
Sorted by: last_message DESC
Grouped by: shop (all TechStore threads together)
Secret inbox:
GET /chat/conversations?type=secret
Returns: DIRECT_SECRET conversations
Sorted by: last_message DESC
Note: content previews NOT shown (encrypted)
Shows: contact name + "Secret Chat 🔒" only
Unread counts:
GET /chat/conversations/unread-counts
Returns:
{ personal: 3, commerce: 7, secret: 1 }
Source: Redis (never DB query)
Used for: tab badges in UI
Secret tab badge: shows count only (no preview)
Shop Side — Shop Inbox
Shop staff logs into NexGate shop dashboard:
Sees: ALL conversations with customers
Sorted by: unread first, then recent
Shop inbox API:
GET /chat/shop/{shopId}/conversations
Auth: requires shop staff JWT
Returns: all COMMERCE_DM for this shop
Shows: customer name, last message, unread
Staff replies:
POST /chat/conversations/{convId}/messages
{ body, attachments }
Sent as: shop-456@nexgate.com (no staff name) ✅
Shop cannot see:
Personal conversations ❌
Secret conversations ❌
Other shops' conversations ❌
JID Mapping Per Conversation Type
DIRECT (alice → juma):
Alice JID: usr-alice@nexgate.com
Juma JID: usr-juma@nexgate.com
Ejabberd: direct XMPP stanza ✅
DIRECT_SECRET (alice → juma, encrypted):
Alice JID: usr-alice@nexgate.com
Juma JID: usr-juma@nexgate.com
Ejabberd: same routing as DIRECT ✅
BUT: payload is OMEMO encrypted
Ejabberd cannot read content ✅
Separate conversation ID from DIRECT
GROUP (warriors group):
Room JID: conv-abc123@conference.nexgate.com
Members: usr-alice, usr-juma, usr-enkiama
Ejabberd: MUC room ✅
COMMERCE_DM (alice → TechStore):
Alice JID: usr-alice@nexgate.com
Shop JID: shop-456@nexgate.com
Staff: shop-456@nexgate.com/staff-1 (internal)
Customer sees: shop-456@nexgate.com ✅
SYSTEM (order notification):
From JID: system@nexgate.com
To JID: usr-alice@nexgate.com
type: headline (no offline storage trigger)
Spring Boot sends via Ejabberd REST
Conversation Routing Rules
Message arrives from: usr-juma@nexgate.com (normal):
→ Check: is this DIRECT or DIRECT_SECRET?
→ Spring Boot checks conversation metadata
→ Route to correct conversation type
→ Inbox: Personal (DIRECT) or Secret (DIRECT_SECRET)
Message arrives from: usr-juma@nexgate.com (OMEMO):
→ Has OMEMO encryption element ✅
→ Route to DIRECT_SECRET conversation
→ Inbox: Secret 🔒
Message arrives from: shop-456@nexgate.com:
→ Route to COMMERCE_DM ✅
→ Inbox: Commerce
Message arrives from: system@nexgate.com:
→ Route to SYSTEM conversation for that order
→ Inbox: Commerce
Message arrives from: conv-abc123@conference.nexgate.com:
→ Route to GROUP conversation ✅
→ Inbox: Personal
4. Cards — Rich Content Type
What Cards Are
Cards are rich interactive content blocks
that travel inside the Text Tunnel.
Like a specialized cargo train
running inside the text tunnel:
Same tunnel (Ejabberd routes it) ✅
Same transport (XMPP stanza) ✅
Different cargo (rich structured data) ✅
Different rendering (UI card) ✅
Every card:
Has a custom xmlns (identifies it) ✅
Has a type field (what kind exactly) ✅
Has a <body> fallback (for basic clients) ✅
Has action buttons (1-3 max) ✅
Renders as visual card in ChatBox ✅
The Six Card Categories
Category 1 — Social Cards
Source: VP Social
xmlns: urn:nexgate:social:1
Types:
POST_CARD → feed post shared in chat
REEL_CARD → short video shared
LIVE_CARD → live stream invite
PROFILE_CARD → user profile share
Category 2 — Commerce Cards
Source: VP Shop
xmlns: urn:nexgate:commerce:1
urn:nexgate:offer:1
urn:nexgate:groupbuy:1
urn:nexgate:installment:1
Types:
PRODUCT_CARD → single product
SHOP_CARD → shop profile
FLASH_SALE_CARD → time-limited sale
INSTALLMENT_PLAN → buy now pay later
CUSTOM_PRICE_OFFER → private negotiated price
item_type: PRODUCT → offer on a product
item_type: TICKET → offer on an event ticket
(item_type field inside <item> determines rendering)
GROUP_BUY → Bei ya pamoja card
item_type: PRODUCT → group buy of a product
item_type: TICKET → group buy of event tickets
Can be shared: in 1:1 DM, group chat, commerce DM
NOT a group feature — it is a CARD ✅
Category 3 — Event Cards
Source: VP Events
xmlns: urn:nexgate:event:1
Types:
EVENT_CARD → event details
TICKET_CARD → purchased ticket
EVENT_REMINDER → reminder notification
Category 4 — System Cards
Source: Spring Boot (automated, no human)
xmlns: urn:nexgate:system:1
Types:
ORDER_CONFIRMATION → order placed ✅
ORDER_STATUS_UPDATE → shipped/delivered
PAYMENT_CONFIRMATION → payment received
REFUND_CARD → refund processed
DISPUTE_CARD → dispute opened
Category 5 — Group Cards
Source: chat_system
xmlns: urn:nexgate:group:1
Types:
GROUP_INVITATION → join group request
user sees card, taps Accept/Decline
this is the ONLY group CARD ✅
Category 6 — Call Signals
Source: chat_system (NOT cards — system signals)
xmlns: urn:nexgate:call:1
Note: These are NOT interactive cards
They are system signals that trigger UI states
(incoming call screen, missed call indicator etc)
Types:
CALL_INITIATED → triggers incoming call screen
CALL_ACCEPTED → call connected
CALL_DECLINED → other side declined
CALL_ENDED → call finished
CALL_MISSED → nobody answered
GROUP_CALL_JOIN_INFO → join info for group call
PARTICIPANT_ADDED → someone added mid-call
HOST_MUTED_YOU → system: you were muted
HOST_REMOVED_YOU → system: you were removed
Category 7 — API Template Cards (Future)
Source: Third-party developers via API
xmlns: urn:nexgate:template:1
Types:
TRANSACTIONAL → order/delivery updates
MARKETING → promotions (opt-in only)
AUTHENTICATION → OTP, verification
Card Anatomy
Every card follows this structure:
┌─────────────────────────────────────┐
│ HEADER │
│ Category icon + source label │
│ e.g. 🛍️ VP Shop │
│─────────────────────────────────────│
│ BODY │
│ Card-specific content │
│ Image, title, price, progress etc │
│─────────────────────────────────────│
│ FOOTER │
│ Action buttons (1-3 max) │
│ Status indicator │
└─────────────────────────────────────┘
Every card also carries a shareable flag:
shareable=true → Forward option shown in UI ✅
shareable=false → Forward option hidden ❌
Cannot be forwarded via API ✅
Set by: Spring Boot (backend decides)
Enforced by: Mobile UI + Spring Boot API
Example — Product Card:
┌─────────────────────────────────────┐
│ 🛍️ VP Shop │
│─────────────────────────────────────│
│ [product image] │
│ Samsung A15 │
│ TZS 450,000 · TechStore │
│ Stock: 12 units │
│─────────────────────────────────────│
│ [View Product] [Chat with Shop] │
└─────────────────────────────────────┘
Example — Order Confirmation:
┌─────────────────────────────────────┐
│ ⚙️ Order Update │
│─────────────────────────────────────│
│ ✅ Order Confirmed │
│ ORD-789 · Samsung A15 │
│ TZS 400,000 · M-PESA │
│─────────────────────────────────────│
│ [Track Order] │
└─────────────────────────────────────┘
Example — Group Buy:
┌─────────────────────────────────────┐
│ 👥 Group Purchase │
│─────────────────────────────────────│
│ Samsung A15 │
│ Public: TZS 450,000 │
│ Group: TZS 350,000 (10 people) │
│ ████████░░ 8 / 10 joined │
│ Expires: 2h 30m │
│─────────────────────────────────────│
│ [Join Group Buy] │
└─────────────────────────────────────┘
Shareable Flag — Default Values Per Card
Card shareable Reason
──────────────────────────────────────────────────────────
PRODUCT_CARD true ✅ share to discover
SHOP_CARD true ✅ share to discover
FLASH_SALE_CARD true ✅ share = more buyers
INSTALLMENT_PLAN false ❌ personal finance
CUSTOM_PRICE_OFFER false ❌ private negotiation
NEVER shareable
GROUP_BUY true ✅ share = more joiners
EVENT_CARD true ✅ invite others
TICKET_CARD false ❌ personal ticket
EVENT_REMINDER false ❌ personal reminder
ORDER_CONFIRMATION false ❌ personal order
ORDER_STATUS_UPDATE false ❌ personal order
PAYMENT_CONFIRMATION false ❌ financial record
REFUND_CARD false ❌ financial record
DISPUTE_CARD false ❌ private dispute
GROUP_INVITATION false ❌ personal invite
POST_CARD true ✅ social sharing
REEL_CARD true ✅ social sharing
LIVE_CARD true ✅ invite others
PROFILE_CARD true ✅ share contact
TEMPLATE (TRANSACTIONAL) false ❌ personal
TEMPLATE (MARKETING) false ❌ personal
TEMPLATE (AUTH/OTP) false ❌ NEVER share OTP
Card Stanza Structure
All cards use the unified <ng> stanza standard.
See Section 15 — NexGate Stanza Standard
for complete stanza examples per card type.
Key rule:
Every card carries <ng xmlns="urn:nexgate:1">
with <meta> + <payload>
card_type field inside <payload> identifies the card
shareable field inside <meta> controls forwarding
Mobile Card Decision Tree
Message arrives:
1. Find <ng xmlns="urn:nexgate:1"> element
2. Read <meta> → message_type = CARD
3. Read <payload> → check card_type field:
card_type = POST_CARD → post preview + [View Post]
card_type = LIVE_CARD → live badge + [Watch Live]
card_type = REEL_CARD → reel preview + [Watch]
card_type = PROFILE_CARD → profile card + [Follow]
card_type = PRODUCT_CARD → product + [View] [Chat]
card_type = SHOP_CARD → shop profile + [Visit]
card_type = FLASH_SALE_CARD → sale card + [Buy Now]
card_type = CUSTOM_PRICE_OFFER:
status=PENDING → offer + timer + [Accept] [Decline]
status=ACCEPTED → "Offer Accepted ✅"
status=DECLINED → "Offer Declined"
status=EXPIRED → "Offer Expired ⏰"
card_type = GROUP_BUY:
→ progress bar + members preview + [Join]
→ check item_type: PRODUCT or TICKET
card_type = EVENT_CARD → event + [View] [Get Ticket]
card_type = TICKET_CARD → ticket QR code + [View]
card_type = ORDER_CONFIRMATION → order + [Track]
card_type = ORDER_STATUS_UPDATE → status badge
card_type = PAYMENT_CONFIRMATION → payment details
card_type = GROUP_INVITATION → [Accept] [Decline]
card_type = CALL_INITIATED → incoming call screen
card_type = GROUP_CALL_JOIN_INFO → [Join] [Decline]
card_type = CALL_MISSED → missed call indicator
card_type = TRANSACTIONAL → template card
card_type = MARKETING → template + unsubscribe
No card_type match?
→ render <body> fallback as plain text ✅
5. Content Types & User Boxes
The Five Content Types
chat_system delivers exactly 5 content types:
Type 1 — Text Plain
What: pure text messages
Tunnel: Text Tunnel
Engine: Ejabberd
Example: "Hello Juma!"
Type 2 — Cards
What: rich interactive content blocks
Tunnel: Text Tunnel (same engine)
Engine: Ejabberd
6 categories: Social, Commerce, Event,
System, Group, Template
Example: Product card, Order update
Type 3 — Media Bundle
What: files users share
Tunnel: Media Tunnel
Engine: File Thunder + Ejabberd
Types: Image, Video, Voice note,
Document, GIF, Sticker
Type 4 — Audio Stream
What: real-time voice
Tunnel: Voice Tunnel
Engine: WebRTC + Coturn (1:1)
LiveKit SFU (group)
Includes: 1:1 voice, group voice
Type 5 — Video Stream
What: real-time video
Tunnel: Video Tunnel
Engine: WebRTC + Coturn (1:1)
LiveKit SFU (group)
Includes: 1:1 video, group video, screen share
Three User-Facing Boxes
Box 1 — Regular ChatBox:
What it shows: normal conversation thread
Security: Level 1 (casual) + Level 2 (commerce)
Delivers:
Type 1: Text Plain → message bubbles
Type 2: Cards → rich interactive blocks
Type 3: Media Bundle → image/video/voice player
Conversations: DMs, Groups, Commerce DMs
Box 2 — Secret ChatBox 🔒:
What it shows: E2EE conversation thread
Security: Level 3 (OMEMO encrypted)
Different UI treatment:
Lock icon in header 🔒
"End-to-end encrypted" banner
No screenshots (Android FLAG_SECURE)
Self-destruct timer (optional) ⏱️
Delivers:
Type 1: Text Plain (encrypted)
Type 3: Media Bundle (encrypted)
NOT: Cards (commerce requires server-side logic)
NOT: Call Signals (calls not E2EE via OMEMO)
Conversations: Secret Chat 1:1 only
Box 3 — CallBox:
What it shows: call screen
Delivers:
Type 4: Audio Stream → voice call UI
Type 5: Video Stream → video call UI
1:1 CallBox:
Two participants
Microphone, camera, hang up controls
Audio ↔ Video switch (no hang up needed)
Group CallBox:
Grid of participant tiles
Raise hand ✋
Emoji reactions 😂❤️🎉
Host controls (mute, remove)
Pin participant
Speaker/grid layout switch
Call Actions — Who Handles What
Action Handler
──────────────────────────────────────────────────────
Mute/unmute mic Local WebRTC (device)
Camera on/off Local WebRTC (device)
Switch front/back cam Local WebRTC (device)
Speaker/earpiece Local OS audio
Hang up Jingle session-terminate
Raise hand ✋ LiveKit metadata broadcast
Lower hand LiveKit metadata broadcast
Emoji reaction 😂 LiveKit sendData() broadcast
Pin participant Local UI only (no server)
Switch grid/speaker Local UI only (no server)
Disable incoming video Local WebRTC (save data)
Mute participant (host) LiveKit SDK + Spring Boot auth
Remove participant LiveKit SDK + Spring Boot auth
Add participant mid-call Spring Boot (new LiveKit room)
→ sends join stanzas to all
→ old WebRTC terminates
→ all migrate to LiveKit
Audio → Video switch Jingle content-add
Video → Audio switch Jingle content-remove
Screen share start Jingle content-add (screen)
Screen share stop Jingle content-remove (screen)
1:1 → Group Call Upgrade Flow
Alice + Bob in 1:1 WebRTC call
Alice taps "Add Participant" → selects Juma
Spring Boot:
1. Creates new LiveKit room
2. Generates tokens for Alice, Bob, Juma
3. Sends ng stanza to Alice:
message_type: CALL
signal_type: GROUP_CALL_JOIN_INFO
reason: UPGRADED_FROM_P2P
livekit_token: eyJ-alice...
See Section 15 Example 11 ✅
4. Sends same to Bob (his token)
5. Sends incoming call to Juma (his token)
Alice + Bob apps:
Receive stanza
Terminate WebRTC P2P
Connect to LiveKit room
Seamless — no hang up needed ✅
Juma's app:
Incoming group call notification
[Join] button
Joins LiveKit room ✅
Result: 3-way group call on LiveKit ✅
6. Where Chat Lives in the Codebase
Package Structure
com.nexgate.backend
├── auth/ ← JWT, XMPP token issuance
├── feed/ ← VP Social
├── shop/ ← VP Shop
├── events/ ← VP Events
├── notification/ ← FCM, Textfy SMS
│
└── chat/ ← EVERYTHING CHAT RELATED
├── config/ ← Ejabberd config, RabbitMQ config
├── controller/ ← REST endpoints for mobile
├── service/ ← Business logic
├── consumer/ ← RabbitMQ event consumers
├── ejabberd/ ← Ejabberd REST client + auth bridge
├── stanza/ ← Custom XMPP stanza builders
├── commerce/ ← Commerce DM flows
├── call/ ← Call management (TURN credentials)
└── model/ ← Chat domain models (NO DB entities here)
Database
nexgate_postgres (port 5432)
├── schema: core ← nexgate_backend owns
│ users, shops, products, orders
│
└── schema: chat ← chat package owns
conversations
messages
message_reactions
message_receipts
calls
offer_sessions
group_invite_links
notification_log
7. The Three Communication Channels
Channel 1 — Spring Boot → Ejabberd (REST API)
Direction: Spring Boot calls Ejabberd
Protocol: HTTP REST (port 5280)
Auth: Admin credentials (stored in HashiCorp Vault)
Used for: Sending stanzas, creating rooms, managing members
Examples:
Send product card to buyer
Create group room on event ticket purchase
Kick user from shop group
Send order confirmation stanza
Flow:
Spring Boot
→ POST http://ejabberd:5280/api/send_message
→ Authorization: Basic admin@nexgate.com:adminpass
→ Body: { from, to, body + custom stanza }
→ Ejabberd routes to recipient ✅
Channel 2 — Ejabberd → Spring Boot (RabbitMQ)
Direction: Ejabberd fires events → Spring Boot consumes
Protocol: RabbitMQ AMQP
Auth: RabbitMQ credentials (stored in Vault)
Used for: Knowing when messages arrive, presence changes, calls
Examples:
Message delivered → Spring Boot persists to DB
User goes offline → Spring Boot updates last seen
Call started → Spring Boot creates call record
Message read → Spring Boot updates receipt status
Flow:
Ejabberd (mod_rabbitmq)
→ publishes to exchange: nexgate.chat
→ routing key: chat.message.inbound
→ Spring Boot consumer picks up
→ persists + triggers notifications ✅
Channel 3 — Mobile → Ejabberd → Spring Boot (JWT)
Direction: Mobile connects to Ejabberd
Ejabberd validates via Spring Boot JWT
Protocol: XMPP over WebSocket/TCP (port 5222) with TLS
Auth: JWT token issued by Spring Boot on login
Used for: User XMPP sessions (all chat activity)
Flow:
1. Mobile → POST /auth/login (Spring Boot)
Spring Boot validates credentials
Issues TWO tokens:
REST JWT → for API calls (7 days)
XMPP JWT → for Ejabberd (24 hours)
2. Mobile → Ejabberd port 5222
Username: usr-kibuti@nexgate.com
Password: <XMPP JWT>
3. Ejabberd validates JWT locally
Uses public key from JWKS endpoint:
GET https://api.nexgate.com/auth/.well-known/jwks.json
Verifies signature + expiry
Allows connection ✅
4. User is in — can send/receive stanzas
8. Full System Diagram
┌─────────────────────────────────────────────────────────────────────┐
│ NexGate Platform │
│ │
│ ┌──────────────┐ ┌─────────────────────────────────────┐ │
│ │ Mobile App │ │ nexgate_backend │ │
│ │ (Android/ │ │ │ │
│ │ iOS) │ │ ┌─────────┐ ┌─────────────────┐ │ │
│ │ │◀────────┼──│ auth/ │ │ chat/ │ │ │
│ │ Smack (XMPP)│ REST │ │ JWT │ │ │ │ │
│ │ OkHttp(REST)│ API │ │ XMPP │ │ controller/ │ │ │
│ │ │ │ │ token │ │ service/ │ │ │
│ └──────┬───────┘ │ └─────────┘ │ consumer/ │ │ │
│ │ │ │ ejabberd/ │ │ │
│ │ XMPP+TLS │ │ stanza/ │ │ │
│ │ port 5222 │ │ commerce/ │ │ │
│ │ │ │ call/ │ │ │
│ ▼ │ └────────┬────────┘ │ │
│ ┌──────────────┐ │ │ │ │
│ │ Ejabberd │ │ Channel 1│ REST API │ │
│ │ Cluster │◀────────┼────────────────────────┘ │ │
│ │ │ Channel1│ │ │
│ │ Node 1 │ │ ┌──────────────────────────────┐ │ │
│ │ Node 2 │─────────┼─▶│ RabbitMQ │ │ │
│ │ │ Channel2│ │ exchange: nexgate.chat │ │ │
│ │ JWT auth │ │ │ queues: │ │ │
│ │ via JWKS │ │ │ chat.message.inbound │ │ │
│ └──────────────┘ │ │ chat.presence │ │ │
│ │ │ │ chat.call │ │ │
│ │ │ └──────────────┬───────────────┘ │ │
│ │ │ │ Channel 2 │ │
│ │ │ ▼ │ │
│ │ │ chat/consumer/ │ │
│ │ │ persists messages │ │
│ │ │ triggers FCM │ │
│ │ │ updates receipts │ │
│ │ └─────────────────────────────────────┘ │
│ │ │ │
│ │ ▼ │
│ │ ┌─────────────────────────────────────┐ │
│ │ │ nexgate_postgres:5432 │ │
│ │ │ schema: core │ schema: chat │ │
│ │ │ users │ conversations │ │
│ │ │ shops │ messages │ │
│ │ │ products │ reactions │ │
│ │ │ orders │ receipts │ │
│ │ │ │ calls │ │
│ │ └─────────────────────────────────────┘ │
│ │ │
│ │ ┌──────────────┐ ┌──────────────────┐ │
│ └────────────────▶│ Coturn │ │ LiveKit │ │
│ WebRTC media │ STUN/TURN │ │ Group Calls + │ │
│ (calls) │ port 3478 │ │ Audio Spaces │ │
│ └──────────────┘ └──────────────────┘ │
└─────────────────────────────────────────────────────────────────────┘
9. Authentication Flow
Login + XMPP Token Issuance
Mobile App Spring Boot Ejabberd
│ │ │
│── POST /auth/login ─────▶│ │
│ { phone, password } │ │
│ │── validate BCrypt │
│ │── check not suspended │
│ │── generate REST JWT │
│ │── generate XMPP JWT │
│ │ payload: │
│ │ { │
│ │ jid: "usr-kibuti │
│ │ @nexgate.com",│
│ │ exp: 1753228800 │
│ │ } │
│ │ signed: RS256 │
│ │ private key (Vault) │
│◀─ { restJwt, xmppJwt } ─│ │
│ │ │
│── XMPP connect ─────────────────────────────────▶│
│ username: usr-kibuti@nexgate.com │
│ password: <xmppJwt> │
│ │ │
│ │◀── GET /auth/.well- │
│ │ known/jwks.json │
│ │ (public key) │
│ │─── { keys: [...] } ──▶│
│ │ │── verify JWT
│ │ │── check expiry
│ │ │── check jid
│◀─────── session opened ─────────────────────────│
│ │ │
JWT Key Management (Multi-Node)
Spring Boot:
Private key → stored in HashiCorp Vault
Used to SIGN XMPP JWTs
Never shared
Ejabberd (all nodes):
Public key → fetched from JWKS endpoint
GET https://api.nexgate.com/auth/.well-known/jwks.json
Used to VERIFY JWTs
All nodes fetch same endpoint ✅
Key rotation:
Spring Boot generates new key pair
Adds both old + new to JWKS endpoint
Issues new tokens with new key
Old tokens valid until expiry (24h)
After 24h → remove old key
Zero downtime rotation ✅
10. Message Flow — 1:1 Chat
Mobile Sends Message
Mobile (Alice) Ejabberd Spring Boot
│ │ │
│── XMPP stanza ───────▶│ │
│ <message │ │
│ to="bob@nexgate.com"│ │
│ type="chat" │ │
│ id="msg-001"> │ │
│ <body>Hello</body> │ │
│ <origin-id │ │
│ xmlns="urn:xmpp:sid:0" │
│ id="msg-001"/> │ │
│ <request │ │
│ xmlns="urn:xmpp:receipts"/> │
│ <markable │ │
│ xmlns="urn:xmpp:chat-markers:0"/> │
│ </message> │ │
│ │ │
│ │── route to Bob ───────▶│(if offline)
│ │ (if online → deliver directly)
│ │ │
│ │── RabbitMQ event ─────▶│
│ │ { │
│ │ event: "message", │
│ │ from: "alice", │
│ │ to: "bob", │
│ │ stanza_id: "...", │
│ │ body: "Hello", │
│ │ timestamp: ... │
│ │ } │
│ │ │── persist to DB
│ │ │── if Bob offline:
│ │ │ send FCM
│ │ │
│◀── stream ACK ────────│ │
│ <a h="N" │ │
│ xmlns="urn:xmpp:sm:3"/> │
Link Handling (Before Send)
When message contains a URL:
1. Client detects URL before sending
2. Calls Link Safety Service (separate service):
safe: true → allow send ✅
safe: false → block, warn user ⚠️
"This link may be unsafe"
3. NexGate internal links (nexgate.com/...):
→ converted to Card automatically
→ no safety check needed (we own these)
4. External links → sent as plain text URL in body
Client fetches Open Graph preview independently
(client side — no server involvement)
Sender sees preview before sending
Receiver app fetches OG independently
Link Safety Service documented separately.
What Spring Boot Does With Event
RabbitMQ consumer receives:
1. Find or create conversation record
2. Insert message into chat.messages
3. Detect message type:
- plain text → just persist
- commerce stanza → trigger offer flow
- system stanza → update order record
4. Check if recipient online:
- Online → delivery handled by Ejabberd
- Offline → send FCM via notification/
5. Update conversation.last_message
6. Update conversation.updated_at
Message Interactions (Stanza Reference)
Edit (XEP-0308):
<message id="edit-002">
<body>corrected text</body>
<replace xmlns="urn:xmpp:message-correct:0"
id="original-msg-id"/>
</message>
Spring Boot: update messages.body, set edited_at
Delete (XEP-0424):
<message id="retract-001">
<apply-to xmlns="urn:xmpp:fasten:0"
id="original-msg-id">
<retract xmlns="urn:xmpp:message-retract:0"/>
</apply-to>
<body>/me retracted a message</body>
</message>
Spring Boot: set messages.deleted_at, body = null
React (XEP-0444):
<message>
<reactions xmlns="urn:xmpp:reactions:0"
id="original-msg-id">
<reaction>👍</reaction>
</reactions>
</message>
Spring Boot: upsert message_reactions
Reply (XEP-0461):
<message>
<body>Thanks!</body>
<reply xmlns="urn:xmpp:reply:0"
to="alice@nexgate.com"
id="original-msg-id"/>
</message>
Spring Boot: set messages.reply_to_id
Forward (XEP-0297):
<message>
<body>Check this</body>
<ng xmlns="urn:nexgate:1">
<meta>
<message_type>TEXT</message_type>
<shareable>true</shareable>
...
</meta>
<payload/>
</ng>
<forwarded xmlns="urn:xmpp:forward:0">
<message from="..." to="...">
<body>original message</body>
</message>
</forwarded>
</message>
Note: Spring Boot checks shareable=true before allowing
Returns 403 if shareable=false ✅
11. Message Flow — Group Chat
Key Differences From 1:1
1:1 chat: Group chat:
to = person JID to = room JID
type = "chat" type = "groupchat"
stanza-id by client stanza-id by ROOM
receipts per person no per-person receipts
reactions ref origin-id reactions ref stanza-id
Group JID format:
{conversationId}@conference.nexgate.com
e.g. conv-abc123@conference.nexgate.com
Group Creation Flow
Mobile creates group:
POST /chat/groups/create
{ name, type: PRIVATE|PUBLIC, description }
Spring Boot:
1. Create conversation record (type=GROUP)
2. Call Ejabberd REST:
POST /api/create_room
{
name: "conv-abc123",
service: "conference.nexgate.com",
options: { persistent: true, members_only: true }
}
3. Creator auto-joined as OWNER
4. Generate invite link token
5. Return { conversationId, inviteLink }
Group Join — Two Mechanisms
Mechanism 1: Consent DM Invitation
Admin selects contacts/followers
Spring Boot sends invitation stanza via Ejabberd:
<message to="juma@nexgate.com">
<nexgate-group-invite xmlns="urn:nexgate:group:1">
<group_id>conv-abc123</group_id>
<group_name>Business Friends</group_name>
<invited_by>Kibuti Mwangi</invited_by>
<expires_at>2026-07-17T10:00:00Z</expires_at>
</nexgate-group-invite>
</message>
Recipient taps Accept:
POST /chat/groups/conv-abc123/join
Spring Boot: ejabberd REST → add_member
48h no response: auto-declined silently
Mechanism 2: Invite Link
PRIVATE group: link → request → admin approves
PUBLIC group: link → instant join
Admin controls: expiry, max joins, revoke
Fan-out
Group message fan-out:
Member sends to room JID
Ejabberd MUC broadcasts to ALL members
Erlang handles fan-out natively
No Redis pub/sub needed
Spring Boot persists via RabbitMQ event
At 500 members:
Still Ejabberd MUC fan-out ✅
Erlang is built for this
12. Commerce DM Flow
Product Card (Seller Attaches in 1:1)
Seller taps [From My Shop] in 1:1 DM attach menu:
Spring Boot builds stanza using ng standard:
card_type: PRODUCT_CARD
sender_type: SHOP
inbox: COMMERCE
shareable: true
See Section 15 Example 2 for full stanza ✅
Rules:
1:1 DM only (never group) ✅
Price = current price at snapshot time
Buyer's app renders product card UI
[View Product] [Chat with Shop] buttons
Custom Price Offer Lifecycle
States: PENDING → ACCEPTED|DECLINED|EXPIRED|COMPLETED
Seller creates offer (1:1 DM only):
POST /chat/commerce/offer/create
{ conversationId, productId, offerPrice, validMinutes: 30 }
Spring Boot:
1. Create offer_sessions record
2. Build ng stanza:
card_type: CUSTOM_PRICE_OFFER
sender_type: SHOP
inbox: COMMERCE
shareable: false (ALWAYS)
expires_at: now + validMinutes
item_type: PRODUCT or TICKET
See Section 15 Example 3 + 4 for full stanza ✅
3. Send via Ejabberd REST
4. Start expiry timer (Redis)
Buyer accepts:
POST /chat/commerce/offer/accept
Spring Boot: update status → ACCEPTED
Trigger checkout flow
Offer expires (30 min):
Spring Boot scheduler fires
Update status → EXPIRED
Send expiry ng stanza:
card_type: OFFER_EXPIRED
priority: SILENT
Buyer's UI: "Offer Expired ⏰"
Bei Ya Pamoja (Group Buy)
NOT a group type — it is a CARD ✅
Can be shared in: DIRECT, GROUP, COMMERCE_DM
Cannot be shared in: DIRECT_SECRET ❌
Spring Boot builds ng stanza:
card_type: GROUP_BUY
item_type: PRODUCT or TICKET
shareable: true
members_preview: max 3 avatars + more_count
See Section 15 Example 5 + 6 for full stanza ✅
Spring Boot updates progress in real-time:
When member joins:
Sends GROUP_BUY_PROGRESS signal (priority: SILENT)
References original card by message_id
App updates card in place — no notification ✅
See Section 15 Example 7 for progress stanza ✅
When target reached → trigger group checkout
When expired → send expired card stanza
Card states:
Initial → In Progress → Joined → Target Reached → Expired
Order Updates (System Messages)
Auto-sent by Spring Boot (no manual action):
Uses ng stanza standard:
sender_type: SYSTEM
message_type: CARD
shareable: false
deletable: false
reactable: false
inbox: COMMERCE
card_type: ORDER_CONFIRMATION or ORDER_STATUS_UPDATE
See Section 15 Example 12 for full stanza ✅
Rules:
Goes directly to commerce thread ✅
Cannot be deleted ✅ (deletable: false in meta)
Cannot be reacted to ✅ (reactable: false in meta)
Cannot be quoted ✅ (quotable: false in meta)
Immutable for legal/audit ✅
13. Call Flow — Voice & Video
1:1 Call (WebRTC + Jingle)
Alice calls Bob:
Mobile (Alice) Spring Boot Ejabberd Bob's Device
│ │ │ │
│── GET /chat/calls/ │ │ │
│ turn-credentials ──▶│ │ │
│ │── generate HMAC │ │
│ │ TURN credentials │ │
│◀── { urls, username, │ │ │
│ credential } │ │ │
│ │ │ │
│── Jingle │ │ │
│ session-initiate ────────────────────────▶│ │
│ <iq type="set"> │ │── route ───────▶│
│ <jingle │ │ │
│ action= │ │ │
│ "session-initiate"│ │ │
│ sid="call-abc"> │ │ │
│ <content │ │ │
│ name="audio"> │ │ │
│ <description │ │ │
│ media="audio">│ │ │
│ <payload-type│ │ │
│ name="opus"/>│ │ │
│ </description> │ │ │
│ <transport │ │ │
│ xmlns= │ │ │
│ "urn:xmpp: │ │ │
│ jingle: │ │ │
│ transports: │ │ │
│ ice-udp:1"> │ │ │
│ <candidate │ │ │
│ type="relay"│ │ │
│ ip="turn. │ │ │
│ nexgate.com"│ │ │
│ port=3478/>│ │ │
│ </transport> │ │ │
│ </content> │ │ │
│ </jingle> │ │ │
│ </iq> │ │ │
│ │ │ │
│ │ │◀─ session- ─────│
│ │ │ accept │
│◀── session-accept ─────────────────────────│ │
│ │ │ │
│◀═══ WebRTC Audio/Video via Coturn ══════════════════════════▶│
│ │ │ │
│── session-terminate ──────────────────────▶│ │
│ │ │ │
│── POST /chat/calls/ │ │ │
│ {callId}/end ───────▶│ │ │
│ │── update call record│ │
Audio ↔ Video Switching (No Hang Up)
Mid-call: Alice enables camera:
Alice → Jingle content-add:
<jingle action="content-add" sid="call-abc">
<content name="video">
<description media="video">
<payload-type name="H264"/>
</description>
</content>
</jingle>
Bob accepts:
Jingle content-accept
Video starts — audio uninterrupted ✅
Switch back (video → audio):
Jingle content-remove
Audio continues ✅
TURN Credentials Generation
Spring Boot generates time-limited TURN credentials:
username = timestamp:userId
credential = HMAC-SHA256(sharedSecret, username)
valid for: 1 hour
TURN server (Coturn) validates:
Checks HMAC signature
Checks timestamp not expired
Allows relay ✅
Coturn config:
use-auth-secret
static-auth-secret = <shared-secret-from-vault>
realm = nexgate.com
Group Call (LiveKit SFU)
3+ people call → use LiveKit (not P2P):
Spring Boot:
1. Create LiveKit room: group-call-{callId}
2. Generate token per participant
3. Send join info via ng stanza:
message_type: CALL
signal_type: GROUP_CALL_JOIN_INFO
priority: HIGH
expires_at: now + 5 minutes
shareable: false
See Section 15 Example 11 for full stanza ✅
Each participant:
Receives stanza → phone rings
Joins → WebRTC to LiveKit
LiveKit SFU forwards streams
EA network strategy:
Simulcast (low/medium/high quality)
Each receiver gets quality network allows
Bad network → low quality (not dropped)
14. Secret Chat — End-to-End Encryption
What Secret Chat Is
Normal chat in NexGate:
Alice sends message → Ejabberd routes → Bob
Spring Boot stores message in DB (readable)
NexGate can read messages if legally required
Good for: commerce DMs, groups, order updates
Secret Chat:
Alice encrypts ON DEVICE → Ejabberd routes → Bob decrypts ON DEVICE
Spring Boot stores ENCRYPTED BLOB (unreadable)
NexGate CANNOT read messages
Keys NEVER leave the device
Good for: personal private conversations
Secret Chat = optional feature
User explicitly chooses to start one
Cannot be forced or auto-converted
Technology — OMEMO (XEP-0384)
OMEMO = Encryption that scales to multiple devices
Based on Signal Protocol (same as WhatsApp)
Built for XMPP natively
Why OMEMO over other options:
Signal Protocol = most secure available ✅
Multi-device support built-in ✅
Standard XEP = Ejabberd supports natively ✅
Forward secrecy ✅
(old messages safe even if key compromised)
Deniability ✅
(cannot prove who sent a message)
Open standard ✅ (auditable)
How OMEMO Works (Simple)
Key Setup (happens once per device):
Alice's phone generates:
Identity Key pair (permanent)
Signed PreKey pair (rotates every week)
One-Time PreKeys (100 per batch)
Alice publishes PUBLIC keys to Ejabberd:
Via XMPP PubSub (XEP-0060)
Ejabberd stores: alice's public keys
Bob can fetch them anytime
Private keys:
NEVER leave Alice's device ✅
Spring Boot NEVER sees them ✅
Ejabberd NEVER sees them ✅
Message Encryption:
Alice wants to send to Bob:
1. Fetch Bob's public keys from Ejabberd
2. Generate session key (Diffie-Hellman)
3. Encrypt message with session key
4. Encrypt session key with Bob's identity key
5. Send encrypted blob via XMPP stanza
Bob receives:
1. Decrypt session key using his private key
2. Decrypt message using session key
3. Display plaintext to Bob ✅
Ejabberd sees: encrypted blob only ❌
Spring Boot stores: encrypted blob only ❌
Nobody between Alice and Bob can read ✅
Multi-Device in Secret Chat
Alice has phone + tablet:
OMEMO encrypts for EACH device separately
Message encrypted for:
Bob's phone key ✅
Bob's tablet key ✅
Alice's tablet key ✅ (carbon copy)
Each device decrypts its own copy
All devices see the message ✅
If Alice adds new device:
Must re-establish sessions
Old messages NOT automatically available
(forward secrecy — by design) ✅
Secret Chat Flow
Alice starts Secret Chat with Bob:
Alice:
Tap Bob's profile
→ "Start Secret Chat" 🔒
→ App checks: do I have Bob's OMEMO keys?
→ If no: fetch from Ejabberd PubSub
→ Establish encrypted session
→ UI shows: "🔒 Secret Chat"
→ "Messages are end-to-end encrypted"
Alice sends message:
App encrypts locally ✅
Sends encrypted stanza:
<message to="bob@nexgate.com" type="chat">
<body>I can't read this</body>
← fallback for non-OMEMO clients
<encrypted xmlns="eu.siacs.conversations.axolotl">
<header sid="473229364">
<key rid="987654321">
BASE64_ENCRYPTED_SESSION_KEY_FOR_BOB
</key>
<key rid="111111111">
BASE64_ENCRYPTED_SESSION_KEY_FOR_ALICE_TABLET
</key>
<iv>BASE64_IV</iv>
</header>
<payload>BASE64_ENCRYPTED_MESSAGE_BODY</payload>
</encrypted>
</message>
Ejabberd:
Routes stanza (cannot read payload) ✅
Spring Boot (RabbitMQ consumer):
Receives event
Detects: has OMEMO encryption ✅
Stores encrypted payload in DB:
messages.body = null
messages.encrypted_payload = BASE64_BLOB
messages.is_e2ee = true
Does NOT attempt to decrypt ✅
Bob's device:
Receives stanza
Decrypts using private key
Displays plaintext ✅
What Spring Boot Stores vs Doesn't
Normal message:
messages.body = "Hello Bob!" ← readable
messages.is_e2ee = false
Secret Chat message:
messages.body = null ← empty ✅
messages.encrypted_payload = "BASE64..." ← blob
messages.is_e2ee = true
messages.sender_id = usr-alice (known)
messages.conversation_id = conv-abc (known)
messages.created_at = timestamp (known)
Spring Boot knows:
WHO sent ✅ (metadata)
WHEN sent ✅ (metadata)
TO WHOM ✅ (metadata)
WHAT conversation ✅ (metadata)
Spring Boot does NOT know:
WHAT was said ❌ (content encrypted)
This is by design ✅
Legal compliance: "we cannot read it" ✅
Secret Chat Rules
Can do in Secret Chat:
✅ Send text messages (encrypted)
✅ Send images (encrypted)
✅ Send voice notes (encrypted)
✅ Send files (encrypted)
✅ Set self-destruct timer
✅ Verify contact's identity (key fingerprint)
Cannot do in Secret Chat:
❌ Forward messages (breaks E2EE chain)
❌ Screenshot (Android FLAG_SECURE)
❌ Quote/reply across devices
❌ Search message content (encrypted in DB)
❌ Commerce stanzas (offer sessions need DB)
❌ Group Secret Chat (Phase 3 — complex)
❌ Web client (keys on device only)
Self-Destruct Timer
Optional feature in Secret Chat:
Alice sets: "Delete after 5 minutes"
Both devices delete locally after timer
Server record remains (encrypted blob)
BUT: server also deletes after timer ✅
Implementation:
Uses ng stanza standard:
message_type: TEXT
encrypted: true
inbox: SECRET
expires_at: now + 300 seconds (self-destruct)
OMEMO payload inside <payload> block
See Section 15 Example 13 for full stanza ✅
Self-destruct carried via expires_at in <meta>:
<expires_at>2026-07-17T10:05:00Z</expires_at>
Spring Boot: sets Redis TTL on message
After TTL: deletes from DB ✅
Device: shows countdown timer in UI ✅
Spring Boot:
Receives event → sets Redis TTL on message:
EXPIRE msg:msg-abc 300
After 300 seconds:
Spring Boot deletes from DB ✅
Recipient device:
Timer shown in UI (countdown)
Local delete after timer ✅
Identity Verification
Users can verify each other's identity:
Compare OMEMO key fingerprints
Out-of-band (meet in person, voice call)
"My fingerprint: A1B2 C3D4 E5F6..."
"Your fingerprint matches ✅"
In UI:
Contact profile → "Verify Security"
Shows: "Your Safety Number with Alice"
64-character fingerprint
QR code option ✅
If verified: show verified badge 🛡️
Why this matters:
Protects against man-in-the-middle attacks
"Is Ejabberd showing me Alice's real keys?"
After verification: guaranteed ✅
Key Management in Spring Boot
Spring Boot role in OMEMO:
Store public keys (not private) ✅
Serve public keys via Ejabberd PubSub ✅
Never store private keys ❌
Public key storage:
Ejabberd PubSub handles automatically
Node: eu.siacs.conversations.axolotl.bundles:{deviceId}
Spring Boot does NOT manage this
Ejabberd handles key distribution ✅
Key rotation:
Device rotates Signed PreKey weekly
Device publishes new PreKey to Ejabberd PubSub
Spring Boot uninvolved ✅
One-Time PreKeys replenished automatically
When running low: device publishes more ✅
Spring Boot only knows:
Which users have OMEMO enabled
Which conversations are Secret Chats
That encrypted blobs exist (not content)
UI Treatment
Secret Chat conversation:
🔒 Lock icon in conversation header
Dark/different color scheme (optional)
"Messages are end-to-end encrypted" banner
Timer icon if self-destruct enabled ⏱️
Starting Secret Chat:
Long press on contact OR
Three dot menu → "Start Secret Chat"
Separate conversation from normal chat
Cannot accidentally send to wrong chat
Incoming Secret Chat:
"Alice wants to start a Secret Chat"
[Accept] [Decline]
Key fingerprint screen:
Settings → Conversations → [name]
→ "View Security Code"
Shows QR + text fingerprint
What's NOT E2EE (Important)
Secret Chat is 1:1 ONLY at launch:
❌ Group Secret Chat (Phase 3)
❌ Commerce DMs (need server-side logic)
❌ Shop conversations (staff audit needed)
❌ System messages (order notifications)
❌ Call content (WebRTC handles separately)
Call security (separate from OMEMO):
WebRTC calls: DTLS-SRTP encrypted ✅
(built into WebRTC standard)
Keys negotiated per call
Nobody can intercept call media ✅
Call metadata: Spring Boot knows
Who called whom ✅ (metadata)
How long ✅ (metadata)
But not WHAT was said ✅
What Secret Chat Supports at Launch
Phase 1 (Launch — all that is needed):
✅ 1:1 Secret Chat (OMEMO)
✅ Self-destruct timer
✅ Identity verification (fingerprint/QR)
✅ Image/file encryption
✅ Voice note encryption
Phase 1 is complete and sufficient.
No Phase 2 or Phase 3 planned.
15. NexGate Stanza Standard
The Element
Every NexGate message carries ONE unified element:
<ng xmlns="urn:nexgate:1">
ng = NexGate (short name)
urn:nexgate:1 = NexGate root namespace version 1
Contains TWO children — ALWAYS:
<meta> → message metadata (always present, all fields)
<payload> → message content (empty for TEXT)
Ejabberd: routes the <message> wrapper — ignores <ng> ✅
Mobile app: reads <ng> to render correctly ✅
Spring Boot: reads <ng> to persist correctly ✅
Basic XMPP clients: see <body> fallback only ✅
The Meta Block — All Fields
<meta>
<sender_type>USER|SHOP|SYSTEM</sender_type>
<sender_id>usr-kibuti</sender_id>
<message_type>TEXT|CARD|MEDIA|CALL</message_type>
<shareable>true|false</shareable>
<deletable>true|false</deletable>
<reactable>true|false</reactable>
<quotable>true|false</quotable>
<priority>NORMAL|HIGH|SILENT</priority>
<inbox>PERSONAL|COMMERCE|SECRET</inbox>
<encrypted>true|false</encrypted>
<expires_at/>
</meta>
sender_type:
USER → regular user
SHOP → shop (staff hidden from customer)
SYSTEM → Spring Boot auto (no human)
message_type:
TEXT → plain text message
CARD → rich interactive card
MEDIA → image/video/voice/file/gif/sticker
CALL → call lifecycle signal
shareable:
true → Forward option shown in UI ✅
false → Forward hidden + API returns 403 ❌
deletable:
true → sender can delete for everyone
false → immutable (system/payment/order)
reactable:
true → emoji reactions allowed
false → no reactions (system/call signals)
quotable:
true → can be replied/quoted
false → cannot quote (system/call signals)
priority:
NORMAL → standard notification
HIGH → ring even in DND (calls)
SILENT → update UI only, no notification
(GROUP_BUY_PROGRESS, card updates)
inbox:
PERSONAL → Personal tab
COMMERCE → Commerce tab
SECRET → Secret tab 🔒
encrypted:
true → OMEMO Secret Chat
false → normal message
expires_at:
empty → lives forever (default)
timestamp → auto-delete after this time
Meta Defaults Per Message Type
Field TEXT CARD MEDIA CALL SYSTEM
──────────────────────────────────────────────────────────
sender_type USER varies USER USER SYSTEM
shareable true varies true false false
deletable true varies true false false
reactable true varies true false false
quotable true varies true false false
priority NORMAL NORMAL NORMAL HIGH NORMAL
inbox PERSONAL varies PERSONAL PERSONAL COMMERCE
encrypted false false false false false
expires_at empty empty empty 5min empty
Complete Stanza Examples
1. Plain Text Message
<message to="bob@nexgate.com" type="chat" id="msg-001">
<body>Hello Bob!</body>
<ng xmlns="urn:nexgate:1">
<meta>
<sender_type>USER</sender_type>
<sender_id>usr-alice</sender_id>
<message_type>TEXT</message_type>
<shareable>true</shareable>
<deletable>true</deletable>
<reactable>true</reactable>
<quotable>true</quotable>
<priority>NORMAL</priority>
<inbox>PERSONAL</inbox>
<encrypted>false</encrypted>
<expires_at/>
</meta>
<payload/>
</ng>
<origin-id xmlns="urn:xmpp:sid:0" id="msg-001"/>
<request xmlns="urn:xmpp:receipts"/>
<markable xmlns="urn:xmpp:chat-markers:0"/>
</message>
2. Product Card
<message to="buyer@nexgate.com" type="chat" id="msg-002">
<body>Check out this product</body>
<ng xmlns="urn:nexgate:1">
<meta>
<sender_type>SHOP</sender_type>
<sender_id>shop-456</sender_id>
<message_type>CARD</message_type>
<shareable>true</shareable>
<deletable>false</deletable>
<reactable>true</reactable>
<quotable>false</quotable>
<priority>NORMAL</priority>
<inbox>COMMERCE</inbox>
<encrypted>false</encrypted>
<expires_at/>
</meta>
<payload>
<card_type>PRODUCT_CARD</card_type>
<product>
<id>prod-123</id>
<name>Samsung A15</name>
<price>450000</price>
<currency>TZS</currency>
<image_url>https://cdn.nexgate.com/img.jpg</image_url>
<shop_name>TechStore</shop_name>
<shop_id>shop-456</shop_id>
<stock>12</stock>
</product>
</payload>
</ng>
<origin-id xmlns="urn:xmpp:sid:0" id="msg-002"/>
<request xmlns="urn:xmpp:receipts"/>
</message>
3. Custom Price Offer (Product)
<message to="buyer@nexgate.com" type="chat" id="msg-003">
<body>Special price offer for you</body>
<ng xmlns="urn:nexgate:1">
<meta>
<sender_type>SHOP</sender_type>
<sender_id>shop-456</sender_id>
<message_type>CARD</message_type>
<shareable>false</shareable>
<deletable>false</deletable>
<reactable>false</reactable>
<quotable>false</quotable>
<priority>NORMAL</priority>
<inbox>COMMERCE</inbox>
<encrypted>false</encrypted>
<expires_at>2026-07-17T11:30:00Z</expires_at>
</meta>
<payload>
<card_type>CUSTOM_PRICE_OFFER</card_type>
<offer_id>offer-abc-123</offer_id>
<status>PENDING</status>
<valid_minutes>30</valid_minutes>
<item>
<item_type>PRODUCT</item_type>
<item_id>prod-123</item_id>
<item_name>Samsung A15</item_name>
<image_url>https://cdn.nexgate.com/img.jpg</image_url>
<shop_name>TechStore</shop_name>
</item>
<pricing>
<public_price>450000</public_price>
<offer_price>400000</offer_price>
<currency>TZS</currency>
<discount_amount>50000</discount_amount>
<discount_pct>11</discount_pct>
</pricing>
</payload>
</ng>
<origin-id xmlns="urn:xmpp:sid:0" id="msg-003"/>
</message>
4. Custom Price Offer (Ticket)
<message to="buyer@nexgate.com" type="chat" id="msg-004">
<body>Special ticket price for you</body>
<ng xmlns="urn:nexgate:1">
<meta>
<sender_type>SYSTEM</sender_type>
<sender_id>org-123</sender_id>
<message_type>CARD</message_type>
<shareable>false</shareable>
<deletable>false</deletable>
<reactable>false</reactable>
<quotable>false</quotable>
<priority>NORMAL</priority>
<inbox>COMMERCE</inbox>
<encrypted>false</encrypted>
<expires_at>2026-07-17T12:00:00Z</expires_at>
</meta>
<payload>
<card_type>CUSTOM_PRICE_OFFER</card_type>
<offer_id>offer-evt-456</offer_id>
<status>PENDING</status>
<valid_minutes>60</valid_minutes>
<item>
<item_type>TICKET</item_type>
<item_id>evt-789</item_id>
<item_name>Dar Tech Summit 2026</item_name>
<image_url>https://cdn.nexgate.com/evt.jpg</image_url>
<event_date>2026-08-15T09:00:00Z</event_date>
<venue>Julius Nyerere ICC, Dar es Salaam</venue>
<ticket_type>VIP</ticket_type>
<quantity>2</quantity>
</item>
<pricing>
<public_price>50000</public_price>
<offer_price>35000</offer_price>
<currency>TZS</currency>
<discount_amount>15000</discount_amount>
<discount_pct>30</discount_pct>
</pricing>
</payload>
</ng>
<origin-id xmlns="urn:xmpp:sid:0" id="msg-004"/>
</message>
5. Group Buy Card (Product) with Members Preview
<message to="conv-abc@conference.nexgate.com"
type="groupchat" id="msg-005">
<body>Group purchase: Samsung A15</body>
<ng xmlns="urn:nexgate:1">
<meta>
<sender_type>USER</sender_type>
<sender_id>usr-kibuti</sender_id>
<message_type>CARD</message_type>
<shareable>true</shareable>
<deletable>false</deletable>
<reactable>true</reactable>
<quotable>false</quotable>
<priority>NORMAL</priority>
<inbox>PERSONAL</inbox>
<encrypted>false</encrypted>
<expires_at>2026-07-17T18:00:00Z</expires_at>
</meta>
<payload>
<card_type>GROUP_BUY</card_type>
<group_buy_id>gb-xyz-789</group_buy_id>
<item>
<item_type>PRODUCT</item_type>
<item_id>prod-123</item_id>
<item_name>Samsung A15</item_name>
<image_url>https://cdn.nexgate.com/img.jpg</image_url>
<shop_name>TechStore</shop_name>
</item>
<pricing>
<public_price>450000</public_price>
<group_price>350000</group_price>
<currency>TZS</currency>
</pricing>
<progress>
<current>8</current>
<target>10</target>
<pct>80</pct>
</progress>
<members_preview>
<member>
<id>usr-kibuti</id>
<name>Kibuti</name>
<avatar>https://cdn.nexgate.com/av1.jpg</avatar>
</member>
<member>
<id>usr-juma</id>
<name>Juma</name>
<avatar>https://cdn.nexgate.com/av2.jpg</avatar>
</member>
<member>
<id>usr-alice</id>
<name>Alice</name>
<avatar>https://cdn.nexgate.com/av3.jpg</avatar>
</member>
<more_count>5</more_count>
</members_preview>
</payload>
</ng>
<origin-id xmlns="urn:xmpp:sid:0" id="msg-005"/>
<request xmlns="urn:xmpp:receipts"/>
</message>
6. Group Buy Card (Ticket)
<message to="bob@nexgate.com" type="chat" id="msg-006">
<body>Group ticket purchase: Dar Tech Summit</body>
<ng xmlns="urn:nexgate:1">
<meta>
<sender_type>USER</sender_type>
<sender_id>usr-alice</sender_id>
<message_type>CARD</message_type>
<shareable>true</shareable>
<deletable>false</deletable>
<reactable>true</reactable>
<quotable>false</quotable>
<priority>NORMAL</priority>
<inbox>PERSONAL</inbox>
<encrypted>false</encrypted>
<expires_at>2026-07-17T20:00:00Z</expires_at>
</meta>
<payload>
<card_type>GROUP_BUY</card_type>
<group_buy_id>gb-evt-001</group_buy_id>
<item>
<item_type>TICKET</item_type>
<item_id>evt-789</item_id>
<item_name>Dar Tech Summit 2026</item_name>
<image_url>https://cdn.nexgate.com/evt.jpg</image_url>
<event_date>2026-08-15T09:00:00Z</event_date>
<venue>Julius Nyerere ICC</venue>
<ticket_type>GENERAL</ticket_type>
</item>
<pricing>
<public_price>25000</public_price>
<group_price>18000</group_price>
<currency>TZS</currency>
</pricing>
<progress>
<current>15</current>
<target>20</target>
<pct>75</pct>
</progress>
<members_preview>
<member>
<id>usr-alice</id>
<name>Alice</name>
<avatar>https://cdn.nexgate.com/av1.jpg</avatar>
</member>
<member>
<id>usr-bob</id>
<name>Bob</name>
<avatar>https://cdn.nexgate.com/av2.jpg</avatar>
</member>
<member>
<id>usr-juma</id>
<name>Juma</name>
<avatar>https://cdn.nexgate.com/av3.jpg</avatar>
</member>
<more_count>12</more_count>
</members_preview>
</payload>
</ng>
<origin-id xmlns="urn:xmpp:sid:0" id="msg-006"/>
</message>
7. Group Buy Progress Update (Silent)
<message to="conv-abc@conference.nexgate.com"
type="groupchat" id="msg-007">
<body>Group buy updated</body>
<ng xmlns="urn:nexgate:1">
<meta>
<sender_type>SYSTEM</sender_type>
<sender_id>system</sender_id>
<message_type>CARD</message_type>
<shareable>false</shareable>
<deletable>false</deletable>
<reactable>false</reactable>
<quotable>false</quotable>
<priority>SILENT</priority>
<inbox>PERSONAL</inbox>
<encrypted>false</encrypted>
<expires_at/>
</meta>
<payload>
<card_type>GROUP_BUY_PROGRESS</card_type>
<group_buy_id>gb-xyz-789</group_buy_id>
<original_message_id>msg-005</original_message_id>
<progress>
<current>9</current>
<target>10</target>
<pct>90</pct>
</progress>
<members_preview>
<member>
<id>usr-kibuti</id>
<name>Kibuti</name>
<avatar>https://cdn.nexgate.com/av1.jpg</avatar>
</member>
<member>
<id>usr-juma</id>
<name>Juma</name>
<avatar>https://cdn.nexgate.com/av2.jpg</avatar>
</member>
<member>
<id>usr-new</id>
<name>Naserian</name>
<avatar>https://cdn.nexgate.com/av4.jpg</avatar>
</member>
<more_count>6</more_count>
</members_preview>
</payload>
</ng>
<no-store xmlns="urn:xmpp:hints"/>
</message>
8. Image (Media)
<message to="bob@nexgate.com" type="chat" id="msg-008">
<body>📷 Photo</body>
<ng xmlns="urn:nexgate:1">
<meta>
<sender_type>USER</sender_type>
<sender_id>usr-alice</sender_id>
<message_type>MEDIA</message_type>
<shareable>true</shareable>
<deletable>true</deletable>
<reactable>true</reactable>
<quotable>false</quotable>
<priority>NORMAL</priority>
<inbox>PERSONAL</inbox>
<encrypted>false</encrypted>
<expires_at/>
</meta>
<payload>
<media_type>IMAGE</media_type>
<file_id>file-abc-123</file_id>
<url>https://cdn.nexgate.com/img.jpg</url>
<thumbnail_url>https://cdn.nexgate.com/thumb.jpg</thumbnail_url>
<mime_type>image/jpeg</mime_type>
<size>245000</size>
<width>1080</width>
<height>720</height>
</payload>
</ng>
<origin-id xmlns="urn:xmpp:sid:0" id="msg-008"/>
<request xmlns="urn:xmpp:receipts"/>
<markable xmlns="urn:xmpp:chat-markers:0"/>
</message>
9. Voice Note (Media)
<message to="bob@nexgate.com" type="chat" id="msg-009">
<body>🎤 Voice note</body>
<ng xmlns="urn:nexgate:1">
<meta>
<sender_type>USER</sender_type>
<sender_id>usr-alice</sender_id>
<message_type>MEDIA</message_type>
<shareable>true</shareable>
<deletable>true</deletable>
<reactable>true</reactable>
<quotable>false</quotable>
<priority>NORMAL</priority>
<inbox>PERSONAL</inbox>
<encrypted>false</encrypted>
<expires_at/>
</meta>
<payload>
<media_type>VOICE_NOTE</media_type>
<file_id>file-voice-456</file_id>
<url>https://cdn.nexgate.com/voice/abc.ogg</url>
<mime_type>audio/ogg</mime_type>
<size>48000</size>
<duration_seconds>15</duration_seconds>
<waveform>0.1,0.4,0.8,0.6,0.3,0.9,0.2,0.5</waveform>
</payload>
</ng>
<origin-id xmlns="urn:xmpp:sid:0" id="msg-009"/>
<request xmlns="urn:xmpp:receipts"/>
</message>
10. Incoming Call Signal
<message to="bob@nexgate.com" type="chat" id="msg-010">
<body>Incoming call from Alice</body>
<ng xmlns="urn:nexgate:1">
<meta>
<sender_type>USER</sender_type>
<sender_id>usr-alice</sender_id>
<message_type>CALL</message_type>
<shareable>false</shareable>
<deletable>false</deletable>
<reactable>false</reactable>
<quotable>false</quotable>
<priority>HIGH</priority>
<inbox>PERSONAL</inbox>
<encrypted>false</encrypted>
<expires_at>2026-07-17T10:05:00Z</expires_at>
</meta>
<payload>
<signal_type>CALL_INITIATED</signal_type>
<call_id>call-abc-789</call_id>
<call_type>VIDEO</call_type>
<caller_name>Alice</caller_name>
<caller_avatar>https://cdn.nexgate.com/av1.jpg</caller_avatar>
</payload>
</ng>
<no-store xmlns="urn:xmpp:hints"/>
</message>
11. Group Call Join Info
<message to="bob@nexgate.com" type="chat" id="msg-011">
<body>Join the group call</body>
<ng xmlns="urn:nexgate:1">
<meta>
<sender_type>SYSTEM</sender_type>
<sender_id>system</sender_id>
<message_type>CALL</message_type>
<shareable>false</shareable>
<deletable>false</deletable>
<reactable>false</reactable>
<quotable>false</quotable>
<priority>HIGH</priority>
<inbox>PERSONAL</inbox>
<encrypted>false</encrypted>
<expires_at>2026-07-17T10:05:00Z</expires_at>
</meta>
<payload>
<signal_type>GROUP_CALL_JOIN_INFO</signal_type>
<call_id>call-group-xyz</call_id>
<call_type>VOICE</call_type>
<livekit_url>wss://livekit.nexgate.com</livekit_url>
<livekit_token>eyJhbGciOiJIUzI1NiJ9...</livekit_token>
<room_id>group-call-xyz</room_id>
<initiated_by>Kibuti Mwangi</initiated_by>
</payload>
</ng>
<no-store xmlns="urn:xmpp:hints"/>
</message>
12. Order Confirmation (System Card)
<message to="buyer@nexgate.com" type="chat" id="msg-012">
<body>Your order has been confirmed</body>
<ng xmlns="urn:nexgate:1">
<meta>
<sender_type>SYSTEM</sender_type>
<sender_id>system</sender_id>
<message_type>CARD</message_type>
<shareable>false</shareable>
<deletable>false</deletable>
<reactable>false</reactable>
<quotable>false</quotable>
<priority>NORMAL</priority>
<inbox>COMMERCE</inbox>
<encrypted>false</encrypted>
<expires_at/>
</meta>
<payload>
<card_type>ORDER_CONFIRMATION</card_type>
<order_id>ORD-789</order_id>
<product_name>Samsung A15</product_name>
<quantity>1</quantity>
<amount_paid>400000</amount_paid>
<currency>TZS</currency>
<payment_method>M-PESA</payment_method>
<status>CONFIRMED</status>
</payload>
</ng>
</message>
13. Secret Chat Message
<message to="bob@nexgate.com" type="chat" id="msg-013">
<body>You have an encrypted message</body>
<ng xmlns="urn:nexgate:1">
<meta>
<sender_type>USER</sender_type>
<sender_id>usr-alice</sender_id>
<message_type>TEXT</message_type>
<shareable>false</shareable>
<deletable>true</deletable>
<reactable>false</reactable>
<quotable>false</quotable>
<priority>NORMAL</priority>
<inbox>SECRET</inbox>
<encrypted>true</encrypted>
<expires_at>2026-07-17T10:35:00Z</expires_at>
</meta>
<payload>
<encrypted xmlns="eu.siacs.conversations.axolotl">
<header sid="473229364">
<key rid="987654321">BASE64_KEY</key>
<iv>BASE64_IV</iv>
</header>
<blob>BASE64_ENCRYPTED_BODY</blob>
</encrypted>
</payload>
</ng>
<origin-id xmlns="urn:xmpp:sid:0" id="msg-013"/>
</message>
14. Group Invitation Card
<message to="juma@nexgate.com" type="chat" id="msg-014">
<body>You have been invited to join a group</body>
<ng xmlns="urn:nexgate:1">
<meta>
<sender_type>USER</sender_type>
<sender_id>usr-kibuti</sender_id>
<message_type>CARD</message_type>
<shareable>false</shareable>
<deletable>false</deletable>
<reactable>false</reactable>
<quotable>false</quotable>
<priority>NORMAL</priority>
<inbox>PERSONAL</inbox>
<encrypted>false</encrypted>
<expires_at>2026-07-19T10:00:00Z</expires_at>
</meta>
<payload>
<card_type>GROUP_INVITATION</card_type>
<group_id>conv-abc123</group_id>
<group_name>Business Friends</group_name>
<group_type>PRIVATE</group_type>
<member_count>47</member_count>
<description>Dar founders discussion</description>
<invited_by>Kibuti Mwangi</invited_by>
</payload>
</ng>
</message>
15. Post Card (Social)
<message to="bob@nexgate.com" type="chat" id="msg-015">
<body>Check out this post</body>
<ng xmlns="urn:nexgate:1">
<meta>
<sender_type>USER</sender_type>
<sender_id>usr-alice</sender_id>
<message_type>CARD</message_type>
<shareable>true</shareable>
<deletable>true</deletable>
<reactable>true</reactable>
<quotable>true</quotable>
<priority>NORMAL</priority>
<inbox>PERSONAL</inbox>
<encrypted>false</encrypted>
<expires_at/>
</meta>
<payload>
<card_type>POST_CARD</card_type>
<post_id>post-789</post_id>
<author_name>Kibuti Mwangi</author_name>
<author_username>kibuti</author_username>
<author_avatar>https://cdn.nexgate.com/av.jpg</author_avatar>
<caption>New Samsung models just arrived!</caption>
<media_url>https://cdn.nexgate.com/post.jpg</media_url>
<media_type>IMAGE</media_type>
<like_count>245</like_count>
<comment_count>12</comment_count>
</payload>
</ng>
<origin-id xmlns="urn:xmpp:sid:0" id="msg-015"/>
<request xmlns="urn:xmpp:receipts"/>
</message>
Standard XEPs — When to Include
XEP element When to include
──────────────────────────────────────────────────────────
<origin-id> All real messages ✅
NOT on CALL signals
NOT on SILENT updates
<request> (receipts) TEXT + CARD + MEDIA ✅
NOT on CALL signals
NOT on SILENT updates
NOT on SYSTEM cards
<markable> (read markers) TEXT + MEDIA ✅
NOT on CARD (no read tick)
NOT on CALL signals
<no-store> (hints) CALL signals ✅
SILENT updates ✅
Typing indicators ✅
Anything ephemeral
The Complete Anatomy
<message> ← XMPP wrapper (Ejabberd routes)
<body>fallback</body> ← ALWAYS present
<ng xmlns="urn:nexgate:1"> ← ALWAYS present
<meta> ← ALWAYS all fields
...
</meta>
<payload> ← empty for TEXT, content for others
...
</payload>
</ng>
<!-- Standard XEPs below (as needed per table above) -->
<origin-id/>
<request/>
<markable/>
<no-store/>
</message>
16. NexGate Custom Namespaces
Complete Registry
All custom stanzas ALWAYS go inside <message>
Ejabberd routes without knowing what is inside
Frontend identifies by xmlns + type field
SOCIAL CARDS (VP Social):
xmlns: urn:nexgate:social:1
Types: POST_CARD, REEL_CARD, LIVE_CARD, PROFILE_CARD
COMMERCE CARDS (VP Shop):
xmlns: urn:nexgate:commerce:1
Types: PRODUCT_CARD, SHOP_CARD, FLASH_SALE_CARD
xmlns: urn:nexgate:offer:1
type: CUSTOM_PRICE_OFFER (always for offer cards)
OFFER_RESPONSE, OFFER_EXPIRED
item_type: PRODUCT | TICKET (for CUSTOM_PRICE_OFFER)
xmlns: urn:nexgate:groupbuy:1
type: GROUP_BUY (always)
item_type: PRODUCT | TICKET
Status types: GROUP_BUY_PROGRESS, GROUP_BUY_COMPLETED
xmlns: urn:nexgate:installment:1
Types: INSTALLMENT_PLAN_CARD
EVENT CARDS (VP Events):
xmlns: urn:nexgate:event:1
Types: EVENT_CARD, TICKET_CARD, EVENT_REMINDER, EVENT_GROUP_BUY
SYSTEM CARDS (Automated):
xmlns: urn:nexgate:system:1
Types: ORDER_CONFIRMATION, ORDER_STATUS_UPDATE,
PAYMENT_CONFIRMATION, REFUND_CARD, DISPUTE_CARD
GROUP CARDS (chat_system):
xmlns: urn:nexgate:group:1
Types: GROUP_INVITATION (only card — user taps Accept/Decline)
CALL SIGNALS (chat_system — NOT cards, system signals):
xmlns: urn:nexgate:call:1
Types: CALL_INITIATED, CALL_ACCEPTED, CALL_DECLINED,
CALL_ENDED, CALL_MISSED, GROUP_CALL_JOIN_INFO,
PARTICIPANT_ADDED, HOST_MUTED_YOU, HOST_REMOVED_YOU
API TEMPLATE CARDS (Third-party Future):
xmlns: urn:nexgate:template:1
Types: TRANSACTIONAL, MARKETING, AUTHENTICATION
MEDIA:
xmlns: urn:nexgate:media:1
Types: IMAGE, VIDEO, VOICE_NOTE, FILE, STICKER, GIF
SECRET CHAT METADATA:
xmlns: urn:nexgate:secret:1
Types: SELF_DESTRUCT_TIMER, KEY_VERIFICATION
EPHEMERAL (always add no-store hint):
xmlns: urn:nexgate:states (recording state)
xmlns: urn:nexgate:forward (forward metadata)
Standard XEPs Used
XEP-0085 chatstates Typing indicators
XEP-0184 urn:xmpp:receipts Delivery ticks
XEP-0333 urn:xmpp:chat-markers:0 Read ticks
XEP-0308 message-correct:0 Edit message
XEP-0424 message-retract:0 Delete message
XEP-0444 urn:xmpp:reactions:0 Emoji reactions
XEP-0297 urn:xmpp:forward:0 Forward message
XEP-0461 urn:xmpp:reply:0 Reply/quote
XEP-0359 urn:xmpp:sid:0 Stable stanza IDs
XEP-0198 urn:xmpp:sm:3 Stream management
XEP-0199 urn:xmpp:ping Keepalive ping
XEP-0334 urn:xmpp:hints no-store hint
XEP-0384 axolotl (OMEMO) E2EE encryption
XEP-0166 urn:xmpp:jingle:1 Call signaling
XEP-0045 muc protocol Group chat
XEP-0280 urn:xmpp:carbons:2 Multi-device sync
XEP-0313 urn:xmpp:mam:2 Message archive
Mobile App Decision Tree
Message arrives:
Has urn:nexgate:social:1 ?
POST_CARD → post preview + [View Post]
LIVE_CARD → live badge + [Watch Live]
Has urn:nexgate:commerce:1 ?
PRODUCT_CARD → product card + [View] [Chat]
Has urn:nexgate:offer:1 ?
PENDING → offer + timer + [Accept] [Decline]
ACCEPTED → "Offer Accepted"
EXPIRED → "Offer Expired"
Has urn:nexgate:groupbuy:1 ?
GROUP_BUY_CARD → progress bar + [Join]
Has urn:nexgate:event:1 ?
EVENT_CARD → event + [View] [Get Ticket]
TICKET_CARD → ticket QR code
Has urn:nexgate:system:1 ?
ORDER_CONFIRMATION → order + [Track]
ORDER_STATUS_UPDATE → status badge
PAYMENT_CONFIRMATION → payment details
Has urn:nexgate:call:1 ?
CALL_INITIATED → incoming call screen
GROUP_CALL_JOIN_INFO → [Join Call] [Decline]
CALL_MISSED → missed call indicator
HOST_MUTED_YOU → "You were muted by host"
HOST_REMOVED_YOU → "You were removed from call"
Has urn:nexgate:group:1 ?
GROUP_INVITATION → [Accept] [Decline]
Has urn:nexgate:media:1 ?
IMAGE → show image
VOICE_NOTE → waveform + play button
STICKER → full size, no bubble
GIF → auto-play loop
VIDEO → video player
FILE → download button
Has urn:nexgate:template:1 ?
TRANSACTIONAL → template card
MARKETING → card + unsubscribe
No xmlns match ?
Render as plain text (use body)
ALWAYS include body as fallback
Basic XMPP clients see fallback text
NexGate app renders rich card
17. RabbitMQ Event Pipeline
Exchange & Queue Setup
Exchange: nexgate.chat (topic exchange)
Queues:
nexgate.chat.messages ← all message events
nexgate.chat.presence ← online/offline events
nexgate.chat.calls ← call start/end events
nexgate.chat.rooms ← group create/destroy events
Routing keys:
chat.message.inbound → message arrived
chat.message.edited → message edited
chat.message.retracted → message deleted
chat.message.reaction → reaction added/removed
chat.presence.online → user came online
chat.presence.offline → user went offline
chat.call.initiated → call started
chat.call.ended → call ended
chat.room.created → group created
chat.room.destroyed → group deleted
Event Payload Structure
chat.message.inbound:
{
event_type: "message.inbound",
from_jid: "alice@nexgate.com/android",
to_jid: "bob@nexgate.com",
stanza_id: "1784107799294256",
origin_id: "msg-abc-123",
conversation_type: "DIRECT|GROUP",
room_jid: null | "conv-abc@conference.nexgate.com",
body: "Hello Bob!",
has_custom_namespace: true,
namespace: "urn:nexgate:commerce:1",
raw_stanza: "<message>...</message>",
timestamp: 1721210400000
}
chat.presence.offline:
{
event_type: "presence.offline",
jid: "alice@nexgate.com/android",
bare_jid: "alice@nexgate.com",
timestamp: 1721210400000
}
chat.call.ended:
{
event_type: "call.ended",
call_id: "call-xyz",
from_jid: "alice@nexgate.com",
to_jid: "bob@nexgate.com",
duration_seconds: 120,
call_type: "VOICE|VIDEO",
ended_reason: "normal|declined|missed|failed"
}
Spring Boot Consumer Responsibilities
@RabbitListener(queues = "nexgate.chat.messages")
fun onMessage(event: ChatMessageEvent) {
// 1. Parse event
// 2. Find or create conversation
// 3. Detect message type:
if (event.namespace == "urn:nexgate:commerce:1") {
// parse product card, link to product
} else if (event.namespace == "urn:nexgate:offer:1") {
// update offer session status
} else {
// plain message
}
// 4. Persist to chat.messages
// 5. Update conversation last_message
// 6. Check recipient online status
// If offline → send FCM via notification/
// 7. For commerce stanzas → trigger business flows
}
@RabbitListener(queues = "nexgate.chat.presence")
fun onPresence(event: PresenceEvent) {
// Update user last_seen in core.users
// Update conversation online indicators
}
18. Shop Inbox & Staff Access
Shop XMPP Identity
Every shop has its own JID:
$techstore → shop-456@nexgate.com
Staff members share this JID:
Staff A connects as: shop-456@nexgate.com/staff-1
Staff B connects as: shop-456@nexgate.com/staff-2
Customer always sees: TechStore (never staff name)
JID resource stripped before delivery ✅
Staff XMPP token:
Issued by Spring Boot when staff logs in
{ jid: "shop-456@nexgate.com", exp: ... }
Short-lived: 8 hours (staff shift)
Shop Inbox Routing
Customer messages shop:
to="shop-456@nexgate.com"
Ejabberd routes to available staff resource
(round-robin or first available)
Staff replies:
from="shop-456@nexgate.com/staff-1"
Spring Boot strips /staff-1 resource
Customer receives:
from="shop-456@nexgate.com" ✅
Customer never knows which staff replied ✅
Audit log:
Spring Boot records which staff_id sent each message
Internal only — never visible to customer
Legal requirement for disputes
Shop Chat Initiation Rules
ALWAYS ALLOWED (transactional — auto):
Order confirmed → system sends to buyer
Order shipped → system sends to buyer
Payment issue → system sends to buyer
ALLOWED (existing relationship):
Previous buyer → shop can message from Orders section
Active follower → shop can send from Customers section
Goes to existing thread OR message request
NEVER ALLOWED:
Cold message to random users ❌
System blocks at API level
"You can only message customers who
interacted with your shop"
19. Offline & Push Notification Flow
Message arrives for Bob (offline):
Ejabberd:
1. Detects Bob offline
2. Stores in Ejabberd offline queue
3. Fires RabbitMQ event: chat.message.inbound
+ is_recipient_offline: true
Spring Boot consumer:
1. Persists message to DB
2. Detects is_recipient_offline = true
3. Calls notification/ package:
- Has FCM token? → send FCM push ✅
- No FCM token? → try Textfy SMS ✅
- Priority: FCM first, SMS fallback
FCM payload:
{
to: "bob-fcm-token",
notification: {
title: "Alice",
body: "Hello Bob!"
},
data: {
conversation_id: "conv-abc",
message_id: "msg-001",
type: "chat"
}
}
Bob comes online:
Ejabberd delivers offline queue
Spring Boot receives delivery event
Updates message status → DELIVERED
Sends <received> receipt to Alice ✅
20. Multi-Device Support
The Problem
Kibuti has:
Phone (android) → alice@nexgate.com/android
Tablet (tablet) → alice@nexgate.com/tablet
Web (browser) → alice@nexgate.com/web
Both logged in simultaneously.
How do messages reach all devices?
How do sent messages sync across devices?
How does read status sync?
Solution — XEP-0280 Message Carbons
Message Carbons = automatic copy to all devices
Kibuti sends message from phone:
Phone → Ejabberd → Bob ✅
Ejabberd ALSO sends carbon copy to:
Kibuti's tablet ✅
Kibuti's web ✅
Result:
All Kibuti's devices see sent messages ✅
Conversation stays in sync ✅
No extra code needed in Spring Boot
Ejabberd handles it via mod_carbons
Enable in ejabberd.yml:
modules:
mod_carbons: {}
Resource Priority
Each device has a priority:
android: priority 10 (highest — phone)
tablet: priority 5
web: priority 1 (lowest)
When Juma sends to Kibuti:
Ejabberd delivers to HIGHEST priority device
= android (phone) gets it first
Carbon copies → all other devices
Priority set during session setup:
<presence>
<priority>10</priority>
</presence>
NexGate mobile sets:
Foreground app: priority 10
Background app: priority 0
Ejabberd routes to foreground device ✅
Read Status Sync Across Devices
Problem:
Kibuti reads message on phone
Tablet still shows unread badge
Solution — XEP-0333 Chat Markers sync via Carbons:
Phone sends <displayed> to Juma
Ejabberd carbons copy to tablet + web
Tablet receives: "Kibuti read this on phone"
Tablet clears unread badge ✅
This is automatic via mod_carbons ✅
XMPP JWT Per Device
Each device gets its OWN XMPP JWT:
Phone connects → POST /auth/login → xmppJwt-1
Tablet connects → POST /auth/refresh → xmppJwt-2
Web connects → POST /auth/login → xmppJwt-3
Each JWT has same JID but different resource:
{ jid: "usr-kibuti@nexgate.com", exp: ... }
Ejabberd assigns resource automatically:
usr-kibuti@nexgate.com/android (phone)
usr-kibuti@nexgate.com/tablet (tablet)
usr-kibuti@nexgate.com/web-abc (browser)
On logout (one device):
That device's JWT invalidated
Other devices unaffected ✅
Active Sessions Tracking
Spring Boot tracks active sessions in Redis:
Key: sessions:usr-kibuti
Value: SET {
"android:gajim.B3HKS5VE",
"tablet:dino.36aa17c4"
}
TTL: 24 hours (JWT expiry)
When checking if user is online:
Redis lookup → instant ✅
No DB query needed ✅
No Ejabberd REST call needed ✅
Updated by:
PresenceConsumer → user online → add to SET
PresenceConsumer → user offline → remove from SET
Multi-Device Message Archive (MAM)
XEP-0313 Message Archive Management:
All messages stored in Ejabberd archive (PostgreSQL)
New device login → fetches message history
"Give me messages since last week"
New device flow:
1. Kibuti logs in on new tablet
2. Tablet connects to Ejabberd
3. Tablet sends MAM query:
<iq type="set">
<query xmlns="urn:xmpp:mam:2">
<x><field var="start">
<value>2026-07-10T00:00:00Z</value>
</field></x>
</query>
</iq>
4. Ejabberd returns all messages since that date
5. Tablet renders full conversation history ✅
Spring Boot also has messages in chat.messages:
Mobile can fetch via REST: GET /chat/messages
Paginated, sorted, filtered
Two sources of truth:
Ejabberd MAM: raw XMPP archive
Spring Boot DB: structured business records
21. Redis — What to Cache & Why
The Golden Rule
Redis is for:
Data that changes frequently
Data read far more than written
Data where 1-2 second staleness is OK
Data that is expensive to compute
Redis is NOT for:
Primary source of truth
Data that must be 100% consistent
Large objects (images, files)
Data that needs complex queries
Cache 1 — Online Presence
Key: presence:usr-kibuti
Value: { status: "online", last_seen: timestamp, devices: [...] }
TTL: 5 minutes (refreshed by heartbeat)
Why Redis:
Checked on EVERY message send
"Is recipient online?" → Redis hit ✅
Without Redis: Ejabberd REST call per message ❌
At 7,000 msg/min: 7,000 Ejabberd REST calls/min ❌
With Redis: 7,000 Redis reads/min (microseconds) ✅
Updated by:
PresenceConsumer when user comes online
PresenceConsumer when user goes offline
Heartbeat pings every 60 seconds
Cache 2 — Active XMPP Sessions
Key: sessions:usr-kibuti
Value: SET of resource strings
TTL: 24 hours (JWT expiry)
Why Redis:
Know which devices are connected
Without Redis: Ejabberd REST call (user_resources)
With Redis: instant SET lookup ✅
Updated by:
PresenceConsumer on connect/disconnect
Cache 3 — Conversation Metadata
Key: conv:conv-abc123
Value: {
name: "Business Friends",
type: "GROUP",
member_count: 47,
last_message: "...",
last_updated: timestamp
}
TTL: 10 minutes
Why Redis:
Inbox list loads conversation metadata for EVERY conversation
User with 50 conversations → 50 DB reads ❌
With Redis: 50 cache hits ✅
Inbox load: <50ms vs 500ms ✅
Updated by:
MessageConsumer when new message arrives
GroupService when group changes
Cache 4 — Unread Count Per Conversation
Key: unread:usr-kibuti:conv-abc123
Value: integer (unread message count)
TTL: none (persist until read)
Why Redis:
Show unread badge on inbox
Without Redis: COUNT query per conversation ❌
With Redis: INCR / SET / GET (nanoseconds) ✅
Updated by:
MessageConsumer: INCR when message arrives for user
ReceiptConsumer: SET 0 when user sends <displayed>
Total unread across all conversations:
Key: unread:total:usr-kibuti
MessageConsumer: INCR
Reset: SET 0 when inbox opened
Cache 5 — Offer Session State
Key: offer:offer-abc123
Value: {
status: "PENDING",
expires_at: timestamp,
offer_price: 400000,
public_price: 450000
}
TTL: 30 minutes (offer validity)
Why Redis:
Offer expiry check on every [Accept] tap
Without Redis: DB read per check ❌
With Redis: instant lookup ✅
TTL = offer auto-expires at Redis level ✅
Updated by:
OfferLifecycleService on state change
Redis TTL handles expiry automatically
Cache 6 — Rate Limiting
Key: ratelimit:msg:usr-kibuti
Value: integer (message count in window)
TTL: 1 minute (sliding window)
Why Redis:
Prevent message spam
Without Redis: DB query per message ❌
With Redis: INCR + TTL = instant ✅
Rules:
Max 60 messages per minute per user
Max 10 group invitations per day
Max 5 concurrent call attempts
Implementation:
INCR ratelimit:msg:usr-kibuti
EXPIRE ratelimit:msg:usr-kibuti 60
If value > 60: reject with 429
Cache 7 — TURN Credentials
Key: turn:usr-kibuti
Value: { username, credential, ttl, urls }
TTL: 1 hour
Why Redis:
User may request TURN credentials multiple times
(call fails, retry, reconnect)
Without Redis: HMAC compute per request
With Redis: cache for 1 hour ✅
Same credentials valid for 1 hour anyway
Cache 8 — User Profile (For Chat Display)
Key: profile:usr-kibuti
Value: {
display_name: "Kibuti Mwangi",
avatar_url: "https://cdn.nexgate.com/...",
username: "kibuti",
is_verified: false
}
TTL: 30 minutes
Why Redis:
Chat inbox shows sender name + avatar
50 conversations × profile = 50 DB reads ❌
With Redis: 50 cache hits ✅
Updated by:
User updates profile → invalidate cache
Cache miss → read from core.users → cache
Cache 9 — Group Membership (Hot Groups)
Key: group:members:conv-abc123
Value: SET of user JIDs
TTL: 5 minutes
Why Redis:
Fan-out notification: "who is in this group?"
For push notifications when group member offline
Without Redis: DB query per group message ❌
With Redis: SMEMBERS in microseconds ✅
Only cache for ACTIVE groups (>10 msg/hour)
Inactive groups: read from DB directly
Redis Key Naming Convention
Pattern: {domain}:{entity}:{id}:{sub}
Examples:
presence:usr-kibuti
sessions:usr-kibuti
conv:conv-abc123
unread:usr-kibuti:conv-abc123
unread:total:usr-kibuti
offer:offer-abc123
ratelimit:msg:usr-kibuti
ratelimit:invite:usr-kibuti
turn:usr-kibuti
profile:usr-kibuti
group:members:conv-abc123
NEVER store:
Raw message content (use DB)
Files or media (use MinIO)
Financial data (use DB with audit)
Auth tokens as plain text (use Vault)
Cache Invalidation Strategy
Write-through (update cache + DB together):
Unread counts ← must be accurate
Offer status ← critical for commerce
Cache-aside (read DB on miss, then cache):
User profiles ← changes rarely
Conversation metadata ← OK if slightly stale
Group membership ← OK if 5 min stale
TTL-based eviction (let it expire):
Presence ← Ejabberd handles truth
TURN credentials ← expire with JWT
Rate limits ← sliding window
Event-based invalidation:
User updates profile → DEL profile:usr-{id}
Group settings change → DEL conv:{id}
Member joins group → SADD group:members:{id}
Member leaves group → SREM group:members:{id}
22. Best Practices & Anti-Patterns
✅ DO — Message Persistence
DO: Persist via RabbitMQ consumer (async)
Message arrives → Ejabberd delivers → fires event
Consumer persists to DB asynchronously
User experience not blocked by DB write ✅
DON'T: Persist via synchronous API call
Message arrives → Spring Boot REST call → DB
DB write blocks message delivery ❌
If DB slow → messages delayed ❌
If DB down → messages lost ❌
✅ DO — Online Status Checks
DO: Check Redis for online status
MessageConsumer:
val isOnline = redis.exists("presence:$userId")
if (!isOnline) sendFCM()
DON'T: Call Ejabberd REST for every message
val resources = ejabberd.getUserResources(userId)
if (resources.isEmpty()) sendFCM()
← This is an HTTP call per message ❌
← At scale: thousands of HTTP calls/min ❌
✅ DO — Stanza Building
DO: Build stanzas in dedicated StanzaBuilder class
val stanza = StanzaBuilder.productCard(product)
ejabberdClient.sendStanza(from, to, stanza)
DON'T: Build XML strings manually scattered in code
val xml = "<message><nexgate-commerce>..." ❌
← Error prone, hard to test, hard to maintain
✅ DO — Offer Expiry
DO: Use Redis TTL for expiry detection
redis.set("offer:$offerId", offer, ttl = 30.minutes)
// TTL fires → OfferExpiryListener → update DB
DON'T: Use scheduled DB queries
@Scheduled(fixedRate = 60000)
fun expireOffers() {
db.query("SELECT * FROM offers WHERE expires_at < NOW()")
}
← Hammers DB every minute ❌
← Redis TTL is instant and free ✅
✅ DO — Conversation Updates
DO: Update conversation in MessageConsumer
MessageConsumer:
1. Persist message
2. Update conversation.last_message (DB)
3. INCR unread count (Redis)
4. Update conv metadata cache (Redis)
DON'T: Fetch conversation on every message render
GET /chat/conversations → DB query every time ❌
Serve from Redis cache ✅
✅ DO — JID Handling
DO: Always strip resource for storage
Store: "alice@nexgate.com" (bare JID)
Never: "alice@nexgate.com/android" (full JID)
val bareJid = fullJid.substringBefore("/")
DON'T: Store full JID with resource
Resource changes between sessions ❌
Queries will break ❌
Exception: call routing (need specific device)
Full JID used only for Jingle call signaling
Not stored in DB
✅ DO — Always Include Fallback
DO: Include <body> fallback in EVERY message
<message>
<body>Check out this product</body> ← fallback ✅
<ng xmlns="urn:nexgate:1">
<meta>...</meta>
<payload>...</payload>
</ng>
</message>
DON'T: Send message without body
<message>
<ng xmlns="urn:nexgate:1">
...
</ng>
</message>
← Basic XMPP clients show nothing ❌
← Debugging impossible ❌
← Violates NexGate stanza standard ❌
✅ DO — Group Message Stanza ID
DO: Use stanza-id (room-assigned) for group references
// Reacting to a group message:
val stanzaId = message.getStanzaId() // by="room@..."
buildReaction(stanzaId, emoji)
DON'T: Use origin-id for group message references
val originId = message.getOriginId() // client-generated
buildReaction(originId, emoji) ← WRONG for groups ❌
Rule:
1:1 chat → reactions/edits reference origin-id
Group chat → reactions/edits reference stanza-id
✅ DO — Shop Staff Privacy
DO: Always strip staff resource from JID
Outbound to customer:
from = "shop-456@nexgate.com" ✅
(resource stripped by StanzaBuilder)
DON'T: Let staff JID leak to customer
from = "shop-456@nexgate.com/staff-john" ❌
Customer now knows staff name ❌
Privacy violation ❌
Implementation:
StanzaBuilder always strips resource
for any stanza from a shop account
✅ DO — Rate Limiting
DO: Rate limit at Redis level before processing
fun sendMessage(userId: String, ...) {
val key = "ratelimit:msg:$userId"
val count = redis.incr(key)
if (count == 1L) redis.expire(key, 60)
if (count > 60) throw RateLimitException()
// proceed with message
}
DON'T: Rate limit at DB level
val recentCount = db.query(
"SELECT COUNT(*) FROM messages
WHERE user_id = ? AND created_at > NOW() - INTERVAL '1 minute'"
)
← DB query per message ❌
← Slow and expensive ❌
✅ DO — Typing Indicators
DO: Add no-store hint to typing stanzas
<message type="chat">
<composing xmlns="http://jabber.org/protocol/chatstates"/>
<no-store xmlns="urn:xmpp:hints"/> ← ALWAYS include
</message>
DON'T: Let typing indicators get stored in MAM
Without no-store: Ejabberd archives the typing indicator
New device fetches history: sees "alice is typing..."
from 3 weeks ago ← makes no sense ❌
✅ DO — Offline Message Handling
DO: Let Ejabberd handle offline queuing + Redis for FCM decision
Ejabberd stores offline messages natively
Spring Boot checks Redis for online status
If offline → FCM via notification/
DON'T: Build your own offline queue in DB
val offlineMessages = db.save(message) ❌
Duplicates Ejabberd's built-in offline storage ❌
Two sources of truth ❌
✅ DO — Gzip Compression
DO: Enable Gzip on ALL REST responses
# application.properties
server.compression.enabled=true
server.compression.min-response-size=1024
server.compression.mime-types=application/json,application/xml
Result:
Chat API responses: 25KB → 6KB ✅
Inbox load on 2G: 1.25s → 0.3s ✅
Zero code change required ✅
DON'T: Send uncompressed JSON to mobile on 2G
25KB per inbox load × 10 loads/day
= 250KB/day just for inbox
On Tanzania 2G data plans: expensive ❌
Anti-Pattern — DB Poll for Events
NEVER DO THIS:
@Scheduled(fixedRate = 1000)
fun checkForNewMessages() {
val messages = db.query(
"SELECT * FROM messages WHERE delivered_at IS NULL"
)
messages.forEach { deliver(it) }
}
← Polls DB every second ❌
← At scale: crushes DB ❌
← High latency (up to 1 second delay) ❌
DO THIS INSTEAD:
RabbitMQ consumer:
Ejabberd fires event → consumer reacts instantly ✅
Zero polling ✅
Sub-millisecond reaction time ✅
Anti-Pattern — N+1 Queries in Inbox
NEVER DO THIS:
val conversations = db.getConversations(userId)
conversations.forEach { conv ->
conv.lastMessage = db.getLastMessage(conv.id) // N queries ❌
conv.unreadCount = db.getUnreadCount(conv.id) // N queries ❌
conv.memberNames = db.getMemberNames(conv.id) // N queries ❌
}
50 conversations = 150 DB queries ❌
DO THIS INSTEAD:
// Redis for unread counts (O(1) per key)
val unreadCounts = redis.mget(convIds.map { "unread:$userId:$it" })
// Single JOIN query for last messages
val conversations = db.query("""
SELECT c.*, m.body as last_body, m.created_at as last_at
FROM chat.conversations c
LEFT JOIN chat.messages m ON m.id = c.last_message_id
WHERE c.user_id = ?
ORDER BY c.updated_at DESC
LIMIT 50
""")
// Profiles from Redis
val profiles = redis.mget(userIds.map { "profile:$it" })
Anti-Pattern — Blocking on Ejabberd REST
NEVER DO THIS (in request path):
fun sendMessage(req: MessageRequest): Response {
val result = ejabberdClient.sendMessage(...) // HTTP call
db.saveMessage(...)
return Response.ok()
}
← HTTP call in request path ❌
← If Ejabberd slow → user waits ❌
← If Ejabberd down → request fails ❌
DO THIS INSTEAD:
fun sendMessage(req: MessageRequest): Response {
// Validate + queue immediately
val messageId = db.saveOutgoing(req)
rabbitMQ.publish("chat.outbound", req)
return Response.ok(messageId) // instant ✅
}
// Async consumer sends to Ejabberd
@RabbitListener(queues = "chat.outbound")
fun sendToEjabberd(req: MessageRequest) {
ejabberdClient.sendMessage(...) // in background ✅
}
23. Package Structure
Full chat/ Package
chat/
├── config/
│ EjabberdConfig.java ← Ejabberd connection settings
│ RabbitMQChatConfig.java ← queues, exchanges, bindings
│ LiveKitConfig.java ← LiveKit server settings
│ CoturnConfig.java ← TURN server settings
│
├── controller/
│ ConversationController.java ← GET /chat/conversations
│ MessageController.java ← GET /chat/messages
│ GroupController.java ← POST /chat/groups/create
│ CallController.java ← GET /chat/calls/turn-credentials
│ ReceiptController.java ← POST /chat/messages/{id}/receipt
│
├── service/
│ ConversationService.java ← conversation lifecycle
│ MessageService.java ← persist, query messages
│ GroupService.java ← group management
│ OfferSessionService.java ← offer lifecycle
│ CallService.java ← call records, TURN creds
│ PresenceService.java ← online status tracking
│
├── consumer/
│ MessageConsumer.java ← handles chat.message.*
│ PresenceConsumer.java ← handles chat.presence.*
│ CallConsumer.java ← handles chat.call.*
│ RoomConsumer.java ← handles chat.room.*
│
├── ejabberd/
│ EjabberdRestClient.java ← HTTP client for Ejabberd API
│ EjabberdAuthController.java ← POST /internal/ejabberd/auth
│ JwksController.java ← GET /auth/.well-known/jwks.json
│ XmppTokenService.java ← generate/validate XMPP JWT
│
├── stanza/
│ StanzaBuilder.java ← builds XMPP stanzas
│ CommerceStanza.java ← product card, offer stanzas
│ GroupBuyStanza.java ← bei ya pamoja stanzas
│ SystemStanza.java ← order update stanzas
│ CallStanza.java ← group call join stanzas
│ MediaStanza.java ← file/image/voice stanzas
│
├── commerce/
│ CommerceDmService.java ← initiate commerce DM
│ OfferLifecycleService.java ← offer state machine
│ GroupBuyService.java ← bei ya pamoja flows
│ ShopInboxService.java ← shop routing, staff access
│
└── call/
TurnCredentialService.java ← HMAC TURN credentials
LiveKitService.java ← group call room management
CallRecordService.java ← persist call records
24. Infrastructure Stack
Docker Containers (Local Development)
Container Port(s) Purpose
──────────────────────────────────────────────────────────────
ejabberd 5222, 5280 XMPP server
coturn 3478 STUN/TURN relay
livekit 7880, 7881 Group calls + Audio Spaces
nexgate_postgres 5432 Main platform DB
ft_postgres 5433 File Thunder DB
ejabberd_postgres 5434 Ejabberd DB (production)
rabbitmq 5672, 15672 Message queue + management
redis 6379 Cache, sessions, rate limits
minio 9000, 9001 Object storage
ft_clamav 3310 Virus scanning
Local processes:
nexgate_backend 8080 Spring Boot (all-in-one)
file_thunder 8084 Media engine
Ejabberd Configuration Summary
ejabberd.yml key settings:
hosts: ["nexgate.com"]
auth_method: jwt
jwt_key: https://api.nexgate.com/auth/.well-known/jwks.json
listen:
port 5222: ejabberd_c2s (XMPP + TLS)
port 5280: ejabberd_http (/api, /admin, /oauth, /ws)
modules:
mod_muc: group chats
mod_mam: message archive
mod_offline: offline message storage
mod_ping: keepalive
mod_stream_mgmt: XEP-0198 reliability
mod_rabbitmq: event publishing
default_db: sql (production)
sql_type: pgsql
sql_server: "ejabberd_postgres"
sql_database: "ejabberd"
Ejabberd ↔ Spring Boot Security
Channel 1 (Spring Boot → Ejabberd REST):
Admin credentials in HashiCorp Vault
Internal Docker network only
Traefik blocks external access to 5280
Channel 2 (Ejabberd → Spring Boot via RabbitMQ):
RabbitMQ credentials in Vault
Internal Docker network only
AMQP not exposed externally
Channel 3 (Mobile → Ejabberd JWT):
JWT signed RS256 (private key in Vault)
Public key via JWKS endpoint (HTTPS only)
XMPP port 5222 with TLS
JWT expires 24 hours
25. Build Order
Week 1 — Foundation
Day 1-2: JWT Auth Infrastructure
EjabberdAuthController.java
POST /internal/ejabberd/auth
Validate XMPP JWT, return 200/401
JwksController.java
GET /auth/.well-known/jwks.json
Return RSA public key
XmppTokenService.java
Generate XMPP JWT on login
Add to auth/LoginResponse
Configure Ejabberd:
auth_method: jwt
jwt_key: JWKS URL
Test: Dino/Gajim connects via JWT ✅
Day 3-4: RabbitMQ Event Pipeline
RabbitMQChatConfig.java
Declare exchange, queues, bindings
MessageConsumer.java
Listen to chat.message.inbound
Persist to chat.messages
PresenceConsumer.java
Listen to chat.presence.*
Update last_seen
Test: Send message in Gajim → appears in DB ✅
Day 5: Conversation Management
ConversationService.java
Find or create conversation
Update last_message
ConversationController.java
GET /chat/conversations (inbox)
GET /chat/conversations/{id}/messages
Week 2 — Core Messaging
Day 1-2: Message Interactions
Handle edit stanzas (XEP-0308)
Handle retract stanzas (XEP-0424)
Handle reaction stanzas (XEP-0444)
Handle reply stanzas (XEP-0461)
MessageService.java for each
Day 3: Group Chat
GroupController.java
POST /chat/groups/create
POST /chat/groups/{id}/join
POST /chat/groups/{id}/decline
DELETE /chat/groups/{id}/leave
GroupService.java
Ejabberd REST: create_room, add_member
Invite link management
Day 4-5: Receipts & Notifications
ReceiptController.java
POST /chat/messages/{id}/receipt
{ status: DELIVERED|READ }
Offline push:
MessageConsumer → detect offline
Call notification/ package
FCM push ✅
Week 3 — Commerce DMs
Day 1-2: Commerce Stanza Sending
EjabberdRestClient.java
sendStanza(from, to, body, customElement)
StanzaBuilder.java
buildProductCard(product)
buildOfferCard(offer)
buildSystemMessage(order)
CommerceDmService.java
initiateCommerceDm(productId, shopId, buyerId)
Day 3-4: Offer Session Lifecycle
OfferLifecycleService.java
createOffer(...)
acceptOffer(offerId)
declineOffer(offerId)
expireOffers() ← scheduled job
Day 5: Shop Inbox
ShopInboxService.java
Route to available staff
Strip staff resource from JID
Audit log staff actions
Week 4 — Calls
Day 1-2: TURN Credentials
TurnCredentialService.java
Generate HMAC-SHA256 credentials
Time-limited (1 hour)
CallController.java
GET /chat/calls/turn-credentials
Day 3-4: Group Calls (LiveKit)
LiveKitService.java
createRoom(callId)
generateToken(userId, roomId)
CallStanza.java
buildGroupCallJoinInfo(callId, token, url)
Spring Boot sends join info stanza
Members receive → join LiveKit room
Day 5: Call Records
CallRecordService.java
onCallInitiated(event)
onCallEnded(event)
getCallHistory(userId)
Week 5 — Polish & Testing
Message search (MAM queries to Ejabberd)
Read count for group messages
Block/unblock users
Message request system
Integration testing with Gajim/Dino
Load testing with sendxmpp
Documentation for mobile dev team
NexGate Chat System — Development Architecture Guide v9.0 QBIT SPARK | chat_system · Spring Boot · Ejabberd · RabbitMQ · Redis · WebRTC · Coturn · LiveKit · OMEMO