NexGate — Private Chat & Calls Phase 2
NexGate Chat — Phase 2
Production Architecture
NexGate / QBIT SPARK | Version 1.0 Ejabberd · WebRTC Calls · Voice & Video · MessagePack · Coturn · Message Interactions
Table of Contents
- Why Phase 2
- What Changes, What Stays
- Full Architecture
- Ejabberd — The Transport Backbone
- Ejabberd ↔ Spring Boot Bridge
- Authentication Flow
- Message Flow — Phase 2
- Voice Calls
- Video Calls
- Coturn — TURN Relay
- MessagePack Encoding
- Broadcast Channels
- MQTT — Mini Apps Foundation
- Message Interactions
- Docker Deployment
- Database Additions
- Migration from Phase 1
1. Why Phase 2
Phase 1 ships fast with Spring Boot WebSocket. It works well up to approximately 50,000 concurrent connections per pod. That is enough for launch and early growth.
Phase 2 is triggered by one or more of these:
Trigger 1 — Concurrent connections
30k+ concurrent WS connections
Spring Boot pods becoming expensive
→ Ejabberd handles 2M on a single node
Trigger 2 — Voice and video calls launch
WebRTC signaling needs a proper protocol
Ejabberd's Jingle (XEP-0166) solves this natively
→ Don't build signaling from scratch
Trigger 3 — Mini Apps groundwork
Third-party developers need real-time events
Ejabberd's built-in MQTT broker is perfect
→ One config change, MQTT is live
Trigger 4 — Team grows
At least one engineer can own Ejabberd ops
→ Don't run Ejabberd solo in production
Phase 2 is not a rewrite. It is an upgrade of one layer only.
2. What Changes, What Stays
STAYS (zero changes): CHANGES (transport layer):
────────────────────────────── ──────────────────────────────
Spring Boot Chat Service ✅ Spring Boot WS Gateway → REMOVED
Spring Boot Main Backend ✅ Ejabberd Cluster → ADDED
PostgreSQL schema ✅ MessagePack encoding → ADDED
Redis usage ✅ Coturn TURN server → ADDED
RabbitMQ exchanges ✅ Voice + Video calls → ADDED
FCM + Textfy escalation ✅ Broadcast channels → ADDED
File Thunder integration ✅ MQTT broker → ACTIVATED
Commerce DM logic ✅
Inbox isolation model ✅
Notification levels ✅
Mobile app API surface ✅ ← app connects to same domain
wss://chat.nexgate.com
what's behind changes invisibly
The mobile developer changes nothing. The domain is the same. The frame format is the same (JSON initially, MessagePack after migration). Ejabberd is invisible to the app.
3. Full Architecture
┌──────────────────────────────────────────────────────────┐
│ NexGate Mobile App │
│ Android / iOS │
└────┬──────────────┬──────────────┬──────────────┬───────┘
│ │ │ │
WebSocket HTTP REST WebRTC HLS Player
XMPP stanzas (unchanged) (calls + (streams —
MessagePack Main Backend spaces) Phase 3)
│ │ │
▼ ▼ │
┌─────────────────────────┐ │
│ Ejabberd Cluster │ │
│ │ │
│ Node 1 Node 2 │ │
│ ┌──────┐ ┌──────┐ │ │
│ │Erlang│◀─▶Erlang│ │ │
│ │ dist │ │ dist │ │ │
│ └──────┘ └──────┘ │ │
│ │ │
│ XMPP/WebSocket │ │
│ Presence (built-in) │ │
│ MUC rooms (XEP-0045) │ │
│ Jingle signaling │◀───────┘
│ (XEP-0166) │ (call signaling
│ MQTT broker │ via WS)
│ Push bridge XEP-0357 │
└──────────┬──────────────┘
│
│ HTTP (auth only — sync)
│ RabbitMQ (all events — async)
│
┌──────────▼──────────────────────────────────────────────┐
│ Spring Boot Chat Service │
│ (unchanged from Phase 1) │
│ │
│ Messages · Receipts · Commerce Context │
│ Notification Router · Call Records │
│ Shop Inbox · Offline Escalation │
└──────────┬──────────────┬──────────────────────────────┘
│ │
▼ ▼
┌──────────────┐ ┌──────────────────────────────────────┐
│ PostgreSQL │ │ RabbitMQ │
│ Redis │ │ chat.message.inbound │
│ (unchanged) │ │ chat.presence │
│ │ │ chat.call.events │
└──────────────┘ │ chat.notify.push │
│ chat.notify.escalation │
└──────────────────────────────────────┘
│
▼
┌─────────────────────────────────┐
│ Coturn TURN Server │
│ (separate VPS) │
│ UDP relay for voice/video │
│ when P2P blocked by EA NAT │
└─────────────────────────────────┘
4. Ejabberd — The Transport Backbone
What Ejabberd Owns in Phase 2
✅ All WebSocket connections (2M concurrent per node)
✅ XMPP stanza routing between users
✅ User presence — online/offline/away (built-in protocol)
✅ Typing indicators (XEP-0085)
✅ Message delivery receipts (XEP-0184)
✅ Multi-User Chat rooms — MUC (XEP-0045)
✅ Voice/video call signaling — Jingle (XEP-0166)
✅ Push notification bridge (XEP-0357 → FCM/APNs)
✅ MQTT broker (Mini Apps events)
✅ Stream management / reconnection (XEP-0198)
✅ Cross-node routing (Erlang distributed — no Redis pub/sub needed)
❌ Does NOT touch:
PostgreSQL (NexGate's schema)
Business logic
Commerce context
Payment processing
File processing
Ejabberd Key Modules Enabled
mod_mam Message Archive Management
Stores message history in its own DB
Clients can sync history on reconnect
mod_muc Multi-User Chat
Group chats, live stream comment rooms
Max 500 members per room (configurable)
Persistent rooms survive server restart
mod_ping Keepalive ping every 30 seconds
Kills dead connections automatically
Critical for EA mobile networks
mod_push Push notification bridge
Connects to FCM/APNs on user disconnect
Replaces manual FCM calls from Chat Service
mod_stun_disco STUN/TURN server discovery
Tells clients where Coturn is
Used for voice/video call setup
mod_mqtt MQTT broker on port 1883
For Mini Apps real-time events (Phase 3)
Zero extra infrastructure needed
mod_http_api REST API on port 5285
Spring Boot calls this to send messages
Admin operations (kick user, create room)
Ejabberd Config Highlights
hosts:
- "nexgate.com"
listen:
- port: 5280 # WebSocket — mobile app connects here
module: ejabberd_http
request_handlers:
/ws: ejabberd_ws
/api: mod_http_api
- port: 5285 # REST API — Spring Boot calls here (internal only)
module: ejabberd_http
ip: "127.0.0.1"
request_handlers:
/api: mod_http_api
- port: 1883 # MQTT — Mini Apps (Phase 3)
module: mod_mqtt
- port: 3478 # STUN — voice/video setup
transport: udp
module: ejabberd_stun
# Auth — Ejabberd calls Spring Boot
auth_method: http
auth_opts:
url: "http://chat-service:8082/internal/ejabberd/auth"
auth_header: "X-Internal-Secret"
auth_header_value: "${EJABBERD_INTERNAL_SECRET}"
# PostgreSQL — Ejabberd's own separate database
sql_type: pgsql
sql_server: "postgres"
sql_database: "ejabberd" # NOT nexgate — separate DB
default_db: sql
modules:
mod_mam:
default: always
db_type: sql
mod_muc:
db_type: sql
max_users: 500
mod_ping:
send_pings: true
ping_interval: 30
timeout_action: kill
mod_push: {}
mod_stun_disco:
credentials_lifetime: 3600
services:
- host: "turn.nexgate.com"
port: 3478
type: turn
secret: "${COTURN_SECRET}"
mod_mqtt: {}
mod_http_api: {}
Two Separate PostgreSQL Databases
postgres instance (same server, two databases):
nexgate ← NexGate application data
messages, conversations, users, orders
Spring Boot owns this entirely
Ejabberd never touches this
ejabberd ← Ejabberd's own operational data
message archive (MAM)
MUC room state
roster data
Spring Boot never touches this
Why separate:
Ejabberd manages its own schema migrations
NexGate schema evolves independently
Clean ownership — no shared tables
Easy to backup independently
5. Ejabberd ↔ Spring Boot Bridge
Communication Rules
Ejabberd → Spring Boot:
Auth events: HTTP (synchronous — must respond fast)
Message events: RabbitMQ (async)
Presence events: RabbitMQ (async)
Call events: RabbitMQ (async)
MUC events: RabbitMQ (async)
Spring Boot → Ejabberd:
Send message to user: Ejabberd REST API (port 5285)
Create MUC room: Ejabberd REST API
Kick user session: Ejabberd REST API
Check user online: Ejabberd REST API
Broadcast to room: Ejabberd REST API
Rule: auth is the ONLY synchronous call
Everything else is async via RabbitMQ
RabbitMQ Events from Ejabberd
Exchange: ejabberd.events (topic)
Routing Key Fired When
──────────────────────────────────────────────────────
chat.message.inbound User sends a message
chat.message.group User sends to MUC room
chat.presence.online User WS connects
chat.presence.offline User WS disconnects
chat.call.initiated Jingle session-initiate received
chat.call.accepted Jingle session-accept received
chat.call.declined Jingle session-declined received
chat.call.ended Jingle session-terminate received
chat.muc.created MUC room created
chat.muc.joined User joined MUC room
chat.muc.left User left MUC room
Spring Boot Internal Endpoints (Ejabberd calls these)
POST /internal/ejabberd/auth
Called on every WebSocket connection
Ejabberd sends: { username, token }
Spring Boot responds: 200 (allow) or 401 (deny)
Must respond in < 200ms (checked in Redis cache first)
All other events arrive via RabbitMQ consumers
No other synchronous HTTP endpoints needed
Spring Boot → Ejabberd REST API Examples
Send system message to user:
POST http://ejabberd:5285/api/send_message
{
"from": "system@nexgate.com",
"to": "usr-123@nexgate.com",
"body": "",
"extra": {
"type": "ORDER_STATUS_UPDATE",
"orderId": "ord-456",
"status": "SHIPPED"
}
}
Create live stream MUC room:
POST http://ejabberd:5285/api/create_room
{
"name": "live-stream-abc",
"service": "conference.nexgate.com",
"host": "nexgate.com"
}
Kick expired session:
POST http://ejabberd:5285/api/kick_session
{
"user": "usr-123",
"host": "nexgate.com",
"resource": "android",
"reason": "Token expired"
}
6. Authentication Flow
Two Tokens Issued at Login
User logs into NexGate
│
▼ POST /auth/login
Main Backend (PONA Auth V3):
Validate credentials
Issue two tokens:
REST JWT (7 days):
Used for all HTTP API calls
Standard Bearer token
XMPP Token (24 hours):
Used only for Ejabberd connection
Contains: userId, JID, expiry
Shorter lifetime — chat sessions refresh more often
│
▼ Both tokens returned to app
WebSocket Connection Auth
App connects WebSocket:
wss://chat.nexgate.com/ws
Header: Authorization: Bearer {XMPP_TOKEN}
│
▼
Ejabberd receives connection
Extracts token from header
│
▼ HTTP POST (sync) → Spring Boot
/internal/ejabberd/auth
{ username: "usr-123", token: "XMPP_TOKEN" }
│
Spring Boot:
Check Redis cache first (fast path):
token:{hash} → valid/invalid (TTL 5min)
If not cached:
Validate JWT signature
Check token type == XMPP
Check user not banned/suspended
Cache result in Redis
Return: 200 { authorized: true, jid: "usr-123@nexgate.com" }
or 401 { authorized: false, reason: "TOKEN_EXPIRED" }
│
Ejabberd:
200 → allow connection
register: usr-123@nexgate.com/android as ONLINE
publish to RabbitMQ: chat.presence.online
401 → reject WebSocket
app shows: "Session expired, please login again"
JID Structure
Every NexGate entity has a JID (Jabber ID):
Personal user:
usr-123@nexgate.com/android ← full JID (user + device)
usr-123@nexgate.com ← bare JID (user only)
Shop identity:
techstore@shops.nexgate.com ← shop JID
Multiple staff auth as this JID
Customer sees "TechStore" — not the staff member
System bot:
system@nexgate.com ← order updates, notifications
MUC rooms:
live-abc@conference.nexgate.com ← live stream chat room
group-xyz@conference.nexgate.com ← group chat room
Multi-device:
usr-123@nexgate.com/android ← phone
usr-123@nexgate.com/ios ← tablet
Both receive messages
READ on one → Ejabberd notifies other to clear notification
XMPP Token Refresh
XMPP token expires every 24 hours
App background service:
At 23 hours → POST /auth/refresh-xmpp-token
Header: Bearer {REST_JWT} (still valid — 7 days)
Response: new XMPP token
Re-auth without reconnecting:
App sends new auth stanza on existing WS connection
Ejabberd re-validates via Spring Boot
No disconnection — seamless for user
7. Message Flow — Phase 2
Inbound Message (User Sends)
[Client A — Android]
│
│ WebSocket frame (MessagePack encoded):
│ {
│ type: MSG_SEND
│ temp_id: "abc-123"
│ to: "usr-456@nexgate.com"
│ conv_id: "conv-789"
│ body: "Habari"
│ content_type: TEXT
│ }
│
▼
[Ejabberd Node 1]
│
├── Validates session (already authed)
├── ACKs client immediately:
│ { temp_id: "abc-123", status: ACK }
├── Routes to usr-456 (if online):
│ Erlang looks up which node holds usr-456
│ If Node 1 → delivers directly
│ If Node 2 → Erlang distributed message (no Redis needed)
│
└── Publishes to RabbitMQ: chat.message.inbound
{
from: "usr-123@nexgate.com",
to: "usr-456@nexgate.com",
conv_id: "conv-789",
body: "Habari",
temp_id: "abc-123",
timestamp: 1719446400
}
.
. (async)
.
[Spring Boot Chat Service]
│ consumes chat.message.inbound
│
├── Authorization check (can A message B?)
├── Resolve message level
├── Write to PostgreSQL (messages table)
├── Write to Redis hot cache (last 50 per conv)
├── Update conversation last_message
│
├── usr-456 online? (check via Ejabberd REST API)
│ YES → DELIVERED receipt after Ejabberd confirms
│ NO → RabbitMQ offline queue + FCM + escalation timer
│
└── Notify sender: tick update
REST API → Ejabberd → WS push to Client A
Client A: ✓✓ (delivered)
Cross-Node Routing — No Redis Pub/Sub Needed
Phase 1 (Spring Boot WS):
Pod 1 holds Client A connection
Pod 2 holds Client B connection
Redis pub/sub needed to bridge pods
Pod 1 publishes → Redis → Pod 2 delivers
Phase 2 (Ejabberd cluster):
Node 1 holds Client A connection
Node 2 holds Client B connection
Erlang distributed messaging bridges nodes
Node 1 → Erlang dist → Node 2 delivers
Redis pub/sub no longer needed for routing
(Redis still used by Chat Service for hot cache)
This is why Ejabberd can do 2M concurrent:
Erlang process per connection (~2KB RAM each)
Native cross-node routing built into the language
No external message bus overhead
8. Voice Calls
Components
Signaling: Ejabberd Jingle (XEP-0166)
coordinates call setup via XMPP stanzas
STUN: Ejabberd built-in (mod_stun_disco)
helps devices find their public IP behind NAT
TURN: Coturn (separate VPS)
relay when P2P impossible (EA carrier NAT)
Transport: WebRTC in mobile app
actual audio stream between devices
Codec: Opus
adaptive 6kbps (2G) → 64kbps (WiFi)
echo cancellation + noise suppression built in
non-negotiable for EA networks
Jingle Signaling Stanzas
<!-- Step 1: Kibuti initiates call to Juma -->
<iq from="kibuti@nexgate.com/android"
to="juma@nexgate.com"
type="set" id="call-001">
<jingle xmlns="urn:xmpp:jingle:1"
action="session-initiate"
sid="session-abc-123"
initiator="kibuti@nexgate.com/android">
<content name="audio">
<description xmlns="urn:xmpp:jingle:apps:rtp:1"
media="audio">
<payload-type id="111" name="opus" clockrate="48000"/>
</description>
<transport xmlns="urn:xmpp:jingle:transports:ice-udp:1"
ufrag="someUfrag"
pwd="somePassword">
<candidate ... /> <!-- Kibuti's ICE candidates -->
</transport>
</content>
</jingle>
</iq>
<!-- Step 2: Juma accepts -->
<iq from="juma@nexgate.com/android"
to="kibuti@nexgate.com/android"
type="set" id="call-002">
<jingle action="session-accept"
sid="session-abc-123">
<!-- Juma's SDP answer + ICE candidates -->
</jingle>
</iq>
<!-- Step 3: Call ends -->
<iq type="set">
<jingle action="session-terminate"
sid="session-abc-123">
<reason><success/></reason>
</jingle>
</iq>
Full Voice Call Flow
[Kibuti — taps Call]
│
▼ GET /chat/calls/turn-credentials
Spring Boot returns:
{
iceServers: [
{ urls: "stun:chat.nexgate.com:3478" },
{ urls: "turn:turn.nexgate.com:3478",
username: "usr-123:1719446400",
credential: "hmac_token" }
],
ttl: 3600
}
│
▼ Initialize WebRTC PeerConnection
Add audio track (Opus codec)
Gather ICE candidates (STUN discovery)
│
▼ Send Jingle session-initiate via Ejabberd WS
Ejabberd routes to Juma
Ejabberd fires RabbitMQ event: chat.call.initiated
│
Spring Boot:
Create call record:
status: RINGING
started_at: now
If Juma offline → FCM HIGH priority:
{ type: INCOMING_CALL, callId, callerName, callType: VOICE }
│
[Juma's phone rings]
Juma taps Answer
│
▼ Juma sends Jingle session-accept via Ejabberd WS
ICE negotiation begins between devices:
│
├── P2P possible? (good network)
│ Direct connection established ✅
│ No Coturn bandwidth used
│
└── P2P blocked? (EA carrier NAT)
Both connect to Coturn relay
Audio flows: Kibuti → Coturn → Juma
│
Call live 🎉
RTCP monitors quality every 200ms:
Good network → Opus 32-64kbps, clear voice
3G → Opus 16kbps, still good
2G → Opus 8kbps, slightly robotic but connected
Very poor → Opus 6kbps, minimum viable
│
Kibuti taps End
│
▼ Jingle session-terminate via Ejabberd WS
Ejabberd fires: chat.call.ended
Spring Boot:
Update call record:
status: COMPLETED
ended_at: now
duration_seconds: calculated
relay_used: true/false
Call State Machine
IDLE
│ user taps Call
▼
INITIATING
│ getting TURN credentials
│ creating WebRTC offer
▼
RINGING ──────────────────────▶ MISSED (45s timeout)
│ Jume answers
▼
CONNECTING
│ ICE negotiation
│ finding best path
▼
CONNECTED ────────────────────▶ RECONNECTING (network drop)
│ call live │ ICE restart
│ │ 10s timeout → FAILED
│ user ends
▼
ENDING
│ Jingle terminate sent
▼
COMPLETED / MISSED / DECLINED / FAILED
Codec Ladder — Opus Adaptive
Network Bitrate Quality
─────────────────────────────────────────────────
WiFi / 4G strong 64 kbps HD voice
4G standard 32 kbps Clear
3G 16 kbps Good enough
2G / Edge 8 kbps Robotic but connected
Barely alive 6 kbps Minimum viable
─────────────────────────────────────────────────
Opus switches automatically based on RTCP feedback
No configuration needed — adaptive by design
9. Video Calls
Same Architecture as Voice + Camera
Everything from voice call applies
Additional components:
Video codec: H.264 (primary)
Hardware accelerated on Tecno, Infinix, Samsung
Low battery drain — GPU handles encoding
Fallback: VP8 (software, more CPU)
Camera: Front camera default (switchable)
Device detects capability at call start
Resolution ladder (adaptive):
─────────────────────────────────────────────────
WiFi 720p 30fps 1.5 Mbps
4G strong 480p 24fps 800 kbps
3G 360p 15fps 400 kbps
2G 240p 10fps 150 kbps
Very poor VIDEO OFF — audio only (Opus)
─────────────────────────────────────────────────
Degradation order (never drops call):
1. Reduce color depth
2. Reduce resolution (720→480→360→240)
3. Reduce frame rate (30→24→15→10fps)
4. Reduce audio bitrate
5. Disable video entirely → audio only
6. Reduce audio to minimum (6kbps Opus)
Device Tier Detection
App detects device capability at call start:
High-end (Pixel, Samsung S series):
H.264 hardware encoder (GPU)
Start at 720p 30fps
Low battery impact
Mid-range (Samsung A series):
H.264 hardware encoder
Start at 480p 24fps
Medium battery impact
Low-end (Tecno Spark, Infinix Hot):
H.264 software encoder (CPU)
Start at 360p 15fps
High battery impact
Show warning: "Video call may drain battery faster"
Auto-disable video after 10min if battery < 20%
Jingle for Video — Additional Content Block
<!-- Video call adds video content block -->
<jingle action="session-initiate" sid="session-xyz">
<!-- Audio block (same as voice) -->
<content name="audio">
<description media="audio">
<payload-type name="opus" clockrate="48000"/>
</description>
<transport .../>
</content>
<!-- Video block (added for video calls) -->
<content name="video">
<description media="video">
<payload-type id="96" name="H264" clockrate="90000"/>
<payload-type id="97" name="VP8" clockrate="90000"/>
</description>
<transport .../>
</content>
</jingle>
10. Coturn — TURN Relay
Why TURN is Mandatory for EA
Direct P2P (ideal):
Both devices negotiate directly
Audio/video flows device-to-device
Ejabberd not involved in media
No bandwidth cost on your servers
EA reality — P2P often blocked:
Vodacom, Airtel, Tigo use CGNAT
Multiple users share one public IP
P2P connection cannot be established
Without TURN → call fails
TURN relay (fallback):
Both devices connect to Coturn
Coturn relays audio/video between them
Call works regardless of carrier NAT
Bandwidth cost on your server (~50KB/min voice)
Coturn Config Highlights
listening-port=3478
tls-listening-port=5349
relay-ip=YOUR_COTURN_VPS_IP
realm=nexgate.com
lt-cred-mech # time-limited credentials
use-auth-secret
static-auth-secret=${COTURN_SECRET} # from Vault
min-port=49152
max-port=65535
TURN Credentials Generation
Credentials are time-limited HMAC tokens
Generated by Spring Boot per call session
Coturn validates them — prevents abuse
Format:
username: {userId}:{expiry_timestamp}
credential: HMAC-SHA1(secret, username)
ttl: 3600 seconds (1 hour per call)
Only NexGate users can use your TURN server
No credential → Coturn rejects connection
Bandwidth Estimation
Voice call via TURN:
Opus 16kbps × 2 directions = ~4KB/min
1 hour call ≈ 240KB per participant
Video call via TURN:
360p H.264 × 2 directions = ~6MB/min
Force 360p max when on relay to control cost
Coturn VPS sizing:
Hetzner CX11 (€4/month, 1vCPU/2GB)
20TB bandwidth included
Handles ~500 concurrent voice relay calls
Upgrade to CX21 at scale
11. MessagePack Encoding
Why Switch from JSON
JSON message frame:
{"type":"MSG_SEND","conv_id":"conv-123456","sender_id":"usr-789012",
"body":"Habari","timestamp":1719446400000,"temp_id":"abc-def-ghi"}
Size: ~140 bytes
Every key repeated as string on every message
Numbers encoded as ASCII characters
Parsing: character by character
MessagePack same message:
[binary representation]
Size: ~50 bytes
Keys encoded as integers (schema registered)
Numbers encoded as actual bytes (int32 = 4 bytes)
Parsing: read fixed byte positions
Result:
60-65% smaller on wire
3-5x faster to parse
Critical for users on 2G/3G with limited data bundles
Migration Strategy (No Breaking Change)
Both formats supported simultaneously:
Client sends header:
Content-Type: application/msgpack → MessagePack
Content-Type: application/json → JSON (default)
Ejabberd detects Content-Type
Routes to appropriate deserializer
Migration flow:
Old app version → sends JSON → works fine
New app version → sends MessagePack → works fine
No forced update required
Gradual migration over 30-60 days
Remove JSON support after 90%+ adoption
12. Broadcast Channels
What They Are
Creator → unlimited followers
One-directional: creator posts, followers receive
Like Telegram channels
No replies from followers (unless creator enables Q&A)
Use cases:
Shop announcement channel ($techstore updates)
Creator content channel (@kibuti posts)
NexGate system channel (platform announcements)
Fan-out Strategy
Small channel (< 10,000 followers):
Write-on-send — Chat Service pushes to each follower
Same as group chat fan-out
Large channel (10,000+ followers):
Lazy fan-out — store message once
Followers fetch on open (read-time delivery)
No per-follower push for casual followers
FCM push only to followers with notifications enabled
Same celebrity bypass pattern as VP Feed:
Hot channels → read-time merge
Normal channels → write-time fan-out
Ejabberd MUC for Channels
Broadcast channel = MUC room with restrictions:
Only owner/admins can send messages
Members are read-only subscribers
mod_muc handles this with role configuration:
Role: moderator → can send
Role: visitor → read only
This means channels are built on the
same MUC infrastructure as group chats
No separate implementation needed
13. MQTT — Mini Apps Foundation
What MQTT Enables
Ejabberd runs MQTT broker on port 1883
No extra infrastructure — already in Ejabberd
Mini Apps subscribe to topics:
orders/{orderId} → real-time order updates
delivery/{trackingId} → GPS delivery tracking
live/{streamId}/viewers → viewer count updates
jiko/{restaurantId} → JikoXpress kitchen events
Spring Boot publishes events:
Order shipped → publish to orders/{orderId}
Mini App receives instantly
No polling needed
MQTT vs XMPP for Mini Apps
XMPP (chat):
Full protocol, complex stanzas
Designed for human conversation
Bidirectional, stateful sessions
Right tool for chat
MQTT (events):
Lightweight pub/sub protocol
Designed for IoT and event streams
Minimal overhead (2-byte header)
Right tool for Mini App events
Works on very limited connections
Both live inside Ejabberd:
Same server, different protocols
Mobile app uses XMPP for chat
Mini Apps use MQTT for events
Zero additional infrastructure
14. Message Interactions
All message interaction features are handled via standard XMPP XEPs. Ejabberd routes the stanzas automatically — Spring Boot handles persistence and business rules via RabbitMQ events.
Overview — All Four Features
Feature XEP Status Ejabberd
────────────────────────────────────────────────────────
Edit message XEP-0308 Stable ✅ auto routed
Delete for everyone XEP-0424 Stable ✅ auto routed
Reactions XEP-0444 Stable ✅ auto routed
Forwarding XEP-0297 Stable ✅ auto routed
Reply to message XEP-0461 Experimental auto routed
Stable stanza IDs XEP-0359 Stable ✅ auto assigned
────────────────────────────────────────────────────────
All routed by Ejabberd
Spring Boot handles: validation, persistence, rules
XEP-0359 — Stable Stanza IDs (Foundation)
Before the features — this XEP is the foundation all others depend on. Every message gets a stable server-assigned ID used by reactions, edits, retractions, and replies to reference the correct message.
<!-- Ejabberd automatically adds stanza-id to every message -->
<message from="kibuti@nexgate.com"
to="juma@nexgate.com"
id="client-generated-id">
<body>Habari</body>
<stanza-id xmlns="urn:xmpp:sid:0"
id="server-stable-id-abc123"
by="nexgate.com"/>
<!-- server-stable-id-abc123 is what reactions/edits reference -->
</message>
Message Editing — XEP-0308
Who can edit: Original sender only
Time window: 15 minutes after send
What: Text body only
Commerce cards: ❌ BLOCKED — financial records are immutable
System messages: ❌ BLOCKED — never editable
<!-- Kibuti edits his message -->
<message from="kibuti@nexgate.com"
to="juma@nexgate.com"
type="chat"
id="edit-002">
<body>Habari yako Juma, vipi biashara?</body>
<replace xmlns="urn:xmpp:message-correct:0"
id="server-stable-id-abc123"/>
<!-- references original message by stanza-id -->
</message>
Flow:
Kibuti edits → stanza sent via Ejabberd WS
Ejabberd routes to Juma (if online)
Ejabberd fires RabbitMQ: chat.message.edited
│
Spring Boot:
Is sender original author? ✅
Within 15 minute window? ✅
Not a commerce/system message? ✅
Update messages.body = new text
Update messages.edited_at = now
Increment messages.edit_count
│
Juma's app:
Receives edit stanza
Updates message in place (same position in thread)
Shows "Edited" label under message
Group chats:
Same stanza sent to MUC room JID
Ejabberd MUC broadcasts to all members
All see updated message simultaneously
Delete for Everyone — XEP-0424
Two delete modes:
Delete for me:
Local filter only
No Ejabberd stanza needed
Spring Boot records: message_deletions (scope: SELF)
Recipient unaffected
Delete for everyone:
XEP-0424 retraction stanza
Ejabberd routes to all recipients
Time window: 15 minutes
Commerce cards: ❌ BLOCKED
System messages: ❌ BLOCKED
<!-- Delete for everyone — retraction stanza -->
<message from="kibuti@nexgate.com"
to="juma@nexgate.com"
type="chat"
id="retract-003">
<apply-to xmlns="urn:xmpp:fasten:0"
id="server-stable-id-abc123">
<retract xmlns="urn:xmpp:message-retract:1"/>
</apply-to>
</message>
Flow:
Kibuti retracts → stanza via Ejabberd WS
Ejabberd routes to Juma
Ejabberd fires RabbitMQ: chat.message.retracted
│
Spring Boot:
Is sender original author? ✅
Within 15 minute window? ✅
Not blocked message type? ✅
Soft delete:
messages.deleted_at = now
messages.deleted_by = usr-kibuti
messages.delete_scope = EVERYONE
body NOT removed (audit trail kept)
│
Juma's app:
Receives retraction stanza
Replaces message with:
"This message was deleted"
Same position in thread
Nothing is ever hard deleted from PostgreSQL:
Legal compliance (EA regulations)
Dispute resolution (order/payment disputes)
Admin investigation (fraud cases)
Soft delete always — hard delete never
Reactions — XEP-0444
Model: One reaction per user per message
Emoji set: Limited set at launch
❤️ 👍 😂 😮 😢 🙏
Change: Send new emoji → replaces old
Remove: Send empty → removes reaction
Commerce cards: ✅ ALLOWED (reactions don't modify content)
System messages: ❌ BLOCKED
<!-- Kibuti reacts 👍 to message -->
<message from="kibuti@nexgate.com"
to="juma@nexgate.com"
type="chat"
id="reaction-001">
<reactions xmlns="urn:xmpp:reactions:0"
id="server-stable-id-abc123">
<reaction>👍</reaction>
</reactions>
</message>
<!-- Kibuti changes to ❤️ -->
<message ...>
<reactions xmlns="urn:xmpp:reactions:0"
id="server-stable-id-abc123">
<reaction>❤️</reaction>
</reactions>
</message>
<!-- Kibuti removes reaction -->
<message ...>
<reactions xmlns="urn:xmpp:reactions:0"
id="server-stable-id-abc123">
<!-- empty = removed -->
</reactions>
</message>
Flow:
Kibuti taps 👍 → reaction stanza via Ejabberd WS
Ejabberd routes to Juma
Ejabberd fires RabbitMQ: chat.message.reaction
│
Spring Boot:
Upsert in message_reactions:
ON CONFLICT (message_id, user_id)
→ update emoji + timestamp
Empty emoji received → delete reaction record
│
Juma's app:
Receives reaction stanza
Updates reaction display below message:
👍 1
Kibuti's own reaction: highlighted
Group chats:
Stanza sent to MUC room
Ejabberd MUC broadcasts to all members
All screens update simultaneously:
👍 3 ❤️ 2 😂 1
Notification:
Reaction on your message → FCM push
"Juma reacted 👍 to your message"
Level: NORMAL (FCM only — no SMS)
Muted conversations → no reaction notification
Message Forwarding — XEP-0297
What it is:
Client creates NEW message in target conversation
Original message wrapped inside as reference
Server never "moves" anything
Forwarded label shown with original sender name
Forward chain tracking:
chain = 1: "↪ Forwarded from Juma Mwangi"
chain = 2-4: "↪ Forwarded"
chain = 5+: "↪ Forwarded many times" (misinformation warning)
Multi-forward: up to 5 conversations per action
Max chain: no hard limit but UI degrades label
Commerce rules:
Product card: ✅ anyone can forward
Custom price offer: ❌ private deal — blocked
Order confirmation: ❌ private record — blocked
Payment record: ❌ private record — blocked
System messages: ❌ blocked
<!-- Kibuti forwards Juma's message to Amina -->
<message from="kibuti@nexgate.com"
to="amina@nexgate.com"
type="chat"
id="fwd-001">
<body>Angalia hii</body>
<forwarded xmlns="urn:xmpp:forward:0">
<delay xmlns="urn:xmpp:delay"
stamp="2026-07-02T10:32:00Z"/>
<!-- original send time preserved -->
<message from="juma@nexgate.com"
to="kibuti@nexgate.com"
type="chat"
id="msg-original-001">
<body>Habari yako rafiki!</body>
</message>
</forwarded>
<!-- NexGate forward metadata -->
<nexgate-forward xmlns="urn:nexgate:forward">
<original_sender_name>Juma Mwangi</original_sender_name>
<forward_chain>1</forward_chain>
</nexgate-forward>
</message>
Flow:
Kibuti taps Forward on Juma's message
Picks Amina's conversation
App creates new message stanza (not routing original)
Sends via Ejabberd WS to Amina
Ejabberd routes normally as new message
Fires RabbitMQ: chat.message.inbound (same as any message)
│
Spring Boot:
Validates forward is allowed (type check)
Creates new messages record:
is_forwarded: true
original_sender_name: "Juma Mwangi"
forward_chain: 1
media_ref: original fileId (no re-upload)
│
Amina's app:
Receives as new message
Renders with forwarded label:
┌────────────────────────────────┐
│ ↪ Forwarded from Juma Mwangi │
│ │
│ Habari yako rafiki! │
│ 10:45 │
└────────────────────────────────┘
Media forwarding:
References original fileId — no re-upload
10 people forward same image
→ 1 file in MinIO, 10 message records
File Thunder serves same file to all
Message Replies — XEP-0461
Reply to a specific message in thread
Like WhatsApp/Telegram quote-reply
Shows original message above reply
Status: Experimental ⚠️
Not yet stable standard
But widely implemented
(Gajim, Monal, many others use it)
Safe to implement — unlikely to change drastically
<!-- Juma replies to Kibuti's specific message -->
<message from="juma@nexgate.com"
to="kibuti@nexgate.com"
type="chat"
id="reply-001">
<body>Nzuri sana, asante!</body>
<reply xmlns="urn:xmpp:reply:0"
to="kibuti@nexgate.com"
id="server-stable-id-abc123"/>
<!-- id references the message being replied to -->
</message>
UI renders:
┌────────────────────────────────┐
│ ┌──────────────────────────┐ │
│ │ Kibuti │ │ ← quoted original
│ │ Habari yako Juma! │ │
│ └──────────────────────────┘ │
│ │
│ Nzuri sana, asante! │
│ 10:47 │
└────────────────────────────────┘
Tap on quote → scroll to original message
Commerce Messages — Interaction Rules Summary
Message type Edit Delete(all) React Forward
────────────────────────────────────────────────────────
Text message ✅ 15m ✅ 15m ✅ ✅
Voice note ❌ ✅ 15m ✅ ✅
Image / Video ❌ ✅ 15m ✅ ✅
Product card ❌ ❌ ✅ ✅
Custom price offer ❌ ❌ ✅ ❌
Order confirmation ❌ ❌ ✅ ❌
Payment confirmation ❌ ❌ ✅ ❌
System message ❌ ❌ ❌ ❌
────────────────────────────────────────────────────────
Why commerce cards are protected:
Immutable negotiation record
Seller cannot change agreed price after the fact
Buyer cannot claim different price was offered
Full audit trail in thread — legally important
RabbitMQ Events — New in Phase 2 for Interactions
Exchange: nexgate.chat (topic) — additions:
Routing Key Fired When
──────────────────────────────────────────────────────
chat.message.edited XEP-0308 received
chat.message.retracted XEP-0424 received
chat.message.reaction XEP-0444 received
chat.message.forwarded XEP-0297 received
chat.message.delete_self delete for me (REST call)
15. Docker Deployment
docker-compose additions for Phase 2
ejabberd:
image: ghcr.io/processone/ejabberd:latest
container_name: ejabberd
restart: unless-stopped
ports:
- "5222:5222" # XMPP TCP
- "5280:5280" # WebSocket + HTTP
- "1883:1883" # MQTT
- "3478:3478/udp" # STUN
volumes:
- ./ejabberd/ejabberd.yml:/home/ejabberd/conf/ejabberd.yml
- ./ejabberd/data:/home/ejabberd/database
- ./ejabberd/logs:/home/ejabberd/logs
environment:
- EJABBERD_BYPASS_WARNINGS=true
depends_on:
- postgres
- rabbitmq
networks:
- nexgate-internal
# Coturn on separate VPS — not in same compose
# Deployed independently on Hetzner CX11
# Connects back to NexGate via internal network
Traefik — WebSocket Routing
# Ejabberd service labels for Traefik
labels:
- "traefik.enable=true"
# App connects here for chat
- "traefik.http.routers.chat.rule=Host(`chat.nexgate.com`)"
- "traefik.http.routers.chat.tls=true"
- "traefik.http.routers.chat.tls.certresolver=letsencrypt"
- "traefik.http.services.chat.loadbalancer.server.port=5280"
# Sticky sessions — CRITICAL for WebSocket
# Same user must always hit same Ejabberd node
- "traefik.http.services.chat.loadbalancer.sticky.cookie=true"
- "traefik.http.services.chat.loadbalancer.sticky.cookie.name=ejabberd_node"
Why sticky sessions:
User connected to Ejabberd Node 1
Next request hits Node 2
→ connection context lost → disconnected
Sticky cookie ensures:
usr-123 always → Node 1
usr-456 always → Node 2
WS sessions stable across load balancer
Ejabberd Cluster Config
# Second node joins cluster
# On node 2's ejabberd.yml:
hosts:
- "nexgate.com"
# Erlang cookie must match on all nodes
# Set via environment variable
# Both nodes discover each other automatically
# Erlang distributed handles the rest
# Result:
# Message to usr-456 arrives on Node 1
# usr-456 connected to Node 2
# Erlang routes internally — transparent
16. Database Additions
calls (new in Phase 2)
calls
─────────────────────────────────────────────
call_id UUID PK
caller_id UUID
receiver_id UUID
conversation_id UUID FK → conversations
type ENUM VOICE / VIDEO
status ENUM RINGING / CONNECTED / COMPLETED /
MISSED / DECLINED / FAILED
started_at TIMESTAMPTZ
answered_at TIMESTAMPTZ
ended_at TIMESTAMPTZ
duration_seconds INT
relay_used BOOLEAN
end_reason ENUM NORMAL / NETWORK / TIMEOUT / DECLINED
call_quality_logs (new in Phase 2)
call_quality_logs
─────────────────────────────────────────────
log_id UUID PK
call_id UUID FK → calls
timestamp TIMESTAMPTZ
direction ENUM OUTBOUND / INBOUND
bitrate_kbps INT
packet_loss_pct DECIMAL
jitter_ms INT
rtt_ms INT
resolution TEXT "360p" "480p" "720p" or null
codec_audio TEXT "opus"
codec_video TEXT "h264" "vp8" or null
broadcast_channels (new in Phase 2)
broadcast_channels
─────────────────────────────────────────────
channel_id UUID PK
owner_id UUID userId or shopId
owner_type ENUM USER / SHOP
name TEXT
description TEXT
avatar_file_id UUID
subscriber_count INT
type ENUM PERSONAL / SHOP / SYSTEM
created_at TIMESTAMPTZ
message_reactions (new in Phase 2)
message_reactions
─────────────────────────────────────────────
id UUID PK
message_id UUID FK → messages
conversation_id UUID FK → conversations
user_id UUID
emoji TEXT "👍" "❤️" "😂" etc
reacted_at TIMESTAMPTZ
Unique constraint: (message_id, user_id)
→ one reaction per user per message
→ upsert on conflict replaces emoji
message_deletions (new in Phase 2)
message_deletions
─────────────────────────────────────────────
id UUID PK
message_id UUID FK → messages
deleted_by UUID userId
scope ENUM SELF / EVERYONE
deleted_at TIMESTAMPTZ
messages table additions (Phase 2)
New columns added to existing messages table:
edited_at TIMESTAMPTZ when last edited
edit_count INT how many times edited
original_body TEXT body before first edit (audit)
deleted_at TIMESTAMPTZ soft delete timestamp
deleted_by UUID who deleted
delete_scope ENUM SELF / EVERYONE
is_forwarded BOOLEAN was this forwarded
forward_chain INT forwarding depth (1,2,3...)
original_sender_name TEXT display name at forward time
original_message_id UUID source message if forwarded
reply_to_id UUID FK → messages (for replies)
stanza_id TEXT Ejabberd XEP-0359 stable ID
17. Migration from Phase 1
Zero Downtime Migration Path
Step 1 — Deploy Ejabberd alongside Phase 1 gateway
Both running simultaneously
Ejabberd on port 5280 (new)
Spring Boot WS gateway on port 8083 (existing)
No traffic to Ejabberd yet
Step 2 — Internal testing
Team connects to Ejabberd directly
Test auth bridge
Test message flow
Test commerce DMs
Confirm RabbitMQ events match expected format
Step 3 — Gradual traffic shift (Traefik)
5% of new connections → Ejabberd
Monitor error rates
25% → 50% → 75% → 100%
Can rollback instantly via Traefik weight
Step 4 — Full switch
chat.nexgate.com → Ejabberd (100%)
Spring Boot WS gateway → decommission
Redis pub/sub routing → no longer needed for delivery
Step 5 — Enable voice calls
App update released
WebRTC + Jingle support in mobile app
Coturn live and tested
Voice calls launch
Step 6 — MessagePack migration
New app version sends MessagePack
Old versions continue JSON
Both supported simultaneously
Monitor adoption → remove JSON after 90%+
Step 7 — Video calls
App update with camera support
Same signaling path as voice
Adaptive video quality enabled
Summary
Phase 2 upgrades the transport layer from Spring Boot WebSocket to Ejabberd without touching a single line of business logic. The schema stays. The commerce flows stay. The notification system stays. The inbox model stays.
Ejabberd brings what Spring Boot WS cannot: 2 million concurrent connections on a single node, built-in XMPP presence protocol, native Jingle signaling for voice and video calls, Erlang distributed cross-node routing without Redis pub/sub, and an MQTT broker for Mini Apps at zero extra infrastructure cost.
Voice calls use WebRTC with Opus audio — adaptive from 64kbps on WiFi to 6kbps on 2G. Coturn TURN relay ensures calls work even behind EA carrier NAT, which blocks most peer-to-peer connections on Vodacom, Airtel, and Tigo networks.
Video calls add H.264 video with hardware acceleration on low-end EA phones, adaptive resolution from 720p down to audio-only when the network demands it.
MessagePack reduces message frame size by 60-65%, saving real money on EA mobile data bundles and reducing server bandwidth costs at scale.
Message interactions — editing, deleting, reactions, forwarding, and replies — are all handled via standard XMPP XEPs routed automatically by Ejabberd. Spring Boot validates business rules (time windows, commerce card protection, author checks) and persists results. Commerce messages are immutable by design: product cards, custom price offers, order confirmations, and payment records cannot be edited or deleted for everyone — they form the legal audit trail of every transaction.
The migration from Phase 1 to Phase 2 is gradual and zero-downtime — Traefik shifts traffic incrementally from Spring Boot WS to Ejabberd, with instant rollback available at any point.
NexGate Chat Platform — Phase 2: Production Architecture v1.0 QBIT SPARK | Ejabberd · Coturn · WebRTC · Voice & Video · Edit · Delete · React · Forward