Event Catalogue — every message NexGate publishes to Kafka
Cluster: Redpanda (Kafka API) · Schema registry: JSON Schema, subject <topic>-value, BACKWARD compatibility
Short Description: TheEverything that crosses the line between the backend and the recommendation service, in both directions. Inbound (§1–§6): the complete list of what the backend puts on Kafka:Kafka — 12 topics, 27 event types. For— when each one: when it fires, what the key is, what the payload carries, and — just as important — what does not cause it.it, Thisand isevery payload field with its type, nullability and allowed values, including the referenceexact forshape of each *.deleted. Outbound (§7): the recommendationeight developer,Redis keys the model writes back, their exact format, and the contractwhat the backend does with each one. It is heldwritten to.to be self-contained: a developer with this document and a Kafka client needs nothing else to start.
Read
06_RECOMMENDATION_DEVELOPER_GUIDE.mdfirst. That guide is how to connect, what to do with the messages and how to write recommendations back. This document is the catalogue — everything that arrives, in detail.06links here rather than repeating it.
Hints:
- Nothing is emitted from service code. Events come from committed database rows, so what you receive always matches what is in the database — a rolled-back transaction publishes nothing.
- A payload is always the full current state, never a diff. Replace what you stored.
- Order per key is guaranteed; order across keys is not. Use
entityVersion, neveroccurredAt. - Fields are only ever added. Ignore what you don't recognise.
- If a topic is ever lost, everything can be republished from the database:
POST /api/admin/feed/backfills.
0. The round trip in one picture
INBOUND — backend ➜ model §1–§6
──────────────────────────────────────────────────────────────────────────────
a row commits
(post · product · shop · event · profile · follow · block · order · wishlist)
└─► outbox ─► relay ─► 9 entity topics ......... compacted, "what exists"
the app calls POST /api/v1/feed/interactions/batch (impressions, views…)
a row commits (like · comment · cart · order paid) (seen by the backend)
└─► nexgate.interactions.v1 ................. 90 days, "what people did"
a feed page is served
└─► nexgate.feed-served.v1 .................. 14 days, "what we showed"
│
▼
┌───────────────────────────────┐
│ RECOMMENDATION SERVICE │
│ yours — we never see inside │
└───────────────┬───────────────┘
│
OUTBOUND — model ➜ backend │ §7
──────────────────────────────────┼───────────────────────────────────────────
▼
SET reco:<surface>:{id} <json> EX 172800 (8 keys, all optional)
│
▼
the feed pipeline reads reco:* while building a session;
missing · invalid · stale ─► trending and interest pools fill in
The boundary never moves. The backend only ever writes Kafka and reads reco:*. The model only ever reads Kafka and writes reco:*. Neither side calls the other over HTTP, so neither can be made slow or unavailable by the other.
1. The envelope
Every message on the nine entity topics has the same eight fields:
{
"eventId": "9f2c1e8a-5b3d-4a91-8c77-2e6f0b4d1a35",
"eventType": "product.updated",
"entityId": "4f2a77e1-0c8b-4d2f-9a11-7b6c3e5d8f90",
"entityVersion": 184022,
"occurredAt": "2026-09-18T10:22:33.120Z",
"schemaVersion": 1,
"source": "nexgate-backend",
"payload": { }
}
| Field | Meaning |
|---|---|
eventId |
Unique per event. Seen twice = a retry; drop the second |
eventType |
One of the 27 in this document |
entityId |
The thing this is about |
entityVersion |
Only ever increases per entity. Keep the highest you have seen and ignore anything lower |
occurredAt |
When it happened. Not an ordering key — clocks and retries make it unreliable for that |
schemaVersion |
1 today |
source |
Always nexgate-backend |
payload |
Full current state, or, for a *.deleted, the id plus a reason |
The interactions and feed-served topics use a slightly different envelope — noted in their sections.
2. How a delete arrives
Always two messages, in this order:
<entity>.deletedwith areason— the entity is gone for you, and the reason says why.- A tombstone: same key,
value = null— so log compaction can drop the key.
"Deleted" does not always mean the row was deleted. It means this entity is no longer something to recommend. A post set to private, a shop that closed, an account that got locked — all arrive as *.deleted with a reason that tells them apart. If it becomes visible again later, you get *.created again.
Tombstones are kept for 1 day. A consumer reading a compacted topic from offset 0 after that sees only live entities.
3. The topics at a glance
| # | Topic | Key | Cleanup | Partitions | Event types |
|---|---|---|---|---|---|
| 1 | nexgate.posts.v1 |
postId |
compact | 24 | post.created · post.updated · post.deleted |
| 2 | nexgate.products.v1 |
productId |
compact | 24 | product.created · product.updated · product.deleted |
| 3 | nexgate.shops.v1 |
shopId |
compact | 12 | shop.created · shop.updated · shop.deleted |
| 4 | nexgate.events.v1 |
eventId |
compact | 12 | event.created · event.updated · event.deleted |
| 5 | nexgate.profiles.v1 |
accountId |
compact | 24 | profile.created · profile.updated · profile.deleted |
| 6 | nexgate.follows.v1 |
followerId:followedId |
compact | 24 | follow.created · follow.deleted |
| 7 | nexgate.blocks.v1 |
blockerId:blockedId |
compact | 12 | block.created · block.deleted |
| 8 | nexgate.orders.v1 |
orderId |
compact | 24 | order.created · order.updated · order.deleted |
| 9 | nexgate.wishlist.v1 |
accountId:productId |
compact | 24 | wishlist.created · wishlist.updated · wishlist.deleted |
| 10 | nexgate.interactions.v1 |
accountId |
delete, 90 d | 96 | interaction |
| 11 | nexgate.feed-served.v1 |
accountId |
delete, 14 d | 48 | feed.served |
| 12 | nexgate.fanout-tasks.v1 |
postId:chunk |
delete, 3 d | 48 | (internal — not for consumers) |
Partition counts are fixed for good. Adding partitions later would move keys to different partitions and break per-key ordering.
Compacted topics are a live catalogue: read from offset 0 and you have the current state of every entity, then keep consuming to stay current.
4. Entity topics
Each section below is complete: every field of the payload, its type, whether it can be null, its allowed values where it is an enum, and the exact shape of the *.deleted payload.
Reading the type column: uuid, string, int, decimal, bool, timestamp (ISO 8601 UTC), json (an opaque object — structure may change, do not depend on it). A field marked nullable can arrive as null; one marked required is on every message of that topic.
Fields are only ever added. The schemas are open by design: ignore any field you do not recognise, and never fail a message because of one.
4.1 nexgate.posts.v1 — posts and reels
postId · compacted · 24 partitions · schema src/main/resources/feed-schemas/nexgate.posts.v1.json
A post exists for you while it is PUBLISHED, not deleted, and PUBLIC or FOLLOWERS.
| Event | Fires when |
|---|---|
post.created |
A draft becomes published, or a hidden post becomes visible again |
post.updated |
A visible post, or any of its child rows, changes |
post.deleted |
It stops being visible — only if it was visible before |
Delete reasons: DELETED (soft-deleted) · UNPUBLISHED (no longer PUBLISHED) · VISIBILITY_RESTRICTED (switched to MENTIONED-only) or PRIVATE)
What counts as a change. Not just the post row — eleven child tables are watched, and a change to any of themone republishes the whole post: media, hashtags, attached productsproducts, /attached shopsshops, /attached events, user andmentions, shop mentions, links, collaborators, and the poll.
What doespublishes NOT publish anythingnothing: likesCount, commentsCount, repostsCount, bookmarksCount, viewsCount, quotesCount, sharesCount, shortClipCount, lastVerifiedAt, updatedAt. Engagement counters move constantly; republishing every post on every like would be most of the topic's traffic on the topic for none of the value. CountsThe counts in the payload are therefore a snapshot from the last real change, not live. For live behaviourbehaviour, use nexgate.interactions.v1.
A draft that is saved, edited and never published produces nothing at all.
Payload
—post.created/post.updated
| Field | Type | Notes |
|---|---|---|
postId |
uuid | required |
postType |
string, nullable | REGULAR · POLL |
status |
string, nullable | PUBLISHED on every message you receive (DRAFT, SCHEDULED, DELETED never reach you) |
author.id |
uuid, nullable | Who wrote it |
author.userName |
string, nullable | Public username. No real name is ever published |
content |
string, nullable | The post text as written |
contentParsed.hashtags[] |
string[] | Normalised, without the #. Any script (Latin, Arabic, CJK …) |
contentParsed.mentionedUserIds[] |
uuid[] | Accounts mentioned in the text |
contentParsed.mentionedShopIds[] |
uuid[] | Shops mentioned in the text |
media[].fileId |
uuid, nullable | File-service id |
media[].mediaType |
string, nullable | IMAGE · VIDEO |
media[].order |
int | Display order within the post, from 0 |
media[].status |
string, |
Processing PENDING |
media[].variants |
json, nullable | Rendition URLs keyed by name (hls, thumb, |
media[] |
bool | true There is no separate reel |
media[].mediaId |
uuid, nullable | The clip's own id — the one the app uses on reel endpoints |
media[].durationMs |
int, nullable | Length in milliseconds, for video |
attachments.products[] |
uuid[] | Products attached to the post |
attachments.shops[] |
uuid[] | Shops attached |
attachments.events[] |
uuid[] | Events attached |
engagement.likesCount |
int | Snapshot, see above |
engagement.commentsCount |
int | Snapshot |
engagement.sharesCount |
int | Snapshot |
engagement.viewsCount |
int | Snapshot |
engagement.bookmarksCount |
int | Snapshot |
engagement.repostsCount |
int | Snapshot |
engagement.quotesCount |
int | Snapshot |
privacySettings.visibility |
string, nullable | PUBLIC or FOLLOWERS on every message you receive |
privacySettings.whoCanComment |
string, nullable | Who may comment |
privacySettings.whoCanRepost |
string, nullable | Who may repost |
quotedPostId |
uuid, nullable | Set when this post quotes or reposts another. A pure repost carries the original's engagement, not its own |
collaboratorIds[] |
uuid[] | Co-authors |
hasPoll |
bool | Whether a poll is attached. Poll options and votes are not published |
externalLinkDomain |
string, nullable | The domain only, never the full URL |
isEdited |
bool | Whether it has been edited since publishing |
createdAt |
timestamp, nullable | When the row was created (may be long before publishing) |
publishedAt |
timestamp, nullable | When it went live — use this for recency, not createdAt |
updatedAt |
timestamp, nullable | Last change |
interests[].id |
uuid | Interest id from the shared list (interest_categories) |
interests[].w |
decimal | Weight: 1.0 from attached products and events, 0.6 from hashtags. An interest reached twice keeps the higher weight. Empty until an admin maps the category or hashtag |
Payload — post.deleted
| Field | Type | Notes |
|---|---|---|
postId |
uuid | required |
authorId |
uuid, nullable | The author, or null when the row itself is gone |
reason |
string | One of the delete reasons above |
Followed by a tombstone: same key, value = null.
4.2 nexgate.products.v1 — products
productId · compacted · 24 partitions
A product exists for you while it is ACTIVE or OUT_OF_STOCK, not deleted, and its shop is visible.
Running out of stock is an update, not a disappearance — stock is in the payload, so you decide what to do with it.
| Event | Fires when |
|---|---|
product.created |
It becomes visible, including when its shop comes back |
product.updated |
The product or its installment plan changes |
product.deleted |
It stops being visible |
Delete reasons: DELETED · UNPUBLISHED (DRAFT, INACTIVE, ARCHIVED) · SHOP_UNAVAILABLE (the shop was suspended, closed, unapproved or deleted)
The shop cascade. When a shop's visibility flips, every product of that shop is republished —, in pages, right at that moment. One shop suspension can therefore produce thousands of product.deleted messages. This is deliberate:Deliberate: a consumer thatwatching only watched the productthis topic would otherwise keep recommending products from a suspended shop.
What doespublishes NOT publishnothing: viewCount, cartAddCount, updatedAt.
Payload
—product.created/product.updated
| Field | Type | Notes |
|---|---|---|
productId |
uuid | required |
productName |
string, nullable | |
productSlug |
string, nullable | URL slug |
productDescription |
string, nullable | |
productType |
string, nullable | PHYSICAL · DIGITAL |
productMedia[].fileId |
uuid, nullable | |
productMedia[].mediaType |
string, nullable | IMAGE · VIDEO |
productMedia[].order |
int, nullable | Display order |
productMedia[].status |
string, nullable | Processing state |
productMedia[].variants |
json, nullable | Rendition URLs |
categoryId |
uuid, nullable | The key to interests — mapped by an admin |
categoryName |
string, nullable | |
parentCategoryId |
uuid, nullable | A category with no mapping inherits its parent's interests |
price |
decimal, nullable | Current price |
comparePrice |
decimal, nullable | "Was" price |
isOnSale |
bool | |
discountPercentage |
decimal, nullable | |
stockQuantity |
int | |
isInStock |
bool | |
isLowStock |
bool | |
showStockToPublic |
bool | Whether the shop displays the number |
status |
string, nullable | ACTIVE or OUT_OF_STOCK on every message you receive |
condition |
string, nullable | NEW · USED_LIKE_NEW · USED_GOOD · USED_FAIR · REFURBISHED · FOR_PARTS |
urgencyTag |
string, nullable | NONE · NEW_ARRIVAL · LIMITED_EDITION · LIMITED_OFFER · FEW_REMAINS |
specifications |
json, |
Free-form, |
colors |
json, |
Free-form, |
hasGroupBuying |
bool | |
groupPrice |
decimal, nullable | Price when bought as a group |
hasInstallments |
bool | |
stockInfo.soldCount |
int | Units sold |
shopId |
uuid, nullable | |
shopName |
string, nullable | |
createdAt |
timestamp, nullable | |
updatedAt |
timestamp, nullable | |
interests[] |
uuid | From the product's category, weight 1.0 |
interests[].w |
decimal |
Payload — product.deleted
| Field | Type | Notes |
|---|---|---|
productId |
uuid | required |
shopId |
uuid, nullable | Useful when the cause was the shop |
reason |
string |
4.3 nexgate.shops.v1 — shops
shopId · compacted · 12 partitions
A shop exists for you while it is ACTIVE or TEMPORARILY_OFFLINE, approved, and not deleted.
| Event | Fires when |
|---|---|
shop.created |
It becomes visible |
shop.updated |
Anything meaningful on the shop changes |
shop.deleted |
It stops being visible |
Delete reasons: DELETED · SUSPENDED · CLOSED (permanently closed)PERMANENTLY_CLOSED) · UNPUBLISHED (PENDING, or approval withdrawn)
Every one of these also republishes the shop's products (§4.2).
What doespublishes NOT publishnothing: subscriberCount, lastSeenTime, updatedAt.
Payload
—shop.created/shop.updated
| Field | Type | Notes |
|---|---|---|
shopId |
uuid | required |
shopName |
string, nullable | |
shopSlug |
string, nullable | |
shopDescription |
string, nullable | |
logo.fileId / .mediaType / .status / .variants |
uuid / string / string / json, nullable | The shop logo |
banner.fileId / .mediaType / .status / .variants |
uuid / string / string / json, nullable | The shop banner |
ownerId |
uuid, nullable | The account that owns the shop |
status |
string, nullable | ACTIVE or TEMPORARILY_OFFLINE on every message you receive |
tempOfflineUntil |
timestamp, nullable | Set while the owner has gone offline deliberately |
city |
string, nullable | |
district |
string, nullable | |
region |
string, nullable | The feed filters by region |
countryCode |
string, nullable | |
isVerified |
bool | |
verificationBadge |
string, nullable | Which badge |
trustScore |
decimal, nullable | Platform trust score |
subscriberCount |
int | Snapshot — a change to it alone does not republish |
approvedAt |
timestamp, nullable | |
createdAt |
timestamp, nullable | |
updatedAt |
timestamp, nullable |
Payload — shop.deleted
| Field | Type | Notes |
|---|---|---|
shopId |
uuid | required |
reason |
string |
Never published: phone, email, street address, landmark, coordinates. Many shops are run from someone's home. Region, district and city stay because the feed filters by them.
4.4 nexgate.events.v1 — events
eventId · compacted · 12 partitions
An event exists for you while it is PUBLISHED or HAPPENING, PUBLIC, and not deleted.
| Event | Fires when |
|---|---|
event.created |
It becomes visible |
event.updated |
The event or its tickets change — including selling out, which is an update, not a removal |
event.deleted |
It stops being visible |
Delete reasons: DELETED · ENDED (COMPLETED) · CANCELLED · UNPUBLISHED (draftDRAFT or any other status) · VISIBILITY_RESTRICTED (PRIVATE or UNLISTED)
Ticket statistics count sold tickets only. Reservations change on every checkout attempt and expire by themselves, so publishing them would be noise.
What doespublishes NOT publishnothing: updatedAt, currentStage, completedStages, rsaKeys.
Payload
—event.created/event.updated
| Field | Type | Notes |
|---|---|---|
eventId |
uuid | required |
title |
string, nullable | |
slug |
string, nullable | |
description |
string, nullable | |
category.id |
uuid, nullable | The key to interests |
category.name |
string, nullable | |
bannerMedia |
json, nullable | Opaque media object |
eventFormat |
string, nullable | IN_PERSON · ONLINE · HYBRID · TBA |
visibility |
string, |
PUBLIC |
status |
string, |
PUBLISHED HAPPENING |
schedule.startDateTime |
timestamp, nullable | |
schedule.endDateTime |
timestamp, nullable | |
schedule.timezone |
string, nullable | IANA zone |
venue.name |
string, nullable | |
venue.address |
string, nullable | A public venue, unlike a shop's address |
venue.latitude / .longitude |
decimal, nullable | |
pricing.minPrice / .maxPrice |
decimal, nullable | Across ticket types |
pricing.pricingType |
string, nullable | How the event is priced |
stats.ticketsSold |
int | Sold only — never reservations |
stats.ticketsAvailable |
int, nullable | |
stats.isSoldOut |
bool | Selling out arrives as event.updated |
organizerId |
uuid, nullable | |
organizerName |
string, nullable | |
linkedProductIds[] |
uuid[] | Products tied to the event |
linkedShopIds[] |
uuid[] | Shops tied to the event |
createdAt |
timestamp, nullable | |
publishedAt |
timestamp, nullable | |
updatedAt |
timestamp, nullable | |
interests[] .w |
uuid / decimal | From the event's category, weight 1.0 |
Payload — event.deleted
| Field | Type | Notes |
|---|---|---|
eventId |
uuid | required |
reason |
string |
Never published: the virtual meeting link, meeting id or passcode (they admit people into the room), and never the event's signing keys.
4.5 nexgate.profiles.v1 — public profile facts
accountId · compacted · 24 partitions
A profile exists for you once sign-up is complete (the account has a real username, not a temp_… placeholder) and while the account is not locked.
| Event | Fires when |
|---|---|
profile.created |
Sign-up completes, or a locked account is unlocked |
profile.updated |
One of the watched fields |
profile.deleted |
The account is locked, or the row is gone |
Delete reasons: LOCKED · DELETED
Only these changes republish: userName, accountType, accountTier, locked, isVerified, verificationBadgeType, and rows in the user's declared interests. A login, a password change or a profile-new profile photo change publishes nothing.
Payload
—profile.created/profile.updated
| Field | Type | Notes |
|---|---|---|
accountId |
uuid | required |
userName |
string, nullable | Public username. Never temp_… on a message you receive |
accountType |
string, nullable | NORMAL · SYSTEM · VERIFIED |
accountTier |
string, nullable | FULL · RESTRICTED · MINOR — MINOR matters: treat these accounts conservatively |
isVerified |
bool | |
verificationBadgeType |
string, nullable | Which badge |
followerCount |
int | Snapshot |
followingCount |
int | Snapshot |
declaredInterestIds[] |
uuid[] | What the person chose at onboarding, on the shared interest list |
createdAt |
timestamp, nullable | Account age — useful for cold-start |
Payload — profile.deleted
| Field | Type | Notes |
|---|---|---|
accountId |
uuid | required |
reason |
string | LOCKED or DELETED |
Never published: first, middle or last name, bio, location, phone, email, birth date, profile photo.
⚠️
Do notNever use theactivecolumn if youeverencounterseeitit.elsewhere. Sign-up sets itfalseand nothing ever sets ittrue, so it isfalseon every real account. It is deliberately not in the payloadfor that reason;— relying on it would have tombstoned every profile on the platform.
4.6 nexgate.follows.v1 — who follows whom
Key followerId:followedId · compacted · 24 partitions
Only ACCEPTED follows exist for you. KeyA follow request (PENDING) publishes nothing; if it is followerId:followedId.never accepted you never hear about it.
| Event | Fires when |
|---|---|
follow.created |
A |
follow.deleted |
Unfollowed, or a request |
A
Payload follow— follow.created
| Field | Type | Notes |
|---|---|---|
followerId |
uuid | |
|
uuid | required |
followedAt |
timestamp, nullable | When it |
Payload about— it.follow.deleted
Payload: followerId (uuid, required) · followedId ·(uuid, followedAt
Deleted payload: followerId · required) · followedIdreason (string)
4.7 nexgate.blocks.v1 — who blocked whom
Key is blockerId:blockedId. · compacted · 12 partitions
A block exists for you the moment the row is written — there is no visibility condition.
| Event | Fires when |
|---|---|
block.created |
Someone blocks someone |
block.deleted |
They unblock |
This topic is not optional. The model must never recommend a blocked author's content in either direction — theA block hides content both ways.
Payloadways: neither person may be recommended the other's content.
Payload — block.created
| Field | Type | Notes |
|---|---|---|
blockerId |
uuid | required — who blocked |
blockedId |
uuid | required — who was blocked |
blockedAt |
timestamp, nullable |
Payload — block.deleted
blockerId (uuid, required) · blockedId ·(uuid, blockedAt
Deleted payload: blockerId · required) · blockedIdreason (string)
Mutes are not on Kafka. A mute is a softer, per-viewer preference applied inside the feed; it is not published as an entity. You do see
MUTE_AUTHORas an interaction (§5).
4.8 nexgate.orders.v1 — what was actually bought
orderId · compacted · 24 partitions
An order exists for you from creation. This is the strongest purchase signal on the platform.
| Event | Fires when |
|---|---|
order.created |
An order is placed |
order.updated |
One of the watched fields |
order.deleted |
The order is soft-deleted (reason DELETED) |
Only these changes republish: productOrderStatus, deliveryStatus, totalAmount, currency, shippedAt, deliveredAt, completedAt, cancelledAt, isDeleted.
Payload
—order.created/order.updated
| Field | Type | Notes |
|---|---|---|
orderId |
uuid | required |
orderNumber |
string, nullable | Human-readable reference |
buyer.accountId |
uuid, nullable | Who bought |
shopId |
uuid, nullable | Who sold |
items[].productId |
uuid, nullable | |
items[].quantity |
int | |
items[].unitPrice |
decimal, nullable | |
items[].subtotal |
decimal, nullable | |
productOrderStatus |
string, nullable | PENDING_PAYMENT · PENDING_SHIPMENT · SHIPPED · DELIVERED · AWAITING_BUYER_CONFIRM · COMPLETED · DISPUTED · CANCELLED · REFUNDED |
deliveryStatus |
string, nullable | PENDING · SHIPPED · DELIVERED · CONFIRMED · IN_TRANSIT · NOT_APPLICABLE |
productOrderSource |
string, nullable | DIRECT_PURCHASE · DIGITAL_PURCHASE · INSTALLMENT · GROUP_PURCHASE · CART_PURCHASE · CHAT_OFFER |
totalAmount |
decimal, nullable | |
currency |
string, nullable | |
orderedAt |
timestamp, nullable | |
shippedAt |
timestamp, nullable | |
deliveredAt |
timestamp, nullable | |
completedAt |
timestamp, nullable | The strongest signal: money settled and the buyer confirmed |
cancelledAt |
timestamp, nullable | |
updatedAt |
timestamp, nullable |
An order leaving
PENDING_PAYMENTfor anything other thanCANCELLED/REFUNDEDalso produces aPURCHASEinteraction per order item (§5). The order topic tells you what the order is; the interaction tells you when the buying happened.
Payload — order.deleted
| Field | Type | Notes |
|---|---|---|
orderId |
uuid | required |
reason |
string | DELETED |
Never published: delivery address or instructions, order notes, cancellation text, payment method, escrow or fee figures, tracking number, confirmation code, chat ids.
Event ticket bookings are not on this
topic yet.topic.They arrive as aPURCHASEnexgate.orders.v1interaction (§5) but thereisnoproductbookingordersentityonly.stream. Noted as a gap inSee §7.8.
4.9 nexgate.wishlist.v1 — saved products
Key is accountId:productId —· acompacted · 24 partitions
A product can be in a person's wishlist once, whatever group it is filed under.
| Event | Fires when |
|---|---|
wishlist.created |
A product is added |
wishlist.updated |
The entry changes (for example, moved to another group) |
wishlist.deleted |
It is removed |
Payload — wishlist.created / wishlist.updated
| Field | Type | Notes |
|---|---|---|
accountId |
uuid | |
productId |
uuid | required |
wishlistId |
uuid, nullable | The row's own id |
groupId |
uuid, nullable | Which of the user's lists it sits in |
addedAt |
timestamp, nullable |
Payload — wishlist.deleted
accountId (uuid, required) · productId ·(uuid, required) · groupId (string)addedAtreason
Never published: the group's name. It is free text a person writes about themselves ("Gifts for Mum"). The group id is enough to tell groups apart.
5. nexgate.interactions.v1 — what people did
accountId · delete, 90 days · 96 partitions · schema nexgate.interactions.v1.json
One event type, interaction,. keyedKeyed by accountId so one person's actions stay in order. Retained 90 days.
This is the only topic anything outside the backend can write to, and it does so through exactly one endpoint: POST /api/v1/feed/interactions/batch (09_NEW_ENDPOINTS.md A8). Every other topic is written from committed database rows, so no client can announce that an entity exists or changed.
The envelope is different here
No entityId and no entityVersion — an interaction is not an entity. Instead:
| Field | Type | Notes |
|---|---|---|
eventId |
uuid | required. Deterministic (UUIDv5 of account + clientEventId), so a re-publish of the same action produces the same id. Deduplicate on it |
eventType |
string | Always the literal interaction |
occurredAt |
timestamp | The device's clock, clamped so it can never be in the future |
receivedAt |
timestamp | The server's clock, when it arrived |
schemaVersion |
int | 1 |
source |
string | nexgate-backend |
payload |
object | Below |
{
"eventId": "…", "eventType": "interaction",
"occurredAt": "2026-09-18T10:22:41.100Z", "receivedAt": "2026-09-18T10:22:41.480Z",
"schemaVersion": 1, "source": "nexgate-backend",
"payload": {
"accountId": "…", "sessionId": "s-8812e0c4", "clientEventId": "c-000185",
"action": "VIEW", "targetType": "POST", "targetId": "…",
"context": { "surface": "REELS", "position": 3, "feedSessionId": "fs-77aa", "source": "RECO", "viaPostId": null, "query": null },
"media": { "dwellMs": null, "watchMs": 9000, "mediaDurationMs": 6000, "loopCount": 1, "soundOn": true, "mediaIndex": 0 },
"client": { "platform": "ANDROID", "appVersion": "2.4.1", "networkType": "CELLULAR" },
"origin": "CLIENT"
}
}
Payload — every field
| Field | Type | Notes |
|---|---|---|
|
uuid | required. Who did it |
sessionId |
string, nullable | The app session, max 64 chars. null |
clientEventId |
string | required. The app's own id, or srv:<action>:<rowId> when the |
action |
string | required. See the table below |
targetType |
string, nullable | POST · PRODUCT · SHOP · EVENT · PROFILE. null only for SEARCH |
targetId |
uuid, nullable | Lower-cased. null only for SEARCH |
context.surface |
string, nullable | FEED · REELS · SEARCH · PRODUCT_PAGE · SHOP_PAGE · EVENT_PAGE · PROFILE_PAGE · RECOMMENDATION · NOTIFICATION · CHAT |
context.position |
int, nullable | Where in the |
context.feedSessionId |
string, nullable | Joins to nexgate.feed-served.v1 — max 64 chars |
context.source |
string, nullable | Why it was shown: FOLLOWING · CELEBRITY · RECO · INTEREST · FALLBACK · SPONSORED · TRENDING · SEARCH. RECO means your list produced it |
context.viaPostId |
uuid, nullable | The post a product was reached through |
context.query |
string, nullable | The search text, for SEARCH |
media.dwellMs |
int, nullable | Time on screen. 0 – 6 h |
media.watchMs |
int, nullable | Time watched. 0 – 6 h |
media.mediaDurationMs |
int, nullable | Length of the media. 0 – 12 h |
media.loopCount |
int, nullable | Replays. 0 – 1000 |
media.soundOn |
bool, nullable | Whether sound was on |
media.mediaIndex |
int, nullable | Which item of a carousel. 0 – 100 |
client.platform |
string, nullable | ANDROID · IOS · WEB |
client.appVersion |
string, nullable | e.g. 2.4.1 |
client.networkType |
string, nullable | WIFI · CELLULAR · OFFLINE · UNKNOWN |
origin |
enum, nullable | CLIENT or SERVER |
media and client are always null on server-produced events — the backend did not witness a screen.
5.1 Two origins, never overlapping
origin |
clientEventId |
|
|---|---|---|
CLIENT |
The app, in a batch |
the app's own id |
SERVER |
The backend, watching committed rows | srv:<action>:<rowId> |
The split is enforced: the batch API rejects every server-owned action with SENT_BY_SERVER, so nothing is ever counted twice.
5.2 Every action, and who sends it, and what triggers it
| Action | Origin | Exact trigger | |
|---|---|---|---|
IMPRESSION |
CLIENT | any | The item was on screen. Use it as the denominator |
VIEW |
CLIENT | any | Real |
SKIP |
CLIENT | any | Scrolled past quickly |
CLICK |
CLIENT | any | Opened the item |
SEARCH |
CLIENT | A context.query |
|
NOT_INTERESTED |
CLIENT | any | Explicit negative |
HIDE |
CLIENT | any | Explicit negative, stronger |
LIKE |
SERVER | POST | A post_likes row |
UNLIKE |
SERVER | POST | A post_likes row deleted |
COMMENT |
SERVER | POST | A post_comments row |
BOOKMARK |
SERVER | POST | A post_bookmarks row |
SHARE |
SERVER | POST | A post_shares row |
REPORT |
SERVER | POST | A post_reports row |
BLOCK |
SERVER | PROFILE | A user_blocks row |
MUTE_AUTHOR |
SERVER | PROFILE | A user_mutes row |
ADD_TO_CART |
SERVER | PRODUCT | A cart_items row |
REMOVE_FROM_CART |
SERVER | PRODUCT | A cart_items row deleted |
PURCHASE |
SERVER | PRODUCT | An order PENDING_PAYMENT for any status other than CANCELLED/REFUNDED — one event per order item |
PURCHASE |
SERVER | EVENT | An event booking became CONFIRMED |
Server events are sent Target types: POST, PRODUCT, SHOP, EVENT, PROFILE.
Surfaces: FEED, REELS, SEARCH, PRODUCT_PAGE, SHOP_PAGE, EVENT_PAGE, PROFILE_PAGE, RECOMMENDATION, NOTIFICATION, CHAT.
Sources (whyafter the itemtransaction wascommits. shown):A FOLLOWING,rolled-back CELEBRITY,like RECO,sends INTEREST,nothing, FALLBACK,and SPONSORED,a TRENDING,like SEARCH.made through any code path is captured — nothing depends on a service remembering to emit.
5.3 Media measurements are raw, on purpose
watchMs: 9000 against mediaDurationMs: 6000 with loopCount: 1 is a rewatch, not bad data. Nothing is clamped to "100%100 %" before you see it, because the fact that someone watched a clip one and a half times is exactly the signal worth having.
5.4 WhatThis youtopic willis not find here
If the event log is briefly unavailable, interactions are dropped and One event type, No Join it to the interactions topic on ⚠️ Everything above travels backend → model. This section is the other direction: model → backend. There is no API call and no callback. The recommendation service writes plain Redis keys; the feed reads them when it builds a session. If the model stops writing, the feed keeps working on trending and interest pools — quieter, never broken. The Every key is optional. A surface whose key is absent runs on trending and interest pools instead. You can ship one surface at a time. One JSON string, written with a single After each successful batch, write the health key: Read, then judged in one of four states: Every read is counted as How your items are ranked against ours. A reco item's score becomes Where they land on the home feed. Your items fill the discovery slots, interleaved with posts from people the user follows in the pattern We have 200 ms. Each reco read has a 200 ms timeout. Past that the source returns nothing for that session and the pools fill in. A slow Redis degrades the feed's quality, never its availability. You get a Redis user that can write only Anything outside Local docker-compose trap. The containers announce internal addresses ( The per-account debug trace is the fastest way to prove the loop end to end: it shows every item the last session produced, its source, its base score and its final score — and every item a filter removed, with the filter's name. Stated plainly so nobody builds on an assumption: Backend to backend: when a post is published, the work of writing it into followers' timelines is chunked and sent over this topic. Key is Ignore it. It carries no signal a model can use, and its shape may change without notice.Interactions are best-effort by design.
counted,counted, not queued — they are signal, not records. A missing view is a missing sample; a queued backlog of stale views would be worse. EntityThe entity topics are the opposite: those are never lost.lost, and can always be rebuilt with a backfill.
6.
nexgate.feed-served.v1 — what the feed actually showedaccountId · delete, 14 days · 48 partitions · no registered schemafeed.served, keyed by accountId. Retained 14 days. This is the ground truth for ranking evaluation: what was shown, in what order, and why.The envelope
entityId, no entityVersion, no receivedAt: eventId, eventType (feed.served), occurredAt, schemaVersion (1), source, payload.{
"eventId": "…",
"eventType": "feed.served",
"occurredAt": "2026-09-18T10:22:31.900Z",
"schemaVersion": 1,
"source": "nexgate-backend",
"payload": {
"feedSessionId": "fs-77aa31c8",
"accountId": "…",
"surface": "FEED",
"items": [
{ "type": "POST", "id": "4f2a77e1-…", "position": 0, "source": "FOLLOWING" },
{ "type": "PRODUCT", "id": "8c31a0d2-…", "position": 1, "source": "RECO" }
]
}
}
Payload — every field
Field
Type
Notes
feedSessionIdstring
Joins to
context.feedSessionId on interactions
accountIduuid
Who it was served to
surfacestring
Currently always the literal
"FEED" — see the warning below
items[].typestring
POST · PRODUCT
items[].iduuid
The item
items[].positionint
Its place on the page, from 0
items[].sourcestring
Why it was there:
FOLLOWING · CELEBRITY · RECO · INTEREST · FALLBACK · SPONSORED · TRENDINGfeedSessionId and you have, per session: what we showed, in which position, from which source — and what the person did about it.
Today this is publishedPublished by the home feed only, and surface is alwayshard-coded the literalto "FEED". Reels, marketplace and events do not publish it yet.it. See §7.8.7. The way back — what the model writes to Redis
7.1 The eight keys
Key
Read by
Items must be
When it is read
reco:home:posts:{accountId}Home feed discovery
post ids
every home session build
reco:home:products:{accountId}Home feed discovery
product ids
every home session build
reco:reels:{accountId}Reels feed
post ids (with a short video)
every reels session build
reco:market:products:{accountId}Marketplace "For You"
product ids
every marketplace session build
reco:events:{accountId}Events "For You"
event ids
every events session build
reco:shops:{accountId}"Shops you might like"
shop ids
every call to the strip
reco:similar:products:{productId}"Similar items" on a product page
product ids
every call to the strip
reco:meta:healthour alerting only
—
on every metrics scrape
{…} braces are part of the key — they are Redis Cluster hash tags, and they decide which node the key lives on. Write them exactly as shown.7.2 The value
SET key <value> EX 172800 (48 h). One command means a reader sees either the whole old list or the whole new one, never half of each.{
"v": 1,
"modelVersion": "ranker-2026-09-14",
"generatedAt": "2026-09-18T02:10:00Z",
"items": [
{ "id": "4f2a77e1-0c8b-4d2f-9a11-7b6c3e5d8f90", "score": 0.83 },
{ "id": "9b10c2d4-7e3f-4a52-b8c1-6d9e0f2a3b47", "score": 0.79 }
]
}
Field
Required
Rule
generatedAtYes
ISO 8601. Without it the whole list is INVALID and ignored
itemsYes
Must be an array. Best first. At most 300 are read; the rest are discarded
items[].idYes
A UUID of the right type for that key. Not a UUID → that item is skipped. Unknown or deleted → dropped silently at read time
items[].scoreNo
Kept when inside
[0, 1]. Missing or out of range → we use rank instead: 1 / (1 + rank), so position 0 → 1.0, 1 → 0.5, 3 → 0.25
modelVersionNo
Carried into our debug trace, so a feed can be explained by the model that produced it
vNo
Format version
SET reco:meta:health '{"finishedAt":"2026-09-18T02:40:00Z","modelVersion":"ranker-2026-09-14"}'
7.3 What the backend does with each list
State
Condition
Used?
FRESH
generatedAt within reco.maxAgeHours (default 30 h)Yes
STALE
Older than that, key still present
Yes — a stale list beats no list. Counted on our dashboards
MISSING
No key
No → fallback pools fill discovery
INVALID
Not JSON, no
items array, or no generatedAtNo → same as missing, but counted separately, so a broken writer is visible rather than silent
feed.reco.lists{list, state}, so we can see per surface how often your lists are fresh, stale, missing or invalid.1 + yourScore. Fallback items score inside [0, 1]. So every recommended item outranks every fallback item in the discovery lane, and the pools only fill what you did not supply. If the same post appears in both, the RECO copy wins.F, F, D, F, D, F, F, D — never the same author twice within 4 slots. A session holds 120 items and is rebuilt on pull-to-refresh or when the user reaches the end, so a newly written list appears at the next session, not mid-scroll.7.4 Access
reco:*:ACL SETUSER reco-writer on >******** ~reco:* -@all +set +get +del +expire +exists +ping +cluster|slots +cluster|shards +cluster|nodes
reco:* answers NOPERM. Locally: user reco-writer, password reco_writer_local, seeds localhost:7001,localhost:7002.
172.29.0.11–16). A cluster client on your laptop follows those announcements and cannot reach them. Run your writer in a container on the nexgate-feed-redis network, or on the host network.7.5 How we both know it is working
Signal
Where
Means
feed.reco.lists{state="FRESH"} risingour dashboard
your lists are being read and used
feed.reco.lists{state="INVALID"} above zeroour dashboard, alert after 15 min
your writer is producing something unparseable
feed.reco.health.age.secondsour dashboard, alert above 26 h
reco:meta:health.finishedAt is old — your batch has stopped. -1 means no batch has ever been reported
source: "RECO" in a feed responsethe app, and
GET /api/admin/feed/debug/{accountId}an item on a real screen came from your list
7.8. Known gaps
Gap
Consequence
feed-served is published by the home feed only, with surface hard-coded to "FEED"No served-ground-truth for reels, marketplace or events. Evaluation on those surfaces has to rely on impressions
Event ticket bookings have no entity topic
A booking arrives as a
PURCHASE interaction with targetType: EVENT, but there is no order-like stream for bookings. nexgate.orders.v1 is product orders only
No JSON Schema is registered for
nexgate.feed-served.v1 or nexgate.fanout-tasks.v1The nine entity topics and interactions have schemas under
src/main/resources/feed-schemas/; these two do not
Interest weights are empty until an admin maps categories
interests: [{id, w}] on posts, products and events is [] for any unmapped category or hashtag. 128 product categories and 12 event categories are still unmapped
Follows and shop subscriptions are not yet interest signals, and
SEARCH is not matched to an interestThe backend's own interest profile is weaker than the design describes. Does not affect what you receive
8.9. nexgate.fanout-tasks.v1 — internalpostId:chunk, retained 3 days, no schema registered.
Doc
What it covers
06_RECOMMENDATION_DEVELOPER_GUIDE.mdHow to connect, consume, and write your lists back into Redis
02_KAFKA_CONTRACT.mdThe original agreement: why these topics, deletes, ordering, versioning, personal data
01_ARCHITECTURE.mdThe design. §4 is the write path and the outbox; §4.6 is what is deliberately left out of payloads
05_INTERACTIONS_CLIENT_GUIDE.mdThe app's side of the interactions topic: exact definitions of every action