Skip to main content

Event Catalogue — every message NexGate publishes to Kafka

Event Catalogue — the full round trip, both directions

Author: Josh S. Sakweli, Backend Lead Team
Last Updated: 2026-09-18
Version: v1.0 (backend branch fet/feed_baking, schema version 1)

Cluster: Redpanda (Kafka API) · Schema registry: JSON Schema, subject <topic>-value, BACKWARD compatibility

Short Description: Everything that crosses the line between the backend and the recommendation service, in both directions. Inbound (§1–§6): the complete list of what the backend puts on Kafka — 12 topics, 27 event types — when each fires, what does not cause it, and every payload field with its type, nullability and allowed values, including the exact shape of each *.deleted. Outbound (§7): the eight Redis keys the model writes back, their exact format, and what the backend does with each one. §8 draws the line between the two jobs — everything the pipeline already does, so the model does not rebuild it. It is written to be self-contained: a developer with this document and a Kafka client needs nothing else to start.

Read 06_RECOMMENDATION_DEVELOPER_GUIDE.md first. That guide is how to connect, what to do with the messages and how to write recommendations back. This document is the catalogue — everything that arrives, in detail. 06 links here rather than repeating it.

Hints:

  • Nothing is emitted from service code. Events come from committed database rows, so what you receive always matches what is in the database — a rolled-back transaction publishes nothing.
  • A payload is always the full current state, never a diff. Replace what you stored.
  • Order per key is guaranteed; order across keys is not. Use entityVersion, never occurredAt.
  • Fields are only ever added. Ignore what you don't recognise.
  • If a topic is ever lost, everything can be republished from the database: POST /api/admin/feed/backfills.

0. The round trip in one picture

  INBOUND — backend ➜ model                                              §1–§6
  ──────────────────────────────────────────────────────────────────────────────
  a row commits
  (post · product · shop · event · profile · follow · block · order · wishlist)
        └─► outbox ─► relay ─► 9 entity topics ......... compacted, "what exists"

  the app calls POST /api/v1/feed/interactions/batch        (impressions, views…)
  a row commits (like · comment · cart · order paid)        (seen by the backend)
        └─► nexgate.interactions.v1 ................. 90 days, "what people did"

  a feed page is served
        └─► nexgate.feed-served.v1 .................. 14 days, "what we showed"

                                    │
                                    ▼
                    ┌───────────────────────────────┐
                    │    RECOMMENDATION SERVICE     │
                    │   yours — we never see inside │
                    └───────────────┬───────────────┘
                                    │
  OUTBOUND — model ➜ backend        │                                       §7
  ──────────────────────────────────┼───────────────────────────────────────────
                                    ▼
        SET reco:<surface>:{id} <json> EX 172800        (8 keys, all optional)
                                    │
                                    ▼
        the feed pipeline reads reco:* while building a session;
        missing · invalid · stale ─► trending and interest pools fill in

The boundary never moves. The backend only ever writes Kafka and reads reco:*. The model only ever reads Kafka and writes reco:*. Neither side calls the other over HTTP, so neither can be made slow or unavailable by the other.


1. The envelope

Every message on the nine entity topics has the same eight fields:

{
  "eventId": "9f2c1e8a-5b3d-4a91-8c77-2e6f0b4d1a35",
  "eventType": "product.updated",
  "entityId": "4f2a77e1-0c8b-4d2f-9a11-7b6c3e5d8f90",
  "entityVersion": 184022,
  "occurredAt": "2026-09-18T10:22:33.120Z",
  "schemaVersion": 1,
  "source": "nexgate-backend",
  "payload": { }
}
Field Meaning
eventId Unique per event. Seen twice = a retry; drop the second
eventType One of the 27 in this document
entityId The thing this is about
entityVersion Only ever increases per entity. Keep the highest you have seen and ignore anything lower
occurredAt When it happened. Not an ordering key — clocks and retries make it unreliable for that
schemaVersion 1 today
source Always nexgate-backend
payload Full current state, or, for a *.deleted, the id plus a reason

The interactions and feed-served topics use a slightly different envelope — noted in their sections.

2. How a delete arrives

Always two messages, in this order:

  1. <entity>.deleted with a reason — the entity is gone for you, and the reason says why.
  2. A tombstone: same key, value = null — so log compaction can drop the key.

"Deleted" does not always mean the row was deleted. It means this entity is no longer something to recommend. A post set to private, a shop that closed, an account that got locked — all arrive as *.deleted with a reason that tells them apart. If it becomes visible again later, you get *.created again.

Tombstones are kept for 1 day. A consumer reading a compacted topic from offset 0 after that sees only live entities.


3. The topics at a glance

# Topic Key Cleanup Partitions Event types
1 nexgate.posts.v1 postId compact 24 post.created · post.updated · post.deleted
2 nexgate.products.v1 productId compact 24 product.created · product.updated · product.deleted
3 nexgate.shops.v1 shopId compact 12 shop.created · shop.updated · shop.deleted
4 nexgate.events.v1 eventId compact 12 event.created · event.updated · event.deleted
5 nexgate.profiles.v1 accountId compact 24 profile.created · profile.updated · profile.deleted
6 nexgate.follows.v1 followerId:followedId compact 24 follow.created · follow.deleted
7 nexgate.blocks.v1 blockerId:blockedId compact 12 block.created · block.deleted
8 nexgate.orders.v1 orderId compact 24 order.created · order.updated · order.deleted
9 nexgate.wishlist.v1 accountId:productId compact 24 wishlist.created · wishlist.updated · wishlist.deleted
10 nexgate.interactions.v1 accountId delete, 90 d 96 interaction
11 nexgate.feed-served.v1 accountId delete, 14 d 48 feed.served
12 nexgate.fanout-tasks.v1 postId:chunk delete, 3 d 48 (internal — not for consumers)

Partition counts are fixed for good. Adding partitions later would move keys to different partitions and break per-key ordering.

Compacted topics are a live catalogue: read from offset 0 and you have the current state of every entity, then keep consuming to stay current.


4. Entity topics

Each section below is complete: every field of the payload, its type, whether it can be null, its allowed values where it is an enum, and the exact shape of the *.deleted payload.

Reading the type column: uuid, string, int, decimal, bool, timestamp (ISO 8601 UTC), json (an opaque object — structure may change, do not depend on it). A field marked nullable can arrive as null; one marked required is on every message of that topic.

Fields are only ever added. The schemas are open by design: ignore any field you do not recognise, and never fail a message because of one.


4.1 nexgate.posts.v1 — posts and reels

postId · compacted · 24 partitions · schema src/main/resources/feed-schemas/nexgate.posts.v1.json

A post exists for you while it is PUBLISHED, not deleted, and PUBLIC or FOLLOWERS.

Event Fires when
post.created A draft becomes published, or a hidden post becomes visible again
post.updated A visible post, or any of its child rows, changes
post.deleted It stops being visible — only if it was visible before

Delete reasons: DELETED (soft-deleted) · UNPUBLISHED (no longer PUBLISHED) · VISIBILITY_RESTRICTED (switched to MENTIONED or PRIVATE)

What counts as a change. Not just the post row — eleven child tables are watched, and a change to any one republishes the whole post: media, hashtags, attached products, attached shops, attached events, user mentions, shop mentions, links, collaborators, and the poll.

What publishes nothing: likesCount, commentsCount, repostsCount, bookmarksCount, viewsCount, quotesCount, sharesCount, shortClipCount, lastVerifiedAt, updatedAt. Engagement counters move constantly; republishing every post on every like would be most of the topic's traffic for none of the value. The counts in the payload are therefore a snapshot from the last real change, not live. For live behaviour, use nexgate.interactions.v1.

A draft that is saved, edited and never published produces nothing at all.

Payload — post.created / post.updated

Field Type Notes
postId uuid required
postType string, nullable REGULAR · POLL
status string, nullable PUBLISHED on every message you receive (DRAFT, SCHEDULED, DELETED never reach you)
author.id uuid, nullable Who wrote it
author.userName string, nullable Public username. No real name is ever published
content string, nullable The post text as written
contentParsed.hashtags[] string[] Normalised, without the #. Any script (Latin, Arabic, CJK …)
contentParsed.mentionedUserIds[] uuid[] Accounts mentioned in the text
contentParsed.mentionedShopIds[] uuid[] Shops mentioned in the text
media[].fileId uuid, nullable File-service id
media[].mediaType string, nullable IMAGE · VIDEO
media[].order int Display order within the post, from 0
media[].status string, nullable Processing state. PENDING means variants are not ready yet
media[].variants json, nullable Rendition URLs keyed by name (hls, thumb, …). Opaque — read keys, do not assume the set
media[].shortClip bool true = this is a reel. There is no separate reel topic
media[].mediaId uuid, nullable The clip's own id — the one the app uses on reel endpoints
media[].durationMs int, nullable Length in milliseconds, for video
attachments.products[] uuid[] Products attached to the post
attachments.shops[] uuid[] Shops attached
attachments.events[] uuid[] Events attached
engagement.likesCount int Snapshot, see above
engagement.commentsCount int Snapshot
engagement.sharesCount int Snapshot
engagement.viewsCount int Snapshot
engagement.bookmarksCount int Snapshot
engagement.repostsCount int Snapshot
engagement.quotesCount int Snapshot
privacySettings.visibility string, nullable PUBLIC or FOLLOWERS on every message you receive
privacySettings.whoCanComment string, nullable Who may comment
privacySettings.whoCanRepost string, nullable Who may repost
quotedPostId uuid, nullable Set when this post quotes or reposts another. A pure repost carries the original's engagement, not its own
collaboratorIds[] uuid[] Co-authors
hasPoll bool Whether a poll is attached. Poll options and votes are not published
externalLinkDomain string, nullable The domain only, never the full URL
isEdited bool Whether it has been edited since publishing
createdAt timestamp, nullable When the row was created (may be long before publishing)
publishedAt timestamp, nullable When it went live — use this for recency, not createdAt
updatedAt timestamp, nullable Last change
interests[].id uuid Interest id from the shared list (interest_categories)
interests[].w decimal Weight: 1.0 from attached products and events, 0.6 from hashtags. An interest reached twice keeps the higher weight. Empty until an admin maps the category or hashtag

Payload — post.deleted

Field Type Notes
postId uuid required
authorId uuid, nullable The author, or null when the row itself is gone
reason string One of the delete reasons above

Followed by a tombstone: same key, value = null.


4.2 nexgate.products.v1 — products

productId · compacted · 24 partitions

A product exists for you while it is ACTIVE or OUT_OF_STOCK, not deleted, and its shop is visible. Running out of stock is an update, not a disappearance — stock is in the payload, so you decide what to do with it.

Event Fires when
product.created It becomes visible, including when its shop comes back
product.updated The product or its installment plan changes
product.deleted It stops being visible

Delete reasons: DELETED · UNPUBLISHED (DRAFT, INACTIVE, ARCHIVED) · SHOP_UNAVAILABLE (shop suspended, closed, unapproved or deleted)

The shop cascade. When a shop's visibility flips, every product of that shop is republished, in pages, at that moment. One suspension can produce thousands of product.deleted messages. Deliberate: a consumer watching only this topic would otherwise keep recommending products from a suspended shop.

What publishes nothing: viewCount, cartAddCount, updatedAt.

Payload — product.created / product.updated

Field Type Notes
productId uuid required
productName string, nullable
productSlug string, nullable URL slug
productDescription string, nullable
productType string, nullable PHYSICAL · DIGITAL
productMedia[].fileId uuid, nullable
productMedia[].mediaType string, nullable IMAGE · VIDEO
productMedia[].order int, nullable Display order
productMedia[].status string, nullable Processing state
productMedia[].variants json, nullable Rendition URLs
categoryId uuid, nullable The key to interests — mapped by an admin
categoryName string, nullable
parentCategoryId uuid, nullable A category with no mapping inherits its parent's interests
price decimal, nullable Current price
comparePrice decimal, nullable "Was" price
isOnSale bool
discountPercentage decimal, nullable
stockQuantity int
isInStock bool
isLowStock bool
showStockToPublic bool Whether the shop displays the number
status string, nullable ACTIVE or OUT_OF_STOCK on every message you receive
condition string, nullable NEW · USED_LIKE_NEW · USED_GOOD · USED_FAIR · REFURBISHED · FOR_PARTS
urgencyTag string, nullable NONE · NEW_ARRIVAL · LIMITED_EDITION · LIMITED_OFFER · FEW_REMAINS
specifications json, nullable Free-form, seller-entered. Opaque
colors json, nullable Free-form, seller-entered. Opaque
hasGroupBuying bool
groupPrice decimal, nullable Price when bought as a group
hasInstallments bool
stockInfo.soldCount int Units sold
shopId uuid, nullable
shopName string, nullable
createdAt timestamp, nullable
updatedAt timestamp, nullable
interests[].id uuid From the product's category, weight 1.0
interests[].w decimal

Payload — product.deleted

Field Type Notes
productId uuid required
shopId uuid, nullable Useful when the cause was the shop
reason string

4.3 nexgate.shops.v1 — shops

shopId · compacted · 12 partitions

A shop exists for you while it is ACTIVE or TEMPORARILY_OFFLINE, approved, and not deleted.

Event Fires when
shop.created It becomes visible
shop.updated Anything meaningful on the shop changes
shop.deleted It stops being visible

Delete reasons: DELETED · SUSPENDED · CLOSED (PERMANENTLY_CLOSED) · UNPUBLISHED (PENDING, or approval withdrawn)

Every one of these also republishes the shop's products (§4.2).

What publishes nothing: subscriberCount, lastSeenTime, updatedAt.

Payload — shop.created / shop.updated

Field Type Notes
shopId uuid required
shopName string, nullable
shopSlug string, nullable
shopDescription string, nullable
logo.fileId / .mediaType / .status / .variants uuid / string / string / json, nullable The shop logo
banner.fileId / .mediaType / .status / .variants uuid / string / string / json, nullable The shop banner
ownerId uuid, nullable The account that owns the shop
status string, nullable ACTIVE or TEMPORARILY_OFFLINE on every message you receive
tempOfflineUntil timestamp, nullable Set while the owner has gone offline deliberately
city string, nullable
district string, nullable
region string, nullable The feed filters by region — this is why location stays
countryCode string, nullable
isVerified bool
verificationBadge string, nullable Which badge
trustScore decimal, nullable Platform trust score
subscriberCount int Snapshot — a change to it alone does not republish
approvedAt timestamp, nullable
createdAt timestamp, nullable
updatedAt timestamp, nullable

Payload — shop.deleted

Field Type Notes
shopId uuid required
reason string

Never published: phone, email, street address, landmark, coordinates. Many shops are run from someone's home.


4.4 nexgate.events.v1 — events

eventId · compacted · 12 partitions

An event exists for you while it is PUBLISHED or HAPPENING, PUBLIC, and not deleted.

Event Fires when
event.created It becomes visible
event.updated The event or its tickets change — including selling out, which is an update, not a removal
event.deleted It stops being visible

Delete reasons: DELETED · ENDED (COMPLETED) · CANCELLED · UNPUBLISHED (DRAFT or any other status) · VISIBILITY_RESTRICTED (PRIVATE or UNLISTED)

Ticket statistics count sold tickets only. Reservations change on every checkout attempt and expire by themselves, so publishing them would be noise.

What publishes nothing: updatedAt, currentStage, completedStages, rsaKeys.

Payload — event.created / event.updated

Field Type Notes
eventId uuid required
title string, nullable
slug string, nullable
description string, nullable
category.id uuid, nullable The key to interests
category.name string, nullable
bannerMedia json, nullable Opaque media object
eventFormat string, nullable IN_PERSON · ONLINE · HYBRID · TBA
visibility string, nullable PUBLIC on every message you receive
status string, nullable PUBLISHED or HAPPENING on every message you receive
schedule.startDateTime timestamp, nullable
schedule.endDateTime timestamp, nullable
schedule.timezone string, nullable IANA zone
venue.name string, nullable
venue.address string, nullable A public venue, unlike a shop's address
venue.latitude / .longitude decimal, nullable
pricing.minPrice / .maxPrice decimal, nullable Across ticket types
pricing.pricingType string, nullable How the event is priced
stats.ticketsSold int Sold only — never reservations
stats.ticketsAvailable int, nullable
stats.isSoldOut bool Selling out arrives as event.updated
organizerId uuid, nullable
organizerName string, nullable
linkedProductIds[] uuid[] Products tied to the event
linkedShopIds[] uuid[] Shops tied to the event
createdAt timestamp, nullable
publishedAt timestamp, nullable
updatedAt timestamp, nullable
interests[].id / .w uuid / decimal From the event's category, weight 1.0

Payload — event.deleted

Field Type Notes
eventId uuid required
reason string

Never published: the virtual meeting link, meeting id or passcode (they admit people into the room), and never the event's signing keys.


4.5 nexgate.profiles.v1 — public profile facts

accountId · compacted · 24 partitions

A profile exists for you once sign-up is complete (a real username, not a temp_… placeholder) and while the account is not locked.

Event Fires when
profile.created Sign-up completes, or a locked account is unlocked
profile.updated One of the watched fields changes, or the person's declared interests change
profile.deleted The account is locked, or the row is gone

Delete reasons: LOCKED · DELETED

Only these changes republish: userName, accountType, accountTier, locked, isVerified, verificationBadgeType, and rows in the user's declared interests. A login, a password change or a new profile photo publishes nothing.

Payload — profile.created / profile.updated

Field Type Notes
accountId uuid required
userName string, nullable Public username. Never temp_… on a message you receive
accountType string, nullable NORMAL · SYSTEM · VERIFIED
accountTier string, nullable FULL · RESTRICTED · MINORMINOR matters: treat these accounts conservatively
isVerified bool
verificationBadgeType string, nullable Which badge
followerCount int Snapshot
followingCount int Snapshot
declaredInterestIds[] uuid[] What the person chose at onboarding, on the shared interest list
createdAt timestamp, nullable Account age — useful for cold-start

Payload — profile.deleted

Field Type Notes
accountId uuid required
reason string LOCKED or DELETED

Never published: first, middle or last name, bio, location, phone, email, birth date, profile photo.

⚠️ Never use the active column if you encounter it elsewhere. Sign-up sets it false and nothing ever sets it true, so it is false on every real account. It is deliberately not in the payload — relying on it would have tombstoned every profile on the platform.


4.6 nexgate.follows.v1 — who follows whom

Key followerId:followedId · compacted · 24 partitions

Only ACCEPTED follows exist for you. A follow request (PENDING) publishes nothing; if it is never accepted you never hear about it.

Event Fires when
follow.created A request is accepted, or a public account is followed
follow.deleted Unfollowed, or a request withdrawn — only if it had been accepted

Payload — follow.created

Field Type Notes
followerId uuid required — the one doing the following
followedId uuid required — the one being followed
followedAt timestamp, nullable When it was accepted

Payload — follow.deleted

followerId (uuid, required) · followedId (uuid, required) · reason (string)


4.7 nexgate.blocks.v1 — who blocked whom

Key blockerId:blockedId · compacted · 12 partitions

A block exists for you the moment the row is written — there is no visibility condition.

Event Fires when
block.created Someone blocks someone
block.deleted They unblock

This topic is not optional. A block hides content both ways: neither person may be recommended the other's content.

Payload — block.created

Field Type Notes
blockerId uuid required — who blocked
blockedId uuid required — who was blocked
blockedAt timestamp, nullable

Payload — block.deleted

blockerId (uuid, required) · blockedId (uuid, required) · reason (string)

Mutes are not on Kafka. A mute is a softer, per-viewer preference applied inside the feed; it is not published as an entity. You do see MUTE_AUTHOR as an interaction (§5).


4.8 nexgate.orders.v1 — what was actually bought

orderId · compacted · 24 partitions

An order exists for you from creation. This is the strongest purchase signal on the platform.

Event Fires when
order.created An order is placed
order.updated One of the watched fields changes
order.deleted The order is soft-deleted (reason DELETED)

Only these changes republish: productOrderStatus, deliveryStatus, totalAmount, currency, shippedAt, deliveredAt, completedAt, cancelledAt, isDeleted.

Payload — order.created / order.updated

Field Type Notes
orderId uuid required
orderNumber string, nullable Human-readable reference
buyer.accountId uuid, nullable Who bought
shopId uuid, nullable Who sold
items[].productId uuid, nullable
items[].quantity int
items[].unitPrice decimal, nullable
items[].subtotal decimal, nullable
productOrderStatus string, nullable PENDING_PAYMENT · PENDING_SHIPMENT · SHIPPED · DELIVERED · AWAITING_BUYER_CONFIRM · COMPLETED · DISPUTED · CANCELLED · REFUNDED
deliveryStatus string, nullable PENDING · SHIPPED · DELIVERED · CONFIRMED · IN_TRANSIT · NOT_APPLICABLE
productOrderSource string, nullable DIRECT_PURCHASE · DIGITAL_PURCHASE · INSTALLMENT · GROUP_PURCHASE · CART_PURCHASE · CHAT_OFFER
totalAmount decimal, nullable
currency string, nullable
orderedAt timestamp, nullable
shippedAt timestamp, nullable
deliveredAt timestamp, nullable
completedAt timestamp, nullable The strongest signal: money settled and the buyer confirmed
cancelledAt timestamp, nullable
updatedAt timestamp, nullable

An order leaving PENDING_PAYMENT for anything other than CANCELLED/REFUNDED also produces a PURCHASE interaction per order item (§5). The order topic tells you what the order is; the interaction tells you when the buying happened.

Payload — order.deleted

Field Type Notes
orderId uuid required
reason string DELETED

Never published: delivery address or instructions, order notes, cancellation text, payment method, escrow or fee figures, tracking number, confirmation code, chat ids.

Event ticket bookings are not on this topic. nexgate.orders.v1 is product orders only. See §8.9.


4.9 nexgate.wishlist.v1 — saved products

Key accountId:productId · compacted · 24 partitions

A product can be in a person's wishlist once, whatever group it is filed under.

Event Fires when
wishlist.created A product is added
wishlist.updated The entry changes (for example, moved to another group)
wishlist.deleted It is removed

Payload — wishlist.created / wishlist.updated

Field Type Notes
accountId uuid required
productId uuid required
wishlistId uuid, nullable The row's own id
groupId uuid, nullable Which of the user's lists it sits in
addedAt timestamp, nullable

Payload — wishlist.deleted

accountId (uuid, required) · productId (uuid, required) · reason (string)

Never published: the group's name. It is free text a person writes about themselves ("Gifts for Mum"). The group id is enough to tell groups apart.


5. nexgate.interactions.v1 — what people did

accountId · delete, 90 days · 96 partitions · schema nexgate.interactions.v1.json

One event type, interaction. Keyed by accountId so one person's actions stay in order.

This is the only topic anything outside the backend can write to, and it does so through exactly one endpoint: POST /api/v1/feed/interactions/batch (09_NEW_ENDPOINTS.md A8). Every other topic is written from committed database rows, so no client can announce that an entity exists or changed.

The envelope is different here

No entityId and no entityVersion — an interaction is not an entity. Instead:

Field Type Notes
eventId uuid required. Deterministic (UUIDv5 of account + clientEventId), so a re-publish of the same action produces the same id. Deduplicate on it
eventType string Always the literal interaction
occurredAt timestamp The device's clock, clamped so it can never be in the future
receivedAt timestamp The server's clock, when it arrived
schemaVersion int 1
source string nexgate-backend
payload object Below
{
  "eventId": "…", "eventType": "interaction",
  "occurredAt": "2026-09-18T10:22:41.100Z", "receivedAt": "2026-09-18T10:22:41.480Z",
  "schemaVersion": 1, "source": "nexgate-backend",
  "payload": {
    "accountId": "…", "sessionId": "s-8812e0c4", "clientEventId": "c-000185",
    "action": "VIEW", "targetType": "POST", "targetId": "…",
    "context": { "surface": "REELS", "position": 3, "feedSessionId": "fs-77aa", "source": "RECO", "viaPostId": null, "query": null },
    "media": { "dwellMs": null, "watchMs": 9000, "mediaDurationMs": 6000, "loopCount": 1, "soundOn": true, "mediaIndex": 0 },
    "client": { "platform": "ANDROID", "appVersion": "2.4.1", "networkType": "CELLULAR" },
    "origin": "CLIENT"
  }
}

Payload — every field

Field Type Notes
accountId uuid required. Who did it
sessionId string, nullable The app session, max 64 chars. null for server-produced events
clientEventId string required. The app's own id, or srv:<action>:<rowId> when the backend produced it
action string required. See the table below
targetType string, nullable POST · PRODUCT · SHOP · EVENT · PROFILE. null only for SEARCH
targetId uuid, nullable Lower-cased. null only for SEARCH
context.surface string, nullable FEED · REELS · SEARCH · PRODUCT_PAGE · SHOP_PAGE · EVENT_PAGE · PROFILE_PAGE · RECOMMENDATION · NOTIFICATION · CHAT
context.position int, nullable Where in the feed the item sat, from 0
context.feedSessionId string, nullable Joins to nexgate.feed-served.v1 — max 64 chars
context.source string, nullable Why it was shown: FOLLOWING · CELEBRITY · RECO · INTEREST · FALLBACK · SPONSORED · TRENDING · SEARCH. RECO means your list produced it
context.viaPostId uuid, nullable The post a product was reached through
context.query string, nullable The search text, for SEARCH
media.dwellMs int, nullable Time on screen. 0 – 6 h
media.watchMs int, nullable Time watched. 0 – 6 h
media.mediaDurationMs int, nullable Length of the media. 0 – 12 h
media.loopCount int, nullable Replays. 0 – 1000
media.soundOn bool, nullable Whether sound was on
media.mediaIndex int, nullable Which item of a carousel. 0 – 100
client.platform string, nullable ANDROID · IOS · WEB
client.appVersion string, nullable e.g. 2.4.1
client.networkType string, nullable WIFI · CELLULAR · OFFLINE · UNKNOWN
origin enum, nullable CLIENT or SERVER

media and client are always null on server-produced events — the backend did not witness a screen.

5.1 Two origins, never overlapping

origin Produced by clientEventId
CLIENT The app, in a batch the app's own id
SERVER The backend, watching committed rows srv:<action>:<rowId>

The split is enforced: the batch API rejects every server-owned action with SENT_BY_SERVER, so nothing is counted twice.

5.2 Every action, who sends it, and what triggers it

Action Origin Target Exact trigger
IMPRESSION CLIENT any The item was on screen. Use it as the denominator
VIEW CLIENT any Real attention: dwell ≥ 5 s, or a reel watched ≥ 50 %
SKIP CLIENT any Scrolled past quickly
CLICK CLIENT any Opened the item
SEARCH CLIENT none A search was run — what matters is context.query
NOT_INTERESTED CLIENT any Explicit negative
HIDE CLIENT any Explicit negative, stronger
LIKE SERVER POST A post_likes row inserted
UNLIKE SERVER POST A post_likes row deleted
COMMENT SERVER POST A post_comments row inserted
BOOKMARK SERVER POST A post_bookmarks row inserted
SHARE SERVER POST A post_shares row inserted
REPORT SERVER POST A post_reports row inserted
BLOCK SERVER PROFILE A user_blocks row inserted
MUTE_AUTHOR SERVER PROFILE A user_mutes row inserted
ADD_TO_CART SERVER PRODUCT A cart_items row inserted
REMOVE_FROM_CART SERVER PRODUCT A cart_items row deleted
PURCHASE SERVER PRODUCT An order left PENDING_PAYMENT for any status other than CANCELLED/REFUNDEDone event per order item
PURCHASE SERVER EVENT An event booking became CONFIRMED

Server events are sent after the transaction commits. A rolled-back like sends nothing, and a like made through any code path is captured — nothing depends on a service remembering to emit.

5.3 Media measurements are raw, on purpose

watchMs: 9000 against mediaDurationMs: 6000 with loopCount: 1 is a rewatch, not bad data. Nothing is clamped to "100 %" before you see it, because the fact that someone watched a clip one and a half times is exactly the signal worth having.

5.4 This topic is best-effort by design

If the event log is briefly unavailable, interactions are dropped and counted, not queued — they are signal, not records. A missing view is a missing sample; a queued backlog of stale views would be worse. The entity topics are the opposite: those are never lost, and can always be rebuilt with a backfill.


6. nexgate.feed-served.v1 — what the feed actually showed

accountId · delete, 14 days · 48 partitions · no registered schema

One event type, feed.served. This is the ground truth for ranking evaluation: what was shown, in what order, and why.

The envelope

No entityId, no entityVersion, no receivedAt: eventId, eventType (feed.served), occurredAt, schemaVersion (1), source, payload.

{
  "eventId": "…",
  "eventType": "feed.served",
  "occurredAt": "2026-09-18T10:22:31.900Z",
  "schemaVersion": 1,
  "source": "nexgate-backend",
  "payload": {
    "feedSessionId": "fs-77aa31c8",
    "accountId": "…",
    "surface": "FEED",
    "items": [
      { "type": "POST", "id": "4f2a77e1-…", "position": 0, "source": "FOLLOWING" },
      { "type": "PRODUCT", "id": "8c31a0d2-…", "position": 1, "source": "RECO" }
    ]
  }
}

Payload — every field

Field Type Notes
feedSessionId string Joins to context.feedSessionId on interactions
accountId uuid Who it was served to
surface string Currently always the literal "FEED" — see the warning below
items[].type string POST · PRODUCT
items[].id uuid The item
items[].position int Its place on the page, from 0
items[].source string Why it was there: FOLLOWING · CELEBRITY · RECO · INTEREST · FALLBACK · SPONSORED · TRENDING

Join it to the interactions topic on feedSessionId and you have, per session: what we showed, in which position, from which source — and what the person did about it.

⚠️ Published by the home feed only, and surface is hard-coded to "FEED". Reels, marketplace and events do not publish it. See §8.9.

7. The way back — what the model writes to Redis

Everything above travels backend → model. This section is the other direction: model → backend.

There is no API call and no callback. The recommendation service writes plain Redis keys; the feed reads them when it builds a session. If the model stops writing, the feed keeps working on trending and interest pools — quieter, never broken.

7.1 The eight keys

Key Read by Items must be When it is read
reco:home:posts:{accountId} Home feed discovery post ids every home session build
reco:home:products:{accountId} Home feed discovery product ids every home session build
reco:reels:{accountId} Reels feed post ids (with a short video) every reels session build
reco:market:products:{accountId} Marketplace "For You" product ids every marketplace session build
reco:events:{accountId} Events "For You" event ids every events session build
reco:shops:{accountId} "Shops you might like" shop ids every call to the strip
reco:similar:products:{productId} "Similar items" on a product page product ids every call to the strip
reco:meta:health our alerting only on every metrics scrape

The {…} braces are part of the key — they are Redis Cluster hash tags, and they decide which node the key lives on. Write them exactly as shown.

Every key is optional. A surface whose key is absent runs on trending and interest pools instead. You can ship one surface at a time.

7.2 The value

One JSON string, written with a single SET key <value> EX 172800 (48 h). One command means a reader sees either the whole old list or the whole new one, never half of each.

{
  "v": 1,
  "modelVersion": "ranker-2026-09-14",
  "generatedAt": "2026-09-18T02:10:00Z",
  "items": [
    { "id": "4f2a77e1-0c8b-4d2f-9a11-7b6c3e5d8f90", "score": 0.83 },
    { "id": "9b10c2d4-7e3f-4a52-b8c1-6d9e0f2a3b47", "score": 0.79 }
  ]
}
Field Required Rule
generatedAt Yes ISO 8601. Without it the whole list is INVALID and ignored
items Yes Must be an array. Best first. At most 300 are read; the rest are discarded
items[].id Yes A UUID of the right type for that key. Not a UUID → that item is skipped. Unknown or deleted → dropped silently at read time
items[].score No Kept when inside [0, 1]. Missing or out of range → we use rank instead: 1 / (1 + rank), so position 0 → 1.0, 1 → 0.5, 3 → 0.25
modelVersion No Carried into our debug trace, so a feed can be explained by the model that produced it
v No Format version

After each successful batch, write the health key:

SET reco:meta:health '{"finishedAt":"2026-09-18T02:40:00Z","modelVersion":"ranker-2026-09-14"}'

7.3 What the backend does with each list

Read, then judged in one of four states:

State Condition Used?
FRESH generatedAt within reco.maxAgeHours (default 30 h) Yes
STALE Older than that, key still present Yes — a stale list beats no list. Counted on our dashboards
MISSING No key No → fallback pools fill discovery
INVALID Not JSON, no items array, or no generatedAt No → same as missing, but counted separately, so a broken writer is visible rather than silent

Every read is counted as feed.reco.lists{list, state}, so we can see per surface how often your lists are fresh, stale, missing or invalid.

How your items are ranked against ours. A reco item's score becomes 1 + yourScore. Fallback items score inside [0, 1]. So every recommended item outranks every fallback item in the discovery lane, and the pools only fill what you did not supply. If the same post appears in both, the RECO copy wins.

Where they land on the home feed. Your items fill the discovery slots, interleaved with posts from people the user follows in the pattern F, F, D, F, D, F, F, D — never the same author twice within 4 slots. A session holds 120 items and is rebuilt on pull-to-refresh or when the user reaches the end, so a newly written list appears at the next session, not mid-scroll.

We have 200 ms. Each reco read has a 200 ms timeout. Past that the source returns nothing for that session and the pools fill in. A slow Redis degrades the feed's quality, never its availability.

7.4 Access

You get a Redis user that can write only reco:*:

ACL SETUSER reco-writer on >******** ~reco:* -@all +set +get +del +expire +exists +ping +cluster|slots +cluster|shards +cluster|nodes

Anything outside reco:* answers NOPERM. Locally: user reco-writer, password reco_writer_local, seeds localhost:7001,localhost:7002.

Local docker-compose trap. The containers announce internal addresses (172.29.0.11–16). A cluster client on your laptop follows those announcements and cannot reach them. Run your writer in a container on the nexgate-feed-redis network, or on the host network.

7.5 How we both know it is working

Signal Where Means
feed.reco.lists{state="FRESH"} rising our dashboard your lists are being read and used
feed.reco.lists{state="INVALID"} above zero our dashboard, alert after 15 min your writer is producing something unparseable
feed.reco.health.age.seconds our dashboard, alert above 26 h reco:meta:health.finishedAt is old — your batch has stopped. -1 means no batch has ever been reported
source: "RECO" in a feed response the app, and GET /api/admin/feed/debug/{accountId} an item on a real screen came from your list

The per-account debug trace is the fastest way to prove the loop end to end: it shows every item the last session produced, its source, its base score and its final score — and every item a filter removed, with the filter's name.


8. Who does what — do not rebuild this

The backend is not a thin pipe that hands the feed to a model. It already does a great deal of work after your list is read, and most of it you should not duplicate — partly because it would be wasted effort, and partly because the backend has to do it anyway for correctness, so doing it twice can only disagree.

8.1 The backend already does all of this

Visibility and safety — applied to every candidate, whatever its source:

FilterRemoves
Blockedeither direction of a block
Mutedauthors the viewer muted
Selfthe viewer's own items
Statusdeleted, unpublished, no longer visible
DiscoveryPublicfollowers-only items reached through discovery
HiddenItemmoderation labels (§4.1, sensitive_media becomes an interstitial rather than a removal)
Stockout-of-stock products where that matters
Ageitems past the surface's age limit
UnfollowedAuthortimeline leftovers from someone since unfollowed
Seen / Servedanything already shown, per surface
HydrationFaileditems whose card could not be loaded

So: send us blocked authors, muted authors, deleted items and already-seen items if you like — we drop them anyway. Spending model capacity on exclusions is wasted. nexgate.blocks.v1 is worth consuming for training (a block is a strong negative signal), not for filtering the output list.

Ranking, mixing and presentation — after your scores are read:

  • Lane interleaving per surface. Home is F,F,D,F,D,F,F,D (follow / discovery), reels D,D,F, marketplace D,F,D,D, events F,D,D. You supply the discovery lane; the in-network lane is ours.
  • Author diversity. An item whose author appeared in the last few slots waits for the first slot where it fits, rather than being dropped.
  • Recency, NewShopBoost, TimeToStart (events) as scorers on top of your score.
  • Sponsored slots, business rules and per-viewer ad caps — commercial placement is not yours to model.
  • Sessions, cursors and paging. A session is built once and paged from; filters re-run at page-read time so something deleted mid-scroll disappears.

Cold start and fallbacks — a viewer with no history, or a surface where your list is missing, stale, invalid or fully filtered, is served from trending pools, interest pools and recency. You never need to produce a list for everyone, and a gap costs quality, not availability.

An interest profile per user, without a model — every item carries interests: [{id, w}], and the backend maintains interest:{accountId} from the interaction stream with decay, a floor for declared interests, and negative weights for HIDE / NOT_INTERESTED. This exists so discovery works before you ship, and so a feed can be explained in words. It is a feature you may use, not a thing to reimplement — though learning your own representation from the raw stream is expected to beat it, which is the point of §C.5.3.

8.2 Recommendation work the backend already does today

This is the part most worth reading, because it is recommendation logic, not plumbing — and it is live right now, without a model. Everything here is a baseline you can use as a feature, or beat, but should not rebuild.

Discovery pools, rebuilt every 10 minutes (FeedPoolsJob, one container at a time). Each is built into a temporary key and RENAMEd into place, so a reader never sees a half-built pool, and each expires after an hour — if the job stops, discovery empties rather than serving hours-old "trending".

PoolHow it is computed
pool:{trending:posts}the last 24 hourly buckets summed with weight 0.9 ^ hoursAgo, top 500. This hour counts ×1, an hour ago ×0.9, two hours ago ×0.81
pool:{trending:products}same decay, over product engagement
pool:{trending:reels}trending posts that have a READY short clip, same scores
pool:{new:reels}newest public clips of the last 7 days, scored by publish time
pool:{newshops}products of shops younger than 30 days with fewer than 2 000 impressions, newest first, 200
pool:{events:upcoming}public events starting within 60 days or happening now, scored 1 + confirmed bookings in the last 7 days
pool:shops:popularshops by subscriber count
pool:cat:{categoryId}best sellers of a category — the fallback behind "similar items"
pool:{int:<interestId>}:poststrending posts carrying that interest, score = trending score × interest weight
pool:{int:<interestId>}:productsin-stock products of that interest's categories, by quantity sold then newest
pool:{int:<interestId>}:eventsupcoming events of that interest's categories, by start date

A per-user interest profile, maintained from the same interaction stream you consume (interest:{accountId}, flushed to user_interest_scores):

score[interest] = decay(score[interest]) + actionWeight × itemWeight
decay halves every 14 days

with VIEW 1, LIKE/BOOKMARK 2, COMMENT/SHARE 3, ADD_TO_CART 4, purchase 6–8, NOT_INTERESTED/HIDE −5, and a floor of 5 for interests declared at onboarding. Each surface reads the user's top 5 interests in proportion to score, plus 10 % explore from outside the top 5 so the profile cannot close in on itself.

What this means for you: a user who has never been modelled still gets interest-based discovery, and every surface has a defensible answer on day one. Your list is not competing with an empty feed — it is competing with this. When both exist, yours wins: a reco item scores 1 + yourScore while every pool item scores inside [0, 1].

Where each surface falls back to, when your key is missing, stale, invalid or fully filtered:

SurfaceFalls back to
Home discoverytrending posts + trending products + interest pools
Reelstrending reels, then new reels
Marketplaceinterest products, trending products, new shops, products seen in posts
Eventsinterest events, then upcoming by bookings
Similar itemspool:cat:{categoryId} — best sellers of the same category
Suggested shopsnew shops (< 30 days), then most subscribers

So a surface you have not shipped yet still works, and a gap in your coverage costs quality, not availability. You can ship one key at a time.

8.3 What we want from you

Candidate generation and ranking, per surfacethe seven reco:* keys of §7.1 — that is the whole contract
Similar itemsreco:similar:products:{productId}, the one key that is per item rather than per person
Shop suggestionsreco:shops:{accountId}
Cross-domain learningthe thing the backend fundamentally cannot do: that people who buy baby products watch parenting reels. One representation of a person from every domain

What "sending" means, precisely: one SET key <json> EX 172800 per key, per user, whenever your batch runs. No API call, no callback, no schema negotiation — the full contract is §7.2, and the backend reads whatever is there the next time it builds a session. You never tell us you have finished; we notice a fresher generatedAt.

Where the baseline already beats an obvious first model: trending with hourly decay, category best-sellers for similar items, and the interest profile are all live. If a first version cannot beat those on a surface, the honest move is to not write that key — the fallback is better, and an absent key costs nothing. feed.reco.lists{list,state} on our dashboards shows per surface how often yours is used.

8.4 The seam, in one sentence

You decide what is worth showing; we decide what is allowed to be shown, in what order, and beside what.

If you find yourself writing exclusion logic, position rules or fallbacks, stop — that code already exists here, and two implementations of it will disagree the first time one of them changes.


9. Known gaps

Stated plainly so nobody builds on an assumption:

Gap Consequence
feed-served is published by the home feed only, with surface hard-coded to "FEED" No served-ground-truth for reels, marketplace or events. Evaluation on those surfaces has to rely on impressions
Event ticket bookings have no entity topic A booking arrives as a PURCHASE interaction with targetType: EVENT, but there is no order-like stream for bookings. nexgate.orders.v1 is product orders only
No JSON Schema is registered for nexgate.feed-served.v1 or nexgate.fanout-tasks.v1 The nine entity topics and interactions have schemas under src/main/resources/feed-schemas/; these two do not
Interest weights are empty until an admin maps categories interests: [{id, w}] on posts, products and events is [] for any unmapped category or hashtag. 128 product categories and 12 event categories are still unmapped
Follows and shop subscriptions are not yet interest signals, and SEARCH is not matched to an interest The backend's own interest profile is weaker than the design describes. Does not affect what you receive

9.10. nexgate.fanout-tasks.v1 — internal

Backend to backend: when a post is published, the work of writing it into followers' timelines is chunked and sent over this topic. Key is postId:chunk, retained 3 days, no schema registered.

Ignore it. It carries no signal a model can use, and its shape may change without notice.


Doc What it covers
06_RECOMMENDATION_DEVELOPER_GUIDE.md How to connect, consume, and write your lists back into Redis
02_KAFKA_CONTRACT.md The original agreement: why these topics, deletes, ordering, versioning, personal data
01_ARCHITECTURE.md The design. §4 is the write path and the outbox; §4.6 is what is deliberately left out of payloads
05_INTERACTIONS_CLIENT_GUIDE.md The app's side of the interactions topic: exact definitions of every action