Event Catalogue — the full round trip, both directions

Author: Josh S. Sakweli, Backend Lead Team
Last Updated: 2026-09-18
Version: v1.0 (backend branch fet/feed_baking, schema version 1)

Cluster: Redpanda (Kafka API) · Schema registry: JSON Schema, subject <topic>-value, BACKWARD compatibility

Short Description: Everything that crosses the line between the backend and the recommendation service, in both directions. Inbound (§1–§6): the complete list of what the backend puts on Kafka — 12 topics, 27 event types — when each fires, what does not cause it, and every payload field with its type, nullability and allowed values, including the exact shape of each *.deleted. Outbound (§7): the eight Redis keys the model writes back, their exact format, and what the backend does with each one. §8 draws the line between the two jobs — everything the pipeline already does, so the model does not rebuild it. It is written to be self-contained: this document and a Kafka client are sufficient to begin.

Read 06_RECOMMENDATION_DEVELOPER_GUIDE.md first. That guide is how to connect, what to do with the messages and how to write recommendations back. This document is the catalogue — everything that arrives, in detail. 06 links here rather than repeating it.

Hints:

0. The round trip in one picture

  INBOUND — backend ➜ model                                              §1–§6
  ──────────────────────────────────────────────────────────────────────────────
  a row commits
  (post · product · shop · event · profile · follow · block · order · wishlist)
        └─► outbox ─► relay ─► 9 entity topics ......... compacted, "what exists"

  the app calls POST /api/v1/feed/interactions/batch        (impressions, views…)
  a row commits (like · comment · cart · order paid)        (seen by the backend)
        └─► nexgate.interactions.v1 ................. 90 days, "what people did"

  a feed page is served
        └─► nexgate.feed-served.v1 .................. 14 days, "what was shown"

                                    │
                                    ▼
                    ┌───────────────────────────────┐
                    │    RECOMMENDATION SERVICE     │
                    │   owned by the ML team       │
                    └───────────────┬───────────────┘
                                    │
  OUTBOUND — model ➜ backend        │                                       §7
  ──────────────────────────────────┼───────────────────────────────────────────
                                    ▼
        SET reco:<surface>:{id} <json> EX 172800        (8 keys, all optional)
                                    │
                                    ▼
        the feed pipeline reads reco:* while building a session;
        missing · invalid · stale ─► trending and interest pools fill in

The boundary never moves. The backend only ever writes Kafka and reads reco:*. The recommendation service only ever reads Kafka and writes reco:*. Neither side calls the other over HTTP, so neither can be made slow or unavailable by the other.

1. The envelope

Every message on the nine entity topics has the same eight fields:

{
  "eventId": "9f2c1e8a-5b3d-4a91-8c77-2e6f0b4d1a35",
  "eventType": "product.updated",
  "entityId": "4f2a77e1-0c8b-4d2f-9a11-7b6c3e5d8f90",
  "entityVersion": 184022,
  "occurredAt": "2026-09-18T10:22:33.120Z",
  "schemaVersion": 1,
  "source": "nexgate-backend",
  "payload": { }
}
Field Meaning
eventId Unique per event. A repeat indicates a retry and is to be discarded
eventType One of the 27 in this document
entityId The thing this is about
entityVersion Only ever increases per entity. The highest value seen is authoritative; lower values are ignored
occurredAt When it happened. Not an ordering key — clocks and retries make it unreliable for that
schemaVersion 1 today
source Always nexgate-backend
payload Full current state, or, for a *.deleted, the id plus a reason

The interactions and feed-served topics use a slightly different envelope — noted in their sections.

2. How a delete arrives

Always two messages, in this order:

  1. <entity>.deleted with a reason — the entity is no longer available to consumers, and the reason states why.
  2. A tombstone: same key, value = null — so log compaction can drop the key.

"Deleted" does not always mean the row was deleted. It means the entity is no longer something to recommend. A post set to private, a shop that closed, an account that was locked all arrive as *.deleted, with a reason that distinguishes them. If the entity becomes visible again, *.created follows.

Tombstones are retained for 1 day. A consumer reading a compacted topic from offset 0 after that interval sees only live entities.

3. The topics at a glance

# Topic Key Cleanup Partitions Event types
1 nexgate.posts.v1 postId compact 24 post.created · post.updated · post.deleted
2 nexgate.products.v1 productId compact 24 product.created · product.updated · product.deleted
3 nexgate.shops.v1 shopId compact 12 shop.created · shop.updated · shop.deleted
4 nexgate.events.v1 eventId compact 12 event.created · event.updated · event.deleted
5 nexgate.profiles.v1 accountId compact 24 profile.created · profile.updated · profile.deleted
6 nexgate.follows.v1 followerId:followedId compact 24 follow.created · follow.deleted
7 nexgate.blocks.v1 blockerId:blockedId compact 12 block.created · block.deleted
8 nexgate.orders.v1 orderId compact 24 order.created · order.updated · order.deleted
9 nexgate.wishlist.v1 accountId:productId compact 24 wishlist.created · wishlist.updated · wishlist.deleted
10 nexgate.interactions.v1 accountId delete, 90 d 96 interaction
11 nexgate.feed-served.v1 accountId delete, 14 d 48 feed.served
12 nexgate.fanout-tasks.v1 postId:chunk delete, 3 d 48 (internal — not for consumers)

Partition counts are fixed for good. Adding partitions later would move keys to different partitions and break per-key ordering.

Compacted topics are a live catalogue: a read from offset 0 yields the current state of every entity, and continued consumption keeps it current.

4. Entity topics

Each section below is complete: every field of the payload, its type, whether it can be null, its allowed values where it is an enum, and the exact shape of the *.deleted payload.

Type column: uuid, string, int, decimal, bool, timestamp (ISO 8601 UTC), json (an opaque object whose structure may change and should not be relied upon). A field marked nullable may arrive as null; one marked required is present on every message of that topic.

Fields are only ever added. The schemas are open by design: unrecognised fields are ignored, and no message should be rejected because of one.

4.1 nexgate.posts.v1 — posts and reels

postId · compacted · 24 partitions · schema src/main/resources/feed-schemas/nexgate.posts.v1.json

A post is available to consumers while it is PUBLISHED, not deleted, and PUBLIC or FOLLOWERS.

Event Fires when
post.created A draft becomes published, or a hidden post becomes visible again
post.updated A visible post, or any of its child rows, changes
post.deleted It stops being visible — only if it was visible before

Delete reasons: DELETED (soft-deleted) · UNPUBLISHED (no longer PUBLISHED) · VISIBILITY_RESTRICTED (switched to MENTIONED or PRIVATE)

What counts as a change. Not just the post row — eleven child tables are watched, and a change to any one republishes the whole post: media, hashtags, attached products, attached shops, attached events, user mentions, shop mentions, links, collaborators, and the poll.

What publishes nothing: likesCount, commentsCount, repostsCount, bookmarksCount, viewsCount, quotesCount, sharesCount, shortClipCount, lastVerifiedAt, updatedAt. Engagement counters move constantly; republishing every post on every like would be most of the topic's traffic for none of the value. The counts in the payload are therefore a snapshot taken at the last substantive change, not a live figure. Live behaviour is carried by nexgate.interactions.v1.

A draft that is saved, edited and never published produces nothing at all.

Payload — post.created / post.updated

Field Type Notes
postId uuid required
postType string, nullable REGULAR · POLL
status string, nullable PUBLISHED on every message delivered; DRAFT, SCHEDULED and DELETED are never published
author.id uuid, nullable Who wrote it
author.userName string, nullable Public username. No real name is ever published
content string, nullable The post text as written
contentParsed.hashtags[] string[] Normalised, without the #. Any script (Latin, Arabic, CJK …)
contentParsed.mentionedUserIds[] uuid[] Accounts mentioned in the text
contentParsed.mentionedShopIds[] uuid[] Shops mentioned in the text
media[].fileId uuid, nullable File-service id
media[].mediaType string, nullable IMAGE · VIDEO
media[].order int Display order within the post, from 0
media[].status string, nullable Processing state. PENDING means variants are not ready yet
media[].variants json, nullable Rendition URLs keyed by name (hls, thumb, …). Opaque — read keys, do not assume the set
media[].shortClip bool true = this is a reel. There is no separate reel topic
media[].mediaId uuid, nullable The clip's own id, as used by the reel endpoints
media[].durationMs int, nullable Length in milliseconds, for video
attachments.products[] uuid[] Products attached to the post
attachments.shops[] uuid[] Shops attached
attachments.events[] uuid[] Events attached
engagement.likesCount int Snapshot, see above
engagement.commentsCount int Snapshot
engagement.sharesCount int Snapshot
engagement.viewsCount int Snapshot
engagement.bookmarksCount int Snapshot
engagement.repostsCount int Snapshot
engagement.quotesCount int Snapshot
privacySettings.visibility string, nullable PUBLIC or FOLLOWERS on every message delivered
privacySettings.whoCanComment string, nullable Who may comment
privacySettings.whoCanRepost string, nullable Who may repost
quotedPostId uuid, nullable Set when this post quotes or reposts another. A pure repost carries the original's engagement, not its own
collaboratorIds[] uuid[] Co-authors
hasPoll bool Whether a poll is attached. Poll options and votes are not published
externalLinkDomain string, nullable The domain only, never the full URL
isEdited bool Whether it has been edited since publishing
createdAt timestamp, nullable When the row was created (may be long before publishing)
publishedAt timestamp, nullable When the post went live. This is the field for recency, not createdAt
updatedAt timestamp, nullable Last change
interests[].id uuid Interest id from the shared list (interest_categories)
interests[].w decimal Weight: 1.0 from attached products and events, 0.6 from hashtags. An interest reached twice keeps the higher weight. Empty until an administrator maps the category or hashtag

Payload — post.deleted

Field Type Notes
postId uuid required
authorId uuid, nullable The author, or null when the row itself is gone
reason string One of the delete reasons above

Followed by a tombstone: same key, value = null.

4.2 nexgate.products.v1 — products

productId · compacted · 24 partitions

A product is available to consumers while it is ACTIVE or OUT_OF_STOCK, not deleted, and its shop is visible. Running out of stock is an update, not a disappearance: stock is carried in the payload and the consumer decides how to treat it.

Event Fires when
product.created It becomes visible, including when its shop comes back
product.updated The product or its installment plan changes
product.deleted It stops being visible

Delete reasons: DELETED · UNPUBLISHED (DRAFT, INACTIVE, ARCHIVED) · SHOP_UNAVAILABLE (shop suspended, closed, unapproved or deleted)

The shop cascade. When a shop's visibility flips, every product of that shop is republished, in pages, at that moment. One suspension can produce thousands of product.deleted messages. Deliberate: a consumer watching only this topic would otherwise keep recommending products from a suspended shop.

What publishes nothing: viewCount, cartAddCount, updatedAt.

Payload — product.created / product.updated

Field Type Notes
productId uuid required
productName string, nullable
productSlug string, nullable URL slug
productDescription string, nullable
productType string, nullable PHYSICAL · DIGITAL
productMedia[].fileId uuid, nullable
productMedia[].mediaType string, nullable IMAGE · VIDEO
productMedia[].order int, nullable Display order
productMedia[].status string, nullable Processing state
productMedia[].variants json, nullable Rendition URLs
categoryId uuid, nullable The key to interests — mapped by an admin
categoryName string, nullable
parentCategoryId uuid, nullable A category with no mapping inherits its parent's interests
price decimal, nullable Current price
comparePrice decimal, nullable "Was" price
isOnSale bool
discountPercentage decimal, nullable
stockQuantity int
isInStock bool
isLowStock bool
showStockToPublic bool Whether the shop displays the number
status string, nullable ACTIVE or OUT_OF_STOCK on every message delivered
condition string, nullable NEW · USED_LIKE_NEW · USED_GOOD · USED_FAIR · REFURBISHED · FOR_PARTS
urgencyTag string, nullable NONE · NEW_ARRIVAL · LIMITED_EDITION · LIMITED_OFFER · FEW_REMAINS
specifications json, nullable Free-form, seller-entered. Opaque
colors json, nullable Free-form, seller-entered. Opaque
hasGroupBuying bool
groupPrice decimal, nullable Price when bought as a group
hasInstallments bool
stockInfo.soldCount int Units sold
shopId uuid, nullable
shopName string, nullable
createdAt timestamp, nullable
updatedAt timestamp, nullable
interests[].id uuid From the product's category, weight 1.0
interests[].w decimal

Payload — product.deleted

Field Type Notes
productId uuid required
shopId uuid, nullable Useful when the cause was the shop
reason string

4.3 nexgate.shops.v1 — shops

shopId · compacted · 12 partitions

A shop is available to consumers while it is ACTIVE or TEMPORARILY_OFFLINE, approved, and not deleted.

Event Fires when
shop.created It becomes visible
shop.updated Anything meaningful on the shop changes
shop.deleted It stops being visible

Delete reasons: DELETED · SUSPENDED · CLOSED (PERMANENTLY_CLOSED) · UNPUBLISHED (PENDING, or approval withdrawn)

Every one of these also republishes the shop's products (§4.2).

What publishes nothing: subscriberCount, lastSeenTime, updatedAt.

Payload — shop.created / shop.updated

Field Type Notes
shopId uuid required
shopName string, nullable
shopSlug string, nullable
shopDescription string, nullable
logo.fileId / .mediaType / .status / .variants uuid / string / string / json, nullable The shop logo
banner.fileId / .mediaType / .status / .variants uuid / string / string / json, nullable The shop banner
ownerId uuid, nullable The account that owns the shop
status string, nullable ACTIVE or TEMPORARILY_OFFLINE on every message delivered
tempOfflineUntil timestamp, nullable Set while the owner has gone offline deliberately
city string, nullable
district string, nullable
region string, nullable The feed filters by region — this is why location stays
countryCode string, nullable
isVerified bool
verificationBadge string, nullable Which badge
trustScore decimal, nullable Platform trust score
subscriberCount int Snapshot — a change to it alone does not republish
approvedAt timestamp, nullable
createdAt timestamp, nullable
updatedAt timestamp, nullable

Payload — shop.deleted

Field Type Notes
shopId uuid required
reason string

Never published: phone, email, street address, landmark, coordinates. Many shops are run from someone's home.

4.4 nexgate.events.v1 — events

eventId · compacted · 12 partitions

An event is available to consumers while it is PUBLISHED or HAPPENING, PUBLIC, and not deleted.

Event Fires when
event.created It becomes visible
event.updated The event or its tickets change — including selling out, which is an update, not a removal
event.deleted It stops being visible

Delete reasons: DELETED · ENDED (COMPLETED) · CANCELLED · UNPUBLISHED (DRAFT or any other status) · VISIBILITY_RESTRICTED (PRIVATE or UNLISTED)

Ticket statistics count sold tickets only. Reservations change on every checkout attempt and expire by themselves, so publishing them would be noise.

What publishes nothing: updatedAt, currentStage, completedStages, rsaKeys.

Payload — event.created / event.updated

Field Type Notes
eventId uuid required
title string, nullable
slug string, nullable
description string, nullable
category.id uuid, nullable The key to interests
category.name string, nullable
bannerMedia json, nullable Opaque media object
eventFormat string, nullable IN_PERSON · ONLINE · HYBRID · TBA
visibility string, nullable PUBLIC on every message delivered
status string, nullable PUBLISHED or HAPPENING on every message delivered
schedule.startDateTime timestamp, nullable
schedule.endDateTime timestamp, nullable
schedule.timezone string, nullable IANA zone
venue.name string, nullable
venue.address string, nullable A public venue, unlike a shop's address
venue.latitude / .longitude decimal, nullable
pricing.minPrice / .maxPrice decimal, nullable Across ticket types
pricing.pricingType string, nullable How the event is priced
stats.ticketsSold int Sold only — never reservations
stats.ticketsAvailable int, nullable
stats.isSoldOut bool Selling out arrives as event.updated
organizerId uuid, nullable
organizerName string, nullable
linkedProductIds[] uuid[] Products tied to the event
linkedShopIds[] uuid[] Shops tied to the event
createdAt timestamp, nullable
publishedAt timestamp, nullable
updatedAt timestamp, nullable
interests[].id / .w uuid / decimal From the event's category, weight 1.0

Payload — event.deleted

Field Type Notes
eventId uuid required
reason string

Never published: the virtual meeting link, meeting id or passcode (they admit people into the room), and never the event's signing keys.

4.5 nexgate.profiles.v1 — public profile facts

accountId · compacted · 24 partitions

A profile is available to consumers once sign-up is complete (a real username, not a temp_… placeholder) and while the account is not locked.

Event Fires when
profile.created Sign-up completes, or a locked account is unlocked
profile.updated One of the watched fields changes, or the person's declared interests change
profile.deleted The account is locked, or the row is gone

Delete reasons: LOCKED · DELETED

Only these changes republish: userName, accountType, accountTier, locked, isVerified, verificationBadgeType, and rows in the user's declared interests. A login, a password change or a new profile photo publishes nothing.

Payload — profile.created / profile.updated

Field Type Notes
accountId uuid required
userName string, nullable Public username. Never temp_… on a delivered message
accountType string, nullable NORMAL · SYSTEM · VERIFIED
accountTier string, nullable FULL · RESTRICTED · MINOR. MINOR denotes a minor's account and warrants conservative treatment
isVerified bool
verificationBadgeType string, nullable Which badge
followerCount int Snapshot
followingCount int Snapshot
declaredInterestIds[] uuid[] What the person chose at onboarding, on the shared interest list
createdAt timestamp, nullable Account age — useful for cold-start

Payload — profile.deleted

Field Type Notes
accountId uuid required
reason string LOCKED or DELETED

Never published: first, middle or last name, bio, location, phone, email, birth date, profile photo.

⚠️ The active column must not be used, wherever it is encountered. Sign-up sets it false and nothing ever sets it true, so it is false on every real account. It is deliberately not in the payload — relying on it would have tombstoned every profile on the platform.

4.6 nexgate.follows.v1 — who follows whom

Key followerId:followedId · compacted · 24 partitions

Only ACCEPTED follows are published. A follow request (PENDING) emits nothing; if it is never accepted, nothing is emitted at all.

Event Fires when
follow.created A request is accepted, or a public account is followed
follow.deleted Unfollowed, or a request withdrawn — only if it had been accepted

Payload — follow.created

Field Type Notes
followerId uuid required — the one doing the following
followedId uuid required — the one being followed
followedAt timestamp, nullable When it was accepted

Payload — follow.deleted

followerId (uuid, required) · followedId (uuid, required) · reason (string)

4.7 nexgate.blocks.v1 — who blocked whom

Key blockerId:blockedId · compacted · 12 partitions

A block is published the moment the row is written — there is no visibility condition.

Event Fires when
block.created Someone blocks someone
block.deleted They unblock

This topic is not optional. A block hides content in both directions: neither party may be recommended the other's content.

Payload — block.created

Field Type Notes
blockerId uuid required — who blocked
blockedId uuid required — who was blocked
blockedAt timestamp, nullable

Payload — block.deleted

blockerId (uuid, required) · blockedId (uuid, required) · reason (string)

Mutes are not published on Kafka. A mute is a per-viewer preference applied inside the feed and is not an entity. It does appear as a MUTE_AUTHOR interaction (§5).

4.8 nexgate.orders.v1 — what was actually bought

orderId · compacted · 24 partitions

An order is published from creation onwards. This is the strongest purchase signal on the platform.

Event Fires when
order.created An order is placed
order.updated One of the watched fields changes
order.deleted The order is soft-deleted (reason DELETED)

Only these changes republish: productOrderStatus, deliveryStatus, totalAmount, currency, shippedAt, deliveredAt, completedAt, cancelledAt, isDeleted.

Payload — order.created / order.updated

Field Type Notes
orderId uuid required
orderNumber string, nullable Human-readable reference
buyer.accountId uuid, nullable Who bought
shopId uuid, nullable Who sold
items[].productId uuid, nullable
items[].quantity int
items[].unitPrice decimal, nullable
items[].subtotal decimal, nullable
productOrderStatus string, nullable PENDING_PAYMENT · PENDING_SHIPMENT · SHIPPED · DELIVERED · AWAITING_BUYER_CONFIRM · COMPLETED · DISPUTED · CANCELLED · REFUNDED
deliveryStatus string, nullable PENDING · SHIPPED · DELIVERED · CONFIRMED · IN_TRANSIT · NOT_APPLICABLE
productOrderSource string, nullable DIRECT_PURCHASE · DIGITAL_PURCHASE · INSTALLMENT · GROUP_PURCHASE · CART_PURCHASE · CHAT_OFFER
totalAmount decimal, nullable
currency string, nullable
orderedAt timestamp, nullable
shippedAt timestamp, nullable
deliveredAt timestamp, nullable
completedAt timestamp, nullable The strongest signal available: payment settled and receipt confirmed by the buyer
cancelledAt timestamp, nullable
updatedAt timestamp, nullable

An order leaving PENDING_PAYMENT for any status other than CANCELLED or REFUNDED also produces a PURCHASE interaction per order item (§5). The order topic describes what the order is; the interaction records when the purchase occurred.

Payload — order.deleted

Field Type Notes
orderId uuid required
reason string DELETED

Never published: delivery address or instructions, order notes, cancellation text, payment method, escrow or fee figures, tracking number, confirmation code, chat ids.

Event ticket bookings are not on this topic. nexgate.orders.v1 is product orders only. See §9.

4.9 nexgate.wishlist.v1 — saved products

Key accountId:productId · compacted · 24 partitions

A product can be in a person's wishlist once, whatever group it is filed under.

Event Fires when
wishlist.created A product is added
wishlist.updated The entry changes (for example, moved to another group)
wishlist.deleted It is removed

Payload — wishlist.created / wishlist.updated

Field Type Notes
accountId uuid required
productId uuid required
wishlistId uuid, nullable The row's own id
groupId uuid, nullable Which of the user's lists it sits in
addedAt timestamp, nullable

Payload — wishlist.deleted

accountId (uuid, required) · productId (uuid, required) · reason (string)

Never published: the group's name. It is free text a person writes about themselves ("Gifts for Mum"). The group id is enough to tell groups apart.

5. nexgate.interactions.v1 — what people did

accountId · delete, 90 days · 96 partitions · schema nexgate.interactions.v1.json

One event type, interaction. Keyed by accountId so one person's actions stay in order.

This is the only topic anything outside the backend can write to, and it does so through exactly one endpoint: POST /api/v1/feed/interactions/batch (09_NEW_ENDPOINTS.md A8). Every other topic is written from committed database rows, so no client can announce that an entity exists or changed.

The envelope is different here

No entityId and no entityVersion — an interaction is not an entity. Instead:

Field Type Notes
eventId uuid required. Deterministic (UUIDv5 of account + clientEventId), so a re-publish of the same action produces the same id. Deduplicate on it
eventType string Always the literal interaction
occurredAt timestamp The device's clock, clamped so it can never be in the future
receivedAt timestamp The server's clock, when it arrived
schemaVersion int 1
source string nexgate-backend
payload object Below 
{
  "eventId": "…", "eventType": "interaction",
  "occurredAt": "2026-09-18T10:22:41.100Z", "receivedAt": "2026-09-18T10:22:41.480Z",
  "schemaVersion": 1, "source": "nexgate-backend",
  "payload": {
    "accountId": "…", "sessionId": "s-8812e0c4", "clientEventId": "c-000185",
    "action": "VIEW", "targetType": "POST", "targetId": "…",
    "context": { "surface": "REELS", "position": 3, "feedSessionId": "fs-77aa", "source": "RECO", "viaPostId": null, "query": null },
    "media": { "dwellMs": null, "watchMs": 9000, "mediaDurationMs": 6000, "loopCount": 1, "soundOn": true, "mediaIndex": 0 },
    "client": { "platform": "ANDROID", "appVersion": "2.4.1", "networkType": "CELLULAR" },
    "origin": "CLIENT"
  }
}

Payload — every field

Field Type Notes
accountId uuid required. Who did it
sessionId string, nullable The app session, max 64 chars. null for server-produced events
clientEventId string required. The app's own id, or srv:<action>:<rowId> when the backend produced it
action string required. See the table below
targetType string, nullable POST · PRODUCT · SHOP · EVENT · PROFILE. null only for SEARCH
targetId uuid, nullable Lower-cased. null only for SEARCH
context.surface string, nullable FEED · REELS · SEARCH · PRODUCT_PAGE · SHOP_PAGE · EVENT_PAGE · PROFILE_PAGE · RECOMMENDATION · NOTIFICATION · CHAT
context.position int, nullable Where in the feed the item sat, from 0
context.feedSessionId string, nullable Joins to nexgate.feed-served.v1 — max 64 chars
context.source string, nullable Why it was shown: FOLLOWING · CELEBRITY · RECO · INTEREST · FALLBACK · SPONSORED · TRENDING · SEARCH. RECO indicates the item came from a supplied list
context.viaPostId uuid, nullable The post a product was reached through
context.query string, nullable The search text, for SEARCH
media.dwellMs int, nullable Time on screen. 0 – 6 h
media.watchMs int, nullable Time watched. 0 – 6 h
media.mediaDurationMs int, nullable Length of the media. 0 – 12 h
media.loopCount int, nullable Replays. 0 – 1000
media.soundOn bool, nullable Whether sound was on
media.mediaIndex int, nullable Which item of a carousel. 0 – 100
client.platform string, nullable ANDROID · IOS · WEB
client.appVersion string, nullable e.g. 2.4.1
client.networkType string, nullable WIFI · CELLULAR · OFFLINE · UNKNOWN
origin enum, nullable CLIENT or SERVER

media and client are always null on server-produced events — the backend did not witness a screen.

5.1 Two origins, never overlapping

origin Produced by clientEventId
CLIENT The app, in a batch the app's own id
SERVER The backend, watching committed rows srv:<action>:<rowId>

The split is enforced: the batch API rejects every server-owned action with SENT_BY_SERVER, so nothing is counted twice.

5.2 Every action, who sends it, and what triggers it

Action Origin Target Exact trigger
IMPRESSION CLIENT any The item was on screen. Use it as the denominator
VIEW CLIENT any Real attention: dwell ≥ 5 s, or a reel watched ≥ 50 %
SKIP CLIENT any Scrolled past quickly
CLICK CLIENT any Opened the item
SEARCH CLIENT none A search was run — what matters is context.query
NOT_INTERESTED CLIENT any Explicit negative
HIDE CLIENT any Explicit negative, stronger
LIKE SERVER POST A post_likes row inserted
UNLIKE SERVER POST A post_likes row deleted
COMMENT SERVER POST A post_comments row inserted
BOOKMARK SERVER POST A post_bookmarks row inserted
SHARE SERVER POST A post_shares row inserted
REPORT SERVER POST A post_reports row inserted
BLOCK SERVER PROFILE A user_blocks row inserted
MUTE_AUTHOR SERVER PROFILE A user_mutes row inserted
ADD_TO_CART SERVER PRODUCT A cart_items row inserted
REMOVE_FROM_CART SERVER PRODUCT A cart_items row deleted
PURCHASE SERVER PRODUCT An order left PENDING_PAYMENT for any status other than CANCELLED/REFUNDED — one event per order item
PURCHASE SERVER EVENT An event booking became CONFIRMED

Server events are sent after the transaction commits. A rolled-back like sends nothing, and a like made through any code path is captured — nothing depends on a service remembering to emit.

5.3 Media measurements are raw, on purpose

watchMs: 9000 against mediaDurationMs: 6000 with loopCount: 1 is a rewatch, not bad data. Nothing is clamped to "100 %" before delivery, because the fact that someone watched a clip one and a half times is exactly the signal worth having.

5.4 This topic is best-effort by design

If the event log is briefly unavailable, interactions are dropped and counted, not queued — they are signal, not records. A missing view is a missing sample; a queued backlog of stale views would be worse. The entity topics are the opposite: those are never lost, and can always be rebuilt with a backfill.

6. nexgate.feed-served.v1 — what the feed actually showed

accountId · delete, 14 days · 48 partitions · no registered schema

One event type, feed.served. This is the ground truth for ranking evaluation: what was shown, in what order, and why.

The envelope

No entityId, no entityVersion, no receivedAt: eventId, eventType (feed.served), occurredAt, schemaVersion (1), source, payload.

{
  "eventId": "…",
  "eventType": "feed.served",
  "occurredAt": "2026-09-18T10:22:31.900Z",
  "schemaVersion": 1,
  "source": "nexgate-backend",
  "payload": {
    "feedSessionId": "fs-77aa31c8",
    "accountId": "…",
    "surface": "FEED",
    "items": [
      { "type": "POST", "id": "4f2a77e1-…", "position": 0, "source": "FOLLOWING" },
      { "type": "PRODUCT", "id": "8c31a0d2-…", "position": 1, "source": "RECO" }
    ]
  }
}

Payload — every field

Field Type Notes
feedSessionId string Joins to context.feedSessionId on interactions
accountId uuid Who it was served to
surface string Currently always the literal "FEED" — see the warning below
items[].type string POST · PRODUCT
items[].id uuid The item
items[].position int Its place on the page, from 0
items[].source string Why it was there: FOLLOWING · CELEBRITY · RECO · INTEREST · FALLBACK · SPONSORED · TRENDING

Joined to the interactions topic on feedSessionId, it gives, per session: what was shown, in which position, from which source, and what the viewer did about it.

⚠️ Published by the home feed only, and surface is hard-coded to "FEED". Reels, marketplace and events do not publish it. See §9.

7. The way back — what the model writes to Redis

Everything above travels backend → model. This section is the other direction: model → backend.

There is no API call and no callback. The recommendation service writes plain Redis keys; the feed reads them when it builds a session. If the model stops writing, the feed keeps working on trending and interest pools — quieter, never broken.

7.1 The eight keys

Key Read by Items must be When it is read
reco:home:posts:{accountId} Home feed discovery post ids every home session build
reco:home:products:{accountId} Home feed discovery product ids every home session build
reco:reels:{accountId} Reels feed post ids (with a short video) every reels session build
reco:market:products:{accountId} Marketplace "For You" product ids every marketplace session build
reco:events:{accountId} Events "For You" event ids every events session build
reco:shops:{accountId} Suggested shops strip shop ids every call to the strip
reco:similar:products:{productId} "Similar items" on a product page product ids every call to the strip
reco:meta:health backend alerting only — on every metrics scrape

The {…} braces are part of the key — they are Redis Cluster hash tags, and they decide which node the key lives on. Write them exactly as shown.

Every key is optional. A surface whose key is absent runs on trending and interest pools instead. You can ship one surface at a time.

7.2 The value

One JSON string, written with a single SET key <value> EX 172800 (48 h). One command means a reader sees either the whole old list or the whole new one, never half of each.

{
  "v": 1,
  "modelVersion": "ranker-2026-09-14",
  "generatedAt": "2026-09-18T02:10:00Z",
  "items": [
    { "id": "4f2a77e1-0c8b-4d2f-9a11-7b6c3e5d8f90", "score": 0.83 },
    { "id": "9b10c2d4-7e3f-4a52-b8c1-6d9e0f2a3b47", "score": 0.79 }
  ]
}
Field Required Rule
generatedAt Yes ISO 8601. Without it the whole list is INVALID and ignored
items Yes Must be an array. Best first. At most 300 are read; the rest are discarded
items[].id Yes A UUID of the right type for that key. Not a UUID → that item is skipped. Unknown or deleted → dropped silently at read time
items[].score No Kept when inside [0, 1]. Missing or out of range → rank is used instead: 1 / (1 + rank), so position 0 → 1.0, 1 → 0.5, 3 → 0.25
modelVersion No Carried into the backend debug trace, so a feed can be explained by the model that produced it
v No Format version

After each successful batch, write the health key:

SET reco:meta:health '{"finishedAt":"2026-09-18T02:40:00Z","modelVersion":"ranker-2026-09-14"}'

7.3 What the backend does with each list

Read, then judged in one of four states:

State Condition Used?
FRESH generatedAt within reco.maxAgeHours (default 30 h) Yes
STALE Older than that, key still present Yes — a stale list beats no list. Counted on the backend dashboards
MISSING No key No → fallback pools fill discovery
INVALID Not JSON, no items array, or no generatedAt No → same as missing, but counted separately, so a broken writer is visible rather than silent

Every read is counted as feed.reco.lists{list, state}, so the state of each list is visible per surface: fresh, stale, missing or invalid.

Where they land on the home feed. Your items fill the discovery slots, interleaved with posts from people the user follows in the pattern F, F, D, F, D, F, F, D — never the same author twice within 4 slots. A session holds 120 items and is rebuilt on pull-to-refresh or when the user reaches the end, so a newly written list appears at the next session, not mid-scroll.

The read budget is 200 ms. Each reco read has a 200 ms timeout. Past that the source returns nothing for that session and the pools fill in. A slow Redis degrades the feed's quality, never its availability.

7.4 Access

You get a Redis user that can write only reco:*:

ACL SETUSER reco-writer on >******** ~reco:* -@all +set +get +del +expire +exists +ping +cluster|slots +cluster|shards +cluster|nodes

Anything outside reco:* answers NOPERM. Locally: user reco-writer, password reco_writer_local, seeds localhost:7001,localhost:7002.

Local docker-compose trap. The containers announce internal addresses (172.29.0.11–16). A cluster client outside the compose network follows those announcements and cannot reach the nodes. The writer should run in a container on the nexgate-feed-redis network, or on the host network.

7.5 Verifying the integration

Signal Where Means
feed.reco.lists{state="FRESH"} rising backend dashboard the lists are being read and used
feed.reco.lists{state="INVALID"} above zero backend dashboard, alert after 15 min the writer is producing something unparseable
feed.reco.health.age.seconds backend dashboard, alert above 26 h reco:meta:health.finishedAt is stale, indicating the batch has stopped. -1 means no batch has ever been reported
source: "RECO" in a feed response the app, and GET /api/admin/feed/debug/{accountId} an item on a live screen originated from a supplied list

The per-account debug trace is the fastest way to prove the loop end to end: it shows every item the last session produced, its source, its base score and its final score — and every item a filter removed, with the filter's name.

8. Who does what — do not rebuild this

The backend is not a thin pipe that hands the feed to a model. It performs a great deal of work after a list is read, and most of it should not be duplicated — partly because the effort is wasted, and partly because the backend must do it regardless for correctness, so a second implementation can only diverge.

8.1 The backend already does all of this

Visibility and safety — applied to every candidate, whatever its source:

Filter Removes
Blocked either direction of a block
Muted authors the viewer muted
Self the viewer's own items
Status deleted, unpublished, no longer visible
DiscoveryPublic followers-only items reached through discovery
HiddenItem moderation labels (§4.1, sensitive_media becomes an interstitial rather than a removal)
Stock out-of-stock products where that matters
Age items past the surface's age limit
UnfollowedAuthor timeline leftovers from someone since unfollowed
Seen / Served anything already shown, per surface
HydrationFailed items whose card could not be loaded

Blocked authors, muted authors, deleted items and already-seen items may therefore be left in a submitted list; the backend removes them regardless. Model capacity spent on exclusions is wasted. nexgate.blocks.v1 is worth consuming for training — a block is a strong negative signal — rather than for filtering the output list.

Ranking, mixing and presentation — applied after the supplied scores are read:

Cold start and fallbacks — a viewer with no history, or a surface whose list is missing, stale, invalid or fully filtered, is served from trending pools, interest pools and recency. You never need to produce a list for everyone, and a gap costs quality, not availability.

An interest profile per user, without a model — every item carries interests: [{id, w}], and the backend maintains interest:{accountId} from the interaction stream with decay, a floor for declared interests, and negative weights for HIDE / NOT_INTERESTED. It exists so that discovery works before any model is delivered, and so that a feed can be explained in plain language. It is available as a feature and is not to be reimplemented — though a learned representation over the raw stream is expected to outperform it, which is the point of §C.5.3.

8.2 Recommendation work the backend already does today

This is the part most worth reading, because it is recommendation logic, not plumbing — and it is live right now, without a model. Everything here is a baseline that may be used as a feature, or improved upon, but should not be rebuilt.

Discovery pools, rebuilt every 10 minutes (FeedPoolsJob, one container at a time). Each is built into a temporary key and RENAMEd into place, so a reader never sees a half-built pool, and each expires after an hour — if the job stops, discovery empties rather than serving hours-old "trending".

Pool How it is computed
pool:{trending:posts} the last 24 hourly buckets summed with weight 0.9 ^ hoursAgo, top 500. This hour counts ×1, an hour ago ×0.9, two hours ago ×0.81
pool:{trending:products} same decay, over product engagement
pool:{trending:reels} trending posts that have a READY short clip, same scores
pool:{new:reels} newest public clips of the last 7 days, scored by publish time
pool:{newshops} products of shops younger than 30 days with fewer than 2 000 impressions, newest first, 200
pool:{events:upcoming} public events starting within 60 days or happening now, scored 1 + confirmed bookings in the last 7 days
pool:shops:popular shops by subscriber count
pool:cat:{categoryId} best sellers of a category — the fallback behind "similar items"
pool:{int:<interestId>}:posts trending posts carrying that interest, score = trending score × interest weight
pool:{int:<interestId>}:products in-stock products of that interest's categories, by quantity sold then newest
pool:{int:<interestId>}:events upcoming events of that interest's categories, by start date

A per-user interest profile, maintained from the same interaction stream (interest:{accountId}, flushed to user_interest_scores):

score[interest] = decay(score[interest]) + actionWeight × itemWeight
decay halves every 14 days

with VIEW 1, LIKE/BOOKMARK 2, COMMENT/SHARE 3, ADD_TO_CART 4, purchase 6–8, NOT_INTERESTED/HIDE −5, and a floor of 5 for interests declared at onboarding. Each surface reads the user's top 5 interests in proportion to score, plus 10 % explore from outside the top 5 so the profile cannot close in on itself.

The consequence: a user who has never been modelled still receives interest-based discovery, and every surface has a defensible answer on day one. A supplied list does not compete with an empty feed; it competes with this baseline. Where both exist, the supplied list wins: a reco item scores 1 + the supplied score, while every pool item scores inside [0, 1].

Fallbacks per surface, used when a key is missing, stale, invalid or fully filtered:

Surface Falls back to
Home discovery trending posts + trending products + interest pools
Reels trending reels, then new reels
Marketplace interest products, trending products, new shops, products seen in posts
Events interest events, then upcoming by bookings
Similar items pool:cat:{categoryId} — best sellers of the same category
Suggested shops new shops (< 30 days), then most subscribers

A surface that has not yet been modelled therefore still works, and a gap in coverage costs quality, not availability. Keys may be delivered one at a time.

8.3 What the backend expects in return
Candidate generation and ranking, per surface the seven reco:* keys of §7.1 — that is the whole contract
Similar items reco:similar:products:{productId}, the one key that is per item rather than per person
Shop suggestions reco:shops:{accountId}
Cross-domain learning the thing the backend fundamentally cannot do: that people who buy baby products watch parenting reels. One representation of a person from every domain

The submission mechanism, precisely: one SET key <json> EX 172800 per key, per user, each time the batch runs. There is no API call, callback or schema negotiation — the full contract is §7.2, and the backend reads whatever is present the next time it builds a session. Completion is never announced; a fresher generatedAt is the signal.

Where the baseline may already exceed an early model: trending with hourly decay, category best-sellers for similar items, and the interest profile are all live. Where an early version cannot beat those on a given surface, the correct decision is to withhold that key — the fallback is stronger, and an absent key costs nothing. feed.reco.lists{list,state} reports, per surface, how often a supplied list is used.

8.4 The seam, in one sentence

The recommendation service decides what is worth showing. The backend decides what is permitted to be shown, in what order, and alongside what.

Exclusion logic, position rules and fallbacks should not be written: that code already exists in the backend, and two implementations will diverge the first time either one changes.

9. Known gaps

Stated plainly so nobody builds on an assumption:

Gap Consequence
feed-served is published by the home feed only, with surface hard-coded to "FEED" No served-ground-truth for reels, marketplace or events. Evaluation on those surfaces has to rely on impressions
Event ticket bookings have no entity topic A booking arrives as a PURCHASE interaction with targetType: EVENT, but there is no order-like stream for bookings. nexgate.orders.v1 is product orders only
No JSON Schema is registered for nexgate.feed-served.v1 or nexgate.fanout-tasks.v1 The nine entity topics and interactions have schemas under src/main/resources/feed-schemas/; these two do not
Interest weights are empty until an admin maps categories interests: [{id, w}] on posts, products and events is [] for any unmapped category or hashtag. 128 product categories and 12 event categories are still unmapped
Follows and shop subscriptions are not yet interest signals, and SEARCH is not matched to an interest The backend's interest profile is weaker than the design describes. The delivered stream is unaffected

10. nexgate.fanout-tasks.v1 — internal

Backend to backend: when a post is published, the work of writing it into followers' timelines is chunked and sent over this topic. Key is postId:chunk, retained 3 days, no schema registered.

Ignore it. It carries no signal a model can use, and its shape may change without notice.

Doc What it covers
06_RECOMMENDATION_DEVELOPER_GUIDE.md Connecting, consuming, and writing lists back into Redis
02_KAFKA_CONTRACT.md The original agreement: why these topics, deletes, ordering, versioning, personal data
01_ARCHITECTURE.md The design. §4 is the write path and the outbox; §4.6 is what is deliberately left out of payloads
05_INTERACTIONS_CLIENT_GUIDE.md The app's side of the interactions topic: exact definitions of every action
Revision #5
Created 18 September 2026 04:45:35 by Admin Qbit
Updated 18 September 2026 09:35:38 by Admin Qbit