Event Catalogue — the full round trip, both directions

Author: Josh S. Sakweli, Backend Lead Team
Last Updated: 2026-09-18
Version: v1.0 (backend branch fet/feed_baking, schema version 1)

Cluster: Redpanda (Kafka API) · Schema registry: JSON Schema, subject <topic>-value, BACKWARD compatibility

Short Description: Everything that crosses the line between the backend and the recommendation service, in both directions. Inbound (§1–§6): the complete list of what the backend puts on Kafka — 12 topics, 27 event types — when each fires, what does not cause it, and every payload field with its type, nullability and allowed values, including the exact shape of each *.deleted. Outbound (§7): the eight Redis keys the model writes back, their exact format, and what the backend does with each one. §8 draws the line between the two jobs — everything the pipeline already does, so the model does not rebuild it. It is written to be self-contained: this document and a Kafka client are sufficient to begin.

Read 06_RECOMMENDATION_DEVELOPER_GUIDE.md first. That guide is how to connect, what to do with the messages and how to write recommendations back. This document is the catalogue — everything that arrives, in detail. 06 links here rather than repeating it.

Hints:

• Nothing is emitted from service code. Events come from committed database rows, so the delivered stream always matches the database — a rolled-back transaction publishes nothing.
• A payload is always the full current state, never a diff. It replaces the stored copy.
• Order per key is guaranteed; order across keys is not. Use entityVersion, never occurredAt.
• Fields are only ever added. Unrecognised fields are to be ignored.
• If a topic is ever lost, everything can be republished from the database: POST /api/admin/feed/backfills.

---

0. The round trip in one picture

  INBOUND — backend ➜ model                                              §1–§6
  ──────────────────────────────────────────────────────────────────────────────
  a row commits
  (post · product · shop · event · profile · follow · block · order · wishlist)
        └─► outbox ─► relay ─► 9 entity topics ......... compacted, "what exists"

  the app calls POST /api/v1/feed/interactions/batch        (impressions, views…)
  a row commits (like · comment · cart · order paid)        (seen by the backend)
        └─► nexgate.interactions.v1 ................. 90 days, "what people did"

  a feed page is served
        └─► nexgate.feed-served.v1 .................. 14 days, "what was shown"

                                    │
                                    ▼
                    ┌───────────────────────────────┐
                    │    RECOMMENDATION SERVICE     │
                    │   owned by the ML team       │
                    └───────────────┬───────────────┘
                                    │
  OUTBOUND — model ➜ backend        │                                       §7
  ──────────────────────────────────┼───────────────────────────────────────────
                                    ▼
        SET reco:<surface>:{id} <json> EX 172800        (8 keys, all optional)
                                    │
                                    ▼
        the feed pipeline reads reco:* while building a session;
        missing · invalid · stale ─► trending and interest pools fill in

The boundary never moves. The backend only ever writes Kafka and reads reco:*. The recommendation service only ever reads Kafka and writes reco:*. Neither side calls the other over HTTP, so neither can be made slow or unavailable by the other.

---

1. The envelope

Every message on the nine entity topics has the same eight fields:

{
  "eventId": "9f2c1e8a-5b3d-4a91-8c77-2e6f0b4d1a35",
  "eventType": "product.updated",
  "entityId": "4f2a77e1-0c8b-4d2f-9a11-7b6c3e5d8f90",
  "entityVersion": 184022,
  "occurredAt": "2026-09-18T10:22:33.120Z",
  "schemaVersion": 1,
  "source": "nexgate-backend",
  "payload": { }
}

Field	Meaning
eventId	Unique per event. A repeat indicates a retry and is to be discarded
eventType	One of the 27 in this document
entityId	The thing this is about
entityVersion	Only ever increases per entity. The highest value seen is authoritative; lower values are ignored
occurredAt	When it happened. Not an ordering key — clocks and retries make it unreliable for that
schemaVersion	1 today
source	Always nexgate-backend
payload	Full current state, or, for a *.deleted, the id plus a reason

The interactions and feed-served topics use a slightly different envelope — noted in their sections.

2. How a delete arrives

Always two messages, in this order:

1. <entity>.deleted with a reason — the entity is no longer available to consumers, and the reason states why.
2. A tombstone: same key, value = null — so log compaction can drop the key.

"Deleted" does not always mean the row was deleted. It means the entity is no longer something to recommend. A post set to private, a shop that closed, an account that was locked all arrive as *.deleted, with a reason that distinguishes them. If the entity becomes visible again, *.created follows.

Tombstones are retained for 1 day. A consumer reading a compacted topic from offset 0 after that interval sees only live entities.

---

3. The topics at a glance

#	Topic	Key	Cleanup	Partitions	Event types
1	nexgate.posts.v1	postId	compact	24	post.created · post.updated · post.deleted
2	nexgate.products.v1	productId	compact	24	product.created · product.updated · product.deleted
3	nexgate.shops.v1	shopId	compact	12	shop.created · shop.updated · shop.deleted
4	nexgate.events.v1	eventId	compact	12	event.created · event.updated · event.deleted
5	nexgate.profiles.v1	accountId	compact	24	profile.created · profile.updated · profile.deleted
6	nexgate.follows.v1	followerId:followedId	compact	24	follow.created · follow.deleted
7	nexgate.blocks.v1	blockerId:blockedId	compact	12	block.created · block.deleted
8	nexgate.orders.v1	orderId	compact	24	order.created · order.updated · order.deleted
9	nexgate.wishlist.v1	accountId:productId	compact	24	wishlist.created · wishlist.updated · wishlist.deleted
10	nexgate.interactions.v1	accountId	delete, 90 d	96	interaction
11	nexgate.feed-served.v1	accountId	delete, 14 d	48	feed.served
12	nexgate.fanout-tasks.v1	postId:chunk	delete, 3 d	48	(internal — not for consumers)

Partition counts are fixed for good. Adding partitions later would move keys to different partitions and break per-key ordering.

Compacted topics are a live catalogue: a read from offset 0 yields the current state of every entity, and continued consumption keeps it current.

---

4. Entity topics

Each section below is complete: every field of the payload, its type, whether it can be null, its allowed values where it is an enum, and the exact shape of the *.deleted payload.

Type column: uuid, string, int, decimal, bool, timestamp (ISO 8601 UTC), json (an opaque object whose structure may change and should not be relied upon). A field marked nullable may arrive as null; one marked required is present on every message of that topic.

Fields are only ever added. The schemas are open by design: unrecognised fields are ignored, and no message should be rejected because of one.

---

4.1 nexgate.posts.v1 — posts and reels

postId · compacted · 24 partitions · schema src/main/resources/feed-schemas/nexgate.posts.v1.json

A post is available to consumers while it is PUBLISHED, not deleted, and PUBLIC or FOLLOWERS.

Event	Fires when
post.created	A draft becomes published, or a hidden post becomes visible again
post.updated	A visible post, or any of its child rows, changes
post.deleted	It stops being visible — only if it was visible before

Delete reasons: DELETED (soft-deleted) · UNPUBLISHED (no longer PUBLISHED) · VISIBILITY_RESTRICTED (switched to MENTIONED or PRIVATE)

What counts as a change. Not just the post row — eleven child tables are watched, and a change to any one republishes the whole post: media, hashtags, attached products, attached shops, attached events, user mentions, shop mentions, links, collaborators, and the poll.

What publishes nothing: likesCount, commentsCount, repostsCount, bookmarksCount, viewsCount, quotesCount, sharesCount, shortClipCount, lastVerifiedAt, updatedAt. Engagement counters move constantly; republishing every post on every like would be most of the topic's traffic for none of the value. The counts in the payload are therefore a snapshot taken at the last substantive change, not a live figure. Live behaviour is carried by nexgate.interactions.v1.

A draft that is saved, edited and never published produces nothing at all.

Payload — post.created / post.updated

Field	Type	Notes
postId	uuid	required
postType	string, nullable	REGULAR · POLL
status	string, nullable	PUBLISHED on every message delivered; DRAFT, SCHEDULED and DELETED are never published
author.id	uuid, nullable	Who wrote it
author.userName	string, nullable	Public username. No real name is ever published
content	string, nullable	The post text as written
contentParsed.hashtags[]	string[]	Normalised, without the #. Any script (Latin, Arabic, CJK …)
contentParsed.mentionedUserIds[]	uuid[]	Accounts mentioned in the text
contentParsed.mentionedShopIds[]	uuid[]	Shops mentioned in the text
media[].fileId	uuid, nullable	File-service id
media[].mediaType	string, nullable	IMAGE · VIDEO
media[].order	int	Display order within the post, from 0
media[].status	string, nullable	Processing state. PENDING means variants are not ready yet
media[].variants	json, nullable	Rendition URLs keyed by name (hls, thumb, …). Opaque — read keys, do not assume the set
media[].shortClip	bool	true = this is a reel. There is no separate reel topic
media[].mediaId	uuid, nullable	The clip's own id, as used by the reel endpoints
media[].durationMs	int, nullable	Length in milliseconds, for video
attachments.products[]	uuid[]	Products attached to the post
attachments.shops[]	uuid[]	Shops attached
attachments.events[]	uuid[]	Events attached
engagement.likesCount	int	Snapshot, see above
engagement.commentsCount	int	Snapshot
engagement.sharesCount	int	Snapshot
engagement.viewsCount	int	Snapshot
engagement.bookmarksCount	int	Snapshot
engagement.repostsCount	int	Snapshot
engagement.quotesCount	int	Snapshot
privacySettings.visibility	string, nullable	PUBLIC or FOLLOWERS on every message delivered
privacySettings.whoCanComment	string, nullable	Who may comment
privacySettings.whoCanRepost	string, nullable	Who may repost
quotedPostId	uuid, nullable	Set when this post quotes or reposts another. A pure repost carries the original's engagement, not its own
collaboratorIds[]	uuid[]	Co-authors
hasPoll	bool	Whether a poll is attached. Poll options and votes are not published
externalLinkDomain	string, nullable	The domain only, never the full URL
isEdited	bool	Whether it has been edited since publishing
createdAt	timestamp, nullable	When the row was created (may be long before publishing)
publishedAt	timestamp, nullable	When the post went live. This is the field for recency, not createdAt
updatedAt	timestamp, nullable	Last change
interests[].id	uuid	Interest id from the shared list (interest_categories)
interests[].w	decimal	Weight: 1.0 from attached products and events, 0.6 from hashtags. An interest reached twice keeps the higher weight. Empty until an administrator maps the category or hashtag

Payload — post.deleted

Field	Type	Notes
postId	uuid	required
authorId	uuid, nullable	The author, or null when the row itself is gone
reason	string	One of the delete reasons above

Followed by a tombstone: same key, value = null.

---

4.2 nexgate.products.v1 — products

productId · compacted · 24 partitions

A product is available to consumers while it is ACTIVE or OUT_OF_STOCK, not deleted, and its shop is visible. Running out of stock is an update, not a disappearance: stock is carried in the payload and the consumer decides how to treat it.

Event	Fires when
product.created	It becomes visible, including when its shop comes back
product.updated	The product or its installment plan changes
product.deleted	It stops being visible

Delete reasons: DELETED · UNPUBLISHED (DRAFT, INACTIVE, ARCHIVED) · SHOP_UNAVAILABLE (shop suspended, closed, unapproved or deleted)

The shop cascade. When a shop's visibility flips, every product of that shop is republished, in pages, at that moment. One suspension can produce thousands of product.deleted messages. Deliberate: a consumer watching only this topic would otherwise keep recommending products from a suspended shop.

What publishes nothing: viewCount, cartAddCount, updatedAt.

Payload — product.created / product.updated

Field	Type	Notes
productId	uuid	required
productName	string, nullable
productSlug	string, nullable	URL slug
productDescription	string, nullable
productType	string, nullable	PHYSICAL · DIGITAL
productMedia[].fileId	uuid, nullable
productMedia[].mediaType	string, nullable	IMAGE · VIDEO
productMedia[].order	int, nullable	Display order
productMedia[].status	string, nullable	Processing state
productMedia[].variants	json, nullable	Rendition URLs
categoryId	uuid, nullable	The key to interests — mapped by an admin
categoryName	string, nullable
parentCategoryId	uuid, nullable	A category with no mapping inherits its parent's interests
price	decimal, nullable	Current price
comparePrice	decimal, nullable	"Was" price
isOnSale	bool
discountPercentage	decimal, nullable
stockQuantity	int
isInStock	bool
isLowStock	bool
showStockToPublic	bool	Whether the shop displays the number
status	string, nullable	ACTIVE or OUT_OF_STOCK on every message delivered
condition	string, nullable	NEW · USED_LIKE_NEW · USED_GOOD · USED_FAIR · REFURBISHED · FOR_PARTS
urgencyTag	string, nullable	NONE · NEW_ARRIVAL · LIMITED_EDITION · LIMITED_OFFER · FEW_REMAINS
specifications	json, nullable	Free-form, seller-entered. Opaque
colors	json, nullable	Free-form, seller-entered. Opaque
hasGroupBuying	bool
groupPrice	decimal, nullable	Price when bought as a group
hasInstallments	bool
stockInfo.soldCount	int	Units sold
shopId	uuid, nullable
shopName	string, nullable
createdAt	timestamp, nullable
updatedAt	timestamp, nullable
interests[].id	uuid	From the product's category, weight 1.0
interests[].w	decimal

Payload — product.deleted

Field	Type	Notes
productId	uuid	required
shopId	uuid, nullable	Useful when the cause was the shop
reason	string

---

4.3 nexgate.shops.v1 — shops

shopId · compacted · 12 partitions

A shop is available to consumers while it is ACTIVE or TEMPORARILY_OFFLINE, approved, and not deleted.

Event	Fires when
shop.created	It becomes visible
shop.updated	Anything meaningful on the shop changes
shop.deleted	It stops being visible

Delete reasons: DELETED · SUSPENDED · CLOSED (PERMANENTLY_CLOSED) · UNPUBLISHED (PENDING, or approval withdrawn)

Every one of these also republishes the shop's products (§4.2).

What publishes nothing: subscriberCount, lastSeenTime, updatedAt.

Payload — shop.created / shop.updated

Field	Type	Notes
shopId	uuid	required
shopName	string, nullable
shopSlug	string, nullable
shopDescription	string, nullable
logo.fileId / .mediaType / .status / .variants	uuid / string / string / json, nullable	The shop logo
banner.fileId / .mediaType / .status / .variants	uuid / string / string / json, nullable	The shop banner
ownerId	uuid, nullable	The account that owns the shop
status	string, nullable	ACTIVE or TEMPORARILY_OFFLINE on every message delivered
tempOfflineUntil	timestamp, nullable	Set while the owner has gone offline deliberately
city	string, nullable
district	string, nullable
region	string, nullable	The feed filters by region — this is why location stays
countryCode	string, nullable
isVerified	bool
verificationBadge	string, nullable	Which badge
trustScore	decimal, nullable	Platform trust score
subscriberCount	int	Snapshot — a change to it alone does not republish
approvedAt	timestamp, nullable
createdAt	timestamp, nullable
updatedAt	timestamp, nullable

Payload — shop.deleted

Field	Type	Notes
shopId	uuid	required
reason	string

Never published: phone, email, street address, landmark, coordinates. Many shops are run from someone's home.

---

4.4 nexgate.events.v1 — events

eventId · compacted · 12 partitions

An event is available to consumers while it is PUBLISHED or HAPPENING, PUBLIC, and not deleted.

Event	Fires when
event.created	It becomes visible
event.updated	The event or its tickets change — including selling out, which is an update, not a removal
event.deleted	It stops being visible

Delete reasons: DELETED · ENDED (COMPLETED) · CANCELLED · UNPUBLISHED (DRAFT or any other status) · VISIBILITY_RESTRICTED (PRIVATE or UNLISTED)

Ticket statistics count sold tickets only. Reservations change on every checkout attempt and expire by themselves, so publishing them would be noise.

What publishes nothing: updatedAt, currentStage, completedStages, rsaKeys.

Payload — event.created / event.updated

Field	Type	Notes
eventId	uuid	required
title	string, nullable
slug	string, nullable
description	string, nullable
category.id	uuid, nullable	The key to interests
category.name	string, nullable
bannerMedia	json, nullable	Opaque media object
eventFormat	string, nullable	IN_PERSON · ONLINE · HYBRID · TBA
visibility	string, nullable	PUBLIC on every message delivered
status	string, nullable	PUBLISHED or HAPPENING on every message delivered
schedule.startDateTime	timestamp, nullable
schedule.endDateTime	timestamp, nullable
schedule.timezone	string, nullable	IANA zone
venue.name	string, nullable
venue.address	string, nullable	A public venue, unlike a shop's address
venue.latitude / .longitude	decimal, nullable
pricing.minPrice / .maxPrice	decimal, nullable	Across ticket types
pricing.pricingType	string, nullable	How the event is priced
stats.ticketsSold	int	Sold only — never reservations
stats.ticketsAvailable	int, nullable
stats.isSoldOut	bool	Selling out arrives as event.updated
organizerId	uuid, nullable
organizerName	string, nullable
linkedProductIds[]	uuid[]	Products tied to the event
linkedShopIds[]	uuid[]	Shops tied to the event
createdAt	timestamp, nullable
publishedAt	timestamp, nullable
updatedAt	timestamp, nullable
interests[].id / .w	uuid / decimal	From the event's category, weight 1.0

Payload — event.deleted

Field	Type	Notes
eventId	uuid	required
reason	string

Never published: the virtual meeting link, meeting id or passcode (they admit people into the room), and never the event's signing keys.

---

4.5 nexgate.profiles.v1 — public profile facts

accountId · compacted · 24 partitions

A profile is available to consumers once sign-up is complete (a real username, not a temp_… placeholder) and while the account is not locked.

Event	Fires when
profile.created	Sign-up completes, or a locked account is unlocked
profile.updated	One of the watched fields changes, or the person's declared interests change
profile.deleted	The account is locked, or the row is gone

Delete reasons: LOCKED · DELETED

Only these changes republish: userName, accountType, accountTier, locked, isVerified, verificationBadgeType, and rows in the user's declared interests. A login, a password change or a new profile photo publishes nothing.

Payload — profile.created / profile.updated

Field	Type	Notes
accountId	uuid	required
userName	string, nullable	Public username. Never temp_… on a delivered message
accountType	string, nullable	NORMAL · SYSTEM · VERIFIED
accountTier	string, nullable	FULL · RESTRICTED · MINOR. MINOR denotes a minor's account and warrants conservative treatment
isVerified	bool
verificationBadgeType	string, nullable	Which badge
followerCount	int	Snapshot
followingCount	int	Snapshot
declaredInterestIds[]	uuid[]	What the person chose at onboarding, on the shared interest list
createdAt	timestamp, nullable	Account age — useful for cold-start

Payload — profile.deleted

Field	Type	Notes
accountId	uuid	required
reason	string	LOCKED or DELETED

Never published: first, middle or last name, bio, location, phone, email, birth date, profile photo.

⚠️ The active column must not be used, wherever it is encountered. Sign-up sets it false and nothing ever sets it true, so it is false on every real account. It is deliberately not in the payload — relying on it would have tombstoned every profile on the platform.

---

4.6 nexgate.follows.v1 — who follows whom

Key followerId:followedId · compacted · 24 partitions

Only ACCEPTED follows are published. A follow request (PENDING) emits nothing; if it is never accepted, nothing is emitted at all.

Event	Fires when
follow.created	A request is accepted, or a public account is followed
follow.deleted	Unfollowed, or a request withdrawn — only if it had been accepted

Payload — follow.created

Field	Type	Notes
followerId	uuid	required — the one doing the following
followedId	uuid	required — the one being followed
followedAt	timestamp, nullable	When it was accepted

Payload — follow.deleted

followerId (uuid, required) · followedId (uuid, required) · reason (string)

---

4.7 nexgate.blocks.v1 — who blocked whom

Key blockerId:blockedId · compacted · 12 partitions

A block is published the moment the row is written — there is no visibility condition.

Event	Fires when
block.created	Someone blocks someone
block.deleted	They unblock

This topic is not optional. A block hides content in both directions: neither party may be recommended the other's content.

Payload — block.created

Field	Type	Notes
blockerId	uuid	required — who blocked
blockedId	uuid	required — who was blocked
blockedAt	timestamp, nullable

Payload — block.deleted

blockerId (uuid, required) · blockedId (uuid, required) · reason (string)

Mutes are not published on Kafka. A mute is a per-viewer preference applied inside the feed and is not an entity. It does appear as a MUTE_AUTHOR interaction (§5).

---

4.8 nexgate.orders.v1 — what was actually bought

orderId · compacted · 24 partitions

An order is published from creation onwards. This is the strongest purchase signal on the platform.

Event	Fires when
order.created	An order is placed
order.updated	One of the watched fields changes
order.deleted	The order is soft-deleted (reason DELETED)

Only these changes republish: productOrderStatus, deliveryStatus, totalAmount, currency, shippedAt, deliveredAt, completedAt, cancelledAt, isDeleted.

Payload — order.created / order.updated

Field	Type	Notes
orderId	uuid	required
orderNumber	string, nullable	Human-readable reference
buyer.accountId	uuid, nullable	Who bought
shopId	uuid, nullable	Who sold
items[].productId	uuid, nullable
items[].quantity	int
items[].unitPrice	decimal, nullable
items[].subtotal	decimal, nullable
productOrderStatus	string, nullable	PENDING_PAYMENT · PENDING_SHIPMENT · SHIPPED · DELIVERED · AWAITING_BUYER_CONFIRM · COMPLETED · DISPUTED · CANCELLED · REFUNDED
deliveryStatus	string, nullable	PENDING · SHIPPED · DELIVERED · CONFIRMED · IN_TRANSIT · NOT_APPLICABLE
productOrderSource	string, nullable	DIRECT_PURCHASE · DIGITAL_PURCHASE · INSTALLMENT · GROUP_PURCHASE · CART_PURCHASE · CHAT_OFFER
totalAmount	decimal, nullable
currency	string, nullable
orderedAt	timestamp, nullable
shippedAt	timestamp, nullable
deliveredAt	timestamp, nullable
completedAt	timestamp, nullable	The strongest signal available: payment settled and receipt confirmed by the buyer
cancelledAt	timestamp, nullable
updatedAt	timestamp, nullable

An order leaving PENDING_PAYMENT for any status other than CANCELLED or REFUNDED also produces a PURCHASE interaction per order item (§5). The order topic describes what the order is; the interaction records when the purchase occurred.

Payload — order.deleted

Field	Type	Notes
orderId	uuid	required
reason	string	DELETED

Never published: delivery address or instructions, order notes, cancellation text, payment method, escrow or fee figures, tracking number, confirmation code, chat ids.

Event ticket bookings are not on this topic. nexgate.orders.v1 is product orders only. See §9.

---

4.9 nexgate.wishlist.v1 — saved products

Key accountId:productId · compacted · 24 partitions

A product can be in a person's wishlist once, whatever group it is filed under.

Event	Fires when
wishlist.created	A product is added
wishlist.updated	The entry changes (for example, moved to another group)
wishlist.deleted	It is removed

Payload — wishlist.created / wishlist.updated

Field	Type	Notes
accountId	uuid	required
productId	uuid	required
wishlistId	uuid, nullable	The row's own id
groupId	uuid, nullable	Which of the user's lists it sits in
addedAt	timestamp, nullable

Payload — wishlist.deleted

accountId (uuid, required) · productId (uuid, required) · reason (string)

Never published: the group's name. It is free text a person writes about themselves ("Gifts for Mum"). The group id is enough to tell groups apart.

---

5. nexgate.interactions.v1 — what people did

accountId · delete, 90 days · 96 partitions · schema nexgate.interactions.v1.json

One event type, interaction. Keyed by accountId so one person's actions stay in order.

This is the only topic anything outside the backend can write to, and it does so through exactly one endpoint: POST /api/v1/feed/interactions/batch (09_NEW_ENDPOINTS.md A8). Every other topic is written from committed database rows, so no client can announce that an entity exists or changed.

The envelope is different here

No entityId and no entityVersion — an interaction is not an entity. Instead:

Field	Type	Notes
eventId	uuid	required. Deterministic (UUIDv5 of account + clientEventId), so a re-publish of the same action produces the same id. Deduplicate on it
eventType	string	Always the literal interaction
occurredAt	timestamp	The device's clock, clamped so it can never be in the future
receivedAt	timestamp	The server's clock, when it arrived
schemaVersion	int	1
source	string	nexgate-backend
payload	object	Below

{
  "eventId": "…", "eventType": "interaction",
  "occurredAt": "2026-09-18T10:22:41.100Z", "receivedAt": "2026-09-18T10:22:41.480Z",
  "schemaVersion": 1, "source": "nexgate-backend",
  "payload": {
    "accountId": "…", "sessionId": "s-8812e0c4", "clientEventId": "c-000185",
    "action": "VIEW", "targetType": "POST", "targetId": "…",
    "context": { "surface": "REELS", "position": 3, "feedSessionId": "fs-77aa", "source": "RECO", "viaPostId": null, "query": null },
    "media": { "dwellMs": null, "watchMs": 9000, "mediaDurationMs": 6000, "loopCount": 1, "soundOn": true, "mediaIndex": 0 },
    "client": { "platform": "ANDROID", "appVersion": "2.4.1", "networkType": "CELLULAR" },
    "origin": "CLIENT"
  }
}

Payload — every field

Field	Type	Notes
accountId	uuid	required. Who did it
sessionId	string, nullable	The app session, max 64 chars. null for server-produced events
clientEventId	string	required. The app's own id, or srv:<action>:<rowId> when the backend produced it
action	string	required. See the table below
targetType	string, nullable	POST · PRODUCT · SHOP · EVENT · PROFILE. null only for SEARCH
targetId	uuid, nullable	Lower-cased. null only for SEARCH
context.surface	string, nullable	FEED · REELS · SEARCH · PRODUCT_PAGE · SHOP_PAGE · EVENT_PAGE · PROFILE_PAGE · RECOMMENDATION · NOTIFICATION · CHAT
context.position	int, nullable	Where in the feed the item sat, from 0
context.feedSessionId	string, nullable	Joins to nexgate.feed-served.v1 — max 64 chars
context.source	string, nullable	Why it was shown: FOLLOWING · CELEBRITY · RECO · INTEREST · FALLBACK · SPONSORED · TRENDING · SEARCH. RECO indicates the item came from a supplied list
context.viaPostId	uuid, nullable	The post a product was reached through
context.query	string, nullable	The search text, for SEARCH
media.dwellMs	int, nullable	Time on screen. 0 – 6 h
media.watchMs	int, nullable	Time watched. 0 – 6 h
media.mediaDurationMs	int, nullable	Length of the media. 0 – 12 h
media.loopCount	int, nullable	Replays. 0 – 1000
media.soundOn	bool, nullable	Whether sound was on
media.mediaIndex	int, nullable	Which item of a carousel. 0 – 100
client.platform	string, nullable	ANDROID · IOS · WEB
client.appVersion	string, nullable	e.g. 2.4.1
client.networkType	string, nullable	WIFI · CELLULAR · OFFLINE · UNKNOWN
origin	enum, nullable	CLIENT or SERVER

media and client are always null on server-produced events — the backend did not witness a screen.

5.1 Two origins, never overlapping

origin	Produced by	clientEventId
CLIENT	The app, in a batch	the app's own id
SERVER	The backend, watching committed rows	srv:<action>:<rowId>

The split is enforced: the batch API rejects every server-owned action with SENT_BY_SERVER, so nothing is counted twice.

5.2 Every action, who sends it, and what triggers it

Action	Origin	Target	Exact trigger
IMPRESSION	CLIENT	any	The item was on screen. Use it as the denominator
VIEW	CLIENT	any	Real attention: dwell ≥ 5 s, or a reel watched ≥ 50 %
SKIP	CLIENT	any	Scrolled past quickly
CLICK	CLIENT	any	Opened the item
SEARCH	CLIENT	none	A search was run — what matters is context.query
NOT_INTERESTED	CLIENT	any	Explicit negative
HIDE	CLIENT	any	Explicit negative, stronger
LIKE	SERVER	POST	A post_likes row inserted
UNLIKE	SERVER	POST	A post_likes row deleted
COMMENT	SERVER	POST	A post_comments row inserted
BOOKMARK	SERVER	POST	A post_bookmarks row inserted
SHARE	SERVER	POST	A post_shares row inserted
REPORT	SERVER	POST	A post_reports row inserted
BLOCK	SERVER	PROFILE	A user_blocks row inserted
MUTE_AUTHOR	SERVER	PROFILE	A user_mutes row inserted
ADD_TO_CART	SERVER	PRODUCT	A cart_items row inserted
REMOVE_FROM_CART	SERVER	PRODUCT	A cart_items row deleted
PURCHASE	SERVER	PRODUCT	An order left PENDING_PAYMENT for any status other than CANCELLED/REFUNDED — one event per order item
PURCHASE	SERVER	EVENT	An event booking became CONFIRMED

Server events are sent after the transaction commits. A rolled-back like sends nothing, and a like made through any code path is captured — nothing depends on a service remembering to emit.

5.3 Media measurements are raw, on purpose

watchMs: 9000 against mediaDurationMs: 6000 with loopCount: 1 is a rewatch, not bad data. Nothing is clamped to "100 %" before delivery, because the fact that someone watched a clip one and a half times is exactly the signal worth having.

5.4 This topic is best-effort by design

If the event log is briefly unavailable, interactions are dropped and counted, not queued — they are signal, not records. A missing view is a missing sample; a queued backlog of stale views would be worse. The entity topics are the opposite: those are never lost, and can always be rebuilt with a backfill.

---

6. nexgate.feed-served.v1 — what the feed actually showed

accountId · delete, 14 days · 48 partitions · no registered schema

One event type, feed.served. This is the ground truth for ranking evaluation: what was shown, in what order, and why.

The envelope

No entityId, no entityVersion, no receivedAt: eventId, eventType (feed.served), occurredAt, schemaVersion (1), source, payload.

{
  "eventId": "…",
  "eventType": "feed.served",
  "occurredAt": "2026-09-18T10:22:31.900Z",
  "schemaVersion": 1,
  "source": "nexgate-backend",
  "payload": {
    "feedSessionId": "fs-77aa31c8",
    "accountId": "…",
    "surface": "FEED",
    "items": [
      { "type": "POST", "id": "4f2a77e1-…", "position": 0, "source": "FOLLOWING" },
      { "type": "PRODUCT", "id": "8c31a0d2-…", "position": 1, "source": "RECO" }
    ]
  }
}

Payload — every field

Field	Type	Notes
feedSessionId	string	Joins to context.feedSessionId on interactions
accountId	uuid	Who it was served to
surface	string	Currently always the literal "FEED" — see the warning below
items[].type	string	POST · PRODUCT
items[].id	uuid	The item
items[].position	int	Its place on the page, from 0
items[].source	string	Why it was there: FOLLOWING · CELEBRITY · RECO · INTEREST · FALLBACK · SPONSORED · TRENDING

Joined to the interactions topic on feedSessionId, it gives, per session: what was shown, in which position, from which source, and what the viewer did about it.

⚠️ Published by the home feed only, and surface is hard-coded to "FEED". Reels, marketplace and events do not publish it. See §9.

7. The way back — what the model writes to Redis

Everything above travels backend → model. This section is the other direction: model → backend.

There is no API call and no callback. The recommendation service writes plain Redis keys; the feed reads them when it builds a session. If the model stops writing, the feed keeps working on trending and interest pools — quieter, never broken.

7.1 The eight keys

Key	Read by	Items must be	When it is read
reco:home:posts:{accountId}	Home feed discovery	post ids	every home session build
reco:home:products:{accountId}	Home feed discovery	product ids	every home session build
reco:reels:{accountId}	Reels feed	post ids (with a short video)	every reels session build
reco:market:products:{accountId}	Marketplace "For You"	product ids	every marketplace session build
reco:events:{accountId}	Events "For You"	event ids	every events session build
reco:shops:{accountId}	Suggested shops strip	shop ids	every call to the strip
reco:similar:products:{productId}	"Similar items" on a product page	product ids	every call to the strip
reco:meta:health	backend alerting only	—	on every metrics scrape

The {…} braces are part of the key — they are Redis Cluster hash tags, and they decide which node the key lives on. Write them exactly as shown.

Every key is optional. A surface whose key is absent runs on trending and interest pools instead. You can ship one surface at a time.

7.2 The value

One JSON string, written with a single SET key <value> EX 172800 (48 h). One command means a reader sees either the whole old list or the whole new one, never half of each.

{
  "v": 1,
  "modelVersion": "ranker-2026-09-14",
  "generatedAt": "2026-09-18T02:10:00Z",
  "items": [
    { "id": "4f2a77e1-0c8b-4d2f-9a11-7b6c3e5d8f90", "score": 0.83 },
    { "id": "9b10c2d4-7e3f-4a52-b8c1-6d9e0f2a3b47", "score": 0.79 }
  ]
}

Field	Required	Rule
generatedAt	Yes	ISO 8601. Without it the whole list is INVALID and ignored
items	Yes	Must be an array. Best first. At most 300 are read; the rest are discarded
items[].id	Yes	A UUID of the right type for that key. Not a UUID → that item is skipped. Unknown or deleted → dropped silently at read time
items[].score	No	Kept when inside [0, 1]. Missing or out of range → rank is used instead: 1 / (1 + rank), so position 0 → 1.0, 1 → 0.5, 3 → 0.25
modelVersion	No	Carried into the backend debug trace, so a feed can be explained by the model that produced it
v	No	Format version

After each successful batch, write the health key:

SET reco:meta:health '{"finishedAt":"2026-09-18T02:40:00Z","modelVersion":"ranker-2026-09-14"}'

7.3 What the backend does with each list

Read, then judged in one of four states:

State	Condition	Used?
FRESH	generatedAt within reco.maxAgeHours (default 30 h)	Yes
STALE	Older than that, key still present	Yes — a stale list beats no list. Counted on the backend dashboards
MISSING	No key	No → fallback pools fill discovery
INVALID	Not JSON, no items array, or no generatedAt	No → same as missing, but counted separately, so a broken writer is visible rather than silent

Every read is counted as feed.reco.lists{list, state}, so the state of each list is visible per surface: fresh, stale, missing or invalid.

How recommended items rank against fallbacks. A reco item's score becomes 1 + the supplied score. Fallback items score inside [0, 1]. Every recommended item therefore outranks every fallback item in the discovery lane, and the pools fill only what was not supplied. Where the same post appears in both, the RECO copy is kept.

Where they land on the home feed. Your items fill the discovery slots, interleaved with posts from people the user follows in the pattern F, F, D, F, D, F, F, D — never the same author twice within 4 slots. A session holds 120 items and is rebuilt on pull-to-refresh or when the user reaches the end, so a newly written list appears at the next session, not mid-scroll.

The read budget is 200 ms. Each reco read has a 200 ms timeout. Past that the source returns nothing for that session and the pools fill in. A slow Redis degrades the feed's quality, never its availability.

7.4 Access

You get a Redis user that can write only reco:*:

ACL SETUSER reco-writer on >******** ~reco:* -@all +set +get +del +expire +exists +ping +cluster|slots +cluster|shards +cluster|nodes

Anything outside reco:* answers NOPERM. Locally: user reco-writer, password reco_writer_local, seeds localhost:7001,localhost:7002.

Local docker-compose trap. The containers announce internal addresses (172.29.0.11–16). A cluster client outside the compose network follows those announcements and cannot reach the nodes. The writer should run in a container on the nexgate-feed-redis network, or on the host network.

7.5 Verifying the integration

Signal	Where	Means
feed.reco.lists{state="FRESH"} rising	backend dashboard	the lists are being read and used
feed.reco.lists{state="INVALID"} above zero	backend dashboard, alert after 15 min	the writer is producing something unparseable
feed.reco.health.age.seconds	backend dashboard, alert above 26 h	reco:meta:health.finishedAt is stale, indicating the batch has stopped. -1 means no batch has ever been reported
source: "RECO" in a feed response	the app, and GET /api/admin/feed/debug/{accountId}	an item on a live screen originated from a supplied list

The per-account debug trace is the fastest way to prove the loop end to end: it shows every item the last session produced, its source, its base score and its final score — and every item a filter removed, with the filter's name.

---

8. Who does what — do not rebuild this

The backend is not a thin pipe that hands the feed to a model. It performs a great deal of work after a list is read, and most of it should not be duplicated — partly because the effort is wasted, and partly because the backend must do it regardless for correctness, so a second implementation can only diverge.

8.1 The backend already does all of this

Visibility and safety — applied to every candidate, whatever its source:

Filter	Removes
Blocked	either direction of a block
Muted	authors the viewer muted
Self	the viewer's own items
Status	deleted, unpublished, no longer visible
DiscoveryPublic	followers-only items reached through discovery
HiddenItem	moderation labels (§4.1, sensitive_media becomes an interstitial rather than a removal)
Stock	out-of-stock products where that matters
Age	items past the surface's age limit
UnfollowedAuthor	timeline leftovers from someone since unfollowed
Seen / Served	anything already shown, per surface
HydrationFailed	items whose card could not be loaded

Blocked authors, muted authors, deleted items and already-seen items may therefore be left in a submitted list; the backend removes them regardless. Model capacity spent on exclusions is wasted. nexgate.blocks.v1 is worth consuming for training — a block is a strong negative signal — rather than for filtering the output list.

Ranking, mixing and presentation — applied after the supplied scores are read:

• Lane interleaving per surface. Home is F,F,D,F,D,F,F,D (follow / discovery), reels D,D,F, marketplace D,F,D,D, events F,D,D. You supply the discovery lane; the in-network lane is ours.
• Author diversity. An item whose author appeared in the last few slots waits for the first slot where it fits, rather than being dropped.
• Recency, NewShopBoost and TimeToStart (events) as scorers applied on top of the supplied score.
• Sponsored slots, business rules and per-viewer ad caps — commercial placement is not a modelling concern.
• Sessions, cursors and paging. A session is built once and paged from; filters re-run at page-read time so something deleted mid-scroll disappears.

Cold start and fallbacks — a viewer with no history, or a surface whose list is missing, stale, invalid or fully filtered, is served from trending pools, interest pools and recency. You never need to produce a list for everyone, and a gap costs quality, not availability.

An interest profile per user, without a model — every item carries interests: [{id, w}], and the backend maintains interest:{accountId} from the interaction stream with decay, a floor for declared interests, and negative weights for HIDE / NOT_INTERESTED. It exists so that discovery works before any model is delivered, and so that a feed can be explained in plain language. It is available as a feature and is not to be reimplemented — though a learned representation over the raw stream is expected to outperform it, which is the point of §C.5.3.

8.2 Recommendation work the backend already does today

This is the part most worth reading, because it is recommendation logic, not plumbing — and it is live right now, without a model. Everything here is a baseline that may be used as a feature, or improved upon, but should not be rebuilt.

Discovery pools, rebuilt every 10 minutes (FeedPoolsJob, one container at a time). Each is built into a temporary key and RENAMEd into place, so a reader never sees a half-built pool, and each expires after an hour — if the job stops, discovery empties rather than serving hours-old "trending".

Pool	How it is computed
pool:{trending:posts}	the last 24 hourly buckets summed with weight 0.9 ^ hoursAgo, top 500. This hour counts ×1, an hour ago ×0.9, two hours ago ×0.81
pool:{trending:products}	same decay, over product engagement
pool:{trending:reels}	trending posts that have a READY short clip, same scores
pool:{new:reels}	newest public clips of the last 7 days, scored by publish time
pool:{newshops}	products of shops younger than 30 days with fewer than 2 000 impressions, newest first, 200
pool:{events:upcoming}	public events starting within 60 days or happening now, scored 1 + confirmed bookings in the last 7 days
pool:shops:popular	shops by subscriber count
pool:cat:{categoryId}	best sellers of a category — the fallback behind "similar items"
pool:{int:<interestId>}:posts	trending posts carrying that interest, score = trending score × interest weight
pool:{int:<interestId>}:products	in-stock products of that interest's categories, by quantity sold then newest
pool:{int:<interestId>}:events	upcoming events of that interest's categories, by start date

A per-user interest profile, maintained from the same interaction stream (interest:{accountId}, flushed to user_interest_scores):

score[interest] = decay(score[interest]) + actionWeight × itemWeight
decay halves every 14 days

with VIEW 1, LIKE/BOOKMARK 2, COMMENT/SHARE 3, ADD_TO_CART 4, purchase 6–8, NOT_INTERESTED/HIDE −5, and a floor of 5 for interests declared at onboarding. Each surface reads the user's top 5 interests in proportion to score, plus 10 % explore from outside the top 5 so the profile cannot close in on itself.

The consequence: a user who has never been modelled still receives interest-based discovery, and every surface has a defensible answer on day one. A supplied list does not compete with an empty feed; it competes with this baseline. Where both exist, the supplied list wins: a reco item scores 1 + the supplied score, while every pool item scores inside [0, 1].

Fallbacks per surface, used when a key is missing, stale, invalid or fully filtered:

Surface	Falls back to
Home discovery	trending posts + trending products + interest pools
Reels	trending reels, then new reels
Marketplace	interest products, trending products, new shops, products seen in posts
Events	interest events, then upcoming by bookings
Similar items	pool:cat:{categoryId} — best sellers of the same category
Suggested shops	new shops (< 30 days), then most subscribers

A surface that has not yet been modelled therefore still works, and a gap in coverage costs quality, not availability. Keys may be delivered one at a time.

8.3 What the backend expects in return

Candidate generation and ranking, per surface	the seven reco:* keys of §7.1 — that is the whole contract
Similar items	reco:similar:products:{productId}, the one key that is per item rather than per person
Shop suggestions	reco:shops:{accountId}
Cross-domain learning	the thing the backend fundamentally cannot do: that people who buy baby products watch parenting reels. One representation of a person from every domain

The submission mechanism, precisely: one SET key <json> EX 172800 per key, per user, each time the batch runs. There is no API call, callback or schema negotiation — the full contract is §7.2, and the backend reads whatever is present the next time it builds a session. Completion is never announced; a fresher generatedAt is the signal.

Where the baseline may already exceed an early model: trending with hourly decay, category best-sellers for similar items, and the interest profile are all live. Where an early version cannot beat those on a given surface, the correct decision is to withhold that key — the fallback is stronger, and an absent key costs nothing. feed.reco.lists{list,state} reports, per surface, how often a supplied list is used.

8.4 The seam, in one sentence

The recommendation service decides what is worth showing. The backend decides what is permitted to be shown, in what order, and alongside what.

Exclusion logic, position rules and fallbacks should not be written: that code already exists in the backend, and two implementations will diverge the first time either one changes.

---

9. Known gaps

Stated plainly so nobody builds on an assumption:

Gap	Consequence
feed-served is published by the home feed only, with surface hard-coded to "FEED"	No served-ground-truth for reels, marketplace or events. Evaluation on those surfaces has to rely on impressions
Event ticket bookings have no entity topic	A booking arrives as a PURCHASE interaction with targetType: EVENT, but there is no order-like stream for bookings. nexgate.orders.v1 is product orders only
No JSON Schema is registered for nexgate.feed-served.v1 or nexgate.fanout-tasks.v1	The nine entity topics and interactions have schemas under src/main/resources/feed-schemas/; these two do not
Interest weights are empty until an admin maps categories	interests: [{id, w}] on posts, products and events is [] for any unmapped category or hashtag. 128 product categories and 12 event categories are still unmapped
Follows and shop subscriptions are not yet interest signals, and SEARCH is not matched to an interest	The backend's interest profile is weaker than the design describes. The delivered stream is unaffected

---

10. nexgate.fanout-tasks.v1 — internal

Backend to backend: when a post is published, the work of writing it into followers' timelines is chunked and sent over this topic. Key is postId:chunk, retained 3 days, no schema registered.

Ignore it. It carries no signal a model can use, and its shape may change without notice.

---

Related documents

Doc	What it covers
06_RECOMMENDATION_DEVELOPER_GUIDE.md	Connecting, consuming, and writing lists back into Redis
02_KAFKA_CONTRACT.md	The original agreement: why these topics, deletes, ordering, versioning, personal data
01_ARCHITECTURE.md	The design. §4 is the write path and the outbox; §4.6 is what is deliberately left out of payloads
05_INTERACTIONS_CLIENT_GUIDE.md	The app's side of the interactions topic: exact definitions of every action