Skip to main content

Event Catalogue — every message NexGate publishes to Kafka

Author: Josh S. Sakweli, Backend Lead Team
Last Updated: 2026-09-18
Version: v1.0 (backend branch fet/feed_baking, schema version 1)

Cluster: Redpanda (Kafka API) · Schema registry: JSON Schema, subject <topic>-value, BACKWARD compatibility

Short Description: TheEverything that crosses the line between the backend and the recommendation service, in both directions. Inbound (§1–§6): the complete list of what the backend puts on Kafka:Kafka 12 topics, 27 event types. For— when each one: when it fires, what the key is, what the payload carries, and — just as important — what does not cause it.it, Thisand isevery payload field with its type, nullability and allowed values, including the referenceexact forshape of each *.deleted. Outbound (§7): the recommendationeight developer,Redis keys the model writes back, their exact format, and the contractwhat the backend does with each one. It is heldwritten to.to be self-contained: a developer with this document and a Kafka client needs nothing else to start.

Read 06_RECOMMENDATION_DEVELOPER_GUIDE.md first. That guide is how to connect, what to do with the messages and how to write recommendations back. This document is the catalogue — everything that arrives, in detail. 06 links here rather than repeating it.

Hints:

  • Nothing is emitted from service code. Events come from committed database rows, so what you receive always matches what is in the database — a rolled-back transaction publishes nothing.
  • A payload is always the full current state, never a diff. Replace what you stored.
  • Order per key is guaranteed; order across keys is not. Use entityVersion, never occurredAt.
  • Fields are only ever added. Ignore what you don't recognise.
  • If a topic is ever lost, everything can be republished from the database: POST /api/admin/feed/backfills.

0. The round trip in one picture

  INBOUND — backend ➜ model                                              §1–§6
  ──────────────────────────────────────────────────────────────────────────────
  a row commits
  (post · product · shop · event · profile · follow · block · order · wishlist)
        └─► outbox ─► relay ─► 9 entity topics ......... compacted, "what exists"

  the app calls POST /api/v1/feed/interactions/batch        (impressions, views…)
  a row commits (like · comment · cart · order paid)        (seen by the backend)
        └─► nexgate.interactions.v1 ................. 90 days, "what people did"

  a feed page is served
        └─► nexgate.feed-served.v1 .................. 14 days, "what we showed"

                                    │
                                    ▼
                    ┌───────────────────────────────┐
                    │    RECOMMENDATION SERVICE     │
                    │   yours — we never see inside │
                    └───────────────┬───────────────┘
                                    │
  OUTBOUND — model ➜ backend        │                                       §7
  ──────────────────────────────────┼───────────────────────────────────────────
                                    ▼
        SET reco:<surface>:{id} <json> EX 172800        (8 keys, all optional)
                                    │
                                    ▼
        the feed pipeline reads reco:* while building a session;
        missing · invalid · stale ─► trending and interest pools fill in

The boundary never moves. The backend only ever writes Kafka and reads reco:*. The model only ever reads Kafka and writes reco:*. Neither side calls the other over HTTP, so neither can be made slow or unavailable by the other.


1. The envelope

Every message on the nine entity topics has the same eight fields:

{
  "eventId": "9f2c1e8a-5b3d-4a91-8c77-2e6f0b4d1a35",
  "eventType": "product.updated",
  "entityId": "4f2a77e1-0c8b-4d2f-9a11-7b6c3e5d8f90",
  "entityVersion": 184022,
  "occurredAt": "2026-09-18T10:22:33.120Z",
  "schemaVersion": 1,
  "source": "nexgate-backend",
  "payload": { }
}
Field Meaning
eventId Unique per event. Seen twice = a retry; drop the second
eventType One of the 27 in this document
entityId The thing this is about
entityVersion Only ever increases per entity. Keep the highest you have seen and ignore anything lower
occurredAt When it happened. Not an ordering key — clocks and retries make it unreliable for that
schemaVersion 1 today
source Always nexgate-backend
payload Full current state, or, for a *.deleted, the id plus a reason

The interactions and feed-served topics use a slightly different envelope — noted in their sections.

2. How a delete arrives

Always two messages, in this order:

  1. <entity>.deleted with a reason — the entity is gone for you, and the reason says why.
  2. A tombstone: same key, value = null — so log compaction can drop the key.

"Deleted" does not always mean the row was deleted. It means this entity is no longer something to recommend. A post set to private, a shop that closed, an account that got locked — all arrive as *.deleted with a reason that tells them apart. If it becomes visible again later, you get *.created again.

Tombstones are kept for 1 day. A consumer reading a compacted topic from offset 0 after that sees only live entities.


3. The topics at a glance

# Topic Key Cleanup Partitions Event types
1 nexgate.posts.v1 postId compact 24 post.created · post.updated · post.deleted
2 nexgate.products.v1 productId compact 24 product.created · product.updated · product.deleted
3 nexgate.shops.v1 shopId compact 12 shop.created · shop.updated · shop.deleted
4 nexgate.events.v1 eventId compact 12 event.created · event.updated · event.deleted
5 nexgate.profiles.v1 accountId compact 24 profile.created · profile.updated · profile.deleted
6 nexgate.follows.v1 followerId:followedId compact 24 follow.created · follow.deleted
7 nexgate.blocks.v1 blockerId:blockedId compact 12 block.created · block.deleted
8 nexgate.orders.v1 orderId compact 24 order.created · order.updated · order.deleted
9 nexgate.wishlist.v1 accountId:productId compact 24 wishlist.created · wishlist.updated · wishlist.deleted
10 nexgate.interactions.v1 accountId delete, 90 d 96 interaction
11 nexgate.feed-served.v1 accountId delete, 14 d 48 feed.served
12 nexgate.fanout-tasks.v1 postId:chunk delete, 3 d 48 (internal — not for consumers)

Partition counts are fixed for good. Adding partitions later would move keys to different partitions and break per-key ordering.

Compacted topics are a live catalogue: read from offset 0 and you have the current state of every entity, then keep consuming to stay current.


4. Entity topics

Each section below is complete: every field of the payload, its type, whether it can be null, its allowed values where it is an enum, and the exact shape of the *.deleted payload.

Reading the type column: uuid, string, int, decimal, bool, timestamp (ISO 8601 UTC), json (an opaque object — structure may change, do not depend on it). A field marked nullable can arrive as null; one marked required is on every message of that topic.

Fields are only ever added. The schemas are open by design: ignore any field you do not recognise, and never fail a message because of one.


4.1 nexgate.posts.v1 — posts and reels

postId · compacted · 24 partitions · schema src/main/resources/feed-schemas/nexgate.posts.v1.json

A post exists for you while it is PUBLISHED, not deleted, and PUBLIC or FOLLOWERS.

Event Fires when
post.created A draft becomes published, or a hidden post becomes visible again
post.updated A visible post, or any of its child rows, changes
post.deleted It stops being visible — only if it was visible before

Delete reasons: DELETED (soft-deleted) · UNPUBLISHED (no longer PUBLISHED) · VISIBILITY_RESTRICTED (switched to MENTIONED-only) or PRIVATE)

What counts as a change. Not just the post row — eleven child tables are watched, and a change to any of themone republishes the whole post: media, hashtags, attached productsproducts, /attached shopsshops, /attached events, user andmentions, shop mentions, links, collaborators, and the poll.

What doespublishes NOT publish anythingnothing: likesCount, commentsCount, repostsCount, bookmarksCount, viewsCount, quotesCount, sharesCount, shortClipCount, lastVerifiedAt, updatedAt. Engagement counters move constantly; republishing every post on every like would be most of the topic's traffic on the topic for none of the value. CountsThe counts in the payload are therefore a snapshot from the last real change, not live. For live behaviourbehaviour, use nexgate.interactions.v1.

A draft that is saved, edited and never published produces nothing at all.

Payload

post.created / post.updated

·userName} mediaType,order,
FieldTypeNotes
postIduuidrequired
postTypestring, nullableREGULAR · postTypePOLL
statusstring, nullablePUBLISHED on every message you receive (DRAFT, SCHEDULED, DELETED never reach you)
author.iduuid, nullableWho wrote it
author.userNamestring, nullablePublic username. No real name is ever published
contentstring, nullableThe post text as written
contentParsed.hashtags[]string[]Normalised, without the #. Any script (Latin, Arabic, CJK …)
contentParsed.mentionedUserIds[]uuid[]Accounts mentioned in the text
contentParsed.mentionedShopIds[]uuid[]Shops mentioned in the text
media[].fileIduuid, nullableFile-service id
media[].mediaTypestring, nullableIMAGE · VIDEO
media[].orderintDisplay order within the post, from 0
media[].status string, author{id,nullable Processing ·state. contentPENDING contentParsed{hashtags[means variants are not ready yet
media[].variantsjson, nullableRendition URLs keyed by name (hls, mentionedUserIds[]thumb, mentionedShopIds[]}…). Opaque — read keys, do not assume the set
media[]{fileId,.shortClip bool true status,= variants, shortClip, mediaId, durationMs} attachments{products[], shops[], events[]} · engagement{…} · privacySettings{visibility, …} quotedPostId · collaboratorIds[] · hasPoll · externalLinkDomain · isEdited createdAt · publishedAt · updatedAt · interests[]{id, w}

A reelthis is a post whose media has shortClip: true.reel. There is no separate reel topic.topic

media[].mediaIduuid, nullableThe clip's own id — the one the app uses on reel endpoints
media[].durationMsint, nullableLength in milliseconds, for video
attachments.products[]uuid[]Products attached to the post
attachments.shops[]uuid[]Shops attached
attachments.events[]uuid[]Events attached
engagement.likesCountintSnapshot, see above
engagement.commentsCountintSnapshot
engagement.sharesCountintSnapshot
engagement.viewsCountintSnapshot
engagement.bookmarksCountintSnapshot
engagement.repostsCountintSnapshot
engagement.quotesCountintSnapshot
privacySettings.visibilitystring, nullablePUBLIC or FOLLOWERS on every message you receive
privacySettings.whoCanCommentstring, nullableWho may comment
privacySettings.whoCanRepoststring, nullableWho may repost
quotedPostIduuid, nullableSet when this post quotes or reposts another. A pure repost carries the original's engagement, not its own
collaboratorIds[]uuid[]Co-authors
hasPollboolWhether a poll is attached. Poll options and votes are not published
externalLinkDomainstring, nullableThe domain only, never the full URL
isEditedboolWhether it has been edited since publishing
createdAttimestamp, nullableWhen the row was created (may be long before publishing)
publishedAttimestamp, nullableWhen it went live — use this for recency, not createdAt
updatedAttimestamp, nullableLast change
interests[].iduuidInterest id from the shared list (interest_categories)
interests[].wdecimalWeight: 1.0 from attached products and events, 0.6 from hashtags. An interest reached twice keeps the higher weight. Empty until an admin maps the category or hashtag

Payload — post.deleted

FieldTypeNotes
postIduuidrequired
authorIduuid, nullableThe author, or null when the row itself is gone
reasonstringOne of the delete reasons above

Followed by a tombstone: same key, value = null.


4.2 nexgate.products.v1 — products

productId · compacted · 24 partitions

A product exists for you while it is ACTIVE or OUT_OF_STOCK, not deleted, and its shop is visible.

Running out of stock is an update, not a disappearance — stock is in the payload, so you decide what to do with it.

Event Fires when
product.created It becomes visible, including when its shop comes back
product.updated The product or its installment plan changes
product.deleted It stops being visible

Delete reasons: DELETED · UNPUBLISHED (DRAFT, INACTIVE, ARCHIVED) · SHOP_UNAVAILABLE (the shop was suspended, closed, unapproved or deleted)

The shop cascade. When a shop's visibility flips, every product of that shop is republished —, in pages, right at that moment. One shop suspension can therefore produce thousands of product.deleted messages. This is deliberate:Deliberate: a consumer thatwatching only watched the productthis topic would otherwise keep recommending products from a suspended shop.

What doespublishes NOT publishnothing: viewCount, cartAddCount, updatedAt.

Payload

product.created / product.updated

price ·isLowStock·showStockToPublicstockInfocondition·urgencyTag·· · · · · · · w}
FieldTypeNotes
productIduuidrequired
productNamestring, nullable
productSlugstring, nullableURL slug
productDescriptionstring, nullable
productTypestring, nullablePHYSICAL · productNameDIGITAL
productMedia[].fileIduuid, nullable
productMedia[].mediaTypestring, nullableIMAGE · productSlugVIDEO
productMedia[].orderint, nullableDisplay order
productMedia[].statusstring, nullableProcessing state
productMedia[].variantsjson, nullableRendition URLs
categoryIduuid, nullableThe key to interests — mapped by an admin
categoryNamestring, nullable
parentCategoryIduuid, nullableA category with no mapping inherits its parent's interests
pricedecimal, nullableCurrent price
comparePricedecimal, nullable"Was" price
isOnSalebool
discountPercentagedecimal, nullable
stockQuantityint
isInStockbool
isLowStockbool
showStockToPublicboolWhether the shop displays the number
statusstring, nullableACTIVE or OUT_OF_STOCK on every message you receive
conditionstring, nullableNEW · productDescriptionUSED_LIKE_NEW · productTypeUSED_GOOD · productMedia[] categoryIdUSED_FAIR · categoryNameREFURBISHED · parentCategoryIdFOR_PARTS
urgencyTagstring, nullableNONE · comparePriceNEW_ARRIVAL · isOnSaleLIMITED_EDITION · discountPercentage stockQuantityLIMITED_OFFER · isInStockFEW_REMAINS
specifications json, ·nullable Free-form, statusseller-entered. ·Opaque
colors json, specificationsnullable Free-form, colorsseller-entered. Opaque
hasGroupBuying bool
groupPrice decimal, nullablePrice when bought as a group
hasInstallments bool
stockInfo.soldCountintUnits sold
shopId uuid, nullable
shopName string, nullable
createdAt timestamp, nullable
updatedAt timestamp, nullable
interests[]{id,.id uuid From the product's category, weight 1.0
interests[].wdecimal

Payload — product.deleted

FieldTypeNotes
productIduuidrequired
shopIduuid, nullableUseful when the cause was the shop
reasonstring

4.3 nexgate.shops.v1 — shops

shopId · compacted · 12 partitions

A shop exists for you while it is ACTIVE or TEMPORARILY_OFFLINE, approved, and not deleted.

Event Fires when
shop.created It becomes visible
shop.updated Anything meaningful on the shop changes
shop.deleted It stops being visible

Delete reasons: DELETED · SUSPENDED · CLOSED (permanently closed)PERMANENTLY_CLOSED) · UNPUBLISHED (PENDING, or approval withdrawn)

Every one of these also republishes the shop's products (§4.2).

What doespublishes NOT publishnothing: subscriberCount, lastSeenTime, updatedAt.

Payload

shop.created / shop.updated

· · · · · · · · · · · · · · ·
FieldTypeNotes
shopId uuid required
shopName string, nullable
shopSlug string, nullable
shopDescription string, nullable
logo.fileId / .mediaType / .status / .variantsuuid / string / string / json, nullableThe shop logo
banner.fileId / .mediaType / .status / .variantsuuid / string / string / json, nullableThe shop banner
ownerId uuid, nullableThe account that owns the shop
status string, nullableACTIVE or TEMPORARILY_OFFLINE on every message you receive
tempOfflineUntil timestamp, nullableSet while the owner has gone offline deliberately
city string, nullable
district string, nullable
regionstring, nullableThe feed filters by region · this is why location stays
countryCode string, nullable
isVerified bool
verificationBadge string, nullableWhich badge
trustScore decimal, nullablePlatform trust score
subscriberCount intSnapshot — a change to it alone does not republish
approvedAt timestamp, nullable
createdAt timestamp, nullable
updatedAt timestamp, nullable

Payload — shop.deleted

FieldTypeNotes
shopIduuidrequired
reasonstring

Never published: phone, email, street address, landmark, coordinates. Many shops are run from someone's home. Region, district and city stay because the feed filters by them.


4.4 nexgate.events.v1 — events

eventId · compacted · 12 partitions

An event exists for you while it is PUBLISHED or HAPPENING, PUBLIC, and not deleted.

Event Fires when
event.created It becomes visible
event.updated The event or its tickets change — including selling out, which is an update, not a removal
event.deleted It stops being visible

Delete reasons: DELETED · ENDED (COMPLETED) · CANCELLED · UNPUBLISHED (draftDRAFT or any other status) · VISIBILITY_RESTRICTED (PRIVATE or UNLISTED)

Ticket statistics count sold tickets only. Reservations change on every checkout attempt and expire by themselves, so publishing them would be noise.

What doespublishes NOT publishnothing: updatedAt, currentStage, completedStages, rsaKeys.

Payload

event.created / event.updated

·category{id,name}·eventFormat schedule{startDateTime,timezone}stats{…,isSoldOut} · · · · · ·
FieldTypeNotes
eventIduuidrequired
titlestring, nullable
slugstring, nullable
descriptionstring, nullable
category.iduuid, nullableThe key to interests
category.namestring, nullable
bannerMediajson, nullableOpaque media object
eventFormatstring, nullableIN_PERSON · titleONLINE · slugHYBRID · descriptionTBA
visibility string, bannerMedianullable PUBLIC ·on visibilityevery ·message you receive
status string, endDateTime,nullable PUBLISHED venue{name,or address,HAPPENING latitude,on longitude}every ·message pricingyou ·receive
schedule.startDateTimetimestamp, nullable
schedule.endDateTimetimestamp, nullable
schedule.timezonestring, nullableIANA zone
venue.namestring, nullable
venue.addressstring, nullableA public venue, unlike a shop's address
venue.latitude / .longitudedecimal, nullable
pricing.minPrice / .maxPricedecimal, nullableAcross ticket types
pricing.pricingTypestring, nullableHow the event is priced
stats.ticketsSoldintSold only — never reservations
stats.ticketsAvailableint, nullable
stats.isSoldOutboolSelling out arrives as event.updated
organizerId uuid, nullable
organizerName string, nullable
linkedProductIds[] uuid[] Products tied to the event
linkedShopIds[] uuid[]Shops tied to the event
createdAt timestamp, nullable
publishedAt timestamp, nullable
updatedAt timestamp, nullable
interests[]{id,.id w}/ .wuuid / decimalFrom the event's category, weight 1.0

Payload — event.deleted

FieldTypeNotes
eventIduuidrequired
reasonstring

Never published: the virtual meeting link, meeting id or passcode (they admit people into the room), and never the event's signing keys.


4.5 nexgate.profiles.v1 — public profile facts

accountId · compacted · 24 partitions

A profile exists for you once sign-up is complete (the account has a real username, not a temp_… placeholder) and while the account is not locked.

Event Fires when
profile.created Sign-up completes, or a locked account is unlocked
profile.updated One of the watched fields below changes, or the person's declared interests change
profile.deleted The account is locked, or the row is gone

Delete reasons: LOCKED · DELETED

Only these changes republish: userName, accountType, accountTier, locked, isVerified, verificationBadgeType, and rows in the user's declared interests. A login, a password change or a profile-new profile photo change publishes nothing.

Payload

profile.created / profile.updated

· · ·
FieldTypeNotes
accountIduuidrequired
userNamestring, nullablePublic username. Never temp_… on a message you receive
accountTypestring, nullableNORMAL · userNameSYSTEM · accountTypeVERIFIED
accountTierstring, nullableFULL · accountTier isVerifiedRESTRICTED · MINOR — MINOR matters: treat these accounts conservatively
isVerifiedbool
verificationBadgeType string, nullableWhich badge
followerCount int Snapshot
followingCount int Snapshot
declaredInterestIds[] uuid[] What the person chose at onboarding, on the shared interest list
createdAt timestamp, nullableAccount age — useful for cold-start

Payload — profile.deleted

FieldTypeNotes
accountIduuidrequired
reasonstringLOCKED or DELETED

Never published: first, middle or last name, bio, location, phone, email, birth date, profile photo.

⚠️ Do notNever use the active column if you everencounter seeit it.elsewhere. Sign-up sets it false and nothing ever sets it true, so it is false on every real account. It is deliberately not in the payload for that reason; relying on it would have tombstoned every profile on the platform.


4.6 nexgate.follows.v1 — who follows whom

Key followerId:followedId · compacted · 24 partitions

Only ACCEPTED follows exist for you. KeyA follow request (PENDING) publishes nothing; if it is followerId:followedId.never accepted you never hear about it.

Event Fires when
follow.created A followrequest is accepted, or a public account is followed
follow.deleted Unfollowed, or a request is withdrawn — only if it had been accepted

A

Payload follow follow.created

publishesnothing. accepted,younever
FieldTypeNotes
followerIduuidrequestrequired (— the one doing the following
PENDINGfollowedId) uuid required If— the one being followed
followedAttimestamp, nullableWhen it iswas neveraccepted
hear

Payload about it.

follow.deleted

Payload: followerId (uuid, required) · followedId ·(uuid, followedAt
Deleted payload: followerId · followedIdrequired) · reason (string)


4.7 nexgate.blocks.v1 — who blocked whom

Key is blockerId:blockedId. · compacted · 12 partitions

A block exists for you the moment the row is written — there is no visibility condition.

Event Fires when
block.created Someone blocks someone
block.deleted They unblock

This topic is not optional. The model must never recommend a blocked author's content in either direction — theA block hides content both ways.

Payloadways: neither person may be recommended the other's content.

Payload — block.created

FieldTypeNotes
blockerIduuidrequired — who blocked
blockedIduuidrequired — who was blocked
blockedAttimestamp, nullable

Payload — block.deleted

blockerId (uuid, required) · blockedId ·(uuid, blockedAt
Deleted payload: blockerId · blockedIdrequired) · reason (string)

Mutes are not on Kafka. A mute is a softer, per-viewer preference applied inside the feed; it is not published as an entity. You do see MUTE_AUTHOR as an interaction (§5).


4.8 nexgate.orders.v1 — what was actually bought

orderId · compacted · 24 partitions

An order exists for you from creation. This is the strongest purchase signal on the platform.

Event Fires when
order.created An order is placed
order.updated One of the watched fields below changes
order.deleted The order is soft-deleted (reason DELETED)

Only these changes republish: productOrderStatus, deliveryStatus, totalAmount, currency, shippedAt, deliveredAt, completedAt, cancelledAt, isDeleted.

Payload

order.created / order.updated

FieldTypeNotes
orderIduuidrequired
orderNumberstring, nullableHuman-readable reference
buyer.accountIduuid, nullableWho bought
shopIduuid, nullableWho sold
items[].productIduuid, nullable
items[].quantityint
items[].unitPricedecimal, nullable
items[].subtotaldecimal, nullable
productOrderStatusstring, nullablePENDING_PAYMENT · orderNumberPENDING_SHIPMENT · buyer{accountId}SHIPPED · shopId items[]{productId, quantity, unitPrice, subtotal} productOrderStatusDELIVERED · deliveryStatusAWAITING_BUYER_CONFIRM · productOrderSource totalAmountCOMPLETED · currency orderedAtDISPUTED · shippedAtCANCELLED · deliveredAtREFUNDED
deliveryStatusstring, nullablePENDING · completedAtSHIPPED · cancelledAtDELIVERED · CONFIRMED · IN_TRANSIT · NOT_APPLICABLE
productOrderSourcestring, nullableDIRECT_PURCHASE · DIGITAL_PURCHASE · INSTALLMENT · GROUP_PURCHASE · CART_PURCHASE · CHAT_OFFER
totalAmountdecimal, nullable
currencystring, nullable
orderedAttimestamp, nullable
shippedAttimestamp, nullable
deliveredAttimestamp, nullable
completedAttimestamp, nullableThe strongest signal: money settled and the buyer confirmed
cancelledAttimestamp, nullable
updatedAt timestamp, nullable

An order leaving PENDING_PAYMENT for anything other than CANCELLED/REFUNDED also produces a PURCHASE interaction per order item (§5). The order topic tells you what the order is; the interaction tells you when the buying happened.

Payload — order.deleted

FieldTypeNotes
orderIduuidrequired
reasonstringDELETED

Never published: delivery address or instructions, order notes, cancellation text, payment method, escrow or fee figures, tracking number, confirmation code, chat ids.

Event ticket bookings are not on this topic yet.topic. They arrive as a PURCHASEnexgate.orders.v1 interaction (§5) but there is noproduct bookingorders entityonly. stream. Noted as a gap inSee §7.8.


4.9 nexgate.wishlist.v1 — saved products

Key is accountId:productId · acompacted · 24 partitions

A product can be in a person's wishlist once, whatever group it is filed under.

Event Fires when
wishlist.created A product is added
wishlist.updated The entry changes (for example, moved to another group)
wishlist.deleted It is removed

Payload — wishlist.created / wishlist.updated

·
FieldTypeNotes
accountIduuidPayloadrequired:
productIduuidrequired
wishlistId uuid, nullableThe row's own id
groupIduuid, nullableWhich of the user's lists it sits in
addedAttimestamp, nullable

Payload — wishlist.deleted

accountId (uuid, required) · productId ·(uuid, groupIdrequired) · addedAtreason (string)

Never published: the group's name. It is free text a person writes about themselves ("Gifts for Mum"). The group id is enough to tell groups apart.


5. nexgate.interactions.v1 — what people did

accountId · delete, 90 days · 96 partitions · schema nexgate.interactions.v1.json

One event type, interaction,. keyedKeyed by accountId so one person's actions stay in order. Retained 90 days.

This is the only topic anything outside the backend can write to, and it does so through exactly one endpoint: POST /api/v1/feed/interactions/batch (09_NEW_ENDPOINTS.md A8). Every other topic is written from committed database rows, so no client can announce that an entity exists or changed.

The envelope is different here

No entityId and no entityVersion — an interaction is not an entity. Instead:

FieldTypeNotes
eventIduuidrequired. Deterministic (UUIDv5 of account + clientEventId), so a re-publish of the same action produces the same id. Deduplicate on it
eventTypestringAlways the literal interaction
occurredAttimestampThe device's clock, clamped so it can never be in the future
receivedAttimestampThe server's clock, when it arrived
schemaVersionint1
sourcestringnexgate-backend
payloadobjectBelow
{
  "eventId": "…", "eventType": "interaction",
  "occurredAt": "2026-09-18T10:22:41.100Z", "receivedAt": "2026-09-18T10:22:41.480Z",
  "schemaVersion": 1, "source": "nexgate-backend",
  "payload": {
    "accountId": "…", "sessionId": "s-8812e0c4", "clientEventId": "c-000185",
    "action": "VIEW", "targetType": "POST", "targetId": "…",
    "context": { "surface": "REELS", "position": 3, "feedSessionId": "fs-77aa", "source": "RECO", "viaPostId": null, "query": null },
    "media": { "dwellMs": null, "watchMs": 9000, "mediaDurationMs": 6000, "loopCount": 1, "soundOn": true, "mediaIndex": 0 },
    "client": { "platform": "ANDROID", "appVersion": "2.4.1", "networkType": "CELLULAR" },
    "origin": "CLIENT"
  }
}

Payload — every field

canneverbe
FieldTypeNotes
occurredAtaccountIduuidrequired. Who did it
sessionIdstring, nullableThe app session, max 64 chars. null isfor server-produced events
clientEventIdstringrequired. The app's own id, or srv:<action>:<rowId> when the device'sbackend time, clamped soproduced it
actionstringrequired. See the table below
targetTypestring, nullablePOST · PRODUCT · SHOP · EVENT · PROFILE. null only for SEARCH
targetIduuid, nullableLower-cased. null only for SEARCH
context.surfacestring, nullableFEED · REELS · SEARCH · PRODUCT_PAGE · SHOP_PAGE · EVENT_PAGE · PROFILE_PAGE · RECOMMENDATION · NOTIFICATION · CHAT
context.positionint, nullableWhere in the future; receivedAt isfeed the server's.item sat, from 0
context.feedSessionIdstring, nullableJoins to nexgate.feed-served.v1 — max 64 chars
context.sourcestring, nullableWhy it was shown: FOLLOWING · CELEBRITY · RECO · INTEREST · FALLBACK · SPONSORED · TRENDING · SEARCH. RECO means your list produced it
context.viaPostIduuid, nullableThe post a product was reached through
context.querystring, nullableThe search text, for SEARCH
media.dwellMsint, nullableTime on screen. 0 – 6 h
media.watchMsint, nullableTime watched. 0 – 6 h
media.mediaDurationMsint, nullableLength of the media. 0 – 12 h
media.loopCountint, nullableReplays. 0 – 1000
media.soundOnbool, nullableWhether sound was on
media.mediaIndexint, nullableWhich item of a carousel. 0 – 100
client.platformstring, nullableANDROID · IOS · WEB
client.appVersionstring, nullablee.g. 2.4.1
client.networkTypestring, nullableWIFI · CELLULAR · OFFLINE · UNKNOWN
originenum, nullableCLIENT or SERVER

media and client are always null on server-produced events — the backend did not witness a screen.

5.1 Two origins, never overlapping

origin WhoProduced produced itby clientEventId
CLIENT The app, in a batch to POST /api/v1/feed/interactions/batch the app's own id
SERVER The backend, watching committed rows srv:<action>:<rowId>

The split is enforced: the batch API rejects every server-owned action with SENT_BY_SERVER, so nothing is ever counted twice.

5.2 Every action, and who sends it, and what triggers it

only inserted/ inserted/ oran
Action Origin MeaningTargetExact trigger
IMPRESSION CLIENT anyThe item was on screen. Use it as the denominator
VIEW CLIENT anyReal attention —attention: dwell ≥ 5 s, or a reel watched ≥ 50 %
SKIP CLIENT anyScrolled past quickly
CLICK CLIENTany Opened the item
SEARCH CLIENT Thenone A actionsearch withwas no targetrun — what matters is context.query
NOT_INTERESTED CLIENT anyExplicit negative
HIDE CLIENT anyExplicit negative, stronger
LIKE / UNLIKE SERVERPOST A post_likes row wasinserted
UNLIKESERVERPOSTA post_likes row deleted
COMMENT SERVER POSTA post_comments row was inserted
BOOKMARK SERVER POSTA post_bookmarks row was inserted
SHARE SERVER POSTA post_shares row was inserted
REPORT SERVER POSTA post_reports row was inserted
BLOCK SERVER PROFILEA user_blocks row was inserted (target PROFILE)
MUTE_AUTHOR SERVER PROFILEA user_mutes row was inserted (target PROFILE)
ADD_TO_CART / REMOVE_FROM_CART SERVERPRODUCT A cart_items row wasinserted
REMOVE_FROM_CARTSERVERPRODUCTA cart_items row deleted
PURCHASE SERVER PRODUCTAn order becameleft paidPENDING_PAYMENT for any status other than CANCELLED/REFUNDEDone event per order item
PURCHASESERVEREVENTAn event booking became CONFIRMED

Server events are sent Target types: POST, PRODUCT, SHOP, EVENT, PROFILE.

Surfaces: FEED, REELS, SEARCH, PRODUCT_PAGE, SHOP_PAGE, EVENT_PAGE, PROFILE_PAGE, RECOMMENDATION, NOTIFICATION, CHAT.

Sources (whyafter the itemtransaction wascommits. shown):A FOLLOWING,rolled-back CELEBRITY,like RECO,sends INTEREST,nothing, FALLBACK,and SPONSORED,a TRENDING,like SEARCH.made through any code path is captured — nothing depends on a service remembering to emit.

5.3 Media measurements are raw, on purpose

watchMs: 9000 against mediaDurationMs: 6000 with loopCount: 1 is a rewatch, not bad data. Nothing is clamped to "100%100 %" before you see it, because the fact that someone watched a clip one and a half times is exactly the signal worth having.

5.4 WhatThis youtopic willis not find here

Interactions are best-effort by design.

If the event log is briefly unavailable, interactions are dropped and counted,counted, not queued — they are signal, not records. A missing view is a missing sample; a queued backlog of stale views would be worse. EntityThe entity topics are the opposite: those are never lost.lost, and can always be rebuilt with a backfill.


6. nexgate.feed-served.v1 — what the feed actually showed

accountId · delete, 14 days · 48 partitions · no registered schema

One event type, feed.served, keyed by accountId. Retained 14 days. This is the ground truth for ranking evaluation: what was shown, in what order, and why.

The envelope

No entityId, no entityVersion, no receivedAt: eventId, eventType (feed.served), occurredAt, schemaVersion (1), source, payload.

{
  "eventId": "…",
  "eventType": "feed.served",
  "occurredAt": "2026-09-18T10:22:31.900Z",
  "schemaVersion": 1,
  "source": "nexgate-backend",
  "payload": {
    "feedSessionId": "fs-77aa31c8",
    "accountId": "…",
    "surface": "FEED",
    "items": [
      { "type": "POST", "id": "4f2a77e1-…", "position": 0, "source": "FOLLOWING" },
      { "type": "PRODUCT", "id": "8c31a0d2-…", "position": 1, "source": "RECO" }
    ]
  }
}

Payload — every field

FieldTypeNotes
feedSessionIdstringJoins to context.feedSessionId on interactions
accountIduuidWho it was served to
surfacestringCurrently always the literal "FEED" — see the warning below
items[].typestringPOST · PRODUCT
items[].iduuidThe item
items[].positionintIts place on the page, from 0
items[].sourcestringWhy it was there: FOLLOWING · CELEBRITY · RECO · INTEREST · FALLBACK · SPONSORED · TRENDING

Join it to the interactions topic on feedSessionId and you have, per session: what we showed, in which position, from which source — and what the person did about it.

⚠️ Today this is publishedPublished by the home feed only, and surface is alwayshard-coded the literalto "FEED". Reels, marketplace and events do not publish it yet.it. See §7.8.

7. The way back — what the model writes to Redis

Everything above travels backend → model. This section is the other direction: model → backend.

There is no API call and no callback. The recommendation service writes plain Redis keys; the feed reads them when it builds a session. If the model stops writing, the feed keeps working on trending and interest pools — quieter, never broken.

7.1 The eight keys

KeyRead byItems must beWhen it is read
reco:home:posts:{accountId}Home feed discoverypost idsevery home session build
reco:home:products:{accountId}Home feed discoveryproduct idsevery home session build
reco:reels:{accountId}Reels feedpost ids (with a short video)every reels session build
reco:market:products:{accountId}Marketplace "For You"product idsevery marketplace session build
reco:events:{accountId}Events "For You"event idsevery events session build
reco:shops:{accountId}"Shops you might like"shop idsevery call to the strip
reco:similar:products:{productId}"Similar items" on a product pageproduct idsevery call to the strip
reco:meta:healthour alerting onlyon every metrics scrape

The {…} braces are part of the key — they are Redis Cluster hash tags, and they decide which node the key lives on. Write them exactly as shown.

Every key is optional. A surface whose key is absent runs on trending and interest pools instead. You can ship one surface at a time.

7.2 The value

One JSON string, written with a single SET key <value> EX 172800 (48 h). One command means a reader sees either the whole old list or the whole new one, never half of each.

{
  "v": 1,
  "modelVersion": "ranker-2026-09-14",
  "generatedAt": "2026-09-18T02:10:00Z",
  "items": [
    { "id": "4f2a77e1-0c8b-4d2f-9a11-7b6c3e5d8f90", "score": 0.83 },
    { "id": "9b10c2d4-7e3f-4a52-b8c1-6d9e0f2a3b47", "score": 0.79 }
  ]
}
FieldRequiredRule
generatedAtYesISO 8601. Without it the whole list is INVALID and ignored
itemsYesMust be an array. Best first. At most 300 are read; the rest are discarded
items[].idYesA UUID of the right type for that key. Not a UUID → that item is skipped. Unknown or deleted → dropped silently at read time
items[].scoreNoKept when inside [0, 1]. Missing or out of range → we use rank instead: 1 / (1 + rank), so position 0 → 1.0, 1 → 0.5, 3 → 0.25
modelVersionNoCarried into our debug trace, so a feed can be explained by the model that produced it
vNoFormat version

After each successful batch, write the health key:

SET reco:meta:health '{"finishedAt":"2026-09-18T02:40:00Z","modelVersion":"ranker-2026-09-14"}'

7.3 What the backend does with each list

Read, then judged in one of four states:

StateConditionUsed?
FRESHgeneratedAt within reco.maxAgeHours (default 30 h)Yes
STALEOlder than that, key still presentYes — a stale list beats no list. Counted on our dashboards
MISSINGNo keyNo → fallback pools fill discovery
INVALIDNot JSON, no items array, or no generatedAtNo → same as missing, but counted separately, so a broken writer is visible rather than silent

Every read is counted as feed.reco.lists{list, state}, so we can see per surface how often your lists are fresh, stale, missing or invalid.

How your items are ranked against ours. A reco item's score becomes 1 + yourScore. Fallback items score inside [0, 1]. So every recommended item outranks every fallback item in the discovery lane, and the pools only fill what you did not supply. If the same post appears in both, the RECO copy wins.

Where they land on the home feed. Your items fill the discovery slots, interleaved with posts from people the user follows in the pattern F, F, D, F, D, F, F, D — never the same author twice within 4 slots. A session holds 120 items and is rebuilt on pull-to-refresh or when the user reaches the end, so a newly written list appears at the next session, not mid-scroll.

We have 200 ms. Each reco read has a 200 ms timeout. Past that the source returns nothing for that session and the pools fill in. A slow Redis degrades the feed's quality, never its availability.

7.4 Access

You get a Redis user that can write only reco:*:

ACL SETUSER reco-writer on >******** ~reco:* -@all +set +get +del +expire +exists +ping +cluster|slots +cluster|shards +cluster|nodes

Anything outside reco:* answers NOPERM. Locally: user reco-writer, password reco_writer_local, seeds localhost:7001,localhost:7002.

Local docker-compose trap. The containers announce internal addresses (172.29.0.11–16). A cluster client on your laptop follows those announcements and cannot reach them. Run your writer in a container on the nexgate-feed-redis network, or on the host network.

7.5 How we both know it is working

SignalWhereMeans
feed.reco.lists{state="FRESH"} risingour dashboardyour lists are being read and used
feed.reco.lists{state="INVALID"} above zeroour dashboard, alert after 15 minyour writer is producing something unparseable
feed.reco.health.age.secondsour dashboard, alert above 26 hreco:meta:health.finishedAt is old — your batch has stopped. -1 means no batch has ever been reported
source: "RECO" in a feed responsethe app, and GET /api/admin/feed/debug/{accountId}an item on a real screen came from your list

The per-account debug trace is the fastest way to prove the loop end to end: it shows every item the last session produced, its source, its base score and its final score — and every item a filter removed, with the filter's name.


7.8. Known gaps

Stated plainly so nobody builds on an assumption:

Gap Consequence
feed-served is published by the home feed only, with surface hard-coded to "FEED" No served-ground-truth for reels, marketplace or events. Evaluation on those surfaces has to rely on impressions
Event ticket bookings have no entity topic A booking arrives as a PURCHASE interaction with targetType: EVENT, but there is no order-like stream for bookings. nexgate.orders.v1 is product orders only
No JSON Schema is registered for nexgate.feed-served.v1 or nexgate.fanout-tasks.v1 The nine entity topics and interactions have schemas under src/main/resources/feed-schemas/; these two do not
Interest weights are empty until an admin maps categories interests: [{id, w}] on posts, products and events is [] for any unmapped category or hashtag. 128 product categories and 12 event categories are still unmapped
Follows and shop subscriptions are not yet interest signals, and SEARCH is not matched to an interest The backend's own interest profile is weaker than the design describes. Does not affect what you receive

8.9. nexgate.fanout-tasks.v1 — internal

Backend to backend: when a post is published, the work of writing it into followers' timelines is chunked and sent over this topic. Key is postId:chunk, retained 3 days, no schema registered.

Ignore it. It carries no signal a model can use, and its shape may change without notice.


Doc What it covers
06_RECOMMENDATION_DEVELOPER_GUIDE.md How to connect, consume, and write your lists back into Redis
02_KAFKA_CONTRACT.md The original agreement: why these topics, deletes, ordering, versioning, personal data
01_ARCHITECTURE.md The design. §4 is the write path and the outbox; §4.6 is what is deliberately left out of payloads
05_INTERACTIONS_CLIENT_GUIDE.md The app's side of the interactions topic: exact definitions of every action