Event Catalogue — every message NexGate publishes to Kafka

Author: Josh S. Sakweli, Backend Lead Team
Last Updated: 2026-09-18
Version: v1.0 (backend branch fet/feed_baking, schema version 1)

Cluster: Redpanda (Kafka API) · Schema registry: JSON Schema, subject <topic>-value, BACKWARD compatibility

Short Description: The complete list of what the backend puts on Kafka: 12 topics, 27 event types. For each one: when it fires, what the key is, what the payload carries, and — just as important — what does not cause it. This is the reference for the recommendation developer, and the contract the backend is held to.

Read 06_RECOMMENDATION_DEVELOPER_GUIDE.md first. That guide is how to connect, what to do with the messages and how to write recommendations back. This document is the catalogue — everything that arrives, in detail. 06 links here rather than repeating it.

Hints:

Nothing is emitted from service code. Events come from committed database rows, so what you receive always matches what is in the database — a rolled-back transaction publishes nothing.
A payload is always the full current state, never a diff. Replace what you stored.
Order per key is guaranteed; order across keys is not. Use entityVersion, never occurredAt.
Fields are only ever added. Ignore what you don't recognise.
If a topic is ever lost, everything can be republished from the database: POST /api/admin/feed/backfills.

1. The envelope

Every message on the nine entity topics has the same eight fields:

{
  "eventId": "9f2c1e8a-5b3d-4a91-8c77-2e6f0b4d1a35",
  "eventType": "product.updated",
  "entityId": "4f2a77e1-0c8b-4d2f-9a11-7b6c3e5d8f90",
  "entityVersion": 184022,
  "occurredAt": "2026-09-18T10:22:33.120Z",
  "schemaVersion": 1,
  "source": "nexgate-backend",
  "payload": { }
}

Field	Meaning
`eventId`	Unique per event. Seen twice = a retry; drop the second
`eventType`	One of the 27 in this document
`entityId`	The thing this is about
`entityVersion`	Only ever increases per entity. Keep the highest you have seen and ignore anything lower
`occurredAt`	When it happened. Not an ordering key — clocks and retries make it unreliable for that
`schemaVersion`	`1` today
`source`	Always `nexgate-backend`
`payload`	Full current state, or, for a `*.deleted`, the id plus a `reason`

The interactions and feed-served topics use a slightly different envelope — noted in their sections.

2. How a delete arrives

Always two messages, in this order:

<entity>.deleted with a reason — the entity is gone for you, and the reason says why.
A tombstone: same key, value = null — so log compaction can drop the key.

"Deleted" does not always mean the row was deleted. It means this entity is no longer something to recommend. A post set to private, a shop that closed, an account that got locked — all arrive as *.deleted with a reason that tells them apart. If it becomes visible again later, you get *.created again.

Tombstones are kept for 1 day. A consumer reading a compacted topic from offset 0 after that sees only live entities.

3. The topics at a glance

#	Topic	Key	Cleanup	Partitions	Event types
1	`nexgate.posts.v1`	`postId`	compact	24	`post.created` · `post.updated` · `post.deleted`
2	`nexgate.products.v1`	`productId`	compact	24	`product.created` · `product.updated` · `product.deleted`
3	`nexgate.shops.v1`	`shopId`	compact	12	`shop.created` · `shop.updated` · `shop.deleted`
4	`nexgate.events.v1`	`eventId`	compact	12	`event.created` · `event.updated` · `event.deleted`
5	`nexgate.profiles.v1`	`accountId`	compact	24	`profile.created` · `profile.updated` · `profile.deleted`
6	`nexgate.follows.v1`	`followerId:followedId`	compact	24	`follow.created` · `follow.deleted`
7	`nexgate.blocks.v1`	`blockerId:blockedId`	compact	12	`block.created` · `block.deleted`
8	`nexgate.orders.v1`	`orderId`	compact	24	`order.created` · `order.updated` · `order.deleted`
9	`nexgate.wishlist.v1`	`accountId:productId`	compact	24	`wishlist.created` · `wishlist.updated` · `wishlist.deleted`
10	`nexgate.interactions.v1`	`accountId`	delete, 90 d	96	`interaction`
11	`nexgate.feed-served.v1`	`accountId`	delete, 14 d	48	`feed.served`
12	`nexgate.fanout-tasks.v1`	`postId:chunk`	delete, 3 d	48	(internal — not for consumers)

Partition counts are fixed for good. Adding partitions later would move keys to different partitions and break per-key ordering.

Compacted topics are a live catalogue: read from offset 0 and you have the current state of every entity, then keep consuming to stay current.

4. Entity topics

4.1 `nexgate.posts.v1` — posts and reels

A post exists for you while it is PUBLISHED, not deleted, and PUBLIC or FOLLOWERS.

Event	Fires when
`post.created`	A draft becomes published, or a hidden post becomes visible again
`post.updated`	A visible post, or any of its child rows, changes
`post.deleted`	It stops being visible — only if it was visible before

Delete reasons: DELETED (soft-deleted) · UNPUBLISHED (no longer PUBLISHED) · VISIBILITY_RESTRICTED (switched to MENTIONED-only)

What counts as a change. Not just the post row — eleven child tables are watched, and a change to any of them republishes the whole post: media, hashtags, attached products / shops / events, user and shop mentions, links, collaborators, and the poll.

What does NOT publish anything: likesCount, commentsCount, repostsCount, bookmarksCount, viewsCount, quotesCount, sharesCount, shortClipCount, lastVerifiedAt, updatedAt. Engagement counters move constantly; republishing every post on every like would be most of the traffic on the topic for none of the value. Counts in the payload are therefore a snapshot from the last real change, not live. For live behaviour use nexgate.interactions.v1.

A draft that is saved, edited and never published produces nothing at all.

Payload

postId · postType · status · author{id, userName} · content
contentParsed{hashtags[], mentionedUserIds[], mentionedShopIds[]}
media[]{fileId, mediaType, order, status, variants, shortClip, mediaId, durationMs}
attachments{products[], shops[], events[]} · engagement{…} · privacySettings{visibility, …}
quotedPostId · collaboratorIds[] · hasPoll · externalLinkDomain · isEdited
createdAt · publishedAt · updatedAt · interests[]{id, w}

A reel is a post whose media has shortClip: true. There is no separate reel topic.

4.2 `nexgate.products.v1` — products

A product exists for you while it is ACTIVE or OUT_OF_STOCK, not deleted, and its shop is visible.

Running out of stock is an update, not a disappearance — stock is in the payload, so you decide what to do with it.

Event	Fires when
`product.created`	It becomes visible, including when its shop comes back
`product.updated`	The product or its installment plan changes
`product.deleted`	It stops being visible

Delete reasons: DELETED · UNPUBLISHED (DRAFT, INACTIVE, ARCHIVED) · SHOP_UNAVAILABLE (the shop was suspended, closed, unapproved or deleted)

The shop cascade. When a shop's visibility flips, every product of that shop is republished — in pages, right at that moment. One shop suspension can therefore produce thousands of product.deleted messages. This is deliberate: a consumer that only watched the product topic would otherwise keep recommending products from a suspended shop.

What does NOT publish: viewCount, cartAddCount, updatedAt.

Payload

productId · productName · productSlug · productDescription · productType · productMedia[]
categoryId · categoryName · parentCategoryId
price · comparePrice · isOnSale · discountPercentage
stockQuantity · isInStock · isLowStock · showStockToPublic · stockInfo
status · condition · urgencyTag · specifications · colors
hasGroupBuying · groupPrice · hasInstallments
shopId · shopName · createdAt · updatedAt · interests[]{id, w}

4.3 `nexgate.shops.v1` — shops

A shop exists for you while it is ACTIVE or TEMPORARILY_OFFLINE, approved, and not deleted.

Event	Fires when
`shop.created`	It becomes visible
`shop.updated`	Anything meaningful on the shop changes
`shop.deleted`	It stops being visible

Delete reasons: DELETED · SUSPENDED · CLOSED (permanently closed) · UNPUBLISHED

Every one of these also republishes the shop's products (§4.2).

What does NOT publish: subscriberCount, lastSeenTime, updatedAt.

Payload

shopId · shopName · shopSlug · shopDescription · logo · banner · ownerId
status · tempOfflineUntil · city · district · region · countryCode
isVerified · verificationBadge · trustScore · subscriberCount
approvedAt · createdAt · updatedAt

Never published: phone, email, street address, landmark, coordinates. Many shops are run from someone's home. Region, district and city stay because the feed filters by them.

4.4 `nexgate.events.v1` — events

An event exists for you while it is PUBLISHED or HAPPENING, PUBLIC, and not deleted.

Event	Fires when
`event.created`	It becomes visible
`event.updated`	The event or its tickets change — including selling out, which is an update, not a removal
`event.deleted`	It stops being visible

Delete reasons: DELETED · ENDED (COMPLETED) · CANCELLED · UNPUBLISHED (draft or any other status) · VISIBILITY_RESTRICTED (PRIVATE or UNLISTED)

Ticket statistics count sold tickets only. Reservations change on every checkout attempt and expire by themselves, so publishing them would be noise.

What does NOT publish: updatedAt, currentStage, completedStages, rsaKeys.

Payload

eventId · title · slug · description · category{id, name} · bannerMedia
eventFormat · visibility · status
schedule{startDateTime, endDateTime, timezone}
venue{name, address, latitude, longitude} · pricing · stats{…, isSoldOut}
organizerId · organizerName · linkedProductIds[] · linkedShopIds[]
createdAt · publishedAt · updatedAt · interests[]{id, w}

Never published: the virtual meeting link, meeting id or passcode (they admit people into the room), and never the event's signing keys.

4.5 `nexgate.profiles.v1` — public profile facts

A profile exists for you once sign-up is complete (the account has a real username, not a temp_… placeholder) and while the account is not locked.

Event	Fires when
`profile.created`	Sign-up completes, or a locked account is unlocked
`profile.updated`	One of the fields below changes, or the person's declared interests change
`profile.deleted`	The account is locked, or the row is gone

Delete reasons: LOCKED · DELETED

Only these changes republish: userName, accountType, accountTier, locked, isVerified, verificationBadgeType, and rows in the user's declared interests. A login, a password change or a profile-photo change publishes nothing.

Payload

accountId · userName · accountType · accountTier
isVerified · verificationBadgeType
followerCount · followingCount · declaredInterestIds[] · createdAt

Never published: first, middle or last name, bio, location, phone, email, birth date, profile photo.

⚠️ Do not use the active column if you ever see it. Sign-up sets it false and nothing ever sets it true, so it is false on every real account. It is not in the payload for that reason; relying on it would have tombstoned every profile on the platform.

4.6 `nexgate.follows.v1` — who follows whom

Only ACCEPTED follows exist for you. Key is followerId:followedId.

Event	Fires when
`follow.created`	A follow is accepted, or a public account is followed
`follow.deleted`	Unfollowed, or a request is withdrawn — only if it had been accepted

A follow request (PENDING) publishes nothing. If it is never accepted, you never hear about it.

Payload: followerId · followedId · followedAt
Deleted payload: followerId · followedId · reason

4.7 `nexgate.blocks.v1` — who blocked whom

Key is blockerId:blockedId. A block exists for you the moment the row is written — there is no visibility condition.

Event	Fires when
`block.created`	Someone blocks someone
`block.deleted`	They unblock

This topic is not optional. The model must never recommend a blocked author's content in either direction — the block hides both ways.

Payload: blockerId · blockedId · blockedAt
Deleted payload: blockerId · blockedId · reason

4.8 `nexgate.orders.v1` — what was actually bought

An order exists for you from creation. This is the strongest purchase signal on the platform.

Event	Fires when
`order.created`	An order is placed
`order.updated`	One of the fields below changes
`order.deleted`	The order is soft-deleted (reason `DELETED`)

Only these changes republish: productOrderStatus, deliveryStatus, totalAmount, currency, shippedAt, deliveredAt, completedAt, cancelledAt, isDeleted.

Payload

orderId · orderNumber · buyer{accountId} · shopId
items[]{productId, quantity, unitPrice, subtotal}
productOrderStatus · deliveryStatus · productOrderSource
totalAmount · currency
orderedAt · shippedAt · deliveredAt · completedAt · cancelledAt · updatedAt

Never published: delivery address or instructions, order notes, cancellation text, payment method, escrow or fee figures, tracking number, confirmation code, chat ids.

Event ticket bookings are not on this topic yet. They arrive as a PURCHASE interaction (§5) but there is no booking entity stream. Noted as a gap in §7.

4.9 `nexgate.wishlist.v1` — saved products

Key is accountId:productId — a product can be in a person's wishlist once, whatever group it is filed under.

Event	Fires when
`wishlist.created`	A product is added
`wishlist.updated`	The entry changes (for example, moved to another group)
`wishlist.deleted`	It is removed

Payload: wishlistId · accountId · productId · groupId · addedAt

Never published: the group's name. It is free text a person writes about themselves ("Gifts for Mum"). The group id is enough to tell groups apart.

5. `nexgate.interactions.v1` — what people did

One event type, interaction, keyed by accountId so one person's actions stay in order. Retained 90 days.

This is the only topic anything outside the backend can write to, and it does so through exactly one endpoint: POST /api/v1/feed/interactions/batch (09_NEW_ENDPOINTS.md A8). Every other topic is written from committed database rows, so no client can announce that an entity exists or changed.

{
  "eventId": "…",
  "eventType": "interaction",
  "occurredAt": "2026-09-18T10:22:41.100Z",
  "receivedAt": "2026-09-18T10:22:41.480Z",
  "schemaVersion": 1,
  "source": "nexgate-backend",
  "payload": {
    "accountId": "…", "sessionId": "s-8812e0c4", "clientEventId": "c-000185",
    "action": "VIEW", "targetType": "POST", "targetId": "…",
    "context": { "surface": "REELS", "position": 3, "feedSessionId": "fs-77aa", "source": "RECO", "viaPostId": null, "query": null },
    "media": { "dwellMs": null, "watchMs": 9000, "mediaDurationMs": 6000, "loopCount": 1, "soundOn": true, "mediaIndex": 0 },
    "client": { "platform": "ANDROID", "appVersion": "2.4.1", "networkType": "CELLULAR" },
    "origin": "CLIENT"
  }
}

occurredAt is the device's time, clamped so it can never be in the future; receivedAt is the server's.

5.1 Two origins, never overlapping

`origin`	Who produced it	`clientEventId`
`CLIENT`	The app, in a batch to `POST /api/v1/feed/interactions/batch`	the app's own id
`SERVER`	The backend, watching committed rows	`srv:<action>:<rowId>`

The split is enforced: the batch API rejects every server-owned action with SENT_BY_SERVER, so nothing is ever counted twice.

5.2 Every action, and who sends it

Action	Origin	Meaning
`IMPRESSION`	CLIENT	The item was on screen. Use it as the denominator
`VIEW`	CLIENT	Real attention — dwell ≥ 5 s, or a reel watched ≥ 50 %
`SKIP`	CLIENT	Scrolled past quickly
`CLICK`	CLIENT	Opened the item
`SEARCH`	CLIENT	The only action with no target — what matters is `context.query`
`NOT_INTERESTED`	CLIENT	Explicit negative
`HIDE`	CLIENT	Explicit negative, stronger
`LIKE` / `UNLIKE`	SERVER	A `post_likes` row was inserted / deleted
`COMMENT`	SERVER	A `post_comments` row was inserted
`BOOKMARK`	SERVER	A `post_bookmarks` row was inserted
`SHARE`	SERVER	A `post_shares` row was inserted
`REPORT`	SERVER	A `post_reports` row was inserted
`BLOCK`	SERVER	A `user_blocks` row was inserted (target `PROFILE`)
`MUTE_AUTHOR`	SERVER	A `user_mutes` row was inserted (target `PROFILE`)
`ADD_TO_CART` / `REMOVE_FROM_CART`	SERVER	A `cart_items` row was inserted / deleted
`PURCHASE`	SERVER	An order became paid — one event per order item — or an event booking became `CONFIRMED`

Target types: POST, PRODUCT, SHOP, EVENT, PROFILE.

Surfaces: FEED, REELS, SEARCH, PRODUCT_PAGE, SHOP_PAGE, EVENT_PAGE, PROFILE_PAGE, RECOMMENDATION, NOTIFICATION, CHAT.

Sources (why the item was shown): FOLLOWING, CELEBRITY, RECO, INTEREST, FALLBACK, SPONSORED, TRENDING, SEARCH.

5.3 Media measurements are raw, on purpose

watchMs: 9000 against mediaDurationMs: 6000 with loopCount: 1 is a rewatch, not bad data. Nothing is clamped to "100%" before you see it, because the fact that someone watched a clip one and a half times is exactly the signal worth having.

5.4 What you will not find here

Interactions are best-effort by design. If the event log is briefly unavailable, interactions are dropped and counted, not queued — they are signal, not records. A missing view is a missing sample; a queued backlog of stale views would be worse. Entity topics are the opposite: those are never lost.

6. `nexgate.feed-served.v1` — what the feed actually showed

One event type, feed.served, keyed by accountId. Retained 14 days. This is the ground truth for ranking evaluation: what was shown, in what order, and why.

{
  "eventId": "…",
  "eventType": "feed.served",
  "occurredAt": "2026-09-18T10:22:31.900Z",
  "schemaVersion": 1,
  "source": "nexgate-backend",
  "payload": {
    "feedSessionId": "fs-77aa31c8",
    "accountId": "…",
    "surface": "FEED",
    "items": [
      { "type": "POST", "id": "4f2a77e1-…", "position": 0, "source": "FOLLOWING" },
      { "type": "PRODUCT", "id": "8c31a0d2-…", "position": 1, "source": "RECO" }
    ]
  }
}

Join it to the interactions topic on feedSessionId and you have, per session: what we showed, in which position, from which source — and what the person did about it.

⚠️ Today this is published by the home feed only, and surface is always the literal "FEED". Reels, marketplace and events do not publish it yet. See §7.

7. Known gaps

Stated plainly so nobody builds on an assumption:

Gap	Consequence
`feed-served` is published by the home feed only, with `surface` hard-coded to `"FEED"`	No served-ground-truth for reels, marketplace or events. Evaluation on those surfaces has to rely on impressions
Event ticket bookings have no entity topic	A booking arrives as a `PURCHASE` interaction with `targetType: EVENT`, but there is no order-like stream for bookings. `nexgate.orders.v1` is product orders only
No JSON Schema is registered for `nexgate.feed-served.v1` or `nexgate.fanout-tasks.v1`	The nine entity topics and interactions have schemas under `src/main/resources/feed-schemas/`; these two do not
Interest weights are empty until an admin maps categories	`interests: [{id, w}]` on posts, products and events is `[]` for any unmapped category or hashtag. 128 product categories and 12 event categories are still unmapped
Follows and shop subscriptions are not yet interest signals, and `SEARCH` is not matched to an interest	The backend's own interest profile is weaker than the design describes. Does not affect what you receive

8. `nexgate.fanout-tasks.v1` — internal

Backend to backend: when a post is published, the work of writing it into followers' timelines is chunked and sent over this topic. Key is postId:chunk, retained 3 days, no schema registered.

Ignore it. It carries no signal a model can use, and its shape may change without notice.

Doc	What it covers
`06_RECOMMENDATION_DEVELOPER_GUIDE.md`	How to connect, consume, and write your lists back into Redis
`02_KAFKA_CONTRACT.md`	The original agreement: why these topics, deletes, ordering, versioning, personal data
`01_ARCHITECTURE.md`	The design. §4 is the write path and the outbox; §4.6 is what is deliberately left out of payloads
`05_INTERACTIONS_CLIENT_GUIDE.md`	The app's side of the interactions topic: exact definitions of every action

Event Catalogue — every message NexGate publishes to Kafka

1. The envelope

2. How a delete arrives

3. The topics at a glance

4. Entity topics

4.1 nexgate.posts.v1 — posts and reels

4.2 nexgate.products.v1 — products

4.3 nexgate.shops.v1 — shops

4.4 nexgate.events.v1 — events

4.5 nexgate.profiles.v1 — public profile facts

4.6 nexgate.follows.v1 — who follows whom

4.7 nexgate.blocks.v1 — who blocked whom

4.8 nexgate.orders.v1 — what was actually bought

4.9 nexgate.wishlist.v1 — saved products

5. nexgate.interactions.v1 — what people did

5.1 Two origins, never overlapping

5.2 Every action, and who sends it

5.3 Media measurements are raw, on purpose

5.4 What you will not find here

6. nexgate.feed-served.v1 — what the feed actually showed

7. Known gaps

8. nexgate.fanout-tasks.v1 — internal

4.1 `nexgate.posts.v1` — posts and reels

4.2 `nexgate.products.v1` — products

4.3 `nexgate.shops.v1` — shops

4.4 `nexgate.events.v1` — events

4.5 `nexgate.profiles.v1` — public profile facts

4.6 `nexgate.follows.v1` — who follows whom

4.7 `nexgate.blocks.v1` — who blocked whom

4.8 `nexgate.orders.v1` — what was actually bought

4.9 `nexgate.wishlist.v1` — saved products

5. `nexgate.interactions.v1` — what people did

6. `nexgate.feed-served.v1` — what the feed actually showed

8. `nexgate.fanout-tasks.v1` — internal