# Files-nexgate-service(2)

# File Thunder Arch & Flow

va# File Thunder — Developer Documentation

File Thunder is the dedicated media-processing microservice for the NexGate / Veepii platform. It handles all file ingestion, processing, storage, and serving-readiness — the main backend never touches raw bytes.

---

## Table of Contents

1. [Architecture Overview](#1-architecture-overview)
2. [Storage: Buckets & Object Keys](#2-storage-buckets--object-keys)
3. [MediaDomain & MediaContext](#3-mediadomain--mediacontext)
4. [Upload Flow — End to End](#4-upload-flow--end-to-end)
5. [The Four Wheels](#5-the-four-wheels)
6. [Progress Tracking (Redis → SSE)](#6-progress-tracking-redis--sse)
7. [Watermarking](#7-watermarking)
8. [CDN & File Serving Per Environment](#8-cdn--file-serving-per-environment)
9. [Security](#9-security)
10. [API Reference & How to Consume](#10-api-reference--how-to-consume)

---

## 1. Architecture Overview

```
┌─────────────────────────────────────────────────────────┐
│                      Main Backend                       │
│  - Only public-facing API                               │
│  - Calls File Thunder over internal HTTP (HMAC signed)  │
│  - Subscribes to Redis for progress → pushes SSE        │
└────────────┬──────────────────────────┬─────────────────┘
             │ HTTP (sync)              │ Redis Pub/Sub (async)
             ▼                          ▼
┌─────────────────────────────────────────────────────────┐
│                     File Thunder                        │
│                                                         │
│   API Profile          Worker Profile                   │
│   ─────────────        ──────────────                   │
│   UploadController     UploadConfirmedWorker            │
│   MediaQueryController ThumbnailWorker                  │
│                        RawFilePurgeJob                  │
│                                                         │
│   Wheels (processing engines)                           │
│   ──────────────────────────                            │
│   ImageWheel  VideoWheel  ScanWheel  (MinIO ops)        │
└────────────┬──────────────────────────┬─────────────────┘
             │                          │
             ▼                          ▼
          MinIO                      RabbitMQ
    (4 private buckets)       (internal event routing)

```

**Key constraints:**

- File Thunder is **internal only** — never exposed to the internet directly
- Main backend is the **only caller** of File Thunder HTTP APIs
- File Thunder **never touches file bytes on upload** — client uploads directly to MinIO via presigned URL
- Redis is **FT-internal** — used for progress pub/sub and nonce replay protection
- RabbitMQ is **FT-internal** — used to route events between listeners and workers

---

## 2. Storage: Buckets & Object Keys

### Buckets

<table id="bkmrk-bucket-purpose-acces"><thead><tr><th>Bucket</th><th>Purpose</th><th>Access</th></tr></thead><tbody><tr><td>`nexgate-raw`</td><td>Temporary upload landing zone — 24h TTL</td><td>Private</td></tr><tr><td>`nexgate-public`</td><td>Processed media served to end users</td><td>Private (CDN in front)</td></tr><tr><td>`nexgate-private`</td><td>Internal system assets (outros, future forensic assets)</td><td>Private</td></tr><tr><td>`nexgate-digital`</td><td>Digital product files — ClamAV scanned, download only</td><td>Private</td></tr></tbody></table>

All buckets are **fully private**. MinIO is never directly accessible from the internet. Public content is served exclusively through the CDN (Cloudflare), which pulls from MinIO origin.

### Object Key Pattern

```
{domain}/{ownerId}/{fileId}/{variant}

```

**Examples:**

```
posts/550e8400-e29b-41d4-a716-446655440000/f7e8d9.../original
posts/550e8400-e29b-41d4-a716-446655440000/f7e8d9.../large.webp
posts/550e8400-e29b-41d4-a716-446655440000/f7e8d9.../thumb.webp
posts/550e8400-e29b-41d4-a716-446655440000/f7e8d9.../hls/master.m3u8
posts/550e8400-e29b-41d4-a716-446655440000/f7e8d9.../hls/360p/360p.m3u8

system/outro/{ownerId}/{height}p.mp4   ← in nexgate-private

```

**Rule:** The database stores **object keys only**, never full URLs. URLs are assembled at the resolver boundary (CDN base URL + key in prod, presigned GET in local).

---

## 3. MediaDomain &amp; MediaContext

### MediaDomain

Defines **where** in the storage hierarchy a file lives. It is the top-level folder in the object key. This is an enum — the main backend must send one of these exact values.

<table id="bkmrk-value-used-for-posts"><thead><tr><th>Value</th><th>Used For</th></tr></thead><tbody><tr><td>`POSTS`</td><td>Social posts — images and videos</td></tr><tr><td>`PROFILES`</td><td>Profile pictures, cover photos</td></tr><tr><td>`MESSAGES`</td><td>Direct message attachments</td></tr><tr><td>`PRODUCTS`</td><td>Product images, product videos, digital downloads</td></tr><tr><td>`SHOPS`</td><td>Shop banners, shop logos</td></tr><tr><td>`EVENTS`</td><td>Event covers, event gallery images</td></tr></tbody></table>

### MediaContext

Defines **how** a file is processed. This drives the entire processing pipeline. No logic is attached to domain — all logic is driven by context.

<table id="bkmrk-value-type-processin"><thead><tr><th>Value</th><th>Type</th><th>Processing Pipeline</th></tr></thead><tbody><tr><td>`SOCIAL_IMAGE`</td><td>Image</td><td>ImageWheel — orient, strip EXIF, WebP variants, blurhash, lqip</td></tr><tr><td>`SOCIAL_VIDEO`</td><td>Video</td><td>VideoWheel — transcode + thumbnail; **social watermark + outro only if duration &lt; 3 min (short clip)**</td></tr><tr><td>`PROFILE_PICTURE`</td><td>Image</td><td>ImageWheel</td></tr><tr><td>`COVER_PHOTO`</td><td>Image</td><td>ImageWheel</td></tr><tr><td>`DM_IMAGE`</td><td>Image</td><td>ImageWheel</td></tr><tr><td>`DM_VIDEO`</td><td>Video</td><td>VideoWheel — transcode, no watermark</td></tr><tr><td>`DM_DOCUMENT`</td><td>Any</td><td>ScanWheel — ClamAV scan → `private` bucket</td></tr><tr><td>`PRODUCT_IMAGE`</td><td>Image</td><td>ImageWheel</td></tr><tr><td>`PRODUCT_VIDEO`</td><td>Video</td><td>VideoWheel — transcode + text watermark, no outro</td></tr><tr><td>`DIGITAL_PRODUCT`</td><td>Any</td><td>ScanWheel — ClamAV scan → `digital` bucket</td></tr><tr><td>`PRODUCT_PREVIEW_IMAGE`</td><td>Image</td><td>ImageWheel — identical pipeline to `PRODUCT_IMAGE`</td></tr><tr><td>`PRODUCT_PREVIEW_VIDEO`</td><td>Video</td><td>VideoWheel — identical pipeline to `PRODUCT_VIDEO`; **watermarkLabel required**</td></tr><tr><td>`PRODUCT_PREVIEW_DOCUMENT`</td><td>Any</td><td>ScanWheel — identical pipeline to `DIGITAL_PRODUCT` → `digital` bucket</td></tr><tr><td>`SHOP_BANNER`</td><td>Image</td><td>ImageWheel</td></tr><tr><td>`SHOP_LOGO`</td><td>Image</td><td>ImageWheel</td></tr><tr><td>`EVENT_COVER`</td><td>Image</td><td>ImageWheel</td></tr><tr><td>`EVENT_GALLERY`</td><td>Image</td><td>ImageWheel</td></tr></tbody></table>

**Validation rules enforced at API layer:**

- Image contexts only accept image MIME types; video contexts only accept video MIME types
- `SOCIAL_VIDEO`, `PRODUCT_VIDEO`, and `PRODUCT_PREVIEW_VIDEO` require `watermarkLabel` field — pass the shop name with `$` prefix (e.g. `$NexGateShop`); for `SOCIAL_VIDEO` pass the user's `@` handle
- Max file sizes: images 20 MB, videos 2 GB, digital products / preview documents 500 MB
- HEIC/HEIF are rejected — parser attack surface too high

---

## 4. Upload Flow — End to End

### Standard Media Upload

```
Main Backend                    File Thunder                 MinIO           Client (Browser)
     │                               │                         │                   │
     │  POST /api/v1/upload/request  │                         │                   │
     │  (HMAC signed)                │                         │                   │
     │──────────────────────────────▶│                         │                   │
     │                               │ validate + create DB    │                   │
     │                               │ record (PENDING)        │                   │
     │                               │ publish PENDING         │                   │
     │                               │ to Redis                │                   │
     │                               │ generate presigned PUT  │                   │
     │                               │─────────────────────────▶                  │
     │  { fileId, presignedUrl }     │◀────────────────────────                   │
     │◀──────────────────────────────│                         │                   │
     │                               │                         │                   │
     │  return presignedUrl          │                         │                   │
     │  to client                    │                         │                   │
     │──────────────────────────────────────────────────────────────────────────▶ │
     │                               │                         │                   │
     │                               │                         │  PUT (file bytes) │
     │                               │                         │◀──────────────────│
     │                               │                         │  200 OK           │
     │                               │                         │──────────────────▶│
     │                               │                         │                   │
     │                               │ ◀── MinIO event ────────│ s3:ObjectCreated  │
     │                               │ MinioEventListener      │                   │
     │                               │ PENDING → UPLOADED      │                   │
     │                               │ publish UPLOADED→Redis  │                   │
     │                               │ publish UploadConfirmed │                   │
     │                               │ to RabbitMQ             │                   │
     │                               │                         │                   │
     │                               │ UploadConfirmedWorker   │                   │
     │                               │ routes by mimeType      │                   │
     │                               │ UPLOADED → PROCESSING   │                   │
     │                               │ publish PROCESSING→Redis│                   │
     │                               │                         │                   │
     │                               │ [ImageWheel or          │                   │
     │                               │  VideoWheel runs]       │                   │
     │                               │                         │                   │
     │                               │ variants uploaded       │                   │
     │                               │ to nexgate-public ──────▶                  │
     │                               │ raw deleted from        │                   │
     │                               │ nexgate-raw ────────────▶                  │
     │                               │ PROCESSING → READY      │                   │
     │                               │ publish READY → Redis   │                   │
     │                               │                         │                   │

```

### User Custom Thumbnail Upload

```
Main Backend                    File Thunder                 MinIO
     │                               │                         │
     │  POST /api/v1/upload/         │                         │
     │  thumbnail/{fileId}           │                         │
     │──────────────────────────────▶│                         │
     │                               │ generate presigned PUT  │
     │                               │ key: .../custom-        │
     │                               │ thumbnail               │
     │  { presignedUrl }             │                         │
     │◀──────────────────────────────│                         │
     │  client uploads thumbnail     │                         │
     │─────────────────────────────────────────────────────────▶
     │                               │◀── MinIO event ─────────│
     │                               │ MinioEventListener      │
     │                               │ detects suffix=         │
     │                               │ "custom-thumbnail"      │
     │                               │ → ThumbnailWorker       │
     │                               │ download → verify magic │
     │                               │ bytes → ImageMagick     │
     │                               │ variants → save to      │
     │                               │ entity.userThumbnail    │
     │                               │ → delete raw            │

```

### File Status Lifecycle

```
PENDING → UPLOADING → UPLOADED → SCANNING → PROCESSING → LIVE_PARTIAL → READY
                                                                ▲
                                              (all videos: after 360p HLS segments
                                               uploaded and partial master.m3u8 written)

```

---

## 5. The Four Wheels

Wheels are the processing engines. Each runs in the worker profile.

---

### Wheel 1 — ImageWheel (ImageMagick)

**Triggered by:** image MIME type on any image context

**Pipeline:**

```
download raw from nexgate-raw
      │
      ▼
auto-orient + strip EXIF
      │
      ├──▶ large.webp    (1920px max width, shrink only, Q85)
      ├──▶ medium.webp   (800px max width, shrink only, Q82)
      ├──▶ thumb.webp    (300px max width, shrink only, Q80)
      ├──▶ og.webp       (1200×630 center crop, Q85)
      │    └─ falls back to medium if source width < 1200px
      ├──▶ blurhash      (encoded from 32×32 downsample)
      └──▶ lqip          (10×10 forced, base64 WebP data URI)

all WebP variants → nexgate-public
keys + blurhash + lqip → media_files.variants (JSONB)
raw deleted from nexgate-raw
status → READY

```

**Skipping logic:** variant is skipped if source is already within bounds (never upscale).

---

### Wheel 2 — VideoWheel (FFmpeg / Jaffree)

**Triggered by:** video MIME type on any video context

**Step 1 — Probe &amp; Validate**

```
download raw from nexgate-raw
magic byte verify (must be real video container)
FFprobe → codedWidth, codedHeight, rotation, duration, codec
displayWidth/displayHeight = rotation-aware (swap for 90°/270°)
size gate: > 2 GB → reject
duration gate: > 4h → reject
entity.shortClip = duration < 3min
route: shortClip=true  → processShort()
       shortClip=false → processLong()

```

**Step 2a — Short Path (&lt; 3 min) — HLS Portrait**

```
transcodeHlsPortrait() runs 3 variants in sequence:
  360p  (CRF 28, 96k audio)  → HLS segments + playlist → upload → status LIVE_PARTIAL
  720p  (CRF 23, 128k audio) → HLS segments + playlist → upload
  1080p (CRF 21, 192k audio) → HLS segments + playlist → upload → final master.m3u8

filter_complex (portrait blur-pad, same as before):
  [0:v] split=2 [tmp_bg][tmp_fg]
  [tmp_bg] scale={W}:{H}:increase, crop={W}:{H}, boxblur=20:1 [bg]
  [tmp_fg] scale={W}:{H}:decrease, setsar=1 [fg]
  [bg][fg] overlay=(W-w)/2:(H-h)/2 [out]
  + rotation transpose prepended if needed

output format: HLS (-f hls), 2-second segments, VOD playlist type
variant keys:  {keyBase}/hls/{name}/{name}.m3u8
master key:    {keyBase}/hls/master.m3u8
segment pattern: {keyBase}/hls/{name}/{name}_%03d.ts

watermarked MP4 download (if profile.watermark()):
  transcode() — MP4 only, not HLS
  if SOCIAL_VIDEO: append outro via concat (no re-encode)
  key: {keyBase}/{name}_watermarked.mp4

skip logic: source displayWidth >= targetW OR displayHeight >= targetH

```

**Step 2b — Long Path (≥ 3 min) — HLS Adaptive**

```
transcodeHls() per variant:
  360p → segments + .m3u8 → upload → partial master.m3u8 → LIVE_PARTIAL
  720p → segments + .m3u8 → upload
  1080p → segments + .m3u8 → upload → full master.m3u8 → READY

segment length: 2s
playlist type: VOD
master.m3u8 key: {keyBase}/hls/master.m3u8
variant keys:   {keyBase}/hls/{name}/{name}.m3u8

No watermark generated — watermark is only produced for short clips (shortClip=true).

```

**Step 3 — Thumbnail &amp; Preview (both short and long)**

```
selectBestFrame():
  5 candidates at 15/30/45/60/75% of duration
  scored by: brightness gate (30–230) + Laplacian variance (sharpness)
  FFmpeg extracts 720px JPEG per candidate → pick winner

buildThumbnailVariants():
  poster.webp      (1280px, Q85)
  thumb.webp       (480px, Q80)
  og.webp          (1200×630 center crop, Q85)
  blurhash         (from 32×32 downsample)
  lqip             (10×10 forced, base64 WebP data URI)
  dominant_color   (#RRGGBB from 1×1 squish)

extractPreviewClip():
  skip first 5% (max 2s) → read 6s → setpts=0.5*PTS → 3s at 2× speed
  blur-pad 360×640, muted
  watermarked (same moving watermark as main video)

all thumbnail variants → nexgate-public
raw deleted from nexgate-raw after all variants done

```

---

### Wheel 3 — ScanWheel (ClamAV)

**Triggered by:** `DIGITAL_PRODUCT`, `DM_DOCUMENT`, and `PRODUCT_PREVIEW_DOCUMENT` contexts only Social content (images and videos) is **never** scanned by ClamAV — FFmpeg/ImageMagick re-encode is the sanitisation.

**Pipeline:**

```
download raw from nexgate-raw
      │
      ▼
SHA-256 hash → check file_hashes table
      │
      ├── hash known + clean → skip scan, copy to target bucket, READY (dedup fast lane)
      │
      └── hash unknown →
            ClamAV scan
            if clean → move to target bucket, save hash, status READY
            if infected → delete raw, status FAILED (VIRUS_DETECTED)

Target bucket by context:
  DIGITAL_PRODUCT        → nexgate-digital
  PRODUCT_PREVIEW_DOCUMENT → nexgate-digital
  DM_DOCUMENT            → nexgate-private

```

---

### Wheel 4 — MinIO Operations (used by all wheels)

Not a standalone service — MinIO operations are helpers used across all wheels:

<table id="bkmrk-operation-used-by-pr"><thead><tr><th>Operation</th><th>Used By</th></tr></thead><tbody><tr><td>Presigned PUT URL</td><td>Upload request endpoint</td></tr><tr><td>Download object</td><td>All wheels (download raw for processing)</td></tr><tr><td>Upload object</td><td>All wheels (upload processed variants)</td></tr><tr><td>Remove object</td><td>All wheels (delete raw after processing)</td></tr><tr><td>Stat object</td><td>OutroService (cache check), ScanWheel (dedup check)</td></tr></tbody></table>

---

## 6. Progress Tracking (Redis → SSE)

File Thunder publishes status changes to Redis. The main backend subscribes and drives SSE to the client. **SSE is the main backend's responsibility — File Thunder only publishes.**

### What File Thunder Does

On every status change:

```java
// 1. Cache current status in Redis (24h TTL)
redisTemplate.opsForValue().set("ft:status:{fileId}", status.name(), 24h);

// 2. Publish to per-file channel
redisTemplate.convertAndSend("ft:progress:{fileId}", status.name());

// 3. Append to timeline in Postgres
entity.timeline.add({ status, at })

```

### What the Main Backend Must Do

```java
// Subscribe to the file's progress channel
redisTemplate.subscribe((message, pattern) -> {
    String status = message.toString();
    sseEmitter.send(SseEmitter.event().data(status));
}, "ft:progress:" + fileId);

```

### Status Channel Key

```
ft:progress:{fileId}     ← subscribe here for live updates
ft:status:{fileId}       ← read here for current status (24h cached)

```

---

## 7. Watermarking

### When Watermarks Are Applied

<table id="bkmrk-context-condition-wa"><thead><tr><th>Context</th><th>Condition</th><th>Watermark type</th></tr></thead><tbody><tr><td>`SOCIAL_VIDEO`</td><td>`shortClip = true` (duration &lt; 3 min)</td><td>Social watermark (logo + watermarkLabel)</td></tr><tr><td>`SOCIAL_VIDEO`</td><td>`shortClip = false` (duration ≥ 3 min)</td><td>**None**</td></tr><tr><td>`PRODUCT_VIDEO`</td><td>Always</td><td>Text-only</td></tr><tr><td>`PRODUCT_PREVIEW_VIDEO`</td><td>Always</td><td>Text-only (same as `PRODUCT_VIDEO`)</td></tr><tr><td>`DM_VIDEO`</td><td>Never</td><td>—</td></tr></tbody></table>

Short-clip detection happens after FFprobe during VideoWheel processing. The `shortClip` flag is persisted on the entity and returned in every media response.

### Moving Watermark

Watermark position cycles through corners on a timer to resist cropping.

**Text-only (PRODUCT\_VIDEO):**

```
drawtext: fontsize=max(14,H/40), white@0.65 + black shadow
position: floor(t/3) mod 4 → top-left, top-right, bottom-right, bottom-left
text: from watermark.text property (default: "NexGate")

```

**Social watermark (SOCIAL\_VIDEO short clips only):**

```
2-point diagonal: Pos A (22%, 28%) ↔ Pos B (58%, 65%) — switches every 5s
Logo: nexgate_logo_white.svg, 36×36, 65% opacity
@ symbol: orange #F06023@0.85 + shadow
watermarkLabel: white@0.85 + shadow, below logo

```

### Outro (SOCIAL\_VIDEO only)

A personalized outro clip is appended to the watermarked variant (not clean variants).

**Generation:**

```
template: outro_template_with_sfx.mp4 (classpath resource)
font: Sora SemiBold 600 (classpath resource)
drawtext: @ in orange #F06023, watermarkLabel in cream #F4EEE9
fade-in: invisible before t=0.75s, fully opaque by t=1.2s
scale to match variant resolution (360p / 720p / 1080p)
append via: -f concat -safe 0 -c copy (no re-encode, fast)

```

**Caching:**

```
Generated once per ownerId + resolution
Cached to: nexgate-private / system/outro/{ownerId}/{height}p.mp4
On next upload: cache hit → download and reuse, skip generation

```

---

## 8. CDN &amp; File Serving Per Environment

All MinIO buckets are private. Access to public content is environment-dependent.

### Environment Matrix

<table id="bkmrk-environment-serving-"><thead><tr><th>Environment</th><th>Serving Method</th><th>Config Value</th></tr></thead><tbody><tr><td>Local dev</td><td>Presigned GET URLs (MinIO direct, short-lived)</td><td>`ft.storage.mode=local`</td></tr><tr><td>Staging</td><td>CDN — `cdn-staging.nexgate.com`</td><td>`ft.storage.mode=cdn`</td></tr><tr><td>Production</td><td>CDN — `cdn.nexgate.com`</td><td>`ft.storage.mode=cdn`</td></tr></tbody></table>

### Local / Dev

- App generates a presigned GET URL from MinIO (e.g. 1-hour expiry)
- URL is returned directly to the client
- MinIO must be reachable by the client
- Good enough for development and manual testing

### Staging &amp; Production (CDN)

- App assembles: `{ft.cdn.base-url} + "/" + objectKey`
- Cloudflare sits in front of MinIO origin
- Cloudflare pulls from MinIO on cache miss using a service credential
- Every subsequent request served from Cloudflare edge — MinIO never hit again
- Cost: Cloudflare bandwidth is free; MinIO only pays VPS bandwidth once per cache miss

### Properties

```properties
# Local
ft.storage.mode=local
ft.cdn.base-url=

# Staging
ft.storage.mode=cdn
ft.cdn.base-url=https://cdn-staging.nexgate.com

# Production
ft.storage.mode=cdn
ft.cdn.base-url=https://cdn.nexgate.com

```

### URL Assembly Logic (resolver boundary)

```java
// local mode → presigned GET from MinIO
// cdn mode   → CDN base + object key
String url = storageMode.equals("cdn")
    ? cdnBaseUrl + "/" + objectKey
    : minioClient.getPresignedObjectUrl(GET, bucket, objectKey, 1h);

```

**Private content (DIGITAL\_PRODUCT, PRODUCT\_PREVIEW\_DOCUMENT, DM\_DOCUMENT):** always presigned GET URLs regardless of environment — these are never cached by CDN. Generated fresh per download request (15-minute expiry, audit logged).

---

## 9. Security

### CORS

Configured via `ft.cors.*` properties. The CORS filter runs **before all other filters** so OPTIONS preflight requests are handled without requiring HMAC headers.

```properties
# Local
ft.cors.allowed-origins=http://localhost:3000

# Production
ft.cors.allowed-origins=https://nexgate.com,https://www.nexgate.com

ft.cors.allowed-methods=GET,POST,PUT,DELETE,OPTIONS
ft.cors.allowed-headers=*
ft.cors.allow-credentials=false
ft.cors.max-age-seconds=3600

```

---

### Service-to-Service HMAC Authentication

Every HTTP request from the main backend to File Thunder must be HMAC-SHA256 signed. File Thunder rejects any unsigned or incorrectly signed request with `401 Unauthorized`.

**Filter order:**

```
Request → CorsFilter (HIGHEST_PRECEDENCE)
        → HmacAuthFilter (HIGHEST_PRECEDENCE + 1)
        → Controllers

```

#### Required Headers

<table id="bkmrk-header-description-e"><thead><tr><th>Header</th><th>Description</th><th>Example</th></tr></thead><tbody><tr><td>`X-Service-Id`</td><td>Identifier of the calling service</td><td>`nexgate-main`</td></tr><tr><td>`X-Timestamp`</td><td>Unix epoch seconds at time of request</td><td>`1718123456`</td></tr><tr><td>`X-Nonce`</td><td>Random UUID — unique per request</td><td>`f47ac10b-58cc-...`</td></tr><tr><td>`X-Signature`</td><td>HMAC-SHA256 hex of the canonical string</td><td>`a3f5c2...`</td></tr></tbody></table>

#### Canonical String

```
METHOD\n
REQUEST_URI\n
TIMESTAMP\n
NONCE\n
HEX(SHA-256(requestBody))

```

**Example for `POST /api/v1/upload/request`:**

```
POST
/api/v1/upload/request
1718123456
f47ac10b-58cc-4372-a567-0e02b2c3d479
e3b0c44298fc1c149afb...   ← SHA-256 of the JSON body

```

#### Signature Computation

```java
// Main backend — how to sign a request
String bodyHash = HexFormat.of().formatHex(
    MessageDigest.getInstance("SHA-256").digest(requestBodyBytes)
);

String canonical = method + "\n" + uri + "\n" + timestamp + "\n" + nonce + "\n" + bodyHash;

Mac mac = Mac.getInstance("HmacSHA256");
mac.init(new SecretKeySpec(sharedSecret.getBytes(UTF_8), "HmacSHA256"));
String signature = HexFormat.of().formatHex(mac.doFinal(canonical.getBytes(UTF_8)));

// Set headers on outgoing request
request.setHeader("X-Service-Id", "nexgate-main");
request.setHeader("X-Timestamp",  String.valueOf(Instant.now().getEpochSecond()));
request.setHeader("X-Nonce",      UUID.randomUUID().toString());
request.setHeader("X-Signature",  signature);

```

#### What File Thunder Verifies (in order)

```
1. All 4 headers present                → 401 if any missing
2. X-Service-Id in allowed list         → 401 if unknown
3. X-Timestamp within ±5 minutes        → 401 if stale (replay protection)
4. X-Nonce not seen before              → 401 if duplicate
   (stored in Redis with 10-min TTL)       (replay attack blocked)
5. Recompute HMAC, timing-safe compare  → 401 if mismatch
   (MessageDigest.isEqual)                 (prevents timing oracle)

```

#### Properties

```properties
ft.security.hmac.secret=<long-random-secret-same-in-both-services>
ft.security.hmac.allowed-service-ids=nexgate-main
ft.security.hmac.timestamp-tolerance-seconds=300
ft.security.hmac.nonce-ttl-seconds=600

```

The `secret` must be identical in both File Thunder and the main backend config. Use a different value per environment (local / staging / prod).

---

## 10. API Reference &amp; How to Consume

Base URL: `http://file-thunder:8081` (internal network only) All requests must include HMAC headers — see Section 9.

---

### POST /api/v1/upload/request

Request a presigned URL to upload a file directly to MinIO.

**Request body:**

```json
{
  "ownerId":          "550e8400-e29b-41d4-a716-446655440000",
  "domain":           "POSTS",
  "context":          "SOCIAL_VIDEO",
  "originalFilename": "my-video.mp4",
  "mimeType":         "video/mp4",
  "fileSizeBytes":    104857600,
  "watermarkLabel":   "@josh_dev"
}

```

`watermarkLabel` is required when `context` is `SOCIAL_VIDEO` (pass `@username`) or `PRODUCT_VIDEO` (pass `$shopName`). All other fields are always required.

**Response:**

```json
{
  "status": "success",
  "message": "Presigned upload URL generated",
  "data": {
    "fileId":           "f7e8d9fa-...",
    "presignedUrl":     "http://minio:9000/nexgate-raw/posts/.../original?X-Amz-...",
    "objectKey":        "posts/{ownerId}/{entityId}/{fileId}/original",
    "bucket":           "nexgate-raw",
    "expiresInSeconds": 1800
  }
}

```

**What to do next:** PUT the file bytes directly to `presignedUrl` from the client browser. No auth headers needed on the PUT — the presigned URL is self-authenticating.

---

### POST /api/v1/upload/thumbnail/{fileId}

Request a presigned URL to replace the system-generated video thumbnail with a user-supplied one. Only valid after the video file has been processed (status READY).

**Response:**

```json
{
  "status": "success",
  "message": "Thumbnail presigned upload URL generated",
  "data": {
    "fileId":           "f7e8d9fa-...",
    "presignedUrl":     "http://minio:9000/nexgate-raw/.../custom-thumbnail?X-Amz-...",
    "objectKey":        "posts/.../custom-thumbnail",
    "bucket":           "nexgate-raw",
    "expiresInSeconds": 1800
  }
}

```

Accepted formats: JPEG, PNG, WebP only (verified by magic bytes).

---

### GET /api/v1/media/{fileId}

Get full metadata for a file, including all processed variants.

**Response — SOCIAL\_VIDEO short clip (`shortClip: true`, duration &lt; 3 min):**

```json
{
  "status": "success",
  "data": {
    "fileId":        "f7e8d9fa-...",
    "ownerId":       "550e8400-...",
    "domain":        "POSTS",
    "context":       "SOCIAL_VIDEO",
    "status":        "READY",
    "mimeType":      "video/mp4",
    "shortClip":     true,
    "variants": {
      "360p_playlist":    "posts/.../hls/360p/360p.m3u8",
      "720p_playlist":    "posts/.../hls/720p/720p.m3u8",
      "1080p_playlist":   "posts/.../hls/1080p/1080p.m3u8",
      "master":           "posts/.../hls/master.m3u8",
      "720p_watermarked": "posts/.../720p_watermarked.mp4",
      "poster":           "posts/.../poster.webp",
      "thumb":            "posts/.../thumb.webp",
      "og":               "posts/.../og.webp",
      "preview":          "posts/.../preview_3s.mp4",
      "blurhash":         "LKO2?U%2Tw=w]~RBVZRi};RPxuwH",
      "lqip":             "data:image/webp;base64,...",
      "dominant_color":   "#1A2B3C"
    },
    "userThumbnail": null,
    "timeline": [
      { "status": "PENDING",     "at": "2026-06-15T10:00:00" },
      { "status": "UPLOADED",    "at": "2026-06-15T10:00:05" },
      { "status": "PROCESSING",  "at": "2026-06-15T10:00:06" },
      { "status": "LIVE_PARTIAL","at": "2026-06-15T10:00:45" },
      { "status": "READY",       "at": "2026-06-15T10:02:10" }
    ],
    "createdAt": "2026-06-15T10:00:00",
    "updatedAt": "2026-06-15T10:02:10"
  }
}

```

**Response — SOCIAL\_VIDEO long clip (`shortClip: false`, duration ≥ 3 min) — HLS:**

```json
{
  "status": "success",
  "data": {
    "fileId":        "f7e8d9fa-...",
    "ownerId":       "550e8400-...",
    "domain":        "POSTS",
    "context":       "SOCIAL_VIDEO",
    "status":        "READY",
    "mimeType":      "video/mp4",
    "shortClip":     false,
    "variants": {
      "360p_playlist":  "posts/.../hls/360p/360p.m3u8",
      "720p_playlist":  "posts/.../hls/720p/720p.m3u8",
      "1080p_playlist": "posts/.../hls/1080p/1080p.m3u8",
      "master":         "posts/.../hls/master.m3u8",
      "poster":         "posts/.../poster.webp",
      "thumb":          "posts/.../thumb.webp",
      "og":             "posts/.../og.webp",
      "preview":        "posts/.../preview_3s.mp4",
      "blurhash":       "LKO2?U%2Tw=w]~RBVZRi};RPxuwH",
      "lqip":           "data:image/webp;base64,...",
      "dominant_color": "#1A2B3C"
    },
    "userThumbnail": null,
    "timeline": [
      { "status": "PENDING",     "at": "2026-06-15T10:00:00" },
      { "status": "UPLOADED",    "at": "2026-06-15T10:00:05" },
      { "status": "PROCESSING",  "at": "2026-06-15T10:00:06" },
      { "status": "LIVE_PARTIAL","at": "2026-06-15T10:01:00" },
      { "status": "READY",       "at": "2026-06-15T10:05:30" }
    ],
    "createdAt": "2026-06-15T10:00:00",
    "updatedAt": "2026-06-15T10:05:30"
  }
}

```

**Thumbnail resolution rule:**

```
userThumbnail != null ? use userThumbnail : use variants

```

Both `userThumbnail` and `variants` use the same key names (`poster`, `thumb`, `og`, `blurhash`, `lqip`, `dominant_color`).

---

### GET /api/v1/media/{fileId}/download?requesterId={uuid}

Generate a time-limited download URL for private files (`DIGITAL_PRODUCT`, `PRODUCT_PREVIEW_DOCUMENT`, `DM_DOCUMENT` only). Every call is audit-logged.

**Response:**

```json
{
  "status": "success",
  "data": {
    "url":              "http://minio:9000/nexgate-digital/...?X-Amz-...",
    "expiresInSeconds": 900
  }
}

```

---

### GET /api/v1/quota/{ownerId}

Get the total storage used by an owner across all their processed files. Updated atomically as each file is processed — never stale.

**Response:**

```json
{
  "status": "success",
  "message": "Storage usage",
  "data": {
    "ownerId":   "550e8400-e29b-41d4-a716-446655440000",
    "usedBytes": 1073741824,
    "usedMb":    1024.0,
    "usedGb":    1.0
  }
}

```

**How quota is tracked:**

- Every wheel (`ImageWheel`, `VideoWheel`, `ScanWheel`) calls `StorageUsageService.trackUsage(ownerId, bytes)` after uploading processed variants
- Tracked bytes = size of the **processed variants**, not the raw upload
- Stored in `user_storage_quota` table, upserted atomically per owner
- `releaseUsage()` is called on hard delete (soft deletes do not release quota)
- Returns `0` if the owner has no tracked usage yet (never uploaded)

**Use this endpoint to:**

- Display storage usage in the main backend dashboard
- Enforce storage limits before allowing a new upload request

---

### Error Responses

```json
{ "error": "Missing required security headers" }          ← 401
{ "error": "Request timestamp out of acceptable window" } ← 401
{ "error": "Nonce already used — possible replay attack"} ← 401
{ "error": "Invalid signature" }                          ← 401

```

Validation errors return 400 with field-level messages via `@ControllerAdvice`.

---

*This document covers the full File Thunder system as of 2026-06-15.**CDN wiring (Cloudflare), Kafka integration, lazy transcode, and forensic watermarks are deferred.*

# Files Handling API (NEW)

**Author**: Josh S. Sakweli, Backend Lead Team  
**Last Updated**: 2026-06-21  
**Version**: v1.0

**Base URL**: `https://your-api-domain.com/api/v1`

**Short Description**: This document covers everything a frontend developer needs to handle files on the NexGate/Veepii platform — uploading, tracking processing progress, reading variant URLs from entity responses, and downloading private files. Files are managed by an internal service called **FileThunder**; the frontend never talks to FileThunder directly. All file-related operations go through the main backend endpoints documented here.

**Hints**:

- All upload endpoints require a valid Bearer token
- Files are **never** uploaded through the backend server — you upload directly to object storage using a time-limited presigned URL returned by the backend
- The backend stores only object key paths; full CDN/MinIO URLs are assembled at response time — always use the URLs in entity responses as-is
- Processing is asynchronous — after uploading, you track progress via SSE or polling, then the entity response will include the resolved variant map once processing is complete
- Private files (digital products, DM documents) are never served via CDN — always request a fresh download URL before initiating a download

---

## How File Handling Works — Big Picture

Understanding this flow will save you from confusion. There are **two separate progress concepts**:

1. **Upload progress** — bytes travelling from the client to object storage. You can show a progress bar for this using `XMLHttpRequest`.
2. **Processing progress** — FileThunder processing those bytes into variants (WebP images, transcoded video resolutions, virus scans). You track this via SSE or polling.

```
┌─────────┐     Step 1: Request presigned URL      ┌──────────────┐
│  Client │ ──────────────────────────────────────► │ Main Backend │
│         │ ◄────────────────────────────────────── │              │
│         │     { fileId, presignedUrl, expiresIn } └──────────────┘
│         │
│         │     Step 2: PUT file bytes directly         ┌─────────┐
│         │ ────────────────────────────────────────►   │  MinIO  │
│         │ ◄────────────────────────────────────────   │         │
│         │     HTTP 200 (upload complete)              └────┬────┘
│         │                                                  │ MinIO fires event
│         │     Step 3: Open SSE stream                      ▼
│         │ ──────────────────────────────────────► ┌──────────────┐
│         │ ◄── status events (PROCESSING, READY) ── │ Main Backend │
└─────────┘                                          └──────────────┘
                                                          │ on READY
                                                          ▼
                                                  Variants saved to entity in DB
                                                  Entity response includes variant URLs

```

---

## Standard Response Format

### Success Response Structure

```json
{
  "success": true,
  "httpStatus": "OK",
  "message": "Operation completed successfully",
  "action_time": "2025-09-23T10:30:45",
  "data": {}
}

```

### Error Response Structure

```json
{
  "success": false,
  "httpStatus": "BAD_REQUEST",
  "message": "Error description",
  "action_time": "2025-09-23T10:30:45",
  "data": "Error description"
}

```

### Standard Response Fields

<table id="bkmrk-field-type-descripti"><thead><tr><th>Field</th><th>Type</th><th>Description</th></tr></thead><tbody><tr><td>`success`</td><td>boolean</td><td>`true` for success, `false` for errors</td></tr><tr><td>`httpStatus`</td><td>string</td><td>HTTP status name (OK, BAD\_REQUEST, NOT\_FOUND, etc.)</td></tr><tr><td>`message`</td><td>string</td><td>Human-readable result description</td></tr><tr><td>`action_time`</td><td>string</td><td>ISO 8601 timestamp of the response</td></tr><tr><td>`data`</td><td>object/string</td><td>Response payload or error details</td></tr></tbody></table>

---

## HTTP Method Badge Standards

- **GET** — <span style="background-color: #28a745; color: white; padding: 4px 8px; border-radius: 4px; font-family: monospace; font-size: 12px; font-weight: bold;">GET</span>
- **POST** — <span style="background-color: #007bff; color: white; padding: 4px 8px; border-radius: 4px; font-family: monospace; font-size: 12px; font-weight: bold;">POST</span>
- **PUT** — <span style="background-color: #ffc107; color: black; padding: 4px 8px; border-radius: 4px; font-family: monospace; font-size: 12px; font-weight: bold;">PUT</span>

---

## File Contexts

Every upload must declare a **context**. Context tells FileThunder what this file is for, which determines how it is processed, what variants are generated, and how it is stored.

<table id="bkmrk-context-domain-what-"><thead><tr><th>Context</th><th>Domain</th><th>What it is</th><th>Who uploads it</th></tr></thead><tbody><tr><td>`SOCIAL_IMAGE`</td><td>Posts</td><td>Image attached to a post or story</td><td>Authenticated user</td></tr><tr><td>`SOCIAL_VIDEO`</td><td>Posts</td><td>Video attached to a post or story</td><td>Authenticated user</td></tr><tr><td>`PROFILE_PICTURE`</td><td>Profiles</td><td>User avatar / profile photo</td><td>Authenticated user</td></tr><tr><td>`COVER_PHOTO`</td><td>Profiles</td><td>Profile cover / banner image</td><td>Authenticated user</td></tr><tr><td>`DM_IMAGE`</td><td>Messages</td><td>Image sent in a direct message</td><td>Authenticated user</td></tr><tr><td>`DM_VIDEO`</td><td>Messages</td><td>Video sent in a direct message</td><td>Authenticated user</td></tr><tr><td>`DM_DOCUMENT`</td><td>Messages</td><td>Document sent in a direct message</td><td>Authenticated user</td></tr><tr><td>`PRODUCT_IMAGE`</td><td>Products</td><td>Product listing photo</td><td>Shop owner</td></tr><tr><td>`PRODUCT_VIDEO`</td><td>Products</td><td>Product demo/preview video</td><td>Shop owner</td></tr><tr><td>`DIGITAL_PRODUCT`</td><td>Products</td><td>Purchasable digital file (PDF, ZIP, software, etc.)</td><td>Shop owner</td></tr><tr><td>`SHOP_BANNER`</td><td>Shops</td><td>Shop header banner image</td><td>Shop owner</td></tr><tr><td>`SHOP_LOGO`</td><td>Shops</td><td>Shop logo / avatar</td><td>Shop owner</td></tr><tr><td>`EVENT_COVER`</td><td>Events</td><td>Event banner/hero image</td><td>Event organiser</td></tr><tr><td>`EVENT_GALLERY`</td><td>Events</td><td>Additional event gallery image</td><td>Event organiser</td></tr></tbody></table>

---

## File Format &amp; Size Constraints

These are the accepted formats and recommended limits per context. FileThunder enforces the actual limits server-side — passing incorrect `mimeType` or oversized files will result in a rejection at the presigned URL stage.

### Images

<table id="bkmrk-context-accepted-mim"><thead><tr><th>Context</th><th>Accepted MIME Types</th><th align="right">Max Size</th><th>Min Dimensions</th><th>Recommended Dimensions</th></tr></thead><tbody><tr><td>`SOCIAL_IMAGE`</td><td>`image/jpeg`, `image/png`, `image/webp`, `image/gif`</td><td align="right">20 MB</td><td>200 × 200 px</td><td>1080 × 1080 px (square) or 1080 × 1920 px (portrait)</td></tr><tr><td>`PROFILE_PICTURE`</td><td>`image/jpeg`, `image/png`, `image/webp`</td><td align="right">10 MB</td><td>100 × 100 px</td><td>400 × 400 px square</td></tr><tr><td>`COVER_PHOTO`</td><td>`image/jpeg`, `image/png`, `image/webp`</td><td align="right">15 MB</td><td>600 × 200 px</td><td>1500 × 500 px</td></tr><tr><td>`DM_IMAGE`</td><td>`image/jpeg`, `image/png`, `image/webp`, `image/gif`</td><td align="right">20 MB</td><td>—</td><td>—</td></tr><tr><td>`PRODUCT_IMAGE`</td><td>`image/jpeg`, `image/png`, `image/webp`</td><td align="right">20 MB</td><td>400 × 400 px</td><td>1000 × 1000 px square</td></tr><tr><td>`SHOP_BANNER`</td><td>`image/jpeg`, `image/png`, `image/webp`</td><td align="right">15 MB</td><td>800 × 200 px</td><td>1200 × 400 px</td></tr><tr><td>`SHOP_LOGO`</td><td>`image/jpeg`, `image/png`, `image/webp`</td><td align="right">5 MB</td><td>100 × 100 px</td><td>400 × 400 px square</td></tr><tr><td>`EVENT_COVER`</td><td>`image/jpeg`, `image/png`, `image/webp`</td><td align="right">15 MB</td><td>800 × 400 px</td><td>1200 × 630 px</td></tr><tr><td>`EVENT_GALLERY`</td><td>`image/jpeg`, `image/png`, `image/webp`</td><td align="right">20 MB</td><td>200 × 200 px</td><td>1080 × 1080 px</td></tr></tbody></table>

### Videos

<table id="bkmrk-context-accepted-mim-1"><thead><tr><th>Context</th><th>Accepted MIME Types</th><th align="right">Max Size</th><th align="right">Max Duration</th><th>Notes</th></tr></thead><tbody><tr><td>`SOCIAL_VIDEO`</td><td>`video/mp4`, `video/quicktime`, `video/webm`, `video/x-msvideo`, `video/x-matroska`</td><td align="right">100 MB</td><td align="right">60 min</td><td>All durations use HLS adaptive streaming</td></tr><tr><td>`DM_VIDEO`</td><td>`video/mp4`, `video/quicktime`, `video/webm`</td><td align="right">100 MB</td><td align="right">60 min</td><td>HLS adaptive streaming, no watermark</td></tr><tr><td>`PRODUCT_VIDEO`</td><td>`video/mp4`, `video/quicktime`, `video/webm`, `video/x-msvideo`, `video/x-matroska`</td><td align="right">100 MB</td><td align="right">60 min</td><td>All durations use HLS adaptive streaming</td></tr></tbody></table>

### Other

<table id="bkmrk-context-accepted-mim-2"><thead><tr><th>Context</th><th>Accepted MIME Types</th><th align="right">Max Size</th><th>Notes</th></tr></thead><tbody><tr><td>`DM_DOCUMENT`</td><td>`application/pdf`, `application/msword`, `application/vnd.openxmlformats-officedocument.wordprocessingml.document`, `application/vnd.ms-excel`, `application/vnd.openxmlformats-officedocument.spreadsheetml.sheet`, `application/zip`, `text/plain`</td><td align="right">50 MB</td><td>ClamAV scanned; no variants generated</td></tr><tr><td>`DIGITAL_PRODUCT`</td><td>Any — PDF, ZIP, EXE, APK, MP3, etc.</td><td align="right">5 GB</td><td>ClamAV dual-scan + SHA-256 deduplication; never CDN served; always presigned download</td></tr></tbody></table>

---

## Processing Status Lifecycle

After the file reaches object storage, FileThunder processes it through these states. You will receive these values from both the SSE stream and the status polling endpoint.

```
PENDING ──► UPLOADING ──► UPLOADED ──► SCANNING ──► PROCESSING ──► READY
                                                                  └──► FAILED
                                                         │
                                                    LIVE_PARTIAL
                                               (video 360p is done,
                                                 higher resolutions
                                                  still processing)

```

<table id="bkmrk-status-meaning-ui-su"><thead><tr><th>Status</th><th>Meaning</th><th>UI suggestion</th></tr></thead><tbody><tr><td>`PENDING`</td><td>Upload slot created, waiting for file bytes</td><td>Show spinner</td></tr><tr><td>`UPLOADING`</td><td>File bytes are being received by storage</td><td>Show upload progress bar (XHR)</td></tr><tr><td>`UPLOADED`</td><td>All bytes received, handing off to processing</td><td>Show spinner</td></tr><tr><td>`SCANNING`</td><td>ClamAV virus scan in progress (digital products / DM docs only)</td><td>"Scanning for safety..."</td></tr><tr><td>`PROCESSING`</td><td>Transcoding / image variant generation in progress</td><td>"Processing..."</td></tr><tr><td>`LIVE_PARTIAL`</td><td>**Videos only** — 360p variant is ready, higher resolutions still processing</td><td>Show video with 360p, overlay "HD processing" badge</td></tr><tr><td>`READY`</td><td>All variants generated and available</td><td>Dismiss progress UI, display media</td></tr><tr><td>`FAILED`</td><td>Processing failed</td><td>Show error, offer re-upload option</td></tr></tbody></table>

---

## Variant Keys

When a file reaches `READY`, its variants are stored in the entity's database record and returned in entity API responses. Here are all possible variant keys and what they contain.

### Image Variants

<table id="bkmrk-key-format-typical-u"><thead><tr><th>Key</th><th>Format</th><th>Typical Use</th></tr></thead><tbody><tr><td>`large`</td><td>WebP URL</td><td>Full-size display, lightbox, detail view</td></tr><tr><td>`medium`</td><td>WebP URL</td><td>Feed cards, grid thumbnails</td></tr><tr><td>`thumb`</td><td>WebP URL</td><td>Tiny previews, avatar chips, comment icons</td></tr><tr><td>`og`</td><td>WebP URL</td><td>`<meta property="og:image">` Open Graph tag</td></tr><tr><td>`blurhash`</td><td>String (e.g. `LGF5?xYk^6...`)</td><td>CSS blur placeholder while image loads</td></tr><tr><td>`lqip`</td><td>`data:image/webp;base64,...` inline data URI</td><td>Inline `<img src>` placeholder, no extra request</td></tr><tr><td>`dominant_color`</td><td>Hex string (e.g. `#F06023`)</td><td>Background color while loading, skeleton screen tint</td></tr></tbody></table>

> **Not every key is present for every context.** `blurhash` and `lqip` are always generated. `large`, `medium`, `og`, and `dominant_color` depend on the context — see the Context → Variants Cheat Sheet at the bottom of this document for the exact set per context.

**Usage priority**: Render `lqip` or apply `blurhash` immediately. Swap in `medium` or `large` once loaded. Use `thumb` for tiny contexts. Always include `og` in page meta where available.

### Video Watermarking

Video processing produces **two separate sets of variants**: clean variants for streaming, and a watermarked variant for download. They are different keys in the variants map.

<table id="bkmrk-context-streaming-va"><thead><tr><th>Context</th><th>Streaming variants (HLS)</th><th>Watermarked download</th><th>Watermark style</th><th>Cycle</th></tr></thead><tbody><tr><td>`SOCIAL_VIDEO`</td><td>`master`, `360p_playlist`, `720p_playlist`, `1080p_playlist`</td><td>`720p_watermarked.mp4` (or `360p_watermarked.mp4`)</td><td>Diagonal 2-point: NexGate logo + `watermarkLabel` overlay</td><td>Every 5 seconds, alternates between upper-left and lower-right</td></tr><tr><td>`PRODUCT_VIDEO`</td><td>`master`, `360p_playlist`, `720p_playlist`, `1080p_playlist`</td><td>`720p_watermarked.mp4` (or `360p_watermarked.mp4`)</td><td>Text-only (`NexGate` label) cycling all 4 corners</td><td>Every 3 seconds</td></tr><tr><td>`DM_VIDEO`</td><td>`master`, `360p_playlist`, `720p_playlist`</td><td>None</td><td>No watermark</td><td>—</td></tr></tbody></table>

**Which watermarked key do you get?**

- If the source video is tall/wide enough for 720p → key is `720p_watermarked`
- If the source is smaller (e.g. a 360p source) → key is `360p_watermarked`
- Always check which key is present rather than assuming 720p

**When to use which**:

- Use `360p_clean` / `720p_clean` / `1080p_clean` for in-app streaming and playback
- Use `720p_watermarked` / `360p_watermarked` when you want to offer a **download** of the video — the watermark protects the content

**Social video outro**: `SOCIAL_VIDEO` files have a personalized branded outro clip appended (`watermarkLabel` + orange accent, no re-encode). The outro is baked into the `720p_watermarked` variant only, not the clean variants.

The `watermarkLabel` is passed to FileThunder by the backend on upload — you send nothing extra for this. For `SOCIAL_VIDEO` it is the user's `@handle`; for `PRODUCT_VIDEO` it is the `$shopName`.

---

### Video Variants — Short Clips (&lt; 3 minutes, HLS)

Short clips use HLS with the same portrait blur-pad filter (blurred background fills the frame). The `shortClip: true` flag in the response tells you it is a short clip.

<table id="bkmrk-key-format-typical-u-1"><thead><tr><th>Key</th><th>Format</th><th>Typical Use</th></tr></thead><tbody><tr><td>`master`</td><td>HLS master playlist URL</td><td>**Default — use this for all playback.** Plug into HLS.js or native `<video>` on Safari; adaptive quality switching is automatic</td></tr><tr><td>`360p_playlist`</td><td>HLS per-rendition playlist URL</td><td>First available at LIVE\_PARTIAL — only use when building a manual quality selector</td></tr><tr><td>`720p_playlist`</td><td>HLS per-rendition playlist URL</td><td>Only use when building a manual quality selector</td></tr><tr><td>`1080p_playlist`</td><td>HLS per-rendition playlist URL</td><td>Only use when building a manual quality selector (only present if source is 1080p-capable)</td></tr><tr><td>`720p_watermarked`</td><td>MP4 URL</td><td>Watermarked download variant — offer this for user downloads</td></tr><tr><td>`360p_watermarked`</td><td>MP4 URL</td><td>Watermarked download fallback — present instead of `720p_watermarked` when source is too small</td></tr><tr><td>`preview`</td><td>MP4 URL</td><td>Auto-generated 3-second silent preview clip (speed-doubled from 6s of footage at ~5% into the video) — use for hover previews on cards</td></tr><tr><td>`poster`</td><td>WebP URL</td><td>Best auto-selected frame — show as video thumbnail before play</td></tr><tr><td>`thumb`</td><td>WebP URL</td><td>Small thumbnail for cards and grids</td></tr><tr><td>`og`</td><td>WebP URL</td><td>1200×630 Open Graph image</td></tr><tr><td>`blurhash`</td><td>String</td><td>BlurHash string for placeholder</td></tr><tr><td>`lqip`</td><td>data URI</td><td>Inline WebP placeholder, no extra request</td></tr><tr><td>`dominant_color`</td><td>Hex string</td><td>Background tint for skeleton screens</td></tr></tbody></table>

### Video Variants — Long Form (≥ 3 minutes, HLS)

<table id="bkmrk-key-format-typical-u-2"><thead><tr><th>Key</th><th>Format</th><th>Typical Use</th></tr></thead><tbody><tr><td>`master`</td><td>HLS master playlist URL</td><td>**Default — use this for all playback.** Plug into HLS.js or native `<video>` on Safari; the player handles quality switching automatically based on bandwidth</td></tr><tr><td>`360p_playlist`</td><td>HLS per-rendition playlist URL</td><td>Only use when building a manual quality selector — swap the player src to this to force 360p</td></tr><tr><td>`720p_playlist`</td><td>HLS per-rendition playlist URL</td><td>Only use when building a manual quality selector — swap the player src to this to force 720p</td></tr><tr><td>`1080p_playlist`</td><td>HLS per-rendition playlist URL</td><td>Only use when building a manual quality selector — swap the player src to this to force 1080p (only present if source is 1080p-capable)</td></tr><tr><td>`preview`</td><td>MP4 URL</td><td>3-second preview clip for card hover effects</td></tr><tr><td>`poster`</td><td>WebP URL</td><td>Best auto-selected frame</td></tr><tr><td>`thumb`</td><td>WebP URL</td><td>Card thumbnail</td></tr><tr><td>`og`</td><td>WebP URL</td><td>Open Graph image</td></tr><tr><td>`blurhash`</td><td>String</td><td>BlurHash placeholder</td></tr><tr><td>`lqip`</td><td>data URI</td><td>Inline placeholder</td></tr><tr><td>`dominant_color`</td><td>Hex string</td><td>Background tint</td></tr></tbody></table>

> **HLS Note**: Always use the `master` key for adaptive streaming — it switches automatically between 360p/720p/1080p based on viewer bandwidth. Long-form videos do **not** include an MP4 fallback — if the environment does not support HLS, you will need to inform the user or handle it at the player level (e.g. HLS.js handles this for most browsers). Long-form videos do not produce a `watermarked` download variant — downloads of long videos are not supported.

---

## The Upload Flow — Step by Step

### Step 1 — Request a presigned upload URL (backend)

Call `POST /api/v1/files/request-upload` with the file metadata. You get back a `fileId` and a `presignedUrl`.

### Step 2 — Upload the file directly to object storage (MinIO/CDN)

Make an HTTP `PUT` request to the `presignedUrl` with the file bytes as the body. This request does **not** go to the backend — it goes directly to object storage. Use `XMLHttpRequest` (not `fetch`) if you want to track upload progress, because XHR exposes an `upload.onprogress` event that `fetch` does not.

The request must include:

- `Content-Type` header matching exactly the `mimeType` you sent in Step 1
- The raw file bytes as the body (no multipart, no form data)
- No `Authorization` header — the presigned URL is self-authenticating

A successful upload returns `HTTP 200` with an empty body.

**Upload progress tracking via XHR**: XHR fires `upload.onprogress` events with `loaded` (bytes sent) and `total` (total bytes). Divide `loaded / total` to get a 0–1 progress value for your progress bar. This tracks **network transfer progress**, not processing progress. The progress bar should complete at 100% when the PUT returns 200, then transition to the processing state UI.

### Step 3 — Track processing progress (SSE or polling)

After the PUT succeeds, open an SSE stream to `GET /api/v1/files/progress/{fileId}`. You will receive server-sent events as FileThunder processes the file. Close the stream when you receive `READY` or `FAILED`.

If SSE is inconvenient in your environment, use `GET /api/v1/files/status/{fileId}` to poll instead. Recommended polling interval: every 2–3 seconds.

### Step 4 — Entity response contains the variants

Once `READY`, the entity API (product, shop, post, etc.) will include the variant map in its response. You do not need to call any file endpoint to get URLs — they come embedded in the entity response.

---

## Endpoints

---

## 1. Request Upload URL

**Purpose**: Generate a presigned PUT URL and a `fileId` for a new file upload. Always call this before any file upload.

**Endpoint**: <span style="background-color: #007bff; color: white; padding: 4px 8px; border-radius: 4px; font-family: monospace; font-size: 12px; font-weight: bold;">POST</span> `{base_url}/files/request-upload`

**Access Level**: 🔒 Protected (Requires authenticated user)

**Authentication**: Bearer Token — `Authorization: Bearer <token>`

**Request Headers**:

<table id="bkmrk-header-type-required"><thead><tr><th>Header</th><th>Type</th><th>Required</th><th>Description</th></tr></thead><tbody><tr><td>`Authorization`</td><td>string</td><td>Yes</td><td>`Bearer <your_jwt_token>`</td></tr><tr><td>`Content-Type`</td><td>string</td><td>Yes</td><td>`application/json`</td></tr></tbody></table>

**Request JSON Sample**:

```json
{
  "context": "PRODUCT_IMAGE",
  "originalFilename": "shoe-red-side.jpg",
  "mimeType": "image/jpeg",
  "fileSizeBytes": 2457600
}

```

**Request Body Parameters**:

<table id="bkmrk-parameter-type-requi"><thead><tr><th>Parameter</th><th>Type</th><th>Required</th><th>Description</th><th>Validation</th></tr></thead><tbody><tr><td>`context`</td><td>string</td><td>Yes</td><td>The purpose of this file — determines processing pipeline and storage bucket</td><td>Must be one of the valid context values listed in the File Contexts section</td></tr><tr><td>`originalFilename`</td><td>string</td><td>Yes</td><td>Original name of the file including extension</td><td>Max 255 characters</td></tr><tr><td>`mimeType`</td><td>string</td><td>Yes</td><td>MIME type of the file exactly as the browser reports it</td><td>Must match the context's accepted formats; see File Format &amp; Size Constraints</td></tr><tr><td>`fileSizeBytes`</td><td>number</td><td>Yes</td><td>File size in bytes</td><td>Must be a positive integer</td></tr></tbody></table>

**Success Response JSON Sample**:

```json
{
  "success": true,
  "httpStatus": "OK",
  "message": "Upload URL generated",
  "action_time": "2026-06-21T14:22:10",
  "data": {
    "fileId": "a3f7c21b-09d4-4e8b-bf12-3c7d09e1f556",
    "presignedUrl": "https://storage.nexgate.com/nexgate-raw/products/.../shoe-red-side.jpg?X-Amz-Signature=...",
    "expiresInSeconds": 3600
  }
}

```

**Success Response Fields**:

<table id="bkmrk-field-description-da"><thead><tr><th>Field</th><th>Description</th></tr></thead><tbody><tr><td>`data.fileId`</td><td>UUID identifying this file — save this, you will need it for SSE tracking and to associate the file with an entity</td></tr><tr><td>`data.presignedUrl`</td><td>The URL to PUT the file bytes to — send directly to this URL, do not proxy through your server</td></tr><tr><td>`data.expiresInSeconds`</td><td>How many seconds before the presigned URL expires (typically 3600 = 1 hour) — start the upload immediately</td></tr></tbody></table>

**Error Response JSON Sample**:

```json
{
  "success": false,
  "httpStatus": "BAD_REQUEST",
  "message": "context is required. Valid values: SOCIAL_IMAGE, SOCIAL_VIDEO, PROFILE_PICTURE, ...",
  "action_time": "2026-06-21T14:22:10",
  "data": "context is required."
}

```

**Standard Error Types**:

- `400 BAD_REQUEST` — Missing or invalid `context`
- `401 UNAUTHORIZED` — Missing or expired Bearer token
- `422 UNPROCESSABLE_ENTITY` — Missing required fields

---

## 2. Upload File to Object Storage (Presigned PUT)

**Purpose**: Upload the raw file bytes directly to object storage using the presigned URL from Step 1. This is **not a backend endpoint** — it is a direct PUT to the object storage URL. It is documented here because understanding this step is essential to the flow.

**Endpoint**: <span style="background-color: #ffc107; color: black; padding: 4px 8px; border-radius: 4px; font-family: monospace; font-size: 12px; font-weight: bold;">PUT</span> `{presignedUrl}` *(the full URL returned in Step 1)*

**Access Level**: 🌐 Self-authenticated (The presigned URL carries authentication in its query parameters — no `Authorization` header needed or allowed)

**Request Headers**:

<table id="bkmrk-header-type-required-1"><thead><tr><th>Header</th><th>Type</th><th>Required</th><th>Description</th></tr></thead><tbody><tr><td>`Content-Type`</td><td>string</td><td>Yes</td><td>Must exactly match the `mimeType` sent in Step 1 — e.g. `image/jpeg`, `video/mp4`</td></tr></tbody></table>

**Request Body**: Raw file bytes. No multipart encoding. No form fields. Just the file content.

**Upload Progress**:

Use `XMLHttpRequest` for this PUT request to gain access to upload progress events. The `XMLHttpRequest.upload` object fires `progress` events containing:

<table id="bkmrk-property-type-descri"><thead><tr><th>Property</th><th>Type</th><th>Description</th></tr></thead><tbody><tr><td>`loaded`</td><td>number</td><td>Bytes sent so far</td></tr><tr><td>`total`</td><td>number</td><td>Total bytes to send (equals your `fileSizeBytes`)</td></tr></tbody></table>

Divide `loaded / total` to get a 0–1 ratio for a progress bar. This tracks **network transfer only** — once the PUT returns 200, transition the UI to the "Processing…" state and begin tracking via SSE (Step 3).

**Success Response**: `HTTP 200` with an empty body. No JSON.

**Failure Responses**:

<table id="bkmrk-status-cause-400-con"><thead><tr><th>Status</th><th>Cause</th></tr></thead><tbody><tr><td>`400`</td><td>`Content-Type` header does not match what was declared in Step 1</td></tr><tr><td>`403`</td><td>Presigned URL has expired — go back to Step 1 and request a new one</td></tr><tr><td>`413`</td><td>File exceeds the size declared in `fileSizeBytes` in Step 1</td></tr></tbody></table>

---

## 3. Stream File Processing Progress (SSE)

**Purpose**: Receive real-time server-sent events as FileThunder processes the uploaded file. Open this stream immediately after the PUT in Step 2 returns 200. The stream closes automatically when `READY` or `FAILED` is reached.

**Endpoint**: <span style="background-color: #28a745; color: white; padding: 4px 8px; border-radius: 4px; font-family: monospace; font-size: 12px; font-weight: bold;">GET</span> `{base_url}/files/progress/{fileId}`

**Access Level**: 🔒 Protected (Only the file owner can stream progress for a given `fileId`)

**Authentication**: Bearer Token — `Authorization: Bearer <token>`

**Request Headers**:

<table id="bkmrk-header-type-required-2"><thead><tr><th>Header</th><th>Type</th><th>Required</th><th>Description</th></tr></thead><tbody><tr><td>`Authorization`</td><td>string</td><td>Yes</td><td>`Bearer <your_jwt_token>`</td></tr><tr><td>`Accept`</td><td>string</td><td>Yes</td><td>`text/event-stream`</td></tr></tbody></table>

**Path Parameters**:

<table id="bkmrk-parameter-type-requi-1"><thead><tr><th>Parameter</th><th>Type</th><th>Required</th><th>Description</th><th>Validation</th></tr></thead><tbody><tr><td>`fileId`</td><td>UUID</td><td>Yes</td><td>The `fileId` returned in Step 1</td><td>Must be a valid UUID owned by the authenticated user</td></tr></tbody></table>

**Response**: This endpoint returns a stream of Server-Sent Events, not a JSON body. Each event has a `name` (the status) and `data` (JSON payload).

**SSE Event Format**:

```
event: PROCESSING
data: {"status":"PROCESSING","fileId":"a3f7c21b-09d4-4e8b-bf12-3c7d09e1f556"}

event: LIVE_PARTIAL
data: {"status":"LIVE_PARTIAL","fileId":"a3f7c21b-09d4-4e8b-bf12-3c7d09e1f556"}

event: READY
data: {"status":"READY","fileId":"a3f7c21b-09d4-4e8b-bf12-3c7d09e1f556"}

```

**SSE Event Sequence**:

*Short video:*

```
PENDING → UPLOADING → UPLOADED → PROCESSING → LIVE_PARTIAL → READY

```

*Image:*

```
PENDING → UPLOADING → UPLOADED → PROCESSING → READY

```

*Long video (≥ 3 minutes):*

```
PENDING → UPLOADING → UPLOADED → PROCESSING → LIVE_PARTIAL → READY

```

*Digital product or DM document:*

```
PENDING → UPLOADING → UPLOADED → SCANNING → READY

```

**Event Handling**:

<table id="bkmrk-event-name-action-pe"><thead><tr><th>Event name</th><th>Action</th></tr></thead><tbody><tr><td>`PENDING`</td><td>Show spinner</td></tr><tr><td>`UPLOADING`</td><td>Show spinner (XHR progress bar already running from Step 2)</td></tr><tr><td>`UPLOADED`</td><td>Show "Processing…" state</td></tr><tr><td>`SCANNING`</td><td>Show "Scanning for safety…" state</td></tr><tr><td>`PROCESSING`</td><td>Show "Processing…" state</td></tr><tr><td>`LIVE_PARTIAL`</td><td>Video 360p is available — can begin playback, show "HD loading" badge</td></tr><tr><td>`READY`</td><td>**Close the SSE connection. Show success. Refresh entity data.**</td></tr><tr><td>`FAILED`</td><td>**Close the SSE connection. Show error. Offer re-upload.**</td></tr></tbody></table>

**SSE Timeout**: The stream has a 5-minute server-side timeout. If your file is still processing after 5 minutes (large video), reconnect to this endpoint — it will immediately send the current status snapshot before continuing to stream.

**Error Responses**:

- `401 UNAUTHORIZED` — Invalid or missing token
- `404 NOT_FOUND` — `fileId` not found or does not belong to the authenticated user

---

## 4. Poll File Processing Status

**Purpose**: An alternative to SSE for environments where persistent connections are difficult (e.g. React Native background states, certain browser extensions). Returns the current processing status as a single JSON response. Poll every 2–3 seconds; stop when `ready` is `true` or `failed` is `true`.

**Endpoint**: <span style="background-color: #28a745; color: white; padding: 4px 8px; border-radius: 4px; font-family: monospace; font-size: 12px; font-weight: bold;">GET</span> `{base_url}/files/status/{fileId}`

**Access Level**: 🔒 Protected (Only the file owner)

**Authentication**: Bearer Token — `Authorization: Bearer <token>`

**Request Headers**:

<table id="bkmrk-header-type-required-3"><thead><tr><th>Header</th><th>Type</th><th>Required</th><th>Description</th></tr></thead><tbody><tr><td>`Authorization`</td><td>string</td><td>Yes</td><td>`Bearer <your_jwt_token>`</td></tr></tbody></table>

**Path Parameters**:

<table id="bkmrk-parameter-type-requi-2"><thead><tr><th>Parameter</th><th>Type</th><th>Required</th><th>Description</th><th>Validation</th></tr></thead><tbody><tr><td>`fileId`</td><td>UUID</td><td>Yes</td><td>The `fileId` returned in Step 1</td><td>Must be a valid UUID owned by the authenticated user</td></tr></tbody></table>

**Success Response JSON Sample**:

```json
{
  "success": true,
  "httpStatus": "OK",
  "message": "Status retrieved",
  "action_time": "2026-06-21T14:23:45",
  "data": {
    "fileId": "a3f7c21b-09d4-4e8b-bf12-3c7d09e1f556",
    "status": "PROCESSING",
    "ready": false,
    "failed": false,
    "processing": true
  }
}

```

**Success Response Fields**:

<table id="bkmrk-field-type-descripti-1"><thead><tr><th>Field</th><th>Type</th><th>Description</th></tr></thead><tbody><tr><td>`data.fileId`</td><td>UUID</td><td>The file identifier</td></tr><tr><td>`data.status`</td><td>string</td><td>Current status — one of `PENDING`, `UPLOADING`, `UPLOADED`, `SCANNING`, `PROCESSING`, `LIVE_PARTIAL`, `READY`, `FAILED`</td></tr><tr><td>`data.ready`</td><td>boolean</td><td>`true` when file is fully processed and variants are available</td></tr><tr><td>`data.failed`</td><td>boolean</td><td>`true` when processing failed — stop polling and show error</td></tr><tr><td>`data.processing`</td><td>boolean</td><td>`true` while processing is in progress — continue polling</td></tr></tbody></table>

**Ready state response**:

```json
{
  "success": true,
  "httpStatus": "OK",
  "message": "Status retrieved",
  "action_time": "2026-06-21T14:24:10",
  "data": {
    "fileId": "a3f7c21b-09d4-4e8b-bf12-3c7d09e1f556",
    "status": "READY",
    "ready": true,
    "failed": false,
    "processing": false
  }
}

```

**Error Response JSON Sample**:

```json
{
  "success": false,
  "httpStatus": "NOT_FOUND",
  "message": "File not found or access denied",
  "action_time": "2026-06-21T14:24:10",
  "data": "File not found or access denied"
}

```

**Standard Error Types**:

- `401 UNAUTHORIZED` — Invalid or missing token
- `404 NOT_FOUND` — `fileId` not found or does not belong to authenticated user

---

## 5. Replace Video Thumbnail

**Purpose**: Upload a custom thumbnail image to replace the auto-selected thumbnail for a video file. Call this after the video has reached `READY` status. Returns a presigned URL — upload the image bytes to it (same pattern as Step 2 above).

**Endpoint**: <span style="background-color: #007bff; color: white; padding: 4px 8px; border-radius: 4px; font-family: monospace; font-size: 12px; font-weight: bold;">POST</span> `{base_url}/files/thumbnail/{fileId}`

**Access Level**: 🔒 Protected (Only the video owner)

**Authentication**: Bearer Token — `Authorization: Bearer <token>`

**Request Headers**:

<table id="bkmrk-header-type-required-4"><thead><tr><th>Header</th><th>Type</th><th>Required</th><th>Description</th></tr></thead><tbody><tr><td>`Authorization`</td><td>string</td><td>Yes</td><td>`Bearer <your_jwt_token>`</td></tr></tbody></table>

**Path Parameters**:

<table id="bkmrk-parameter-type-requi-3"><thead><tr><th>Parameter</th><th>Type</th><th>Required</th><th>Description</th><th>Validation</th></tr></thead><tbody><tr><td>`fileId`</td><td>UUID</td><td>Yes</td><td>The `fileId` of the video whose thumbnail you want to replace</td><td>Must be a valid video file owned by the authenticated user</td></tr></tbody></table>

**Request Body**: None — this is a POST with no body.

**Success Response JSON Sample**:

```json
{
  "success": true,
  "httpStatus": "OK",
  "message": "Thumbnail upload URL generated",
  "action_time": "2026-06-21T14:25:00",
  "data": {
    "fileId": "a3f7c21b-09d4-4e8b-bf12-3c7d09e1f556",
    "presignedUrl": "https://storage.nexgate.com/nexgate-raw/products/.../thumb-a3f7c21b.jpg?X-Amz-Signature=...",
    "expiresInSeconds": 3600
  }
}

```

**Success Response Fields**:

<table id="bkmrk-field-description-da-1"><thead><tr><th>Field</th><th>Description</th></tr></thead><tbody><tr><td>`data.fileId`</td><td>The video file's ID</td></tr><tr><td>`data.presignedUrl`</td><td>PUT the thumbnail image bytes here — same process as the main upload in Step 2</td></tr><tr><td>`data.expiresInSeconds`</td><td>Seconds until the presigned URL expires</td></tr></tbody></table>

**After uploading**: PUT `image/jpeg` or `image/png` bytes to the `presignedUrl`. FileThunder will generate the thumbnail variants and update the video's effective thumbnail. Refresh the entity response to get the updated thumbnail URLs.

**Error Response JSON Sample**:

```json
{
  "success": false,
  "httpStatus": "UNAUTHORIZED",
  "message": "Token has expired",
  "action_time": "2026-06-21T14:25:00",
  "data": "Token has expired"
}

```

**Standard Error Types**:

- `401 UNAUTHORIZED` — Invalid or missing token
- `404 NOT_FOUND` — `fileId` not found

---

## 6. List Digital Files Available for Download (Order)

**Purpose**: For a purchased digital product order, list all the files the buyer is entitled to download, along with download availability and remaining download counts.

**Endpoint**: <span style="background-color: #28a745; color: white; padding: 4px 8px; border-radius: 4px; font-family: monospace; font-size: 12px; font-weight: bold;">GET</span> `{base_url}/e-commerce/orders/{orderId}/downloads`

**Access Level**: 🔒 Protected (Only the order buyer)

**Authentication**: Bearer Token — `Authorization: Bearer <token>`

**Request Headers**:

<table id="bkmrk-header-type-required-5"><thead><tr><th>Header</th><th>Type</th><th>Required</th><th>Description</th></tr></thead><tbody><tr><td>`Authorization`</td><td>string</td><td>Yes</td><td>`Bearer <your_jwt_token>`</td></tr></tbody></table>

**Path Parameters**:

<table id="bkmrk-parameter-type-requi-4"><thead><tr><th>Parameter</th><th>Type</th><th>Required</th><th>Description</th><th>Validation</th></tr></thead><tbody><tr><td>`orderId`</td><td>UUID</td><td>Yes</td><td>The order ID containing digital files</td><td>Must be an order owned by the authenticated buyer</td></tr></tbody></table>

**Success Response JSON Sample**:

```json
{
  "success": true,
  "httpStatus": "OK",
  "message": "3 file(s) available for download",
  "action_time": "2026-06-21T14:26:00",
  "data": [
    {
      "fileId": "b9c1d33e-22f4-4a0b-9e67-1d4f22c3a881",
      "fileName": "nexgate-design-kit-v2.zip",
      "contentType": "application/zip",
      "fileSize": 52428800,
      "downloadCount": 1,
      "downloadsRemaining": 4,
      "accessExpiresAt": "2026-12-21T00:00:00",
      "canDownload": true
    },
    {
      "fileId": "c2e5f44a-33g5-5b1c-af78-2e5g33d4b992",
      "fileName": "user-manual.pdf",
      "contentType": "application/pdf",
      "fileSize": 1048576,
      "downloadCount": 0,
      "downloadsRemaining": 5,
      "accessExpiresAt": "2026-12-21T00:00:00",
      "canDownload": true
    }
  ]
}

```

**Success Response Fields**:

<table id="bkmrk-field-type-descripti-2"><thead><tr><th>Field</th><th>Type</th><th>Description</th></tr></thead><tbody><tr><td>`[].fileId`</td><td>UUID</td><td>File identifier — use this in the next endpoint to get a download URL</td></tr><tr><td>`[].fileName`</td><td>string</td><td>Original filename including extension</td></tr><tr><td>`[].contentType`</td><td>string</td><td>MIME type of the file</td></tr><tr><td>`[].fileSize`</td><td>number</td><td>File size in bytes</td></tr><tr><td>`[].downloadCount`</td><td>number</td><td>How many times the buyer has already downloaded this file</td></tr><tr><td>`[].downloadsRemaining`</td><td>number</td><td>How many more downloads the buyer is allowed</td></tr><tr><td>`[].accessExpiresAt`</td><td>string</td><td>ISO 8601 datetime after which download access expires</td></tr><tr><td>`[].canDownload`</td><td>boolean</td><td>`false` if download limit reached or access has expired — show disabled button</td></tr></tbody></table>

**Error Response JSON Sample**:

```json
{
  "success": false,
  "httpStatus": "NOT_FOUND",
  "message": "Order not found",
  "action_time": "2026-06-21T14:26:00",
  "data": "Order not found"
}

```

**Standard Error Types**:

- `400 BAD_REQUEST` — Order does not contain digital files, or buyer access is expired
- `401 UNAUTHORIZED` — Invalid or missing token
- `404 NOT_FOUND` — Order not found or does not belong to authenticated user

---

## 7. Generate Digital File Download URL

**Purpose**: Generate a time-limited, single-use download URL for a specific digital file in an order. Call this endpoint when the user clicks the download button — do not cache this URL. Open it immediately in a new tab or trigger a browser download.

**Endpoint**: <span style="background-color: #28a745; color: white; padding: 4px 8px; border-radius: 4px; font-family: monospace; font-size: 12px; font-weight: bold;">GET</span> `{base_url}/e-commerce/orders/{orderId}/downloads/{fileId}`

**Access Level**: 🔒 Protected (Only the order buyer)

**Authentication**: Bearer Token — `Authorization: Bearer <token>`

**Request Headers**:

<table id="bkmrk-header-type-required-6"><thead><tr><th>Header</th><th>Type</th><th>Required</th><th>Description</th></tr></thead><tbody><tr><td>`Authorization`</td><td>string</td><td>Yes</td><td>`Bearer <your_jwt_token>`</td></tr></tbody></table>

**Path Parameters**:

<table id="bkmrk-parameter-type-requi-5"><thead><tr><th>Parameter</th><th>Type</th><th>Required</th><th>Description</th><th>Validation</th></tr></thead><tbody><tr><td>`orderId`</td><td>UUID</td><td>Yes</td><td>The order ID</td><td>Must belong to the authenticated buyer</td></tr><tr><td>`fileId`</td><td>UUID</td><td>Yes</td><td>The specific file to download (from endpoint 6)</td><td>Must be a file within the specified order</td></tr></tbody></table>

**Success Response JSON Sample**:

```json
{
  "success": true,
  "httpStatus": "OK",
  "message": "Download URL generated — link expires in 15 minutes",
  "action_time": "2026-06-21T14:27:00",
  "data": {
    "fileId": "b9c1d33e-22f4-4a0b-9e67-1d4f22c3a881",
    "fileName": "nexgate-design-kit-v2.zip",
    "downloadUrl": "https://storage.nexgate.com/nexgate-digital/products/.../nexgate-design-kit-v2.zip?X-Amz-Signature=...&X-Amz-Expires=300",
    "expiresAt": "2026-06-21T14:32:00",
    "downloadsRemaining": 3,
    "downloadCount": 2
  }
}

```

**Success Response Fields**:

<table id="bkmrk-field-type-descripti-3"><thead><tr><th>Field</th><th>Type</th><th>Description</th></tr></thead><tbody><tr><td>`data.fileId`</td><td>UUID</td><td>File identifier</td></tr><tr><td>`data.fileName`</td><td>string</td><td>Filename to use when saving locally</td></tr><tr><td>`data.downloadUrl`</td><td>string</td><td>Pre-signed download URL — valid for **15 minutes only**. Open immediately. Do not cache.</td></tr><tr><td>`data.expiresAt`</td><td>string</td><td>ISO 8601 datetime when the download URL expires</td></tr><tr><td>`data.downloadsRemaining`</td><td>number</td><td>How many downloads remain after this one</td></tr><tr><td>`data.downloadCount`</td><td>number</td><td>Total downloads made so far including this one</td></tr></tbody></table>

**Error Response JSON Sample**:

```json
{
  "success": false,
  "httpStatus": "BAD_REQUEST",
  "message": "Download limit reached for this file",
  "action_time": "2026-06-21T14:27:00",
  "data": "Download limit reached for this file"
}

```

**Standard Error Types**:

- `400 BAD_REQUEST` — Download limit reached, access expired, or buyer is not eligible
- `401 UNAUTHORIZED` — Invalid or missing token
- `404 NOT_FOUND` — Order or file not found

---

## How Variants Appear in Entity Responses

You never call a dedicated endpoint to get variant URLs. When an entity (product, post, shop, profile, event) is fetched via its own API, the response already contains the assembled variant map. Here is what to expect per entity type:

### Product Image Variants

```json
{
  "productImageVariants": [
    {
      "large": "https://cdn.nexgate.com/products/owner123/fileabc/large.webp",
      "medium": "https://cdn.nexgate.com/products/owner123/fileabc/medium.webp",
      "thumb": "https://cdn.nexgate.com/products/owner123/fileabc/thumb.webp",
      "og": "https://cdn.nexgate.com/products/owner123/fileabc/og.webp",
      "blurhash": "LGF5?xYk^6#M@-5c,1J5@[or[Q6",
      "lqip": "data:image/webp;base64,/9j/4AAQ...",
      "dominant_color": "#C8A882"
    }
  ]
}

```

`productImageVariants` is an array — one entry per uploaded image. Index 0 is the primary image.

### Video Variants (short clip, HLS)

```json
{
  "previewVariants": {
    "master": "https://cdn.nexgate.com/products/owner123/filevid/hls/master.m3u8",
    "360p_playlist": "https://cdn.nexgate.com/products/owner123/filevid/hls/360p/360p.m3u8",
    "720p_playlist": "https://cdn.nexgate.com/products/owner123/filevid/hls/720p/720p.m3u8",
    "1080p_playlist": "https://cdn.nexgate.com/products/owner123/filevid/hls/1080p/1080p.m3u8",
    "720p_watermarked": "https://cdn.nexgate.com/products/owner123/filevid/720p_watermarked.mp4",
    "preview": "https://cdn.nexgate.com/products/owner123/filevid/preview_3s.mp4",
    "poster": "https://cdn.nexgate.com/products/owner123/filevid/poster.webp",
    "thumb": "https://cdn.nexgate.com/products/owner123/filevid/thumb.webp",
    "og": "https://cdn.nexgate.com/products/owner123/filevid/og.webp",
    "blurhash": "LGF5?xYk^6#M@-5c,1J5@[or[Q6",
    "lqip": "data:image/webp;base64,/9j/4AAQ...",
    "dominant_color": "#1A1A2E"
  }
}

```

> If the source video was too small for 720p, `720p_watermarked` will be absent and `360p_watermarked` will be present instead. Always check which watermarked key exists before rendering a download button.

### Video Variants (long form / HLS)

```json
{
  "previewVariants": {
    "master": "https://cdn.nexgate.com/products/owner123/filevid/hls/master.m3u8",
    "360p_playlist": "https://cdn.nexgate.com/products/owner123/filevid/hls/360p/360p.m3u8",
    "720p_playlist": "https://cdn.nexgate.com/products/owner123/filevid/hls/720p/720p.m3u8",
    "1080p_playlist": "https://cdn.nexgate.com/products/owner123/filevid/hls/1080p/1080p.m3u8",
    "preview": "https://cdn.nexgate.com/products/owner123/filevid/preview_3s.mp4",
    "poster": "https://cdn.nexgate.com/products/owner123/filevid/poster.webp",
    "thumb": "https://cdn.nexgate.com/products/owner123/filevid/thumb.webp",
    "og": "https://cdn.nexgate.com/products/owner123/filevid/og.webp",
    "blurhash": "LGF5?xYk^6#M@-5c,1J5@[or[Q6",
    "lqip": "data:image/webp;base64,/9j/4AAQ...",
    "dominant_color": "#1A1A2E"
  }
}

```

> Feed `master` into HLS.js or a native `<video>` src on Safari. Long-form videos do not include an MP4 fallback and do not have a watermarked download variant.

### LIVE\_PARTIAL state (video still processing)

When you get an entity response while the video is in `LIVE_PARTIAL`:

- **Short clips and long form (both HLS)**: only `360p_playlist` and `master` (pointing to the 360p-only rendition) are present. Higher renditions, thumbnail keys, and the watermarked MP4 download arrive once `READY`.

You can safely begin playback in both cases and show a "HD loading" badge.

### Null variants

If a file is still in `PROCESSING` and you fetch the entity, `variants` fields may be `null` or an empty array. Always null-check before rendering. Display a placeholder (blurhash, dominant\_color, or a skeleton) until variants are available.

---

## Best Practices

### Upload

- **Always validate before requesting a presigned URL.** Check file size and MIME type on the client before making the Step 1 request. Rejecting obvious bad inputs early avoids wasted presigned URL slots.
- **Start the SSE connection before the PUT finishes.** Open the SSE stream as soon as you receive `fileId`, so you do not miss early status events like `UPLOADING`.
- **Never proxy the file through your frontend server.** The presigned URL uploads directly to object storage. Do not relay bytes through a backend API route.
- **Handle presigned URL expiry.** If a user leaves the upload screen and comes back, the presigned URL may have expired. Call Step 1 again to get a fresh one.
- **Send exactly the declared `mimeType` as `Content-Type`** in the PUT. A mismatch causes a `400` from object storage.

### Progress UI

- Use **XHR upload progress** for the bytes-in-flight phase (Step 2). Show a numeric percentage or progress bar.
- Transition to a **spinner or indeterminate progress** at 100% upload — processing time is unpredictable.
- For **short images**, processing is usually 2–5 seconds.
- For **short videos (&lt; 3 min)**, processing typically takes 30 seconds to 2 minutes.
- For **long videos (≥ 3 min)**, processing can take 5–15 minutes. Show `LIVE_PARTIAL` content early at 360p with an overlay badge.
- On `FAILED`, offer a clear re-upload action — do not auto-retry silently.

### SSE vs Polling

<table id="bkmrk-use-sse-when-use-pol"><thead><tr><th>Use SSE when</th><th>Use polling when</th></tr></thead><tbody><tr><td>Web browser context</td><td>React Native / mobile apps</td></tr><tr><td>Single in-progress upload screen</td><td>Background upload (app minimised)</td></tr><tr><td>You can keep the tab open</td><td>Network conditions drop connections often</td></tr></tbody></table>

SSE is preferred — it is lower overhead and gives instant events. Polling at 2-second intervals is a reliable fallback.

### Variants

- **Always show a placeholder first.** Use `lqip` as an inline `src` or apply `blurhash` as a CSS background immediately — before the real image loads.
- **Use `dominant_color`** as the background tint on skeleton screens and image containers before any image variant is available.
- **Use `medium` for feed cards and grids**, not `large`. `large` is for detail views and lightboxes only.
- **Always set `og` in Open Graph meta tags** when rendering shareable pages (products, posts, events).
- **For HLS video**, prefer `master.m3u8` on supported browsers. Long-form videos have no MP4 fallback — use HLS.js to cover non-native HLS environments (most Android browsers, older Chrome).
- **Do not hardcode or cache variant URLs.** They come from entity responses. If a CDN or storage URL base changes, your app automatically picks up the new URLs without any code change.

### Digital Downloads

- **Never store download URLs.** They expire in 15 minutes. Always call endpoint 7 on each user-initiated download click.
- **Check `canDownload` from endpoint 6** before showing the download button. If `false`, show a disabled button with a reason ("Download limit reached" or "Access expired").
- **Open the `downloadUrl` in a new browser tab** or trigger a native download — do not `fetch()` it through your app.
- **Re-fetch endpoint 6 after each download** to update the `downloadsRemaining` count shown to the user.

---

## Quick Reference

### Context → Variants Cheat Sheet

<table id="bkmrk-context-variants-gen"><thead><tr><th>Context</th><th>Variants Generated</th></tr></thead><tbody><tr><td>`SOCIAL_IMAGE`</td><td>large, medium, thumb, og, blurhash, lqip, dominant\_color</td></tr><tr><td>`SOCIAL_VIDEO` (short, HLS)</td><td>master, 360p\_playlist, 720p\_playlist, 1080p\_playlist, 720p\_watermarked (or 360p\_watermarked), preview, poster, thumb, og, blurhash, lqip, dominant\_color</td></tr><tr><td>`SOCIAL_VIDEO` (long, HLS)</td><td>master, 360p\_playlist, 720p\_playlist, 1080p\_playlist, preview, poster, thumb, og, blurhash, lqip, dominant\_color</td></tr><tr><td>`PROFILE_PICTURE`</td><td>medium, thumb, blurhash, lqip</td></tr><tr><td>`COVER_PHOTO`</td><td>large, thumb, og, blurhash, lqip</td></tr><tr><td>`DM_IMAGE`</td><td>medium, thumb, blurhash, lqip</td></tr><tr><td>`DM_VIDEO`</td><td>master, 360p\_playlist, 720p\_playlist</td></tr><tr><td>`DM_DOCUMENT`</td><td>*(none — secure download only)*</td></tr><tr><td>`PRODUCT_IMAGE`</td><td>large, medium, thumb, og, blurhash, lqip, dominant\_color</td></tr><tr><td>`PRODUCT_VIDEO` (short, HLS)</td><td>master, 360p\_playlist, 720p\_playlist, 1080p\_playlist, 720p\_watermarked (or 360p\_watermarked), preview, poster, thumb, og, blurhash, lqip, dominant\_color</td></tr><tr><td>`PRODUCT_VIDEO` (long, HLS)</td><td>master, 360p\_playlist, 720p\_playlist, 1080p\_playlist, preview, poster, thumb, og, blurhash, lqip, dominant\_color</td></tr><tr><td>`DIGITAL_PRODUCT`</td><td>*(none — secure download only)*</td></tr><tr><td>`SHOP_BANNER`</td><td>large, thumb, og, blurhash, lqip</td></tr><tr><td>`SHOP_LOGO`</td><td>medium, thumb, blurhash, lqip</td></tr><tr><td>`EVENT_COVER`</td><td>large, medium, thumb, og, blurhash, lqip</td></tr><tr><td>`EVENT_GALLERY`</td><td>large, medium, thumb, blurhash, lqip</td></tr></tbody></table>

### Status Codes

<table id="bkmrk-code-meaning-200-ok-"><thead><tr><th>Code</th><th>Meaning</th></tr></thead><tbody><tr><td>`200 OK`</td><td>Success</td></tr><tr><td>`400 BAD_REQUEST`</td><td>Invalid request data, limit reached, or item already exists</td></tr><tr><td>`401 UNAUTHORIZED`</td><td>Missing, expired, or invalid token</td></tr><tr><td>`403 FORBIDDEN`</td><td>Authenticated but not permitted (not the file owner, not the buyer)</td></tr><tr><td>`404 NOT_FOUND`</td><td>File, order, or resource not found</td></tr><tr><td>`422 UNPROCESSABLE_ENTITY`</td><td>Validation error — response `data` will contain field-level error messages</td></tr><tr><td>`500 INTERNAL_SERVER_ERROR`</td><td>Server error — report to backend team with `action_time`</td></tr></tbody></table>

### Authentication

All endpoints require: `Authorization: Bearer <your_jwt_token>`

Tokens are obtained from the auth endpoints (see Auth API documentation). A `401` means the token is expired or invalid — redirect the user to login.

# CDN & File Delivery — Full Walkthrough

This is the learning-session writeup for how Cloudflare fits into File Thunder's delivery story. It complements `FILE_THUNDER.md` §8 (which is the terse reference version) with full end-to-end walkthroughs, diagrams, and setup steps.

**Nothing in this doc changes the processing pipeline.** FFmpeg/ImageMagick/ClamAV/watermarking stay exactly as they are. This is only about the *last mile* — how a finished file reaches a viewer.

---

## Table of Contents

1. [The Core Idea, In One Picture](#1-the-core-idea-in-one-picture)
2. [The Three Delivery Lanes](#2-the-three-delivery-lanes)
3. [Category → Lane Mapping](#3-category--lane-mapping)
4. [How Cloudflare Physically Sits in the Middle](#4-how-cloudflare-physically-sits-in-the-middle)
5. [Walkthrough A — Social Video (Lane 1, Public/Cached)](#5-walkthrough-a--social-video-lane-1-publiccached)
6. [Walkthrough B — Direct Message Image (Lane 2, Private/Bypassed)](#6-walkthrough-b--direct-message-image-lane-2-privatebypassed)
7. [Group Chats — Same Lane, Different Scale](#7-group-chats--same-lane-different-scale)
8. [Voice Notes &amp; Live Streaming — Do the Same Rules Apply?](#8-voice-notes--live-streaming--do-the-same-rules-apply)
9. [Video/Audio Calls — Not a File Thunder Concern](#9-videoaudio-calls--not-a-file-thunder-concern)
10. [Walkthrough C — Digital Product Purchase (Lane 3, Protected/Audited)](#10-walkthrough-c--digital-product-purchase-lane-3-protectedaudited)
11. [Cache Invalidation — The Only Sharp Edge](#11-cache-invalidation--the-only-sharp-edge)
12. [Cloudflare Setup Checklist](#12-cloudflare-setup-checklist)
13. [Video-at-Scale Caveat](#13-video-at-scale-caveat)

---

## 1. The Core Idea, In One Picture

Cloudflare is **not storage**. It's a worldwide network of caches sitting in front of your one real origin (MinIO). Think of MinIO as a single warehouse, and Cloudflare as thousands of small local libraries that keep copies of the popular stuff close to whoever's asking.

```
                         ┌───────────────────────────────────────────┐
                         │              CLOUDFLARE EDGE               │
                         │   (hundreds of mini-caches worldwide)      │
                         │                                             │
   Bob in Nairobi ──────▶│   Nairobi edge node                        │
                         │      │                                     │
                         │      │  cache HIT  → serve instantly        │
                         │      │  cache MISS → ask origin, cache it   │
                         └──────┼─────────────────────────────────────┘
                                │
                                │  (only on a cache miss, via secure tunnel)
                                ▼
                     ┌─────────────────────┐
                     │   MinIO (your VPS)   │
                     │   nexgate-public     │◀── ONLY this bucket is tunnel-reachable
                     │   nexgate-private    │◀── never exposed to Cloudflare
                     │   nexgate-digital    │◀── never exposed to Cloudflare
                     │   nexgate-raw        │◀── never exposed to anything external
                     └─────────────────────┘

```

Every request after the first one, from anyone near that edge node, never touches your VPS at all.

---

## 2. The Three Delivery Lanes

Every file File Thunder ever produces falls into exactly one lane. The lane is decided by **who is allowed to see the file**, nothing else.

```
┌────────────────────┬───────────────────────┬────────────────────────┬───────────────────────┐
│                     │  LANE 1: PUBLIC       │  LANE 2: PRIVATE       │  LANE 3: PROTECTED     │
├────────────────────┼───────────────────────┼────────────────────────┼───────────────────────┤
│ Who can view it     │ Anyone                │ The 1-2 people in the  │ Only the verified      │
│                     │                       │ conversation           │ buyer                  │
├────────────────────┼───────────────────────┼────────────────────────┼───────────────────────┤
│ Cloudflare cache?   │ YES — cached forever  │ NO — always bypassed   │ NO — always bypassed   │
├────────────────────┼───────────────────────┼────────────────────────┼───────────────────────┤
│ URL type            │ Plain permanent URL   │ Signed, 15-min expiry  │ Signed, 15-min expiry  │
│                     │ cdn.nexgate.com/key   │ straight from MinIO    │ + audit log + re-scan  │
├────────────────────┼───────────────────────┼────────────────────────┼───────────────────────┤
│ Bucket              │ nexgate-public        │ nexgate-private        │ nexgate-digital        │
├────────────────────┼───────────────────────┼────────────────────────┼───────────────────────┤
│ Who assembles URL   │ Main backend, no call │ Main backend calls FT  │ Main backend calls FT  │
│                     │ to File Thunder needed│ generateDownloadUrl()  │ generateDownloadUrl()  │
└────────────────────┴───────────────────────┴────────────────────────┴───────────────────────┘

```

**Why Lane 2/3 can never be cached:** their URLs are unique per request (a signature + expiry baked into the query string). Caching is keyed on the URL — a signed URL is never requested twice, so there's nothing to cache, and worse, a shared cache is the last place private bytes should sit.

---

## 3. Category → Lane Mapping

Every `MediaContext` value maps to exactly one lane:

```
LANE 1 — PUBLIC / CDN-CACHED (nexgate-public)
├── SOCIAL_IMAGE, SOCIAL_VIDEO
├── PROFILE_PICTURE, COVER_PHOTO
├── PRODUCT_IMAGE, PRODUCT_VIDEO
├── PRODUCT_PREVIEW_IMAGE, PRODUCT_PREVIEW_VIDEO   ← teasers, meant to be public
├── SHOP_BANNER, SHOP_LOGO
└── EVENT_COVER, EVENT_GALLERY

LANE 2 — PRIVATE / SIGNED, NEVER CACHED (nexgate-private)
├── DM_IMAGE
├── DM_VIDEO
└── DM_DOCUMENT

LANE 3 — PROTECTED / SIGNED + AUDITED, NEVER CACHED (nexgate-digital)
├── DIGITAL_PRODUCT
└── PRODUCT_PREVIEW_DOCUMENT   ← a document preview is still gated like a purchase

```

This matches `DOWNLOAD_CONTEXTS` already defined in `MediaQueryServiceImpl` — Lane 2 + Lane 3 together are exactly the contexts that go through `generateDownloadUrl()`. Everything else is Lane 1.

---

## 4. How Cloudflare Physically Sits in the Middle

Three ingredients, none of which touch your Java code:

1. **A subdomain**: `cdn.nexgate.com` (and `cdn-staging.nexgate.com` for staging).
2. **DNS pointed at Cloudflare** for that subdomain, "proxied" (orange cloud ON) — this is what makes Cloudflare the middleman instead of a pass-through record.
3. **A Cloudflare Tunnel** (`cloudflared`, a small background process on your VPS) that opens an *outbound* connection from your server to Cloudflare. This is the important safety property:

```
   WITHOUT a tunnel                       WITH a tunnel
   ─────────────────                       ─────────────
   Internet ──▶ your VPS:9000              Internet ──▶ Cloudflare ──▶ tunnel ──▶ your VPS
   (port must be open to the world,        (port 9000 is never opened to the world —
    firewall rules, real attack surface)    the VPS "calls out", nobody calls in)

```

The tunnel is configured to route **only** to `nexgate-public`. `nexgate-private`, `nexgate-digital`, and `nexgate-raw` are not part of the tunnel's routing table — Cloudflare has no path to them even if it wanted one.

---

## 5. Walkthrough A — Social Video (Lane 1, Public/Cached)

Alice posts a 40-second clip. Bob, her friend, watches it later.

```
 ALICE (upload)                                    BOB (view, later)
 ──────────────                                    ─────────────────
 1. Tap "post"                                     6. Opens app, scrolls to Alice's post
        │                                                  │
        ▼                                                  ▼
 2. Main backend → File Thunder:                   7. Main backend reads post from its DB,
    "need upload slot for SOCIAL_VIDEO"                sees context = SOCIAL_VIDEO (Lane 1)
        │                                                  │
        ▼                                                  ▼
 3. File Thunder → presigned PUT URL              8. No call to File Thunder needed —
    → nexgate-raw (private, temp)                     main backend just glues:
        │                                              cdn.nexgate.com + stored objectKey
        ▼                                                  │
 4. Phone uploads raw bytes directly to MinIO              ▼
    (File Thunder never touches the bytes)         9. Bob's phone requests that URL
        │                                                  │
        ▼                                                  ▼
 5. MinIO event → File Thunder:                   10. Nearest Cloudflare edge (e.g. Nairobi):
    PENDING → UPLOADED → VideoWheel                    - cache MISS (first viewer ever)
      - FFmpeg: short-path transcode                      → tunnel → MinIO → fetch once
        (360p/720p/1080p, watermark, outro)              - cached at that edge from now on
      - thumbnail + blurhash + preview clip                   │
      - LIVE_PARTIAL after 360p, READY after all           ▼
      - raw original deleted                       11. Every later viewer near Nairobi:
      - keys written to Postgres (not URLs)             cache HIT — instant, MinIO untouched

```

**Key point:** steps 1–5 (upload + processing) are entirely unrelated to CDN — that pipeline already works today. The CDN only enters at step 7 onward, and it's a pure "glue the domain onto the key" operation. No signing, no expiry, no per-viewer logic, because it's public by design.

If the clip had been ≥3 minutes (HLS path), step 10/11 repeats per segment: the player first fetches `master.m3u8` from the CDN, then a stream of `.ts` segments, each cached the same way.

---

## 6. Walkthrough B — Direct Message Image (Lane 2, Private/Bypassed)

Alice sends Bob a private photo in DM.

```
 1. Alice sends photo → uploads to nexgate-raw (same presigned-upload pattern as Lane 1)
 2. ImageWheel processes it → variants written to nexgate-private (NOT nexgate-public)
 3. Bob opens the conversation
        │
        ▼
 4. Main backend checks: is Bob actually a participant in this conversation? (authorization)
        │
        ▼
 5. Main backend calls File Thunder → generateDownloadUrl(fileId, requesterId=Bob)
        │
        ▼
 6. File Thunder issues a PRESIGNED GET URL — 15 min expiry, straight from MinIO
        │
        ▼
 7. Bob's phone fetches directly from MinIO — the CDN is never involved in this request at all
        │
        ▼
 8. URL expires after 15 minutes — if Bob reopens the chat later, a fresh one is issued

```

If a stranger somehow obtained this URL, it would already be expired or tied to a bucket that Cloudflare's tunnel can't even route to — two independent layers of protection.

---

## 7. Group Chats — Same Lane, Different Scale

Group chat media (photos/videos/documents shared in a group conversation, not a 1:1 DM) still belongs in **Lane 2** — same bucket (`nexgate-private`), same `generateDownloadUrl()` call, never cached. File Thunder doesn't actually know or care whether it's a 1:1 DM or a 300-person group; it only trusts whatever `requesterId` the main backend passes in.

**What actually changes is who's allowed to ask:**

```
1:1 DM:        "is requesterId one of the 2 people in this conversation?"
Group chat:    "is requesterId a CURRENT member of this group?"

```

That authorization check lives entirely in the main backend, before it ever calls File Thunder — nothing to design differently on the File Thunder side for correctness.

### The real new problem: fan-out

In a 1:1 DM, at most 2 people will ever request a given file — cheap no matter how it's delivered. In a group chat, one photo sent to a 300-person group means up to 300 people opening it:

```
300 members → 300 separate calls to generateDownloadUrl()
            → 300 separate presigned URLs generated
            → 300 separate direct-to-MinIO fetches
            → none of it cached (Lane 2 is always cache-bypassed) — all of it hits the one VPS

```

That's a materially different load profile than a private DM, and it's worth designing around before group chat ships — but the fix is about *reducing redundant origin calls*, not about caching private files at a shared edge.

### Option 1 — do nothing special (fine for small/medium groups)

A few dozen members fetching a photo once is trivial load for MinIO on a VPS. If groups stay small (family/friend-sized, not broadcast channels), Lane 2 as-is is genuinely fine — no need to build anything extra ahead of time.

### Option 2 — share ONE signed URL instead of one per member (for large groups)

Instead of every member independently triggering `generateDownloadUrl()`, main backend generates the presigned URL **once** — when the message is sent, or on first view — caches that single URL (e.g. in Redis, keyed by message/file id), and hands the *same* URL to every member who opens the chat until it expires (15 min). This turns "300 origin hits" into "1 origin hit, 300 people reusing the same short-lived link":

```
Member 1 opens chat ──▶ main backend: URL cached? NO ──▶ call generateDownloadUrl() ──▶ cache it
Member 2 opens chat ──▶ main backend: URL cached? YES ─▶ reuse same URL, no MinIO call
Member 3-300 ...      ──▶ same — reuse until the 15-min expiry, then regenerate once

```

Still never touches Cloudflare's shared cache — this is purely about cutting redundant presigned-URL generation and redundant MinIO traffic, which is a much simpler problem than "how do we safely cache private files."

### When a "group" stops being private

If a group grows into "public broadcast channel" territory (thousands of members, more like a public feed than a private conversation), that's usually a sign the content has drifted out of Lane 2 entirely — worth revisiting whether it should be public (Lane 1) or a members-only public tier, rather than forcing an increasingly public thing through the private/signed lane. Not a decision to make now — just a pattern to recognize if it comes up.

---

## 8. Voice Notes &amp; Live Streaming — Do the Same Rules Apply?

Neither exists in File Thunder today (no `MediaContext` for either, no live-ingest infra). They split sharply: one fits the existing model perfectly, the other breaks a core assumption.

### Voice notes — yes, exactly the same rules

A voice note is just a short, finished audio file. It doesn't break anything this guide relies on: upload once → process once → immutable key → serve. It drops straight into the existing lanes:

```
Voice note in a DM/group chat  → Lane 2  (same as DM_IMAGE/DM_VIDEO — private bucket, 15-min
                                           presigned URL, never cached)
Voice note on a public post     → Lane 1  (same as SOCIAL_IMAGE — public bucket, plain CDN URL,
                                           cached forever)

```

The only new work is a new `MediaContext` (e.g. `DM_VOICE_NOTE`) and a small audio wheel (FFmpeg re-encode to a consistent format/bitrate, maybe waveform-peaks generation for the UI — same spirit as ImageWheel's blurhash). **Delivery-wise, there is nothing new to design.**

### Live streaming — no, this breaks a core assumption

Everything so far relies on one idea: **a file is finished, then served.** Upload → process → immutable key → cache forever. Live streaming violates that at its core — the "file" is still being created while people are watching it. That changes several things at once:

1. **No presigned-PUT-to-MinIO upload.** A live stream needs a real-time ingest protocol (RTMP, SRT, or WebRTC) hitting an ingest server continuously — not a client uploading one finished file.
2. **The manifest never stabilizes.** VOD HLS's `master.m3u8` flips once (`LIVE_PARTIAL` → `READY`) and never changes again — that's why "short-cache the manifest, cache segments forever" (§10) works for VOD. A *live* HLS playlist is a **sliding window** — rewritten every few seconds to point at only the newest 3-5 segments, continuously, for the whole stream duration. That manifest needs a near-zero cache TTL (1-2s) the entire time, not a one-time flip.
3. **Individual segments are still cacheable** — each 2-6 second `.ts` chunk is immutable once written, so Cloudflare still helps a lot for segment traffic, just not for the manifest.
4. **Private/restricted live streams** (e.g. a paid live-shopping event — Lane 2/3 equivalent) can't use one 15-minute signed link the way a DM photo does, because the session usually runs longer than that. This needs rotating/refreshing tokens or signed cookies for the stream's duration — new machinery, not something the current design already provides.

Because of points 1 and 4, this genuinely isn't "reuse the existing lanes" — it needs its own ingest + packaging pipeline. Building that in-house (RTMP ingest server + real-time FFmpeg transcode + rolling manifest) is a serious lift, well beyond a delivery-lane decision. The pragmatic path most platforms take at this stage is to **not build live ingest in-house** and use a managed service (Cloudflare Stream Live, Mux, Agora, etc.) that hands back an HLS URL to embed — which then *does* slide back into the Lane 1 viewing model, just with someone else running the real-time part.

**Bottom line:** voice notes are a non-event — add the context, reuse the lanes. Live streaming is a separate architectural track that shouldn't be forced into this document's model; buying that piece is worth strongly considering before building it in-house.

---

## 9. Video/Audio Calls — Not a File Thunder Concern

This is the sharpest split of all. Everything else in this guide — even live streaming — is about media that gets **stored** (however briefly) and then **fetched by someone else, later or from elsewhere**. A 1:1 or group call is neither.

1. **Nothing is ever "delivered" through the storage layer.** A call is devices talking to each other in as close to real time as possible — 100-300ms round trip matters, not "eventually arrives." Routing it through MinIO → Cloudflare → viewer would add multiple seconds of latency. Wrong tool entirely.
2. **The transport is peer-to-peer or SFU, not HTTP/CDN.** Calls use **WebRTC**: media streams connect device-to-device directly when possible, and go through a lightweight **SFU (Selective Forwarding Unit)** relay when a direct connection isn't possible (NAT/firewall issues) — the SFU forwards packets, it doesn't store them. A completely different network model from "upload a file, cache it, serve it."
3. **Normally nothing to process, scan, or watermark.** No FFmpeg wheel, no ClamAV, no thumbnails — there's no file, just a live signal that's gone the instant it's forwarded.
4. **Cloudflare's role, if any, is different too**: not caching — it would be **TURN/STUN relay**(helping two devices behind NATs find each other, or relaying when they can't connect directly). Cloudflare Calls / Cloudflare Realtime is their product here, but it conceptually replaces the SFU, not the CDN edge cache used everywhere else in this guide.

Calls need their own **separate service**, not a feature of File Thunder — worth calling out explicitly so this doesn't get scoped into "file delivery" work by mistake later.

**The one place calls do touch File Thunder:** if a call gets recorded and saved afterward (e.g. a missed-call "voicemail," or a saved video-call recording), it's no longer a live call at that point — it's a finished file, and it goes right back into the existing model: probably `DM_VIDEO`or a new `CALL_RECORDING` context, Lane 2, same rules as any other private media.

---

## 10. Walkthrough C — Digital Product Purchase (Lane 3, Protected/Audited)

Alice sells a paid PDF course. Bob buys it.

```
 1. Bob completes payment → main backend confirms order in its own DB
        │
        ▼
 2. Main backend calls File Thunder → generateDownloadUrl(fileId, requesterId=Bob)
        │
        ▼
 3. File Thunder checks:
      - context is DIGITAL_PRODUCT? ✓ (only DIGITAL_PRODUCT / DM_DOCUMENT / PRODUCT_PREVIEW_DOCUMENT
        are allowed through this endpoint at all)
      - file status == READY? ✓
        │
        ▼
 4. File Thunder writes a DownloadAuditEntity row:
      fileId, ownerId, requesterId=Bob, objectKey, requestedAt, expiresAt
      (a permanent record of exactly who downloaded what, and when)
        │
        ▼
 5. Presigned GET URL issued — 15 min expiry, bucket = nexgate-digital
        │
        ▼
 6. Bob downloads directly from MinIO — never cached, never at a shared edge

```

The audit row is what lets you answer "who downloaded this file, and when" months later — important for a paid-goods platform if there's ever a dispute or a leak investigation.

---

## 11. Cache Invalidation — The Only Sharp Edge

Because object keys are unique per `fileId`/variant and are **never overwritten**, a cached copy at the edge can basically never go stale — new content always gets a new key. That removes 99% of the usual "how do I bust the cache" headache. Three known exceptions:

<table id="bkmrk-case-problem-fix-hls"><thead><tr><th>Case</th><th>Problem</th><th>Fix</th></tr></thead><tbody><tr><td>HLS `master.m3u8`</td><td>Same key, first written when only 360p is ready (`LIVE_PARTIAL`), then rewritten when all qualities finish (`READY`)</td><td>Short cache lifetime on this one file only (10–30s). The `.ts` segments never change — cache those forever</td></tr><tr><td>User thumbnail overwrite</td><td>`user-poster.webp` reuses the same key on re-upload</td><td>Prefer versioning the key (`user-poster-v2.webp`) over a cache purge call</td></tr><tr><td>Profile picture / cover / shop logo</td><td>Same "identity slot gets overwritten" pattern</td><td>Same fix — version the key rather than relying on purge</td></tr></tbody></table>

Versioning the key is preferred over calling Cloudflare's purge API because it needs no extra network call and can never race (old URL still valid until the new one is swapped in by the app).

---

## 12. Cloudflare Setup Checklist

Concrete steps, in order, when it's time to actually turn this on:

1. **Add the domain to Cloudflare** (or subdomain, if the root domain is hosted elsewhere).
2. **Create `cdn.nexgate.com` DNS record**, proxied (orange cloud ON). Create `cdn-staging.nexgate.com` the same way for staging.
3. **Install `cloudflared` on the VPS**, authenticate it to your Cloudflare account, create a tunnel, and route it to `http://localhost:9000` (MinIO) — but only for requests that resolve to the `nexgate-public` bucket path.
4. **Re-enable anonymous read, but only on `nexgate-public`** (the other three buckets stay fully private — do not touch their bucket policy).
5. **Firewall the MinIO port** so it's not reachable from the raw internet at all — the tunnel is the only path in. (If a tunnel isn't used yet, at minimum restrict to Cloudflare's published IP ranges.)
6. **Set a Cache Rule** in Cloudflare for `cdn.nexgate.com/*`: cache everything, respect `Cache-Control: public, max-age=31536000, immutable` from origin — except `*/master.m3u8`, which gets its own short-TTL rule (10–30s).
7. **Flip the app config** (already scaffolded per `FILE_THUNDER.md` §8): ```properties
    ft.storage.mode=cdn
    ft.cdn.base-url=https://cdn.nexgate.com
    
    ```
8. **Smoke test**: upload a test image, confirm the CDN URL 200s, re-request it and confirm the `cf-cache-status` response header flips from `MISS` to `HIT` on the second request.

---

## 13. Video-at-Scale Caveat

Cloudflare's free/pro plans restrict using the CDN as a bulk *video* host at large scale (their terms are written around "no non-HTML heavy media" abuse). Images, thumbnails, blurhash/lqip, and HLS manifests are completely fine on the free tier — this whole design is ideal for those. But if social video volume grows large, the two Cloudflare-native upgrade paths to know about later are:

- **Cloudflare R2** — S3-compatible object storage, zero egress fee to Cloudflare's own edge. Natural fit since File Thunder already stores keys, not URLs — swapping the origin later doesn't change any application logic.
- **Cloudflare Stream** — paid, purpose-built for HLS/video delivery with its own player and encoding.

Not a decision to make now — just something to keep in mind as a later migration, not a redesign.