Cut — Post-Production Platform Architecture

Cut integrates Twelve Labs for video indexing, semantic search, transcript generation, and visual detection.

Indexing: point Cut at a video and it builds a searchable index — transcription, visual (scene/shot) embeddings, and on-screen text (OCR) embeddings, with automatic language detection or an explicit language hint.

Video search: natural-language queries ("chef plating dishes") search across visual content, spoken conversation, on-screen text, and logos, with a confidence threshold and result ranking by score or timestamp.

Transcript segments: word-level timestamps with confidence scores and optional speaker diarization, exportable as SRT/VTT captions.

Visual search capabilities:

• Find frames containing a specific person
• Find frames matching a scene description ("crowded restaurant kitchen")
• Find frames with on-screen text ("menu board with prices")
• Find frames with brand logos
• Search by visual similarity across the entire video corpus

Example queries: "Find all scenes with chef Gordon plating dishes," "Find any frame where the menu board is visible," "Find all takes where talent says the word 'special'," "Find all instances of the company logo in the lower third."

Cut's audio processing handles transcription via Whisper AI and stem separation via Demucs.

Whisper transcription: choose a model size (tiny through large), transcribe or translate, with optional voice activity detection and per-word timestamps. Output as JSON, SRT, VTT, or plain text. GPU acceleration is used for medium/large models, and transcription runs as a background batch job so editing isn't blocked.

Demucs stem separation: splits a track into 4 stems (vocals/drums/bass/other) using the htdemucs model, exported as WAV. Runs on GPU-accelerated batch workers for fast turnaround, and stem downloads are available via time-limited secure links.

Progress broadcasting: long-running audio jobs stream live progress (downloading → processing → uploading) back to the editor.

Cut's smart organization auto-tags media using AI analysis and provides face detection, visual similarity clustering, and rule-based organization.

Tag sources: manual tagging plus 7 AI-driven sources — detected objects, scene type, spoken transcript, on-screen text (OCR), face detection, dominant color palette, and audio classification (music/speech/noise) — alongside a rule engine for custom logic. Content-type labels (interview/B-roll/promo) are a tag category populated by those sources, not a separate detector.

Tag categories: object, person, location, action, emotion, color, style, content type, technical (resolution/frame rate/codec), and fully custom tags.

Each tag carries a confidence score and, where relevant, timestamps or bounding boxes showing exactly where in the frame and timeline it applies.

Face detection & grouping: faces are clustered across your media, you assign a name to a cluster once, and new footage featuring that person auto-tags going forward.

Rule-based organization: define conditions (filename, duration, date created, and more) and actions (add/remove tag, move to collection, set a custom field) — with a dry-run mode so you can preview a rule before it runs automatically.

Analysis runs as a background pipeline with automatic retry on transient failures, and previously-analyzed media is skipped so re-running organization doesn't waste time.

Cut's review portal generates secure review links for stakeholders with configurable watermarks, expiry, and download restrictions.

Review sessions carry an expiration, optional view-count limits, and per-session watermark configuration (text, position — center/corner/tiled — opacity, font size). Download settings can restrict quality (proxy/high/original), set an expiration, or cap the number of downloads. Optional view notifications alert you the moment a stakeholder opens the link.

Token security: each review link uses a cryptographically random token, stored in a way that makes the token space computationally infeasible to guess or enumerate.

Watermarking: applied at serve time — not baked into your source media — so the same original file can be shared with different watermarks per recipient. Supports both video and image formats, including watermarking of live HLS streams (in the manifest and segments).

Stakeholder notifications: get notified when someone views a share (who, when, approximate location). The review workflow supports Approve / Request Changes / Reject, with comments timestamped to specific frames.

Cut's timeline manages the nonlinear editing model, track management, and clip manipulation.

Structure: a timeline holds a frame rate (23.976 through 60fps), a resolution (1080p, 4K, and beyond), one or more video/audio/subtitle tracks, and clips referencing your source media with independent in/out points, speed ramping (0.25×–4×), transitions, filters, color grades, opacity, and volume.

Editing operations: trim, split, delete (with ripple or gap-close), extract (lift to clipboard), overwrite, insert (ripple forward), slip, slide, frame-accurate snapping, and transitions (cross-dissolve, wipe, dip-to-black) — all non-destructive.