Media Architecture & Technical Specs

Aeion integrates with Twelve Labs to power natural language video search.

What indexing returns: a video ID, the index timestamp, and the video's duration, so every indexed video is trackable and searchable going forward.

What search returns: for each match — video ID, confidence score, the matching start/end timestamps, and a thumbnail.

Indexing Pipeline:

1. Upload video URL to Twelve Labs
2. Wait for indexing (async, status polling)
3. Index complete → semantic search available

Natural Language Search Examples:

• "find videos showing a person running"
• "scenes with blue sky and ocean"
• "clip of someone typing on keyboard"
• "moment when a car drives through a puddle"

Search Parameters:

• A natural language query description
• A minimum confidence threshold (default: 0.1)
• Search granularity: clip (segment) or shot (camera change)

Multi-Index Support:

Multiple indexes are supported (e.g. marketing, training, archive), each acting as a separate namespace for search — so a training-video archive and a marketing asset library never surface irrelevant cross-matches.

Aeion uses vision AI for content moderation, checked per category against two global thresholds.

Configuration:

Moderation can be turned on or off per tenant, with a category list — adult, racy, violence, spoof, medical — each independently toggled on or off (adult and violence are on by default; racy, medical, and spoof are opt-in). Two thresholds apply across whichever categories are enabled: a warn threshold (default 0.6) and a block threshold (default 0.9). You choose whether flagged uploads are auto-rejected or routed to a review queue.

Result:

Every check returns a per-category score, an overall score (the highest score among enabled categories), whether the content is allowed, and a reason string when it's flagged or blocked — so downstream systems can act without re-implementing the logic.

Detection Flow:

1. Vision AI analyzes the image and returns a score per category (adult, racy, violence, spoof, medical)
2. The overall score is the max across enabled categories
3. If the overall score is at or above the block threshold, the content is blocked
4. Otherwise, if it's at or above the warn threshold, it's flagged for review
5. If detection itself fails, the content is allowed by default (fail-open) rather than blocking uploads on an AI outage

One call runs the full AI analysis pipeline across image, video, and audio — no need to orchestrate each provider separately.

What you can request (images): tags, a generated description, NSFW check, object detection, face detection, OCR text extraction, and dominant colors.

What you can request (video): semantic indexing (Twelve Labs), transcription, and auto-generated chapters.

What you can request (audio): transcription.

Each analysis type can be toggled independently, with its own configuration (e.g. NSFW thresholds, transcription language), and every result — tags, description, NSFW status, detected objects/faces, OCR text, transcript, chapters, detected activities — comes back in a single unified response.

Provider Auto-Selection:

The analyzer routes each request to the right configured AI provider automatically, and falls back gracefully if a provider is unavailable.