Collab Architecture & Specs
LiveKit-based SFU with WHIP/RTMP ingress and room composite egress, Whisper-powered AI transcription with structured action items, a 5-rule DLP compliance engine, a 5-state presence machine, and Bridge-backed local recording + whiteboard camera capture.
Performance & Collab Metrics
Message Delivery P50
8ms
Presence Update Latency
<50ms
Transcription Turnaround
<30s for 60-min recording
LiveKit Egress Start
<3s from request to recording
WHIP Ingress Setup
<2s (RFC 9725 compliant)
Channel List Load
<100ms (cursor pagination)
Webhook Verification
<1ms (HMAC-SHA256)
1. LiveKit Video — SFU, Egress & Ingress
| Capability | Details |
|---|---|
| Room composite egress | Composites an entire room into a single MP4 (or RTMP/HLS stream) |
| Track composite egress | Composites specific tracks — e.g., speaker camera + screen share — to a stream destination |
| Egress status | Live status polling from request through completion |
| Ingress type | Details |
| --- | --- |
| WHIP | RFC 9725-compliant push ingress for OBS/FFmpeg/browser encoders; passthrough mode available for lower CPU use |
| RTMP | Standard broadcast ingress (OBS, Wirecast, professional cameras); transcoding on by default (H.264/AAC → WebRTC) |
2. AI Transcription — Whisper AI + Structured Output
| Field | Description |
|---|---|
| `text` | Full transcript text |
| `vtt` | WebVTT captions for subtitle support |
| `segments` | Timestamped text segments |
3. DLP — 5-Rule Compliance Engine
| Pattern | Category |
|---|---|
| SSN | PII |
| Credit card number | Financial |
| Email address | PII |
| Phone number | PII |
| IP address | PII |
4. Presence — 5-State Machine
| Transition | Trigger |
|---|---|
| Online → Idle | No activity for a configurable period (5 minutes by default) |
| Idle → Offline | Session ends |
| Any → Do-Not-Disturb / Invisible | Explicit user action |
| Offline → Online | Session reconnects |
5. Bridge — Local Hardware Integration
The Aeion Bridge desktop companion unlocks native hardware workflows: local screen+audio recording to disk (avoids cloud upload), whiteboard camera capture, screen capture for chat sharing, and audio device enumeration.
Local Recording (progressive enhancement):
Recording captures screen + audio directly to a local folder in MP4, with live status available while it's running and a completed file (path, size, duration) once stopped. When Bridge isn't connected, none of this is offered — every other feature keeps working fully in the browser.
Whiteboard Camera Capture:
Captures a 1920×1080 JPEG photo of a physical whiteboard through a connected USB camera via Bridge — useful for pulling a hand-drawn whiteboard into the meeting record without a dedicated capture device.
Audio Device Selection:
Lists every available microphone and speaker so users can pick the right input/output for a call.
6. Messaging — Threading, Search & Reactions
Messaging handles channel-based routing, threading (parent/reply linkage), full-text search, and reactions, with real-time delivery to every connected client. AI message intelligence (sentiment analysis, language detection, smart compose) runs as a separate layer over message history rather than being baked into the core messaging path, so it can be upgraded independently of the base messaging engine.
Message Threading:
Replies are linked to their parent message and paginated with cursor-based lookups, so long threads load quickly regardless of size.
7. Channels — Multi-Type + Access Control
| Property | What it controls |
|---|---|
| Type | text / voice / forum / announcement / direct message / group DM |
| Privacy | public or private, independent of type |
| Membership | role-based (owner, admin, moderator, member, guest) |
| Pinned messages | Per-channel pin list |
| Settings | Slow mode, auto-delete messages, thread duration, allow threads/reactions/files, max file size |
8. Recording — Meeting Lifecycle
Recording management covers the full lifecycle — creation, status updates, transcription triggering, download URL generation, and deletion.
Recording Lifecycle: recording → processing → completed (or failed). Recordings older than 90 days are cleaned up automatically by default, configurable per tenant.
Each recording tracks its room, duration, participant count, and — once transcription finishes — the transcript text, WebVTT captions, and language.
Each recording exposes its status (recording / processing / completed / failed) via the API, so you can poll for completion rather than assume a fixed processing time.
bash
# Complete collab flow — meeting to action items
# 1. Generate a LiveKit room join token for a participantcurl -X GET https://api.aeionos.com/api/v1/collab/video/rooms/standup-may-11/token \ -H "Authorization: Bearer $AEION_API_KEY" \ -H "x-tenant-id: $TENANT_ID"
# 2. Start server-side room composite recordingcurl -X POST https://api.aeionos.com/api/v1/collab/recordings/room/standup-may-11/start \ -H "Authorization: Bearer $AEION_API_KEY" \ -H "x-tenant-id: $TENANT_ID" \ -H "Content-Type: application/json"
# 3. Transcribe once the recording completescurl -X POST https://api.aeionos.com/api/v1/collab/recordings/rec_abc123/transcribe \ -H "Authorization: Bearer $AEION_API_KEY" \ -H "x-tenant-id: $TENANT_ID" \ -H "Content-Type: application/json"
# 4. Summarize the transcript into action itemscurl -X POST https://api.aeionos.com/api/v1/collab/recordings/rec_abc123/summarize \ -H "Authorization: Bearer $AEION_API_KEY" \ -H "x-tenant-id: $TENANT_ID"# → { summary, keyPoints[], actionItems[{task, assignee, dueDate}] }
# 5. Update presence for the meeting hostcurl -X PUT https://api.aeionos.com/api/v1/collab/presence/status \ -H "Authorization: Bearer $AEION_API_KEY" \ -H "x-tenant-id: $TENANT_ID" \ -H "Content-Type: application/json"Aeion Collab's LiveKit integration is the foundation for the Cut module's streaming surface. Cut resolves the same `collab_livekit` service directly — its WHIP/RTMP ingress routes and streaming routes reuse Collab's room, ingress, and egress plumbing rather than standing up a second LiveKit integration.
Native phone system (PSTN/SIP) calling — through providers like Twilio, Telnyx, Vonage, or Plivo — is on our roadmap; there's no telephony calling in the module today. Real-time audio/video runs entirely through the shipped LiveKit stack (SFU + WHIP/RTMP ingress + egress); inbound-call recordings will feed the same transcription pipeline once telephony ships.
The whiteboard persists board state as periodic snapshots and broadcasts live edits and cursor positions to every viewer in real time, so collaborators see each other's changes and presence as they happen. Today, concurrent edits resolve last-write-wins at the snapshot level. True conflict-free merge for simultaneous or offline editing is on our roadmap.
Incoming LiveKit webhooks are verified via an HMAC-SHA256 signature check before processing — invalid or missing signatures are rejected immediately with a 401. Custom integration webhooks (whiteboard collaboration server, third-party providers) use a configurable shared-secret header check instead.