Motion Studio Architecture & Technical Specs

Aeion's motion pipeline orchestrates AI-powered 3D generation across four capabilities, all routed through the Neural Bus.

Pipeline 1: Generative 3D Mesh

Send a text prompt (and optionally a reference image) to the generate-3D-mesh endpoint. The Neural Bus routes the request to the best available generation provider and returns a ready-to-use glTF/GLB model URL plus a reusable model token.

Prompt-based generation: "Create a floating crystal orb with internal glow" → glTF/GLB model. Image-based generation: Upload reference image → Neural Bus generates 3D mesh from 2D.

Pipeline 2: Audio-Driven Lip-Sync

Send an audio file to the lip-sync endpoint. A deterministic, zero-ML DSP analyzer reads the waveform's per-frame RMS energy and returns a jawOpen amplitude plus a coarse viseme category for each frame — pure signal analysis, no network call, no model.

Lip-Sync Track Output: each frame carries a timestamp, a jawOpen weight (0.0–1.0), and a viseme hint. It's a deliberately simple, deterministic baseline (same audio in, same track out) rather than a full per-channel facial-blendshape model.

Pipeline 3: Text-to-Animation

Describe a motion in plain language and target a rig format. The pipeline maps it to real keyframe animation data on a professional motion-capture skeleton.

Target Rig Support: industry-standard rig formats, plus custom user-uploaded skeletons.

Pipeline 4: Edge WASM Physics Engine (Rapier)

Submit a scene's bodies, joints, and gravity settings targeting browser or edge_worker. The request runs through Aeion's AST compiler, which models the scene on Rapier's rigid-body/joint/impulse primitives and emits a portable physics-logic source graph.

Status note: source generation is live today, on the same AST-compiler engine that already assembles real, deployable WASM binaries for other targets (e.g. Aeion's pricing/CPQ engine). Packaging the physics target specifically into a runnable browser/edge WASM binary needs the Rapier runtime + asc toolchain wired in and is rolling out next — today this pipeline returns generated source rather than a binary.

Physics Scene Config:

typescript { bodies: Array<{ type: "rigid" | "static" | "kinematic"; shape: "box" | "sphere" | "capsule" | "convex"; position: [number, number, number]; size?: [number, number, number]; mass?: number; restitution?: number; // Bounciness friction?: number; }>; joints: Array<{ type: "revolute" | "prismatic" | "ball" | "fixed"; bodyA: string; bodyB: string; anchorA: [number, number, number]; anchorB: [number, number, number]; }>; gravity: [number, number, number]; solverIterations: number; } `

Uploaded 3D assets are optimized off the request path, on a background job queue with schema-validated payloads.

Why schema validation at the queue boundary matters: it prevents drift between the code that enqueues a job and the code that processes it. If a job's shape doesn't match what the worker expects, it fails fast and routes to a dead-letter queue on the first attempt — instead of silently burning retries against malformed data.

Per-file deduplication: each optimization job is keyed by the source file, so re-uploading the same file never queues a duplicate optimization pass.

Reliability guarantees:

• Structured, validated job payloads (no silent shape drift between producer and consumer)
• Automatic dedup — re-uploads don't double-queue
• Bounded concurrency so optimization jobs don't starve other background work
• Graceful shutdown — in-flight jobs drain cleanly on redeploys

The Motion API exposes four generation endpoints plus publish/unpublish.

POST /api/v1/motion-studio/motion/generate-3d-mesh

bash POST /api/v1/motion-studio/motion/generate-3d-mesh { "prompt": "Create a floating crystal orb with internal glow" } { "prompt": "A low-poly geometric fox", "sourceImageUrl": "https://cdn.example.com/fox.jpg" }

Response:

typescript { "data": { "success": true, "modelUrl": "https://cdn.example.com/models/orb.glb", "modelToken": "tok_xyz123" } }

POST /api/v1/motion-studio/motion/lip-sync

bash POST /api/v1/motion-studio/motion/lip-sync { "audioUrl": "https://cdn.example.com/audio/intro.mp3" }

Response shape (from the deterministic waveform-energy analyzer):

typescript { "data": { "success": true, "blendshapesData": { "method": "amplitude_envelope_v1", "frameRate": 60, "frameCount": 750, "durationMs": 12500, "frames": [ { "t": 0, "jawOpen": 0.0, "viseme": "sil" }, { "t": 33, "jawOpen": 0.62, "viseme": "aa" } // ... one entry per frame ] }, "animationId": "lipsync_1715364000000" } }

POST /api/v1/motion-studio/motion/generate-animation

bash POST /api/v1/motion-studio/motion/generate-animation { "prompt": "A character walking with a bouncy stride", "targetRig": "mixamo_standard" }

POST /api/v1/motion-studio/motion/compile-physics

bash POST /api/v1/motion-studio/motion/compile-physics { "sceneData": { "bodies": [ { "type": "rigid", "shape": "sphere", "position": [0, 10, 0], "mass": 1 } ], "gravity": [0, -9.81, 0] }, "targetEnvironment": "browser" }

Today this returns generated physics-logic source (see status note above) rather than a runnable binary; the deployable/binary-packaging response shape ships once the Rapier + asc toolchain integration lands.

Publish/Unpublish Scenes:

```bash POST /api/v1/motion-studio/motion/scenes/{id}/publish # Returns: { data: { status: "published", embedToken: "embed_xxx_yyy" } }

POST /api/v1/motion-studio/scenes/{id}/unpublish # Returns: { data: { status: "draft", embedToken: null } } ```

Embed Tokens: a unique, opaque token generated on every publish.

Every Spline scene loaded into the editor exposes its authored variables through a Live Variables panel: the panel polls the loaded scene roughly every 800ms, lists each variable with its inferred type (string / number / boolean), and lets an editor click any value to edit it live against the running scene — useful for testing how a variable drives the visuals before wiring it up.

Today this is a manual inspection/editing surface for the person authoring the scene. Piping live Aeion OS business data (order events, revenue figures, inventory levels) into a scene's variables is done via the specialized commerce/data nodes in the node graph described below, not through an automatic collection-to-variable binding layer.

Much of Motion Studio runs entirely in the browser editor, independent of any backend pipeline:

• PBR Shader Architect — a visual node graph compiles directly to live GPU shader code (Fresnel terms, noise, SDF, vector math) and hot-swaps into the scene's material — no recompile/deploy step.
• Timeline authoring — record-mode auto-keyframing and onion-skinning for hand-animation.
• Cinematic camera director — keyframed camera moves, focus targets, and scrollytelling triggers.
• Audio-reactive — real-time frequency analysis drives scene parameters from an audio source.
• Specialized nodes — Gaussian-splat, XR-controller, and commerce (cart-add / variant) nodes wire 3D scenes to data, hardware, and checkout.
• Multiplayer — live collaborative cursors and co-editing in the same scene.
• One-click embed — publishing mints a stable embed token so a scene drops into any page.

(These ship in the admin studio today, independent of the server-side generation pipelines above — see the AI/Neural-Bus roadmap.)