Character Engine Architecture & Specs
A deep dive into the Phase 5 Cognitive Architecture, the 800+ RNN muscle simulation, and the procedural V2 Geometry pipeline that powers Aeion digital humans.
Simulation & Cognitive Metrics
Motor Network Pass
~0.5ms for 800 muscles (@ 120Hz)
FACS Morph Targets
54 (Full ARKit-52 + Identity)
Memory Capacity
10,000 Experience Buffer (Q-Learning)
Canonical Geometry
2,726 Vertices / 3.8 MB (Head)
1. The V2 Procedural Anatomy Stack
Aeion has completely deprecated artist-authored meshes in favor of mathematical determinism. - Canonical Generation: The head and body generators produce geometry at build time from deterministic primitive libraries (ellipsoids, swept profiles) — no scan, no artist pass. The head mesh includes 54 fully parameterized FACS morph targets (ARKit-52 standard plus custom asymmetric identity overlays). - Responsive 7-Layer PBR Skin: The skin shader supports multi-lobe subsurface scattering, dual-lobe specular, and dynamic peach fuzz. Crucially, its surface response is bound to the cognitive state. When the Limbic system registers high arousal and exertion, the pipeline dynamically elevates blood-flow and sweat, physically altering the character's complexion in real-time. - Autonomic Microdynamics: The engine runs a continuous idle-life background process. It calculates layered, non-repeating cycles for blinking, breathing, micro-saccades, and pupil hippus, injecting them directly into the morph pipeline without relying on baked animations.
typescript
// Example: A Cognitive System evaluating a goal via A* Searchimport { cognitiveArchitectureService } from "@aeion/spatial";
// The Prefrontal Cortex defines a high-priority survival goalconst goal = {id: "goal-1",name: "Seek Shelter",priority: 0.95,preconditions: [{ variable: "weather", operator: "equals", value: "storm" }],desiredEffects: [{ variable: "isSheltered", operator: "equals", value: true }],interruptible: false};
// The GOAP engine generates a hierarchical action plancognitiveArchitectureService.setGoal(system.id, goal);
// Meanwhile, the Motor Cortex translates the high-level "run" intent into raw muscle forcesfor (const [muscleId, targetForce] of desiredForces) {const activation = calculateNeuralActivation(muscleId, targetForce);activations.set(muscleId, activation);}2. Phase 5 Cognitive Architecture
The true differentiator of the Aeion engine is its multi-region brain simulation, heavily inspired by the ACT-R architecture and human neurobiology. - Goal-Oriented Action Planning (GOAP): Instead of traversing massive, brittle behavior trees, the Prefrontal Cortex maintains a prioritized Goal Stack and a pool of available actions with cost and precondition metadata, designed around an A planning model. When a goal is pushed, the planner selects the lowest-cost action sequence whose preconditions are satisfiable against current world state. - Reinforcement Learning & Memory: Characters utilize continuous Q-learning paired with an experience-replay buffer. When a character completes an interaction, they store an episode — emotional valence, arousal, and significance included — into long-term memory. The engine queries these past episodes when evaluating future actions, allowing NPCs to hold grudges, form alliances, and adapt to the player. - Theory of Mind:* The Social Intelligence subsystem allows characters to maintain independent mental models of other agents. They calculate Trust, Intimacy, and Dominance vectors, enabling complex social contagion where panic or joy can spread organically through a crowd based on relational networks.
3. Smart Object NLP & Neural Voice Pipeline
The engine bridges conversational AI with physical 3D world interaction seamlessly. - Procedural Viseme Lip-Sync: When a character speaks, the audio and dialogue transcript are run through a phoneme-timing pass that emits a cue timeline (Preston Blair mouth shapes), which the animation pipeline maps to blendshape weights with temporal coarticulation blending, manipulating the FACS morphs in real time without any baked animation clips. - Grammar-Based Smart Objects: Influenced by classic MUD/MOO text architectures, the NLP layer allows agents (and players) to interact with the environment via natural language. The parser handles complex multi-action sequences, flow-control, and spatial superlatives ("pick up the red object nearest the door"), automatically delegating the physical interaction to the object-interaction system.
Observability is critical for AI. The Character Engine emits a strict stream of events (`emotion_changed`, `gaze_changed`, `gesture_triggered`, `spawned`) via two channels. If an MCP AI agent triggers a change on the backend, it hits the Tenant Event Bus. If a designer scrubs a timeline in Motion Studio, it fires a Browser CustomEvent. This unified stream allows for perfect backend recording and playback of all NPC cognitive and physiological states.
The Prefrontal Cortex's working-memory buffer is bounded to a fixed capacity of 7 active items — a deliberate nod to Miller's Law (7±2). Each cognitive system can optionally link to a character's genome record, tying physiological traits (skin, hair, eye color, body proportions) to the same identity used elsewhere in the render pipeline, while cognitive parameters like working-memory capacity and learning rate remain uniform system constants rather than per-genome variables.