Quickstart: Breathe Life into the Void
Initialize the cognitive system, mount the procedural renderer, set GOAP goals, store episodic memories, drive characters via MCP tools, monitor emotional state — the full setup in 6 steps, in under 30 minutes.
Register Your Game's Archetypes
| Field | Purpose |
|---|---|
| `id` | Globally unique (`<gameId>:<localName>`) |
| `gameId` | Scopes the archetype to your world — different games can use the same local name without collision |
| `personalityDefaults` | Big Five anchors (openness, conscientiousness, extraversion, agreeableness, neuroticism) |
| `genomeDefaults` | Partial genome overrides merged onto `DEFAULT_GENOME` at spawn |
| `bodyConstraints` | Per-axis body soft-range overrides (height, build, etc.) |
| `faceConstraints` | Per-morph face soft-range overrides |
| `tags` | Free-form tags for creator-tool filtering |
Mount the Procedural Renderer
In your React frontend, mount the CharacterRenderer3D component. It consumes a MetaHumanState value that you keep in sync (via store, controller, or your own state strategy).
```tsx import { CharacterRenderer3D } from "@aeion/spatial"; import type { MetaHumanState } from "@aeion/spatial";
function MyExperience({ characterState, }: { characterState: MetaHumanState | null; }) { return ( <CharacterRenderer3D characterId="char_999" characterState={characterState} quality="high" // "low" | "medium" | "high" shadows={true} enableAutomaticLOD={true} smoothing={0.15} onDistanceChange={(dist) => { / drive LOD or fade-out decisions / }} /> ); } ```
The renderer automatically:
- Builds canonical V2 geometry — math-generated procedural mesh; no artist-authored 3D models required
- Attaches a multi-layer PBR skin shader — base + subsurface scattering + blood flow + sweat/dirt overlays
- Runs autonomic microdynamics — blinking, breathing, micro-saccades, pupil hippus
- Drives emotional facial morphs — FACS-mapped morphs respond to the limbic system's PAD state
- Manages LOD — full detail close-in; simplified mesh at distance (controlled by
quality+enableAutomaticLOD)
Procedural character generation:
- No artist 3D models required
- Unlimited variations via parametric genome
- Lightweight asset footprint vs scanned MetaHuman alternatives
- Sub-100ms generation per character
Set High-Level Goals (GOAP)
Instead of micromanaging animations, set goals — the prefrontal cortex plans the action sequence.
```typescript await cognitiveArchitectureService.setGoal(brain.id, { goalId: "welcome_visitor", description: "Greet the new visitor and offer assistance", preconditions: { visitorPresent: true, characterIdle: true }, desiredEffects: { visitorEngaged: true, visitorGreeted: true, }, priority: 80, deadline: Date.now() + 30_000, });
// The character will: // 1. Detect the visitor via the sensory cortex // 2. Plan a path through available actions (turn-toward, approach, smile, wave, speak-greeting) // 3. Execute the sequence via the motor cortex // 4. Re-plan if the visitor moves ```
Common action categories the engine ships with:
- Locomotion — walk-toward, run-toward, turn, stop, sit, stand
- Communication — smile, wave, nod, frown, speak (via lipsync + voice services)
- Manipulation — pick-up, put-down, hand-over, point-at
- Social — approach, retreat, observe, mirror-posture
Custom actions — register your own with preconditions + effects on the cognitiveArchitectureService. Action registries are per-character-system, so different archetypes can have different action vocabularies.
Plan visualization — Admin → Spatial → Characters → [character] → Plan View shows the current goal + planned action sequence + step progress. Useful for debugging unexpected behavior.
Trigger Episodic Memories
| Tier | Retention |
|---|---|
| **Working** | Last 30 seconds; high-fidelity context |
| **Short-term** | Last 30 minutes; recent events still actionable |
| **Long-term** | Permanently stored; consolidated over hours/days |
| **Procedural** | Action skills (how to walk, dance); never forgotten |
Drive Characters via MCP Tools
| Tool | What it does |
|---|---|
| `spatial_npc_spawn` | Spawn an NPC in a scene. Required: `sessionId`, `sceneId`, `characterId`, `position`. Optional: `archetype`, `displayName`, `initialEmotion`. |
| `spatial_npc_set_emotion` | Drive emotion via the EmotionDriver — one of 21 canonical values, intensity in [0, 1], `transitionMs` controls blend speed. Facial morphs + autonomic responses + social contagion follow. |
| `spatial_npc_set_appearance` | Adjust narrative tags (e.g., add `veteran_soldier` to stamp scar morphs) and clothing HSL. Either `addTags`, `removeTags`, or `clothingHSL` required. |
| `spatial_npc_trigger_gesture` | Play one of 8 canonical body gestures. Bone-rotation deltas blend on top of idle animation and spring back after the ~1.1-2.4 s duration. |
| `spatial_npc_gaze_at` | Direct head + eyes toward a world-space point. Mirror of the designer-side `CharacterGazeAtNode` — designer-driven + agent-driven gaze converge on the same field. |
Monitor Emotional State + Behavior
The cognitive architecture service exposes character state for monitoring + debugging via the admin surface.
Per-character live state available:
pad: { pleasure, arousal, dominance }— current emotional statecurrentGoal: { id, description, progress }currentAction: { name, progress, blockedBy? }lastMemoryRecall: { episodeId, similarity }sensorState: { visibleObjects, audibleSpeakers }locomotionState: { position, velocity, facing }
Admin dashboard surfaces:
- Character roster — all active characters in the scene
- Per-character state — PAD visualized as an octahedron, current emotion label
- Behavior trace — last N actions with timing + outcome
- Memory recall log — what memories were retrieved when
- Plan visualization — current GOAP plan + step progress
- Health metrics — frame time, render cost, CPU/GPU profile
Recording + replay:
- Every action + state change emits an event that the temporal log captures
- Replay sessions deterministically (useful for debugging "this NPC did X" player reports)
- Export sessions for QA / training data / behavioral research
Performance targets:
- 50+ concurrent NPCs per scene at 60 FPS on consumer hardware (M1, Ryzen 5 + integrated GPU)
- Aggressive LOD scales to 200+ background NPCs alongside 5-10 fully-cognitive foreground NPCs
- Profile via Admin → Spatial → Performance
Integration Recipes
Spatial → character placement. Characters drop into any Spatial scene. Define spawn points in the scene editor; characters initialize there on scene load.
Realtime → multi-user with NPCs. Multi-user sessions can include NPCs as additional participants. Each user sees the same character state synchronized via CRDT.
Cut → character-driven scenes. Pre-rendered scenes can include cognitive characters that react to script direction in real-time. Useful for previs + early-stage production.
Claw → LLM-backed character. AeionClaw serves as the LLM brain — customer talks to the character, Claw answers with full module-context awareness, the character physicalizes the conversation via MCP tools.
Production → character casting. Aeion Production includes character casting alongside human casting — pre-generate characters that match story descriptions.
Lip-sync + voice. Voice synthesis generates speech audio; the lip-sync service maps phonemes onto facial morphs in real-time, no monolingual artifacts even for non-English languages.
Troubleshooting
Character not responding to GOAP goals. Check that the goal's preconditions are met (Admin → Spatial → Characters → [character] → Plan View). Common cause: precondition checking a state that no longer holds. Update preconditions or pre-set the required state.
Memory recall finding irrelevant episodes. Tune the similarity threshold for memory matches. Default 0.6; raise to 0.8 for stricter matches. Verify context tags are consistent — variation in tagging dilutes recall quality.
Procedural mesh looking off (proportions, etc.). Adjust the genome parameters — the character generation tool exposes per-axis knobs for body proportions, facial features. Refresh the character after parameter changes.
Emotion expressions too subtle / too dramatic. Tune the emotion-to-morph mapping intensity. Admin → Spatial → Character → [character] → Emotion Settings → Expression Intensity slider.
Multi-character scene low FPS. Reduce LOD aggressively for background characters via enableAutomaticLOD + lower quality. Keep only the nearest 5-10 characters at full quality.
Archetype not found at spawn. Confirm the archetype was registered for the same gameId your spawn call references. Archetype IDs follow <gameId>:<localName> — check the prefix matches your registration.
Frequently Asked Questions
No. The Character Engine V2 is 100% procedural. CanonicalHeadMeshV2 and CanonicalBodyMeshV2 generate mathematically — unlimited variations of humans without ever opening Maya or Blender.
Highly optimized — calculates body force activations sub-millisecond per character. Total memory footprint for a fully active brain (~10K episodic memories included) sits in the low hundreds of MB.
Via Model Context Protocol (MCP). External agents call the five spatial_npc_* tools (spawn, set_emotion, set_appearance, trigger_gesture, gaze_at). Each tool emits a `spatial.npc.*` event so the rest of the system stays in sync.
Yes — with aggressive LOD. Production deployments mix 200+ background NPCs with 5-10 fully-cognitive foreground NPCs at 60 FPS. The sensory-attention model decides which NPCs deserve full processing each frame.
Yes. Voice synthesis supports multiple languages; the lip-sync service maps target-language phonemes onto facial morphs — no monolingual artifacts.
Modern WebGL2 / WebGPU browser on consumer laptop hardware. Mobile: recent flagship iPhone or Pixel recommended for multi-character scenes.
Animal rigs supported via custom skeletal templates. Common animals ship as presets. For exotic species, define the skeletal template + the engine handles animation procedurally.
Yes — virtual concierge (retail / hospitality), training simulations, telehealth patient avatars, sign-language avatars, animated keynote characters.
Sign-language characters supported via the sign-language motion library. Audio description of character actions for vision-impaired users. Closed-caption integration for character speech.
Yes. Long-term + episodic memory persist in the database. When a user returns days later, the character remembers them, prior interactions, preferences. Storage is per-character + per-user pairing.