AeionClaw + Ollama — Self-Hosted LLM
Run AeionClaw fully on-premises with Ollama. No external API calls; data never leaves your infrastructure; zero per-token cost beyond hardware amortization. Required for HIPAA-bound deployments, classified networks, air-gapped environments, or organizations with strict data-sovereignty requirements. Llama 3.3, DeepSeek, Qwen, Mistral, Phi — pick the model that fits your hardware + use case.
When to Use Local LLMs
`
Model Selection + Hardware Requirements
`
Setup Walkthrough
```bash
Performance Tuning + Tool-Use Reliability
`
FAQ
Yes — Ollama-only config means all inference happens on your hardware. AeionClaw confirms this in the provider config (`no_training_required: true` + Ollama base_url is local). All other Aeion data is also tenant-local. Fully air-gappable.
Configure failover chain. For air-gapped deployments, fallback to a second Ollama node (HA pair). For hybrid (HIPAA persona on Ollama, other personas on Claude), failover to commercial when the workload allows.
~$15-25K one-time for an A100 80GB box running Llama 3.3 70B handles 4-12K conversations/day. Amortized over 3 years vs commercial API spend: crossover at ~5K conversations/day in commercial cost.
For high-stakes tool dispatch (destructive actions, financial operations), the 12-14% reliability gap matters. Mitigations: tighter validation, more retries, human-in-the-loop guardrails. For HIPAA / classified, the gap is acceptable because commercial isn't an option. For everyone else, hybrid (commercial primary + Ollama failover) is often the best of both.
Yes — Ollama supports LoRA + full fine-tuning. AeionClaw stores the fine-tuned model ID per persona. Training data + compute live on your hardware. Typical brand-voice tune: 100-500 examples, 2-6 hours on an A100.
Ollama handles tensor parallelism via the underlying llama.cpp. 4x A100 80GB can run Mistral Large 2 (123B) at production speed. Configure in Ollama via `--num-gpu N`.
AeionClaw doesn't have a dedicated vLLM or TGI adapter today — Ollama is the supported self-hosted path (its adapter is the one built to point at an arbitrary base URL and speak Ollama's API shape).
Vast.ai / RunPod / Lambda Labs offer A100/H100 hourly. For occasional use, cheaper than ownership. For 24/7 production, ownership amortizes. Point AeionClaw's Ollama provider at whichever rented box's address — there's no built-in fleet-management or auto-scaling for rented GPU instances.
Yes — Ollama runs natively on Metal. Llama 3.3 8B at decent speed (40-80 tok/sec on M2 Max). Not production-ready (single user, RAM-limited) but excellent for development + air-gapped demos.
Identical to commercial — every Claw call is written to the audit log and spend ledger (with cost recorded as $0 for local inference). Audit story unchanged; data sovereignty stronger.