Aeion AI for Chatbot Apps
The infrastructure layer for ChatGPT-style products. Streaming SSE responses, automatic multi-provider fallback when one provider rate-limits, per-tenant + per-user cost caps to prevent runaway bills, full conversation history in `ai_usage_logs`, BYOM provider keys (no Aeion-side billing or markup). Real engine; no third-party gateway in the hot path.
What Chatbot Products Actually Need
A production chatbot stack has five hard requirements:
End-to-End Chatbot Flow
```typescript
Recommended Provider Stack
What experienced teams reach for in a production chatbot.
Tenant Cost Cap Pattern
ai_budgets today is a per-tenant cap — one active budget record per tenant, not separately scoped per-user or per-conversation. Set the platform-wide ceiling; build any finer-grained (per-user, per-conversation) limiting yourself in your own application logic.
FAQ
Aeion's conversation-management layer includes context-aware truncation — it keeps the most-recent N turns plus the system prompt and drops older turns. For long conversations where older context matters, pair with embeddings + RAG (re-fetch relevant older turns by semantic search).
Yes — Aeion exposes a unified tool-use abstraction. Define tools with a schema; Aeion translates per-provider (OpenAI's function-calling, Claude's tool_use, Gemini's functionDeclarations).
Aeion's tenant + user + conversation IDs scope everything. Each conversation is isolated; there's no cross-conversation context leakage even at scale.
Typical: $1-2 / user / month hard cap on free tier. With Gemini Flash-Lite at $0.04 / 1M out, $1 = 25M output tokens = ~16K assistant responses. Sufficient for most casual users.
Yes — `ai_usage_logs` queryable per-user. The admin UI at `/admin/ai/usage` surfaces per-user dashboards; customer-tenant admins can see their own users' usage.
Use OpenAI's `omni-moderation-latest` (free) as a pre-filter; combine with Gemini's safety settings + Anthropic's built-in refusal behavior. Q4 2026 roadmap adds Lakera / NVIDIA NeMo guardrails as pluggable adapters.