AI provider toggle (Anthropic ↔ Ollama)
Switch the LLM backend at the platform level (gh#778). Anthropic for production-quality output; Ollama for sandbox/preprod testing without per-call cost.
Where it lives
In AI Cockpit → Setup → AI Provider.
| Provider | When to use |
|---|---|
| Anthropic | Production. Best quality. Per-call cost via Anthropic API. |
| Ollama | Sandbox / preprod / demo. Self-hosted (Mac mini). No per-call cost. Faster iteration during testing. |
Per-env defaults
- prod — Anthropic only. The Ollama toggle is hidden on prod (gh#822 — pre-fix the toggle rendered but prod backend ignored it; UI-gating gap closed).
- preprod / sandbox / demo — Ollama default, with operator toggle to Anthropic if they need to test against the real model.
Setup gate
Before switching to Anthropic in a non-prod env, the operator must set the Anthropic API key in the Setup tab. The toggle is disabled until a key is present.
Key save flow (gh#778):
- Open AI Cockpit → Setup.
- Tap + Add Anthropic key.
- Paste the key.
- Save.
- Toggle Anthropic → confirms key is valid + saves provider preference.
The earlier key-save gate bug is fixed in gh#778 — keys persist correctly.
Ollama model
The sandbox Ollama instance runs at the Mac mini at OLLAMA_BASE_URL (rotating cloudflared tunnel — see memory/reference_sandbox_local_llm_ollama.md for the fallback). Current default model: hermes3:8b (gh#217f0660 — synced fallback to hermes3:8b from qwen2.5:14b).
What stays Anthropic-only
Some AI surfaces require Claude-grade reasoning + Anthropic-only features (extended thinking, large context):
- Pulse Copilot orchestrator + specialist routing (always Anthropic, even in sandbox).
- Agentic Copilot writes (gh#830) — Anthropic-only.
These hardcode Anthropic regardless of the toggle.
Cost transparency
The AI Cockpit → Cost dashboard shows per-call cost in € for Anthropic calls and "0 (Ollama)" for Ollama calls — so operator can see exactly when the toggle saved money during test runs.
Implements: gh#778 (AI provider toggle + Setup key-save gate), gh#822 (hide Ollama toggle on prod). Related: AI Cockpit.