Models
Declare ai.model bindings — known providers auto-resolve baseUrl; Anthropic and Google OpenAI-compat have documented limits.
An ai.model binding names a logical model (smart, local, …), the wire model id, and how
to reach it (provider, optional baseUrl / apiKey / driverId).
For developers swapping OpenRouter for a self-hosted OpenAI-compatible /v1 — one logical
name, different bindings per environment.
The one rule
Known OpenAI-compatible provider names resolve baseUrl automatically. Explicit baseUrl
always wins. Unknown providers require baseUrl — they fail loud. Per-binding apiKey isolates
tokens when several providers run together.
Smallest Example
Declare cloud + BYO bindings
import { ai } from "okengine";
export const smart = ai.model("smart", {
provider: "openrouter",
model: "openrouter/free",
apiKey: process.env.OPENROUTER_API_KEY,
});
export const local = ai.model("local", {
provider: "openai-compatible",
model: process.env.OKE_AI_MODEL ?? "your-model-id",
...(process.env.OKE_AI_URL?.trim() ? { baseUrl: process.env.OKE_AI_URL.trim() } : {}),
});OpenRouter fills https://openrouter.ai/api/v1 without typing it. Any
OpenAI-compatible /v1 uses the same driver — set OKE_AI_URL yourself;
Compose does not manage inference.
Attach a versioned prompt
export const triage = smart.prompt("ticket-triage", {
version: 1,
via: ["smart", "local"],
});Ask from a Flow
import { on, flow, http } from "okengine";
import { triage } from "@/core/ai";
export const classify = on(
http.post(),
flow({
do: async (input, fx) => await fx.ask(triage, input),
}),
);fx.ask uses the prompt’s model chain. Recovery via via opens a separate client per
binding — keys and base URLs do not mix.
Progressive Patterns
Zero Docker — registry fills the URL:
export const smart = ai.model("smart", {
provider: "openrouter",
model: "openrouter/free",
apiKey: process.env.OPENROUTER_API_KEY,
});See OpenRouter for router aliases (openrouter/auto, fusion, …).
Options
| Option | Type | Default | Meaning |
|---|---|---|---|
provider | string | — | Registry name or exempt/local label |
model | string | — | Wire model id sent to the provider |
baseUrl | string | auto / omit | Override (always wins over registry) |
apiKey | string | — | Per-binding key (isolates multi-provider apps) |
driverId | string | app default | Protocol driver (openai-compatible, anthropic, mock, …) |
tier | string | — | Optional app label (not the registry status below) |
Empty name throws TypeError: ai.model: name is required.
Verified providers
These names auto-resolve a verified OpenAI-compatible baseUrl:
| Provider | Base URL (auto) |
|---|---|
openai | https://api.openai.com/v1 |
openrouter | https://openrouter.ai/api/v1 |
groq | https://api.groq.com/openai/v1 |
together | https://api.together.ai/v1 |
deepinfra | https://api.deepinfra.com/v1/openai |
xai | https://api.x.ai/v1 |
mistral | https://api.mistral.ai/v1 |
deepseek | https://api.deepseek.com |
vercel | https://ai-gateway.vercel.sh/v1 |
Not registered (pass baseUrl if you still need them): Cloudflare Workers AI
(account-scoped URL); Meta (retired Llama OpenAI-compat API / unverified Muse Spark
host — do not guess a URL).
Exempt labels (no registry URL): mock, local, openai-compatible.
Limited compatibility
anthropic, google, and alias gemini also auto-resolve a URL, with documented limits.
Prefer native Anthropic for production Claude.
| Provider | Base URL (auto) |
|---|---|
anthropic | https://api.anthropic.com/v1 |
google / gemini | https://generativelanguage.googleapis.com/v1beta/openai |
Anthropic OpenAI-compat is evaluation-only
Anthropic’s OpenAI-compatible endpoint is for testing/comparison only. tools[].function.strict
is ignored; n must be 1; no embeddings here. Production: driverId: "anthropic" (native
Messages API).
Google OpenAI-compat tool schemas
Tool/parameter schemas are not full OpenAI JSON Schema fidelity. Complex Flow-as-tool schemas can fail — do not rely on this path for agent tool calling in production.
Declare-time and oke extract both warn when these providers auto-resolve.
Native vs OpenAI-compat Anthropic
| Path | When |
|---|---|
driverId: "anthropic" | Production Claude — native Messages API |
provider: "anthropic" without native driver | Quick eval via openai-compatible + registry URL |
They are not interchangeable for agents that depend on reliable tool calling.
Streaming
fx.stream(model, { prompt, data?, via? }) records asks and yields token chunks. Drivers
that do not support stream throw
ai: model "…" (driver …) does not support stream.
for await (const chunk of fx.stream(smart, { prompt: "Say hello" })) {
// chunk is a string token
}Per-environment drivers
See AI Overview. create-oke does not pin
drivers.ai by default — unset dev/test → mock; prod must declare.
Troubleshooting
The name is not in the registry (and not an exempt local label). Pass baseUrl, or use a known
provider id from the tables above. Exact message lists known providers and notes that Cloudflare
and Meta always need baseUrl.
Expected for anthropic / google / gemini when auto-resolving the OpenAI-compat URL. Switch
to native driverId: "anthropic" for production Claude.
The binding has no resolvable client and boot has no default AI driver. Pin drivers.ai or set
OKE_AI_DRIVER / OKE_AI_URL for the active environment.
Pass a non-empty logical name ("smart", "local", …).
Learn more
- OpenRouter — free / auto / fusion routers
- Prompts —
model.promptandvia - AI — element overview and
fxsurface