Decisions

A decision picks labels for one piece of work: which queue a ticket belongs in, and whether it needs a person today. One request asks every question. The result records whether the model or a person supplied each value.

Quick start

import { ai, gate } from "okengine";

export const ops = gate.policy("ops", ({ operator }) => operator.id !== null);

export const triage = ai.decision("triage", {
  review: ops,
  ask: {
    team: ai.choice("Which team owns this ticket?", {
      billing: "Billing",
      technical: "Technical",
    }),
    urgent: ai.boolean("Does this need a person today?", {
      true: "A person should see it today",
      false: "It can wait",
    }),
  },
});

import { on, flow, http, signal } from "okengine";
import { z } from "zod";

export const ticketOpened = signal.broadcast("ticket.opened");

export const open = on(
  http.post({ in: z.object({ text: z.string().min(1) }) }).public(),
  flow({
    do: async ({ text }, fx) => {
      await fx.emit(ticketOpened, { text });
      return { accepted: true };
    },
  }),
);

import { on, flow } from "okengine";
import { triage } from "@/core/ai";
import { ticketOpened } from "./open";

export const route = on(
  ticketOpened,
  flow({
    durable: true,
    do: async (input, fx) => fx.decide(triage, input),
  }),
);

curl -X POST http://127.0.0.1:6533/console/decisions/resolve \
  -H "Authorization: $OKE_OPERATOR_AUTHORIZATION" \
  -H "content-type: application/json" \
  -d '{"id":"<review id>","values":{"team":"billing","urgent":false}}'

{
  "team": "billing",
  "urgent": false,
  "$": {
    "meta": {
      "model": "typesafe/jev-1.13-20260917",
      "provider": "openrouter",
      "usage": { "inputTokens": 120, "outputTokens": 40 }
    },
    "team": {
      "how": "reviewed",
      "p": 0.42,
      "raw": { "billing": 0.2, "technical": 0.2, "none_of_these": 0.6 }
    },
    "urgent": { "how": "reviewed", "p": 0.51, "raw": { "noul": 0.51 } }
  }
}

Options

Lifecycle

A decision is in one list state. Drift is one flag per decision. Suspending triage leaves route alone.

learning  --labels-->  candidate ready  --promote-->  certified
                              \                           /
                               \---- drift ----> suspended

oke dev reloads oke-decisions.lock.json when the file changes. Production reads it once, at boot.

The runtime never raises a threshold and never invents a certificate. Missing lock, a stale question, a model version mismatch, an uncertified locale, low confidence, drift, and an outage all take the non-auto path.

How a person resolves a review

Open Console at /flows/decisions on port 6533. Each row is learning, candidate ready, certified, or suspended for that decision. The queue shows how long each row has been waiting. The resolve form only accepts a choice option, none_of_these, or a score level. An audit row submits labelOnly: true. A second resolve is Conflict. A lease collision retries. A failed label write is listed on the page.

An audit row is a label only. The original Flow already returned. A review row is the park: submitting it wakes the consumer.

The review gate must allow the caller. Console resolve runs on the operator plane: it passes the signed-in operator id and the session auth into the gate, and it may resolve any tenant. A gate that checks operator.id matches that call.

A review body must include every question that was not auto. An audit row (labelOnly: true) must include every question, including ones that were already auto. A choice may be none_of_these. A partial body is 422: review values do not match the open questions.

curl -X POST http://127.0.0.1:6533/console/decisions/resolve \
  -H "Authorization: $OKE_OPERATOR_AUTHORIZATION" \
  -H "content-type: application/json" \
  -d '{"id":"<audit id>","labelOnly":true,"values":{"team":"billing","urgent":true}}'

The same 409 split applies to tool approval. Retry only JournalLeaseBusy.

Certify and promote

maxError is the highest fraction of wrong answers you will accept among the ones the model is allowed to return alone. 0.05 means at most 5 in 100 of those automatic answers.

audit is how often an automatic answer is also queued for a person, from 0 to 1. 0.1 queues about one in ten. Those labels are how the next certificate is fit. Omitting audit when autonomy is set fails to compile.

risk is how sure the statistical test must be before it accepts a threshold. Default 0.1. A smaller risk demands more labels.

Calibration rescales the provider's raw scores so p means a probability. Choice and score use a temperature. Yes/no uses a curve fit on the provider score. The lockfile stores that fit next to the question.

The test, once: for each confidence cutoff from 0.50 to 0.99 (50 cutoffs), count mistakes on labels that cleared it. A binomial test passes only when those mistakes are few enough to show the error rate is under maxError. The loosest cutoff that passes is the threshold. If none pass, that question cannot return auto.

With risk at 0.1 and no mistakes, the label count is about log(0.1 / 50) / log(1 − maxError):

A seed file is JSONL. expect is the label for each question. locale is optional.

{"id":"t1","input":{"text":"I was charged twice"},"expect":{"team":"billing","urgent":true}}
{"id":"t2","input":{"text":"The app crashes on save"},"expect":{"team":"technical","urgent":false},"locale":"en"}

Point evals at that file, then certify. Run oke build first. oke eval --certify reads oke.manifest.json from the app root. Prompt evals do not run when the flag is present. It writes oke-decisions.lock.json in that root (OKE_ROOT_DIR, or rootDir), not in the shell's current directory. Boot reads that same file. Declared decisions without a project root fail at boot.

how: "auto" is what the Flow sees after that file is loaded and the question clears its threshold:

{ "team": "technical", "$": { "team": { "how": "auto", "p": 0.91, "audited": true } } }

OKE_ROOT_DIR=/srv/app oke eval --certify

The empty-string key is the slice used when locale is absent. A locale function gets its own key. hash is the question text and options. Edit the question and the hash no longer matches: the runtime stops returning auto until you certify again.

{
  "decisions": {
    "triage": {
      "model": "typesafe/jev-1.13-20260917",
      "certifiedAt": 1710000000000,
      "questions": {
        "team": {
          "": {
            "hash": "…",
            "calibrator": { "kind": "temperature", "t": 1.2 },
            "threshold": 0.8,
            "metrics": {
              "labels": 122,
              "accepted": 122,
              "errors": 0,
              "maxError": 0.05,
              "delta": 0.1
            }
          }
        }
      }
    }
  }
}

After production labels exist, the clock stores a candidate on the journal. It survives a restart only when that journal is a file or Postgres. A memory journal drops it when the process exits. Promote fetches it. It does not recompute the certificate. Other decisions already in the file stay. This decision's slot is replaced. Promote clears drift for this decision only.

OKE_ORIGIN=http://127.0.0.1:6530 \
OKE_OPERATOR_AUTHORIZATION='Bearer …' \
oke decide promote triage \
  --origin "$OKE_ORIGIN" \
  --authorization "$OKE_OPERATOR_AUTHORIZATION" \
  --lock "$OKE_ROOT_DIR/oke-decisions.lock.json"

That GET is /_oke/decisions/triage/candidate. The caller must be an operator. A missing candidate is 404. The route exists only when a decision declares autonomy.

oke decide labels <name> --export writes reviewed labels as seed JSONL (input, expect, locale). It is operator-only and limited to the caller's tenant. The file keeps the decision's declared in fields. Secret and redacted fields are masked. The command prints that the file contains production data.

oke decide labels triage --export --out triage.labels.jsonl

Patterns

export const triage = ai.decision("triage", {
  review: ops,
  ask: {
    team: ai.choice("Which team owns this ticket?", {
      billing: "Billing",
      technical: "Technical",
    }),
  },
});

Provider

The key is a Vault secret. fx.decide reads OPENROUTER_API_KEY, or TYPESAFE_API_KEY when driverId is "typesafe". Declare that secret on the Flow when you write an effects block. An empty value throws DecisionConfigError.

The deadline defaults to 30 seconds. 400, 401, 402, 403, 404, and 422 throw DecisionRequestError and leave the breaker closed. 429 and 529 retry up to 3 attempts and honor Retry-After. A wait over 60 seconds is an outage.

Network errors, timeouts, and HTTP 5xx count toward the breaker. Three failures open it for 30 seconds. An open breaker is an outage: review parks, abstain returns null.

Troubleshooting

Learn more

• Events — tool approval uses the same 409 split
• Gate — the policy on review
• fx — fx.decide effect row
• CLI — oke decide promote and oke eval --certify

Next