Kubernetes
Run an OKE app as a plain Deployment — PgDog, readiness vs liveness probes, graceful SIGTERM, shared drivers, and honest multi-instance limits.
OKE apps are safe to run as more than one replica when shared drivers back Clock, Journal, Gate, and Store. Kubernetes is full orchestration with the largest ecosystem — use a Deployment, never a StatefulSet.
The one rule
Treat every replica as replaceable. Put durable state in Postgres / Redis / S3, probe GET /_/ready for readiness and app GET /health for liveness, and set
terminationGracePeriodSeconds high enough for lease release on SIGTERM.
Quick start
Deploy as a Deployment
Use a plain Kubernetes Deployment (or compatible replica set). Do not
use a StatefulSet — there is no per-pod volume identity, ordered startup, or
sticky network id to preserve. Scale with replicas: N.
Share backing services
Pin docker/prod drivers to shared stores:
| Concern | Driver | Shared backend |
|---|---|---|
| SQL / durable journal / clock | postgres | Postgres (optionally via PgDog) |
| Gate rate counters | redis (drivers.store.kv) | Redis / Valkey / Dragonfly (images["store.kv"]) |
| Files | s3 | S3-compatible object store |
| Signal | see known limits below | — |
Probe readiness and liveness
- Readiness:
GET /_/ready— returns200 { ready: true }only after boot and the durable orphan scan finish. While booting or scanning orphans it returns503 { ready: false, reason: "booting" | "orphan_scan" }. - Liveness: app-authored
GET /health(create-oke ships one) — cheap “process is alive,” not orphan-scan aware.
Wire readinessProbe to /_/ready and livenessProbe to /health.
PgDog pooling
Bun.SQL defaults to 10 connections per process — fine for one pod.
Scale the Deployment and N × pool can exceed Postgres max_connections.
When both store.sql and pgdog are pinned (docker/prod default in templates),
DATABASE_URL points at PgDog on port 6432 — transaction pooling, wire-protocol
transparent to Bun.SQL / Drizzle. Point the cluster Service at that pooler.
Why PgDog. Transaction pooling fixes the connection math. Naive poolers can
leak session state (SET, RLS vars, LISTEN/NOTIFY) across clients; PgDog
re-applies those under transaction mode.
Read-replica routing (BEGIN READ ONLY → replica) is documented readiness —
not configured as a generated Kubernetes manifest this round.
Redis-protocol images (Redis · Valkey · Dragonfly)
images["store.kv"] defaults to Redis (most mature). Pin Valkey or Dragonfly
the same way — driver id stays redis, same REDIS_URL. Image table + one-line
licenses: Store · KV.
Readiness vs liveness
| Endpoint | Role | Kubernetes |
|---|---|---|
GET /_/ready | Kernel readiness (booting → orphan_scan → ready) | readinessProbe — no traffic until Ready |
GET /health | App liveness (create-oke) — cheap “process alive” | livenessProbe — restart if the process is dead |
Consequence: orphan scan delays Ready (and traffic) on purpose so a new
pod is not hit mid-takeover. Do not point liveness at /_/ready — a long orphan
resume would kill the pod instead of waiting.
Graceful shutdown
On SIGTERM / SIGINT, installGracefulShutdown (wired by oke dev’s app
runner; call it next to createBunRuntime().serve in custom entries):
- Stops accepting new connections
- Releases Clock cron leases and Journal run leases held by this instance
- Closes element runtimes and exits
Signal message leases have no proactive release today — survivors reclaim after
the visibility TTL (default 30s). Set terminationGracePeriodSeconds ≥ your
longest lease TTL (typically ≥ 30s) so release finishes before kubelet SIGKILL.
Ingress (TLS / routing)
Use cluster-native Ingress or Gateway API for HTTPS.
Compose images.proxy recipes (Caddy / Traefik) target plain Docker and Swarm —
see Reverse proxy when you are not on Kubernetes.
Still set serve allowedHosts to the public hostname
(Security).
Known multi-instance limits
Be honest about what is still process-local:
| Surface | Multi-instance today |
|---|---|
Clock (postgres / shared file) | Shared leader election |
Journal (postgres / shared file) | Shared durable runs + orphan resume |
Gate rates (drivers.store.kv: redis) | Shared counters |
Store SQL / KV redis / files s3 | Shared |
Store files fs | Per-instance filesystem — boot warns; use s3 when scaled |
Signal redis | Emit relays to Redis; consume / live / drain are process-local outbox until Streams consume ships |
| Channel suppression / consent / receipts | Default stores are process-local — opt-out on replica A is invisible to B until a durable driver ships |
Troubleshooting
Check readiness: /_/ready stays 503 with reason: "orphan_scan" until the
durable orphan resume finishes. A long resume delays Ready; that is intentional
so traffic does not hit a pod mid-takeover.
drivers.store.kv must be redis with REDIS_URL set. A missing URL now
fails boot (no silent memory fallback). Declaring memory for local is fine;
docker/prod should stay on redis.
drivers.store.files: "fs" is single-host. Switch docker/prod to s3
(create-oke templates already do).
Liveness is pointed at /_/ready. Move liveness to /health; keep readiness
on /_/ready so the pod stays Alive while Waiting for Ready.
Learn more
- Deployment — choose compose, Swarm, or Kubernetes
- Docker — plain compose
- Docker Swarm —
docker stack deploy - Reverse proxy — Caddy / Traefik (non-K8s)
- Store — PgDog and facets
- Clock — shared CronStore / leader election
- Flow — durable journal
- Signal — delivery modes and redis honesty
- Channel — suppression process-local limit
- CLI —
oke docker --prod,oke start
Next
Docker Swarm
Deploy the generated compose stack with docker stack deploy — PgDog, HEALTHCHECK readiness, graceful SIGTERM, known multi-instance limits, and optional Caddy or Traefik.
Reverse proxy
Opt-in Caddy or Traefik for TLS termination and routing on plain Docker and Swarm — custom domains, Let's Encrypt, and the socket-proxy security model.