Technology
Decision API: The Fastest Path From Raw State to a Typed Verdict
Most teams that try to get structure out of a chat model end up writing the same layer twice: once to coax the model into a format, and again to defend every downstream system from what happens when it does not comply. Prompt-level JSON mode helps, but it does not change the fundamental shape of the interaction. You are still asking a text generator to behave like a function, and then paying a parser, a retry loop, and a few hundred milliseconds of latency for the privilege.
A decision API takes a different position. The schema is not a suggestion in the prompt — it is the contract of the endpoint. You send state, you declare the questions in code, and you get back typed, schema-valid answers in a single pass, each one carrying a calibrated confidence your application can act on. For a technical audience already comfortable with REST, this is a far more natural primitive than a conversational model wearing a JSON costume.
What a Decision API Actually Is
A decision API is an HTTP service that accepts raw state — plain text, structured JSON, or up to 16 images and video frames — along with a set of questions defined in code, and returns answers expressed as concrete types rather than prose. DecisionNode implements this as a single POST /v1/decide endpoint, backed by a “System One” decision model built specifically to produce calibrated, typed output instead of generated text. There is no system prompt to tune, no temperature to argue about, and no free-form response to sanitize.
The contrast with the two approaches most teams already run is worth making explicit. Hand-written rule engines are fast and deterministic but brittle: they handle the cases someone thought of, and they accumulate exceptions until the rule file becomes tribal knowledge nobody wants to touch. Chat-based LLM calls handle ambiguity well but are slow, non-deterministic, and structurally unreliable — the right answer expressed in the wrong shape still breaks production. The decision api model closes the gap: it keeps the ambiguity tolerance of a learned model while behaving operationally like a function call.
Because the questions are declared in code rather than in a prompt string, the request itself becomes the specification. A routing decision asks for choice over a fixed label set. A risk assessment asks for truth, a calibrated probability between 0.00 and 1.00. A severity check asks for score on an ordered scale. The model returns exactly those fields, in the requested form, on the first attempt.
That first-pass guarantee matters more than it sounds. Every retry you remove from the pipeline removes a tail-latency spike, a partial failure mode, and an alert nobody wants to page on. It also removes the quiet drift that appears when someone edits a prompt in staging and forgets that six services downstream relied on the old wording.
Typed Outputs, Calibrated Confidence, and Threshold Logic
The value of a decision API is not just that answers are typed — it is that the types are designed around the decisions software actually makes. The five core output shapes cover most production needs: choice returns one label from your list, score returns a level on an ordered scale, truth returns a calibrated probability between 0.00 and 1.00, number returns a value on a grid, and point and box return locations inside an image. Combined with text, JSON, and up to 16 image or video-frame inputs, that vocabulary covers routing, scoring, gating, moderation, and visual inspection without inventing a new response format each time.
Calibration is the part most engineers underestimate. A raw softmax score tells you how confident the model was relative to its other options; a calibrated confidence tells you something about the world — that among answers reported at 0.80, roughly 80 percent turn out to be correct. That distinction is what makes threshold logic safe to write. Code acts instantly above a threshold, and below it either re-checks with more context, escalates to a human, or takes a deliberately safe path. You are not guessing whether the model “seemed sure”; you are applying a policy to a number with known behavior.
Consider a support queue. A single request can ask three things at once: choice for the routing destination across your label set, score for urgency on a four-level scale, and truth for whether the ticket involves a billing dispute. Answers arriving at 0.95 confidence get auto-routed in milliseconds. Answers arriving at 0.55 get queued for a second look. Nothing in the application parses a sentence, strips a code fence, or falls back to a regex because the model added a friendly preamble.
Determinism completes the picture. The same request returns the same answer, which means decision logic becomes testable in the ordinary way: fixtures in, fixtures out, regression suite green or red. It also means cached results stay valid, retries are idempotent, and an incident review can replay the exact request that produced a bad outcome instead of theorizing about sampling.
Speed, Determinism, and Cost at Decision Scale
Decision APIs live inside request paths where a chat completion would be a luxury. DecisionNode Flash runs at roughly 5 ms server-side and 19 ms end to end, while DecisionNode MoE runs at roughly 9 ms. Those numbers sit comfortably inside an API gateway, a checkout flow, or a control loop — places where a 900 ms generative call would need a queue, a spinner, and an apology. Pricing follows the same logic: $0.035 per 1M input tokens for Flash and $0.042 per 1M input tokens for MoE, with output tokens always free, batch discounts available, and a free daily tier for development and low-volume production paths.
The economics become clear at volume. A platform making a million decisions a month, each sending a few hundred tokens of ticket text or order context, is working with roughly 200 million input tokens — on the order of a few dollars in input spend. Output tokens, which are where chat models quietly accumulate cost, cost nothing here because there are none to generate. For teams that have been budgeting per-completion retries and fallback paths, the comparison is not close.
Determinism and cost reinforce each other in a less obvious way. Because identical requests return identical answers, aggressive caching is safe: a shipping-eligibility check for the same order state does not need to hit the endpoint twice. Because answers are typed, a cached result can be validated against a schema on read, so a stale or corrupt entry fails loudly instead of propagating a malformed object downstream.
For loops that need more than about 10 decisions per second — drones, robots, simulators, continuous machine monitoring — stateless HTTP round trips are the wrong shape. DecisionNode offers sessions over WebSocket for exactly that regime, keeping the decision channel open so per-decision overhead does not dominate the control loop.
Real-World Patterns and Production Implementation
The pattern that appears most often is replacing rules that have outgrown themselves. Ticket routing and triage, fraud and promo-abuse checks, order and shipping decisions, moderation, scoring and gating — these are all places where a hand-written ruleset started as a reasonable heuristic and slowly became a liability. A decision API keeps the hard edges (your label set, your thresholds, your fallback paths) in code, while moving the ambiguous judgment into a model that returns a calibrated number you can act on.
Visual decisions follow the same model with the image inputs. Quality control on production lines, warehouse monitoring, and traffic analysis all depend on locating things in frames rather than describing them. The point and box output types handle localization directly, and optional boosters — human age estimation, AI-generated image detection, face detection and recognition, OCR, depth, and segmentation — can be layered onto the same request instead of chaining separate services.
AI agents deserve their own mention, because the failure mode is well known: an agent that reasons fluently and then acts on a guess. The inverse pattern — decide before acting — uses the decision API as a gate. The agent asks a typed question, receives a calibrated answer, and proceeds only when confidence clears a threshold. Below threshold it asks for more context, chooses the reversible action, or hands control to a human. That is a far more defensible architecture than trusting a language model’s self-reported certainty in prose.
Implementation is deliberately boring, which is a compliment. SDKs cover TypeScript, Python, Go, Ruby, and curl; MCP tools and webhooks let agents and event-driven systems integrate without bespoke glue. The practices that hold up in production are the same ones that hold up anywhere else: run new decision types in shadow mode against live traffic before enforcing them, log confidence alongside outcomes so you can watch calibration drift, set thresholds per decision rather than globally, and use batch processing for backfills and offline scoring. Product and ops teams get a predictable knob to turn; platform engineers get an endpoint that behaves like the rest of their infrastructure.
Copenhagen-born environmental journalist now living in Vancouver’s coastal rainforest. Freya writes about ocean conservation, eco-architecture, and mindful tech use. She paddleboards to clear her thoughts and photographs misty mornings to pair with her articles.