TL;DR
- The previous post on Jev left a claim hanging: most enterprise AI traffic is routing, triage, classification and gating — decisions, not prose.
- If that is true, the monolithic agent loop is the wrong shape. You are running your branch logic through a sampling process.
- Jev splits the agent into two planes: a decision plane that never generates, and a generation plane that does nothing else — with a control layer in code between them.
- Control flow moves out of natural language and back into code, where it can be tested, versioned and diffed.
- The accuracy figures are modest on purpose-built tasks (67.8% vs 74.1% on the vendor’s own dashboard). The architecture that follows is a cascade with escalation, not a swap.
- Calibration — not speed — is the part you test before you trust.
The Sentence That Needed a Product
The line that stuck from the Jev launch was this one:
Most production AI traffic in an enterprise is not creative. It is routing, triage, classification and gating. Running it through an autoregressive engine is a category error: you pay for a paragraph to obtain a label, and you inherit the latency of text nobody reads. It is a little like commissioning a 4,000-word memorandum to establish whether the office lights are off.
That was a diagnosis. It described a cost structure most enterprises have, cannot see, and have quietly agreed to live with. What it did not have was a treatment.
Jev is the treatment — a model that takes a state and a set of typed questions and returns typed answers: probabilities over options you supply, with nothing generated in between. But the model is not the interesting part. The interesting part is what happens to the shape of your system once the decision layer is no longer inside the language model.
That shape is different enough from what most teams have built that it is worth walking through properly.
The Architecture We Actually Built
Almost every production agent in an enterprise today is one model doing four jobs:
- Comprehend the state.
- Decide what to do next.
- Execute — usually by emitting a tool call as text.
- Narrate the result.
Steps 2 and 4 share an engine, and that is the problem. In a ReAct-style loop, the branch is prose. The agent thinks “I should look up the customer’s order first” and then emits get_order(...). The terminal condition is prose too — some variant of “I have completed the task” that your harness greps for and hopes it parses.
So consider what most enterprise agents actually are: the control flow is written in natural language, and interpreted by a sampling process.
That is a strange place to keep your branch logic. And it is expensive in a specific way. Output tokens are the costly kind. In a decision-shaped step you pay those output tokens to obtain a label, and you inherit the latency of a paragraph that never gets displayed to anyone.
Now count the decisions in one support-agent run:
- Which queue does this belong in?
- Is this a refund or a replacement?
- Is this customer authorised for that amount?
- Has this already been resolved somewhere else?
- Is this reply good enough to send?
Five branches. Essentially zero prose. Today you generate a sentence or two to obtain each one — and the sentence is the unreliable part, because a prompt edit three weeks ago can silently change how the branch resolves.
What Jev Does, In One Paragraph
You send a state — text, JSON, a document — and a set of questions. Questions come in three primitives: choice (pick one of up to 255 options, returns choice plus probabilities and confidence), score (place the state on an ordered 2–10 level rubric, returns score, legend and a distribution), and noul (the probability a yes/no statement is true, returned as a single number). All questions about a state are evaluated in parallel, so asking five costs barely more than asking one. Nothing is generated: every possible answer was enumerated by you in advance, which is why the output matches its schema by construction.
The vendor-reported economics: 70–500ms end-to-end, $0.042 per million input tokens with output free, and headline workload figures of 193.6x faster / 444.6x cheaper. Treat the last pair as a ceiling — a model that emits nothing is trivially fast on latency benchmarks. The durable claim is the pricing model, because it follows from the architecture rather than from a benchmark run.
That is the difference between calling a function and commissioning a memorandum.
The Split: Two Planes and a Control Layer
Once decisions have their own engine, the agent stops being one loop and becomes three layers:
┌───────────────────────────────────────┐
│ CONTROL LAYER │
│ (code) │
│ thresholds · escalation · retries │
│ audit log · deterministic backstops │
└──────┬─────────────────────────┬──────┘
│ │
┌──────────────────────▼──────┐ ┌──────────────▼──────────────────┐
│ DECISION PLANE │ │ GENERATION PLANE │
│ (Jev) │ │ (LLM) │
│─────────────────────────────│ │─────────────────────────────────│
│ route triage │ │ write synthesise │
│ classify score │ │ explain reason │
│ gate verify │ │ code plan │
│─────────────────────────────│ │─────────────────────────────────│
│ typed · calibrated │ │ free-form · unconstrained │
│ 70–500ms · ~free │ │ seconds · expensive │
│ no rationale │ │ rationale included │
└─────────────────────────────┘ └─────────────────────────────────┘Two things are worth noticing about this picture.
The decision plane carries most of the calls; the generation plane carries most of the tokens. Those are different bills. Separating them lets you optimise each one against what actually limits it — latency for the first, quality per token for the second.
The architecture now lives in the control layer, not the model. Which brings back the line from the reliability chasm: an agent isn’t an LLM with tools, it’s a system where the LLM is one component. Jev makes that sentence literally true instead of aspirational, because it forces the branch decisions out of the model and into code where you can see them.
Six Things That Change
1. Control flow moves back into code
A branch becomes a typed value you read and act on, rather than a sentence you hope is stable.
Before, routing policy lived in a system prompt: “Classify the ticket into one of billing, technical, or account.” Nobody could review it, nobody could diff it, and any unrelated prompt edit could perturb it.
After, the option set and the criteria are declared in advance and live in version control next to the code that branches on them. You can unit test the branch. You can diff a policy change and see exactly what moved.
The uncomfortable implication is the useful one: your routing rules become reviewable artifacts, which means they become somebody’s job to review. Teams that have been quietly changing customer routing by editing a prompt will need to start treating that as a policy change with a review and an approval. That is a governance improvement, and it will feel like friction for about a month.
2. Pre-action checks finally get a numeric input
Ask anyone who has built guardrails how they work, and you get one of two answers. Either they are deterministic — regex, allowlists, schema validation — or they are a careful instruction in the system prompt asking the model to please be careful, which is not a control, it is a wish.
A calibrated scorer gives you a third option. One documented example: an ambiguous rm -rf came back classified as irreversible at 0.56 probability, with only 0.33 confidence in that judgement. A number you can threshold, and a number that tells you the model itself is unsure — which is exactly the signature you want on the path to a human.
But note the second-order risk, because it is easy to miss. A gate that reads attacker-controlled text is a new attack surface. Adversarial content inside the state can shift the answers. So the decision plane adds a probabilistic check; it does not replace the deterministic ones. If your only defence against a destructive tool call is a model reading a document an attacker wrote, you have moved the problem rather than solved it. The allowlist stays.
3. The reliability math changes shape
The compounding problem is real and unforgiving: an agent at 95% per-action reliability succeeds on 36% of 20-step tasks. The instinct is to read Jev as a fix for that. It is not, and the honest version is more interesting.
Jev is less accurate than the frontier models it is priced against — on the vendor’s own benchmark, on every task. So the shape of the improvement is not “each step is now 99%.”
It is that the system now knows when it doesn’t know. A 95%-accurate step that reports 0.4 confidence is a step you can escalate. A 95%-accurate step that reports nothing forces you to trust it equally with the step that reports 0.98. Calibration converts an invisible failure into a visible branch, and a visible branch is one you can route around.
Two honest caveats. Calibration is aggregate: 0.9 does not mean this answer is right, it means the answers you gave at 0.9 were right about 90% of the time. And it is a property you must verify on your own data — a confidence score you have not tested is decoration.
4. Cost collapses, so fan-out becomes the cheaper instinct
At $0.042 per million input tokens with free output, a 300-token support ticket costs about $0.0000126 — roughly $1.26 per 100,000 tickets. At that price the arithmetic that has shaped your designs for two years inverts.
The old instinct: cram everything into one prompt, because every additional model call is another round trip and another pile of tokens. Hence the enormous classifier prompts, the “do these five things in one response” instructions, the fragile JSON you parse out of prose.
The new instinct: ask thirteen narrow questions about one state in a single call. TypeSafe measured a 13-question batch at 12.2x cheaper and 10x faster than thirteen sequential calls, and latency barely moves when you add questions, because they run in parallel against one shared read of the state.
That is not just cheaper. Thirteen questions with their own criteria are thirteen testable units, where one prompt doing five jobs is a single untestable blob. Cheap fan-out is better engineering, not just better economics.
5. Decisions come in under the interaction threshold
70–500ms, most near 100ms, against seconds for a frontier model. There is a real line around 300ms: below it, a decision can sit inline on the critical path and nobody notices. Above a second, it has to be designed around, hidden behind a spinner, or moved off the request entirely.
Most enterprise guardrails today are sampled or deferred precisely because they are too slow to run on every call. That is how you end up with the sentence nobody wants to say out loud: we check about 5% of tool calls. At decision-plane latency you can check all of them. Inline is the difference between a control and a sample.
6. Governance improves and degrades in the same move
Better: the decision policy becomes a first-class artifact. Options and criteria are declared in advance, so a reviewer reads the rubric instead of inferring intent from a system prompt. It is versioned, diffable, and it does not change when someone tidies the prompt for unrelated reasons.
Worse: there is no per-decision rationale, because there was no reasoning. And schema-valid is not the same as correct — the model will return a well-formed label that routes to the wrong queue. The “no hallucination” claim is a guarantee about type, not about truth. A confidently wrong gate is worse than no gate at all, because it fails silently and it fails at scale.
For decisions touching regulation, audit or user trust, the absent reasoning trace is disqualifying on its own. For a routing hop or a queue selection, nobody was going to read the explanation anyway.
The Accuracy Problem, Stated Plainly
This is the part that decides whether you should care, so it is worth being blunt.
On the vendor’s own dashboard — 711 cases across four tasks, reference answers averaged from two frontier models rather than ground truth — Jev scored 67.8% overall against 74.1% for the best comparator. Invoice processing: 61.8% vs 79.1%. Customer service: 76.0% vs 78.3%. The gap is consistent across every task, and the vendor publishes it.
Independent testing on a twelve-passage classification task with defects planted in it: Jev caught 6 of 7; a frontier model caught all 7 — at roughly 25x the latency and a small fraction of the cost, but not at parity, and not at parity on the thing that matters most, which is recall on the defects.
So: a naive swap loses. Every vendor chart in this category is a latency-and-cost chart. If you replace the LLM with a decision model somewhere accuracy is the binding constraint, you will ship a worse system that is faster and cheaper, and finance will be delighted for about a quarter.
Which is why the architecture the numbers justify is not a replacement. It is a cascade:
state ──▶ ┌──────────────┐ p ≥ threshold ┌─────────────────┐
│ DECISION │─────────────────▶│ act │
│ PLANE (Jev) │ └─────────────────┘
└──────┬───────┘
│ p < threshold
▼
┌──────────────┐ ┌──────────────────────────┐
│ ESCALATE │───────▶│ GENERATION PLANE (LLM) │
│ │ │ or human reviewer │
└──────────────┘ └──────────────────────────┘
│
▼
log the probability vectorJev decides first, cheaply, on everything. Anything under threshold goes up a level. The probability vector is logged with every decision.
You get 70–80% of your volume at decision-plane economics and keep frontier judgement for the remainder — and you get it because the cheap model reports calibrated confidence rather than plausible prose. That last clause is the whole trick. A cheap model that cannot tell you how sure it is cannot be cascaded with; you would have no signal to route on.
Where It Fits, And Where It Doesn’t
| Use case | Decision shape | Wrong answer costs | Fit |
|---|---|---|---|
| Support ticket triage and routing | choice + score | little | Strong |
| Tool-call and irreversible-action gating | noul | a lot — thresholded, with a deterministic backstop | Strong |
| Model routing inside the agent loop | choice | little | Strong |
| Scoring and verifying LLM output | score | moderate | Strong |
| Document, claims and invoice processing | choice + extraction checks | a lot — escalation mandatory | Conditional |
| Compliance labelling and PII classification | noul + choice | a lot — audit design required | Conditional |
| Planning, synthesis, code, novel reasoning | — | — | No |
| Any decision requiring an auditable rationale | — | — | No |
The pattern in that table is not the vertical you are in. It is the cost of being wrong, and whether anyone will ever ask why.
A Reference Shape
Take supplier invoice intake — the same question the original demo asked, run as a production system rather than a demo.
Read. Parse deterministically where the format allows it, and use the generation plane for the messy remainder. The extraction is a writing job; it is genuinely what the LLM is for.
Decide. One call, several questions, evaluated against the same state:
POST /v1/systemone
{
"model": "jev-latest",
"state": "<invoice text, PO reference, vendor history, prior payments>",
"questions": {
"routing": { "type": "choice", "options": ["straight_through", "review", "reject"],
"criteria": "Straight through only when the PO matches exactly..." },
"p_fraud": { "type": "noul", "instructions": "This invoice is fraudulent." },
"p_duplicate":{ "type": "noul", "instructions": "This invoice duplicates a prior payment." },
"po_match": { "type": "score", "levels": 5,
"criteria": "How well the line items match the purchase order." }
}
}Branch in code, on numbers rather than on sentences:
const { routing, p_fraud, p_duplicate, po_match } = await jev(state, questions);
// Thresholds below are illustrative. Test your own before trusting any of them.
if (p_fraud.noul > 0.9 || p_duplicate.noul > 0.9) return reject();
if (routing.confidence < 0.7 || po_match.score < 3 || po_match.confidence < 0.6)
return escalate({ reason: "low confidence", routing, po_match });
// Escalation summary is a writing job — generation plane.
if (needsReviewerNote) await llm.summarise(state, routing);
return straightThrough(routing);Log the probability vector with the decision. That log is now your audit trail, and it is a better one than the prose you were never reading: it records what the system believed, how strongly, and which threshold it crossed. When a reviewer overrides a decision, you have the exact number that produced it — which is also the data you will need to tune the threshold later.
Note what happened to the control flow. It is if statements. It is testable, reviewable, and it does not change when someone edits a prompt.
What To Do On Monday
1. Measure the ratio. Instrument thirty days of LLM calls and tag each as decision-shaped or generation-shaped. Almost nobody has this number, and it is the whole business case. In most enterprise stacks it lands between 60% and 80% decision-shaped. Measured, not asserted.
2. Test calibration, not accuracy. Build a labelled set from production traces, bucket predictions by reported confidence, and check the observed hit rate in each bucket. If your 0.9 bucket is right 60% of the time, you own a scorer, not a calibrated model — and every threshold you set on it is a guess wearing a number. This single test determines whether the cascade works.
3. Start in the quadrant. Wrong answer cheap, slow answer expensive. Earn the right to move closer to decisions that matter.
4. Write thresholds in code and version them. Thresholds are policy. Policy that is not versioned is not policy.
5. Keep the LLM. This is a split, not a replacement. Once decision traffic stops competing with it for budget, the generation plane gets more useful, not less — because you can finally afford to give it the context and the care that writing actually deserves.
The Point
The quotable claim is 200x. The defensible one is narrower and considerably more useful: the decision layer was never the language model’s job, and enterprises have been paying generative prices to obtain a branch statement.
Once you accept the split, the architecture reorganises around it. Deterministic where it can be. Generative where it must be. A calibrated probability and an explicit threshold in between — which is a much better place to put a boundary than a sentence in a system prompt.
The office lights question does not need a memorandum. It needs a switch, a policy about who is allowed to touch it, and a log of who did. For the first time, all three are things you can actually buy.
The interesting claim is not that a model got faster. It is that a layer of your architecture turned out to be in the wrong place.
Frequently Asked Questions
Why does Jev change agent architecture at all?
Because most enterprise agent traffic is decision-shaped — routing, triage, classification, gating — and today every one of those decisions is produced by an autoregressive model generating text. That makes your control flow natural language interpreted by a sampling process. When decisions have their own engine, they become typed values your code branches on, which moves control flow back into software where it can be tested, versioned and reviewed.
What is a “decision plane” in agent architecture?
It is the layer of an agent responsible for structured choices rather than prose: which queue, which model, which tool, is this safe, is this good enough. In the Jev architecture it is separated from the generation plane (which writes, reasons and synthesises) and joined to both by a control layer in code that holds thresholds, escalation rules and the audit log. The decision plane carries most of the calls; the generation plane carries most of the tokens.
Can Jev replace the LLM in my agent?
No, and attempting it will make your system worse. On the vendor’s own benchmark Jev scores 67.8% against 74.1% for the best comparable LLM, and 61.8% against 79.1% on invoice processing; independent testing found it caught 6 of 7 planted defects where a frontier model caught all 7. The architecture the numbers justify is a cascade: the decision model handles everything it is confident about, and low-confidence cases escalate to the LLM or a human.
How should I set confidence thresholds?
In code, with the values in version control, and only after testing calibration on your own labelled data. Bucket predictions by reported confidence and check the observed hit rate in each bucket. If the 0.9 bucket is right 60% of the time, no threshold is meaningful yet. Because calibration is aggregate, a 0.9 confidence means the answers given at 0.9 were right about 90% of the time — not that any individual answer is correct.
What is the risk of using a decision model as a guardrail?
Three. Adversarial content in the state can shift answers, so a probabilistic gate reading attacker-controlled text is a new attack surface — keep the deterministic allowlist. Schema-valid is not the same as correct: the model can return a well-formed label that routes to the wrong destination, and the “no hallucination” claim is about output type, not truth. And there is no reasoning trace, so a wrong gate fails silently. Confidently wrong is worse than no gate when it happens at scale.
Which enterprise use cases fit Jev best?
Start where a wrong answer is cheap and a slow answer is expensive: support ticket triage and routing, model routing inside the agent loop, scoring and verifying LLM output, and tool-call guardrails backed by deterministic checks. Document and invoice processing fits conditionally, with escalation treated as mandatory. Long-horizon planning, synthesis, code generation and any decision requiring an auditable rationale do not fit at all.
About the Author
Vinci Rufus is a technologist and writer focused on building reliable AI systems and agent architectures. He writes about the practical challenges of production AI implementations and the architectural patterns that separate demo agents from production-ready systems. His work covers the model that refuses to write, the reliability chasm in AI agents, and agentic workflow design.