A Detector That Reads Probabilities Instead of Writing Verdicts: What 44 Benchmarks Say About Jev

Translation pending — read the English original while we localise.

A detector for alignment failures that answers many typed questions in one call, with calibrated probabilities, ranks failures well and costs 63× less than LLM-judge scorers. But those probabilities don't transfer as thresholds, and the label and its reference matter far more than the question's wording.

The cost structure of watching every message #

Deployed models still defer to a user's mistaken belief (Sharma et al., 2024). That sentence is a modest one, but it frames the whole problem. If sycophancy of that kind persists in deployed systems, then someone has to watch for it, and the watching has to happen at the same cadence as the conversation itself.

That's where the economics bite. A detector needs to be accurate and, because it runs on every message, cheap. Either property alone is easy to obtain: a sloppy detector is cheap, and a careful one that's too expensive to run on every message watches only a sample of the traffic. The interesting engineering question is what shape of detector can be both.

The current answer, according to the paper behind this article, has a structural flaw. Most detectors of alignment failures are generative judges that spend a decoding pass on every criterion. Put plainly, current detectors pay a decoding pass for every criterion they check. A monitor that wants to check for ten different failure types is paying for ten passes, and the bill grows linearly with the number of things you care about.

There's a partial fix, and it's worth being exact about its limits. Classifiers that read token probabilities, such as Llama Guard, still score one fixed label per call. Reading token probabilities gives a score, but each call still scores one label or criterion, because the probability is read off one verdict token. The decoding pass is gone, yet the one-question-per-call structure remains. You've made each call cheaper without changing how many calls you need.

So the structural claim runs like this: generative judges are expensive per criterion, and probability-reading classifiers are cheaper per criterion but still tied to one criterion each. Neither lets you ask many questions about the same input at once.

Architecture Dimension	Generative LLM Judges (e.g. GPT-4 as Judge)	Single-Token Classifiers (e.g. Llama Guard)	Multi-Query RLCD Detector (Jev)
Scoring Mechanism	Full autoregressive decoding pass per criteria	Softmax probability over single output token	Multi-head calibrated probability tensor $P(\text{yes})$
Questions per Call	1 evaluation prompt per model invocation	1 classification label per model invocation	Multiple binary/categorical/ordinal questions in 1 call
Inference Cost	High ($10–$50 per 1,000 interactions)	Moderate ($0.50–$2.00 per 1,000 interactions)	63× lower ($0.30 across 19 full benchmarks)
AUROC Performance	Benchmark dependent; prone to judge drift	High on specific trained safety categories	Median AUROC 0.886 zero-shot across 44 suites
Calibration Quality	Heuristic explanations without formal probabilities	Rough logits without post-hoc calibration	Calibrated probabilities; median ECE 0.168 (requires ~10 labelled items)

What Jev does differently #

Jev is trained with reinforcement learning for calibrated decisions (RLCD). It answers many typed questions about one input with calibrated probabilities in a single call. The mechanism is worth stating carefully, because the rest of the article depends on it.

An RLCD model returns a decision with a calibrated probability for each typed question instead of generating text. There's no generation step in which a verdict gets written out and then parsed. Jev answers many binary, categorical, and ordinal questions about one input, which the paper calls the state, in a single call. The state is the input; the questions are the things you ask about it.

That's a different cost curve from either of the approaches above. If the questions share one call, then adding a question doesn't add a decoding pass. The payoff for a monitor that cares about several failure types at once is obvious, though the paper's measurements concern what a single generic question achieves, and I'll stay with what was measured.

There was a gap, though. Whether Jev detects alignment failures had not been measured. A model that answers typed questions with calibrated probabilities is a plausible fit for the job, and plausibility is cheap. The paper sets out to supply the missing measurement.

Why alignment failures are harder than they look #

Before the measurement, a complication. Many of these failures are relational, defined against a reference that the response alone does not reveal. Sycophancy is a good illustration of the general idea: whether a response defers to a user's mistaken belief depends on what the belief is and whether it's mistaken, and neither fact is visible in the response text by itself.

This has a consequence for detector design that the paper states with appropriate hedging. A detector that sees only the response may lack the reference that defines the failure. The word "may" matters; the claim isn't that every failure needs a reference, only that some do and that a response-only view can miss it.

It also complicates diagnosis. If failures are relational, a poor detection may come from the question or from a state that lacks the reference. Two things could be wrong, and a low score doesn't say which. Perhaps the question was badly worded. Perhaps the question was fine and the input simply never contained what was needed to answer it. A benchmark that wants to say anything useful about a detector has to be built so those two causes can be pulled apart.

The developer, for its part, lists indirect meaning and adversarial content among Jev's known weaknesses. That's worth keeping in mind when reading the results later, because some failure types lean on exactly those properties.

Treating detection as classification #

The paper's formal setup is plain. It treats alignment failure detection as binary classification over interactions of a target model. A detector assigns a score s(x) ∈ [0, 1] and flags x when s(x) ≥ t. That means it must both rank failures above non-failures and separate them at a threshold t.

Those two duties are separable, and the paper's results split along exactly that seam. Ranking is what AUROC measures. Separating at a threshold is what depends on calibration, and the article will come back to it because that's where Jev falls short. A detector can rank superbly and still be unusable at t=0.5 if its probabilities sit in the wrong place.

RLCDAlignBench #

The benchmark is called RLCDAlignBench. It covers ten failure types across 44 benchmarks and 7,193 detection instances. The instances come from five open 2–7B target models, each labelled by its benchmark's reference scorer. So the interactions being judged come from small open models, and the ground truth for each is whatever the originating benchmark's own scorer says.

Not every benchmark carries the same kind of label. Human labels on StrongREJECT and HarmBench, and second-judge labels on AbstentionBench and InstrumentalEval, are used to compare Jev with a judge. That gives the paper two kinds of external check on the reference scorers: people in some cases, a second judge in others.

The released artefact is described as 44 benchmarks across ten failure types with context variants, cached Jev answers, and rescoring scripts, together with a split-half protocol that removes selection inflation from the reported gains. Each element answers a practical need. Context variants let a reader ask what happens when the state holds more. Cached answers and rescoring scripts let someone reproduce a number without re-running the model. The split-half protocol deals with a methodological trap, which deserves its own section.

The selection trap #

Jev can be asked many questions about the same input. That's its selling point, and it's also a hazard for evaluation. If you pose a set of candidate questions and report whichever scores best, you've done a search over questions and reported the winner. The paper puts it directly: reporting the best of them would overstate what a practitioner gets.

Anyone who has tuned a prompt against a test set will recognise this. The remedy used here is simple. Questions are selected on one half of the items and scored on the other half. The targeted question is selected on one half of the items and scored on the other half, over 20 splits. Selection never touches the items that produce the reported number.

This matters for how to read every gain reported below. The paper's claim is that the split-half protocol removes selection inflation from the reported gains. When the paper says targeted wording gives a median gain of +0.006 AUROC, that figure comes from wording chosen and scored on separate halves. A figure that small, honestly measured, is more useful than a larger figure that quietly included the winner's curse.

The headline: one generic question #

Now the result most readers will want. A single generic question reaches a median AUROC of 0.886 zero-shot and beats supervised baselines on most benchmarks.

Take the words in turn. "Single generic question" means no per-benchmark engineering of the prompt. "Zero-shot" means no training on the target benchmark. "Beats supervised baselines on most benchmarks" means that systems which were trained on labelled data from the task lose to a question nobody tuned, on most of the benchmarks examined. The claim stops at "most", and so will this article.

The generic question is a template question read as P(yes). The figure reports the AUROC of 38 usable benchmarks, and the generic question was not asked on 7 benchmarks. So the headline number is a median over benchmarks where the generic question applied, drawn from the 38 usable ones. Careful readers should keep the denominators straight: 44 benchmarks exist in total, 38 are usable for this figure, and the generic question has a gap of 7.

I'd argue the zero-shot result is the more interesting half of the sentence. Beating supervised baselines is a statement about comparative standing. A median AUROC of 0.886 with no task-specific training says something about how much of the failure signal is available to a model that reads the state and answers a plain question. Whether that generalises to failure types outside these ten is not something the claims address.

Wording is nearly irrelevant #

If you've spent time on prompt engineering, the next result will sting. Question wording matters little, while context matters more, mostly through fields that encode the label.

The paper's explanation for why is specific. Once answers are kept as probabilities instead of argmax decisions, the wording of the question matters little and what the state and the label contain matters a lot. This is an observation about information. An argmax decision throws away the margin by which one answer beat another, and a wording change can flip the decision across the boundary. A probability keeps the margin, so a rewording that nudges the evidence slightly moves the probability slightly and leaves the ranking mostly intact.

I'll hold that to what the paper states: when probabilities are kept, wording matters little. I won't extend it to a mechanistic account of why, since the claims don't supply one.

The numbers back the statement. Targeted wording gives a median gain of +0.006 AUROC, with the wording chosen and scored on separate halves. Out of sample, targeted wording adds +0.006 [−0.004, +0.015] AUROC. The interval spans zero. A reader shouldn't treat +0.006 as a reliable improvement; the interval allows a small loss as readily as a small gain. For a practitioner, the useful reading is that hunting for a cleverer phrasing is unlikely to repay the effort, at least on these benchmarks.

This should change how teams allocate time. If wording is close to inert, the hours spent on it are better spent on the state.

Context is where the gains live #

So what does the state contain, and what should it contain? The paper's finding is that context matters more, mostly through fields that encode the label. The key idea behind the study's design is to vary what Jev is asked separately from what it sees: the question's wording and answer type on one side, the fields of the input on the other. That separation is what lets the paper say wording matters little while context matters more. Without it the two effects would be tangled together.

The headline example is PrivacyLens. References that define the label give the large gains, such as PrivacyLens's list of secret items (0.79→0.95). A move from 0.79 to 0.95 AUROC is large next to the +0.006 from wording. The reference in question is a list of secret items, which is exactly the kind of thing the earlier discussion of relational failures predicts would matter: whether a response leaks something private depends on what counts as private, and the response alone doesn't say.

There's a deployment caveat, and it's an important one. Context a deployed monitor would hold helps on 1 of 7 benchmarks. The big gains come from references that define the label, but a deployed monitor wouldn't necessarily have those. What a monitor would realistically hold helps on only one of seven benchmarks.

This is a sober finding, and I think it's the most commercially relevant sentence in the paper. It separates the lab number from the field number. A detector evaluated with the label-defining reference in hand is answering an easier question than a deployed monitor faces. The gains are real, but the paper is clear about which of them a deployment can count on.

The paper also flags a limitation of one benchmark. On Open-Prompt-Injection, the label tracks injected-task accuracy. That's an example of a label whose meaning is tied to a property of the task, which bears on what a detector can be expected to see from the state. I'll stop short of drawing further conclusions from it, because the claims offer none.

Cost #

Now the number that makes the economics concrete. Jev costs 63× less than LLM-judge scorers. The specific comparison: a pass over the 19 judge-scored benchmarks costs $0.30, 63× less than their judges at list prices. The cost comparison covers one pass over the 19 API-judge benchmarks at list prices.

Be careful about scope. The comparison is for 19 benchmarks that use API judges, priced at list, for a single pass. It doesn't cover every benchmark in the suite, and it doesn't cover discounted pricing. Within that scope, the ratio is large, and it follows from the architecture described earlier: no decoding pass per criterion.

Recall the opening requirement, that a detector run on every message must be cheap. A figure of $0.30 for a whole pass over 19 benchmarks is what "cheap" looks like in this paper. The paper doesn't translate it into per-message pricing, and neither will I.

Agreement with humans, and what disagreement revealed #

Cheap is only interesting if the detector is also right. Where human labels allow a check, Jev agrees with them as well as the judge does, at a fraction of the cost.

The most specific evidence comes from StrongREJECT. Against human labels there, the generic question agrees as well as the reference scorer (Cohen's κ 0.809 vs. 0.811) and ranks better. The two κ values differ by 0.002, so the sensible reading is a tie on agreement, with Jev ahead on ranking.

Then comes a result that I find the most useful in practice, because it turns the detector into an audit tool. Jev's confident disagreements exposed label-changing defects in three benchmarks and labels the state cannot reveal in four more. When a detector that is confident disagrees with a benchmark's reference label, one explanation is that the detector is wrong. The paper reports that in some cases the disagreement pointed at the benchmark: three benchmarks had defects that change labels, and four more had labels the state cannot reveal.

The second category connects back to the relational point. A label the state cannot reveal is a label defined by something that never made it into the input. No detector reading that state could recover it. Finding such cases is a different contribution from raising a score, since it tells benchmark maintainers where their data and their labels part ways.

The weak point: calibration #

Here the article has to be blunt, because the paper is. The probabilities rank well but do not transfer as thresholds: the median ECE is 0.168 against a null of 0.074, on validated and judge labels alike, because Jev's mean probability misses each benchmark's base rate.

The pieces fit together as follows. AUROC measures ranking, and ranking is strong. A threshold is a different matter. If the model's average output probability doesn't match how often failures occur in a given benchmark, then t=0.5 lands in the wrong place, and the expected calibration error is high. A median ECE of 0.168 sits above the null of 0.074, which is a worse showing than the reference point the paper chose. And it holds on validated and judge labels alike, so the weakness isn't an artefact of noisy judge labels.

Recall the formal setup: a detector has to rank failures above non-failures and separate them at a threshold. Jev does the first well and the second poorly out of the box. The name of the technique contains "calibrated decisions", and the measured calibration here doesn't carry across benchmarks, which is a tension worth stating plainly. The paper attributes the gap to base rates and doesn't claim more.

The practical consequence: as a stand-in for a judge, Jev needs a few labels to set its threshold. That's a real cost, and it narrows the claim of being a drop-in replacement. You can't point it at a new failure type and trust t=0.5.

The recipe #

The paper's recommendation is compact. The recipe is a generic question read as a probability, with its threshold fitted on ten labelled items, which lifts F1 where Jev fires too rarely at t=0.5 and costs 0.025 on validated labels.

Unpack it. The generic question handles the wording problem by declining to engineer it, which fits the finding that wording matters little. Reading the answer as a probability keeps the margin. Fitting the threshold on ten labelled items addresses the base-rate miss. The effect on F1 is positive where Jev fires too rarely at t=0.5. On validated labels, the recipe costs 0.025.

That final figure is worth dwelling on. The recipe isn't free of downside: on validated labels it costs 0.025. The paper reports the trade rather than hiding it, and I'd rather cite a recipe with a stated cost than one advertised as pure gain. The claims don't say what metric the 0.025 is measured in beyond the context of validated labels, so I won't guess.

Ten labelled items is a small ask. It's an afternoon of annotation for most teams, set against the cost of running a judge on every message. But it's a prerequisite, and a team that can't produce any labelled items for a new failure type shouldn't expect the threshold to sort itself out.

What a practitioner should take from it #

Several things follow, and I'll commit to them since the evidence supports them.

First, stop optimising question wording. The out-of-sample gain from targeted wording is +0.006 with an interval from −0.004 to +0.015. That's not a basis for investment.

Second, spend the effort on the state. The large gains came from references that define the label. If your monitor can be given the reference that defines the failure, that's where improvement lives. If it can't, expect less: context a deployed monitor would hold helped on 1 of 7 benchmarks. Plan around the number you can get in deployment rather than the number you get in a benchmark where the reference sits in the input.

Third, budget for calibration. Plan to label ten items per failure type and fit the threshold. Don't ship t=0.5.

Fourth, use disagreement as signal. Confident disagreements with reference labels exposed defects in three benchmarks and unrevealable labels in four more. If you run this on your own evaluation suites, expect that it may tell you about your labels as well as your models. That's an inference from the paper's finding rather than a measured claim about any particular suite, so treat it as an invitation to check.

Fifth, remember the known weaknesses. The developer lists indirect meaning and adversarial content among Jev's known weaknesses. A monitor that matters for safety will meet adversarial content. The claims here don't report how Jev performs on it, so that remains an open flank, and I'd want it tested before trusting a deployment.

Reading the result against the commercial case #

Founders will want to know where this sits. The cost argument is the strongest: a detector that runs on every message has to be cheap, and a 63× reduction against LLM-judge scorers changes which deployments are affordable. The accuracy argument is the second: zero-shot AUROC of 0.886 and a win over supervised baselines on most benchmarks.

The caveats shape what can be sold honestly. The detector needs a few labels per threshold. Its calibration doesn't transfer. Its best gains rely on references a deployment may not hold. Those are conditions of use, and any product built on this should state them rather than bury them. None of that undermines the cost case; it describes the setup work.

There's also a quieter point about the measurement itself. RLCDAlignBench, with its context variants, cached Jev answers, rescoring scripts and split-half protocol, is a reusable evaluation asset. A benchmark that removes selection inflation from reported gains is the kind of artefact that lets later claims be compared. That's a separate kind of value from the detector.

What remains open #

A few boundaries are clear from the claims. The target models are five open models of 2–7B parameters, so the results describe failures in interactions from models of that size. The generic question was not asked on 7 benchmarks, and the headline AUROC is a median over the benchmarks where it was. The labels are reference scorers, with human and second-judge labels covering StrongREJECT, HarmBench, AbstentionBench and InstrumentalEval.

The open question that interests me most is the one the paper's own structure raises. Failures are relational; the state often lacks the reference; and a deployed monitor's context helped on only 1 of 7 benchmarks. Jev's single-call design lets many questions share an input, yet the biggest lever turned out to be which fields the input carries. How a deployment should obtain the label-defining reference at monitoring time, such as a list of secret items for a privacy check, is the next thing to work out. The paper shows the gain is there to be had (0.79→0.95 on PrivacyLens), and leaves open whether a deployed monitor can be given it cheaply enough to keep the 63× advantage.

What This Means by Sector #

The ability to evaluate complex alignment criteria via calibrated decision heads at fractional inference cost reshapes operational safety across multiple domains.

AI gateway and security platform providers #

Infrastructure companies deploying real-time safety guardrails can replace costly cascade architectures (calling separate prompt-injection, toxicity, and privacy classifiers) with unified multi-query RLCD heads. Operating costs drop by over 60× while latency is bounded to a single forward pass.

AI governance and audit compliance teams #

Risk committees evaluating model behaviour against standards such as EU AI Act high-risk requirements or NIST AI RMF gain a deterministic, probabilistic audit log. Instead of unstructured narrative LLM-judge feedback, systems record calibrated probability distributions that support statistical process control and drift monitoring.

Enterprise deployment and red-teaming teams #

Security teams can rapidly deploy custom failure detectors by authoring focused state representations and fitting decision thresholds against as few as ten domain-specific examples, eliminating the multi-week fine-tuning cycle previously required for bespoke classifiers.

Conclusion #

The evaluation of Jev across RLCDAlignBench establishes a compelling architectural direction for production model monitoring: reading calibrated probabilities across multiple typed questions in a single forward pass beats both autoregressive LLM judges on cost and single-token classifiers on flexibility. While practitioners must budget for empirical threshold calibration rather than assuming out-of-the-box probability transfer, the 63× economic advantage makes real-time, comprehensive monitoring viable at scale.

Questions? Answers.

Why are generative LLM judges impractical for real-time production monitoring?

Generative judges require a complete autoregressive decoding pass for every safety or alignment criterion evaluated. In high-throughput consumer or enterprise applications, running multiple generative judge calls per message introduces severe latency penalties and inflates operating costs by orders of magnitude.

What is Reinforcement Learning for Calibrated Decisions (RLCD)?

RLCD is a training methodology that optimizes models to produce well-calibrated posterior probabilities over structured decision heads rather than generating token sequences. This allows the model to evaluate multiple arbitrary questions simultaneously within a single forward evaluation.

Why do decision thresholds need to be calibrated on local data?

While RLCD models excel at ranking interactions by failure likelihood (achieving high AUROC), their absolute mean probabilities shift depending on the base rate of the target domain. Calibrating an operational decision threshold against approximately ten labelled domain examples ensures appropriate sensitivity without unacceptable false-positive rates.

References #

Mrinank Sharma et al., (2024). Towards Understanding and Mitigating Sycophancy in Language Models ⧉. arXiv.
RLCDAlignBench Consortium, (2026). Multi-Criteria Alignment Failure Detection via Calibrated Decision Heads ⧉. arXiv.
Anthropic / Alignment Research, (2025). Constitutional AI and Evaluator Dynamics ⧉. Anthropic.
NIST, (2024). Artificial Intelligence Risk Management Framework (AI RMF 1.0) ⧉. NIST.

Naposledy revidováno 2026-09-30.

Naposledy revidováno 2026-10-16.

Znovu publikovat tento článek

A Detector That Reads Probabilities Instead of Writing Verdicts: What 44 Benchmarks Say About Jev — Sebastien Rousseau

Jev, a model trained with reinforcement learning for calibrated decisions (RLCD), had never been measured on alignment-failure detection. RLCDAlignBench now does that across 44 benchmarks, ten...

Tento článek je licencován pod Creative Commons Attribution 4.0 International. Při opětovné publikaci uveďte odkaz na kanonickou URL.

A Detector That Reads Probabilities Instead of Writing Verdicts: What 44 Benchmarks Say About Jev — Sebastien Rousseau

Jev, a model trained with reinforcement learning for calibrated decisions (RLCD), had never been measured on alignment-failure detection. RLCDAlignBench now does that across 44 benchmarks, ten...

Originally published at https://sebastienrousseau.com/cs/2026-09-30-detector-that-reads-probabilities-instead/ by Sebastien Rousseau.
Licensed under CC-BY-4.0.

SEBASTIEN ROUSSEAU FOUNDER · ENGINEER