Confidence Gate

A 22M parameter reranker answers first. Drag the threshold, and the questions it is not sure enough about go to Laya instead. Every number below is a recorded run, not a mock.

Customer message

Gate threshold

0.60

An answer passes when its confidence reaches the threshold. That is the model's own confidence when it reports one, which Laya does and the reranker does not; otherwise the top probability of a Choice or max(p, 1 - p) of a yes/no. Score is never gated.

Route

Answers

The same thing in code

What this demo is not hiding

Tier 1 is weak, not only cheap. On 150 rows of ag_news, a four-way choice, the reranker scores 0.393 where guessing scores 0.25, and this Laya checkpoint scores 0.953. That gap is the whole reason for the gate, and it is measured in system-one-bench.
Escalating is not the same as being right. On the 500s message both tiers answer billing, which is wrong. The gate routes on confidence, and a model can be confidently wrong.
Yes/no answers move with the names of the two options. This demo leaves them at the default true and false. Naming them lifts this checkpoint by 5.4 points on 150 sst2 rows, and there is a wording that collapses the answer to a constant 0.000 at confidence 1.0 until they are named (laya#1244).
Nothing here is calibrated. The probabilities are whatever the backend returned. Reranker scores are normalized, not calibrated, and this checkpoint ships temperatures that pydecide clamps, which it warns about at load.
Score answers are never gated, because an expected value over levels has no single confidence number behind it.
These are recorded runs, because Gradio Spaces need a paid plan and static Spaces do not. app.py in this repo is the live version: clone it, pip install -r requirements.txt, and it runs both tiers locally.