Nothing is generated
The answer is one of the options you declared, scored in one pass.
| Needle | LLM call | |
|---|---|---|
| output | 0 tokens | every token billed |
| sampling | none | varies run to run |
| parsing | none | JSON, may retry |
Needle · private preview
Stop asking an LLM to pick an option
A decision model behind one API call. Typed questions in, probabilities out.
LLM call
{"action": "refund"} · parsed
about 2,400 msNeedle41 ms
refund 0.97
deny 0.02
escalate 0.01
- median latency, measured
- 41 ms
- output tokens, ever
- 0
- accuracy on requests neither model had seen, vs 85.8%
- 97.0%
- answers changed when the options are shuffled, vs 8.3%
- 0%
Why a forward pass
What changes when nothing is generated
If a step in your pipeline asks a language model to choose, rate or approve something, it is paying for generation to get a label. Two things change when the label comes from a forward pass instead.
You decide how sure is sure enough
Every answer carries its probability. Drag the bar to set where automation stops and a person starts.
Threshold at 0.80
- Charged twice for order 4417 refund 0.97 → auto → human
- Site down for our whole team escalate 0.94 → auto → human
- Wrong size, want an exchange exchange 0.88 → auto → human
- Promo code did not apply refund 0.76 → auto → human
- Cancel and refund, I was told I could refund 0.54 → auto → human
- Is the blue one back in stock? answer 0.66 → auto → human
3 handled automatically 3 to a person
Where it fits
Three requests, every answer
Customer was charged twice for order 4417 and asks for the duplicate to be refunded.
action · choice refund 0.97
urgent · claim 0.22
severity · score 0.61 of 2
User: How did last quarter's revenue compare to forecast? Tools: sql, web_search, calculator.
tool · choice sql 0.91
clarify · claim 0.12
risk · score 0.07 of 2
Forum post: Selling 2 concert tickets, DM me. Payment by gift card only, no refunds!!
action · choice flag 0.63
scam · claim 0.86
toxicity · score 0.05 of 2
How it answers
Three question types, one pass
Every question declares its own answer space, so the model can only return an option you defined.
The primitives in the docsThree primitives
Choice "choice"
refund 0.97
deny 0.03
→ refund confidence 0.94
Score "score"
lowmediumhigh
→ 0.61 confidence 0.22
Claim "claim"
0 false true 1
→ 0.22 P(true)
Switching
Already on System One? Change one line
Needle accepts System One requests. noul works as an alias for claim, and every answer comes back under the name its question used.
- base_url = "https://<your System One endpoint>" + base_url = "https://api.gaugenumerics.com/v1"
Measured in the open
Every result published, including the losses
The same requests sent to Needle and to a commercial decision API ("reference" in the charts), checked against known-correct answers after each round of training (the letters E to L).
Progress by training stage
Needle, stage L +11.8 pts vs reference
Needle, stage K +9.9 pts vs reference
Needle, stage L -24.5 pts vs reference
Needle, stage L -3.9 pts vs reference
Join the preview
$0.20 per million input tokens, the first 10M each month free. Output is always free, because there is none.