demo The gate failed, so this build went red. Two rows regressed — open either one below to see what the judge said about the answer.
Run 7CN4RVCP
completed
ci
20 Aug 2026, 04:02
83.1%
▼ 12.0 vs baseline
Gate failed.
- score 0.8310 is below the minimum 0.9000
- 2 rows regressed against the baseline, the most allowed is 0
2 of 8 rows below full pass. Pass rate 75.0%.
8 rows × 24 samples
cost $0.0071
judge $0.0029 4m 0s
a50e928 · main · Switch the support agent to Haiku
github
| Input | Samples | Score | vs baseline | Cost inc. judge |
|---|---|---|---|---|
|
Can I get a refund without a receipt?
anthropic/claude-haiku-4-5
|
0/3 | 42.0% | — | $0.0009 |
|
Do you price match other retailers?
anthropic/claude-haiku-4-5
|
1/3 | 55.0% | — | $0.0009 |
|
Is there a warranty on clearance items?
anthropic/claude-haiku-4-5
|
3/3 | 91.0% | — | $0.0009 |
|
Can I change the delivery address after ordering?
anthropic/claude-haiku-4-5
|
3/3 | 93.0% | — | $0.0009 |
|
Do you ship to Australia?
anthropic/claude-haiku-4-5
|
3/3 | 95.0% | — | $0.0009 |
|
Do you offer student discount?
anthropic/claude-haiku-4-5
|
3/3 | 95.0% | — | $0.0009 |
|
What time do you close on a Tuesday?
anthropic/claude-haiku-4-5
|
3/3 | 97.0% | — | $0.0009 |
|
I bought a lamp last week, can I still return it?
anthropic/claude-haiku-4-5
|
3/3 | 99.0% | — | $0.0009 |