Features

The complete eval analytics platform.

Everything below is organised by the step of the loop it serves: define, annotate, calibrate, correlate, then act on what you find.

Define

Get from nothing to a written, scoreable standard.

Eval Builder

Start from the evals you already have, or write new ones by labelling the traces they will grade. Every eval is built on a scoring rubric: what good looks like, written down. Severity runs on anchored scales with behavioural anchors, so two people reading the rubric score the same trace the same way, and the headline number is binary at the threshold you choose.

evals · support agent

resolutionimported

refund_handlingimported

tonebuilt

promise_keptproposed

Discovered Evals

Analyzes your outcomes for failure patterns nobody thought to write down. Statistical discovery alongside you, not instead of you. Candidates arrive with the evidence attached: where the pattern shows up, how often, and what it moves.

new eval candidate · found in 212 traces

pattern"agent promises follow-up, none scheduled"

evidence4.1% of traces · 3.2x more common in churned accounts

actionpropose as eval · send 20 examples to annotation

Annotate

Your judgement, captured at the lowest cost to you.

Intelligent Annotation

A Bayesian model picks the traces to annotate that'll best use your time. Queues are sized for completion rather than coverage, mix random samples with the cases where a label changes the most, and track the provenance of every label so the estimates built on them stay honest.

annotation queue · next up 14 traces, est. 25 min · disagreement queue 3

Failure Taxonomy

As you label, failures are named and grouped into a taxonomy of failure modes, sized by how often they happen and what they cost you. The taxonomy becomes the shared language for everything downstream: segment views, recommendations, and the conversation about what to ship next.

failure modes · escalations, last 30 days

refund_not_supported42%

promise_no_followup27%

policy_hallucinated18%

other13%

Calibrate

Judges you can defend, on every trace.

Calibrated Judges

Every judge ships with its agreement rate against your labels. New labels arrive, the judge recalibrates. Updates are backtested against everything you have labelled, so it learns the pattern, not the one example. Heavier traffic runs as a waterfall: a fast model screens everything, a stronger model adjudicates what it flags.

refund_judge · 600 labelled traces

judge · pass

judge · fail

you · pass

412

you · fail

134

91% agreement with your labels. 54 disagreements go back to the queue.

Transparent Verdicts

Every verdict carries a short reason and an evidence quote from the trace, so a score is something you can check, not something you have to trust. The rubric and the full judge prompt are always visible, and judge changes are versioned: a step in the chart is a labelled event, not a mystery.

frustration · severity 3 · judge v4

reasonuser restates the same request after a failed attempt

evidence"I already told you the booking number twice"

configrubric · full prompt · version history

Correlate

Proof that the number matters.

Evals That Matter

Tests every eval against the business metrics you care about, so you have a clear view of which evals move them and by how much.

eval score vs. 90-day retention

resolution_judger = 0.71

response_lengthr = 0.04

Honest Rates

The judge's raw series stays frozen: a number that moves when someone reviews it is not a measurement. Your labels produce a corrected estimate with an interval next to the headline, so the point you quote survives contact with a spot check.

frustration rate · last 30 days

judge says39.7%

estimate29.4% (22.5 to 36.3)

basiscorrected on your randomly sampled labels

Act

From a moving score to a shipped change.

Segmentation

Which intents, users, and flows fail which evals. Scores you can act on, instead of an average that hides the problem inside it. Cut by your own dimensions: task type, account, channel, or anything your traces carry.

resolution by intent · overall 82%

password_reset94%

order_status88%

subscription79%

billing_dispute54%

Recommendations

A diagnosis and a proposed change, grounded in your own failure taxonomy. Not a chart for you to interpret on a Friday afternoon. The worst cases surface immediately, with the conversation attached, so the experiences that deserve a same-day reaction get one.

top failure cluster this week

finding12% of escalations are refund requests the agent cannot handle

changeadd a refund-lookup skill

impactest. +6pts resolution on billing_dispute

Under it all

The substrate everything above runs on.

Built for Coding Agents

Your coding agent connects over MCP and retrieves the cases that matter: failing traces, disagreements, whole failure clusters. Ready to analyze, or to feed whatever automation you run next.

mcp · twotail

get_traces(eval=refund_judge, result=fail, last_24h) get_failures(mode=refund_not_supported) get_disagreements(judge=refund_judge) get_correlations(outcome=retention)

Easy Integration

Send OpenTelemetry traces from any framework, or turn on the Langfuse integration and keep everything where it is. No SDK, no code changes.

two ways in

directOTLP/JSON to the TwoTail API · POST /api/v1/traces

langfuseturn the integration on · same feed, different job

See it on your traces.

Setup in 10 minutes. First calibrated judge within a week.

Book a demo Take the eval maturity test