Evals for AI agents

Stop losing customersto poor agent quality.

Vibe-coded monitoring drowns you in alerts you can’t trust.
TwoTail gives you calibrated LLM judges and proves them against your business outcomes, so you spot problems before your customers do.

Book a demo
Social media agent

On-brand posts

96%
pass rate
Legal agent

Citations are real

99.1%
pass rate
Recruiting agent

Candidate fit

94%
pass rate
Outbound sales agent

Personalised opener

84%
pass rate · down 6 pts
Car dealership agent

Test drive booked

23%
of chats
The problem

Your agent makes mistakes. Your customers notice first.

Support agent

A customer asks the same question three times, then gives up.

Recruiting agent

It scores a strong candidate 4/10, and only a recruiter would spot it.

Lead-gen agent

It books meetings, but not the ones your customer wanted.

Research agent

You change the prompt, the model or a tool. One thing gets better, and two quietly get worse.

How it works

Your vibe-coded monitoring doesn't cut it.

TwoTail is the agent quality system your growing agent startup deserves. It's built on evals: automated checks that grade every trace.

Judged

Monitor quality in production

Hear about issues as they start, not when customers complain.

Every trace scored as it comes in. A Slack alert when a score moves.

Vibe-coded monitoring

  • A prompt someone wrote once, never checked against your experts
  • Needs rewriting every time the agent changes
  • Flags non-issues and misses the failures nobody wrote down

With TwoTail

  • Judges calibrated to your experts, not a prompt someone wrote once
  • Kept up to date as your agent changes
  • Always searching your traces for new use cases and failure modes

# agent-quality

TwoTailAPP9:41

repeat_question is up on support agent: 11% of traces today, from 4% last week. Mostly billing_dispute.

11% of traces
last 7 days

View 14 tracesMute

Calibrated

Align judges to your experts

Scores your expert agrees with, and a number for how often.

Your expert labels the few traces the judge is least sure about. The judge learns from them.

Vibe-coded monitoring

  • Your expert scrolls hundreds of traces, or nobody looks at all
  • An observability tool built for engineers, not for labelling
  • No agreement rate, so you never find out whether the judge is right
  • Labelling takes a week, so it never happens

With TwoTail

  • We pick the few traces worth your expert's time
  • A viewer built for labelling, not an observability tool built for engineers
  • Agreement tracked, so you know when a score can be trusted
  • Labelling takes an hour, not a week

candidate_score · 10 of 14 traces labelled

Expert✓✕✓✓✕✓✓✕✓✓
Judge✓✕✓✓✓✓✓✕✓✓
91%agreement with your expert
up from 64% before calibration
Proven

Measure what moves the business

Know which issues hurt your outcomes, and which to ignore.

Eval scores tested against retention, conversion or your customers' results, ranked by impact.

Vibe-coded monitoring

  • Flags the visible problems, which are often the wrong ones
  • A frustrated power user gets flagged; a quiet user who churns goes unnoticed
  • You fix what is loudest, not what costs you

With TwoTail

  • A map from quality to outcomes, tested statistically
  • Hypotheses ranked by impact, with what to do about each one
  • The first thing you fix is the thing that moves the business

shopping assistant · northstar: revenue

recommended_in_stockasked_about_budgetmessage_toneorders completedreturn visitsnorthstarrevenue0.580.210.04
Tested

Test before you ship

Know whether a change helps before it ships.

Test sets built from production traces, graded by the same judges.

Vibe-coded monitoring

  • Generic test sets built from capabilities, not from what customers do
  • Graded by checks nobody has calibrated
  • Drifts from production, so the tests pass and production still breaks

With TwoTail

  • Test sets drawn from production and selected by impact
  • Graded by the same judges you trust in production
  • Updates as production changes, so a pass means something

model A vs model B

citation_accuracy48 / 48
tool_loops3 new fails
answer_length+31%
Timothy, founder of TwoTail
Meet the founder

Your Forward Deployed Analyst

I’ve worked for some of Europe’s highest growth scaleups, and founded 3 AI startups. When you use TwoTail, you get me too. A powerful tool combined with an experienced operator figuring out the perfect way to measure and optimize your agent.

Read the full story
What customers say
We used to rely on a Claude routine to find issues with our agent, but it was unreliable and I started ignoring it. Now we have evals to measure when agent quality drops - and we are confident that when it pings us on Slack it’s not a false positive.
Harold
Founder, Sliq
Tim is a deep thinker and brilliant problem solver, I would highly recommend him to anyone trying to solve a multi-faceted issue that may seem daunting, he has the craft to sort it out!
Xavier
Founder, DarkMatter
Timothy is an expert in data modeling. With his help, we now have a clear understanding of how users behave. We highly recommend working with him.
Randy
Founder, Recharge360
Because they have so much experience in the field, they proactively find the edge cases and implement systems so those are handled correctly.
Marlon
Founder, Assembly
Integration

Ten minutes, and no code changes.

TwoTail reads the traces you are already producing. Point OpenTelemetry at it, or turn on the Langfuse integration and leave everything where it is.

Direct, via OpenTelemetryStandard OTLP/JSON from any framework, straight to the TwoTail API. Or paste the setup prompt into your coding agent.
Through LangfuseAlready tracing with Langfuse? Turn the integration on and TwoTail reads the same feed. Nothing moves.
prompt
Add TwoTail tracing to this agent.
TwoTail accepts standard OpenTelemetry
spans, no custom conventions required.

## Configuration
endpoint  https://www.twotail.ai/api/v1/traces
header    X-API-Key: tt_live_…
Connected · traces arriving
Works with your stack OpenTelemetry Langfuse LangChain OpenAI Agents SDK Claude Agent SDK Vercel AI SDK

Common questions.

You can, and you should. Claude Code reads traces, proposes failure taxonomies, and writes eval criteria well. What it cannot do is tell you whether its judgement is right: no ground truth from you, no agreement rate, a fresh taxonomy on every rerun, and a sample rather than your whole corpus. TwoTail is what it calls to get those. Your agent does the analysis; TwoTail makes it defensible.

Both, on the same judges. In production, TwoTail scores every session, not a sample, so you see what your agent is doing to customers today. Before a release, you run those judges over test sets built from real sessions. Most teams start with production, because that is where the surprises are.

Langfuse gives you somewhere to put scores. It has no opinion about which scores you should have, or whether they mean anything. TwoTail adds the two parts it does not have: labelling queues that pick the few sessions your expert needs to see, and a map from your scores to the business outcomes they move. Most of our customers run TwoTail on top of Langfuse: same OTel feed, different job.

Not out of the box. In specialised fields, frontier judges agree with you as little as 64% of the time. Calibrated against a few hundred well-chosen labels, they hold up, and the agreement rate is visible before you trust a single score.

Your data is stored in isolated Postgres databases with row-level security. Each account's data is fully segregated, and all connections are encrypted in transit.

Plans are priced for growth-stage teams, with a free trial once you're set up. It starts with a quality kickoff. Full details on the pricing page.

How we start

Start with a quality kickoff.

30 minutes on how to measure quality for your agent. You leave with a plan: what to measure, how to trace it, and which evals to set up. Yours to keep, whatever you decide.

If it's a fit, we set it up for you within two weeks.

Book a demo
Quality plansupport agent
1
What to measure

repeat_question handover_missed refund_policy

2
How to trace it

One span per conversation turn, tool calls as child spans. Sent over OpenTelemetry, or read from your Langfuse feed.

3
Which evals to set up

One LLM judge per measure, calibrated on traces your team labels.

Yours to keep, whatever you decide.