Beyond "Not Escalated": How Not to Measure AI Support Success

Containment rate: the metric that says (almost) nothing

If you ask any AI support vendor how they measure success, chances are they'll tell you it's containment rate: the percentage of tickets the AI closed out without ever pulling in a human agent.

It's a seductive metric. It's simple, it slots neatly into a dashboard, and it climbs over time almost without anyone trying too hard.

It's also nearly meaningless.

A "not escalated" ticket isn't a promise of anything. It might mean the AI nailed the answer. Or it might mean the customer got tired, closed the chat window, and went looking for the answer somewhere else. Maybe with a competitor.

Quiet doesn't mean happy

Here's the real cost of using non-escalation as your only signal: it confuses the absence of complaints with the presence of satisfaction.

Think about it from the customer's side. When an AI doesn't solve the problem, most people won't push back or ask for a human. They just leave. That quiet drop-off, what's known in customer experience as silent churn, leaves no trace in a system that only counts escalations.

You end up with a dashboard that reads "98% resolved" while, in parallel reviews slip, retention drops and the support team feels blind about it.

A customer walks away, and the data says it was a win.

The real pain points of measuring it this way

When resolution is measured only by escalation (yes/no), here's what actually happens:

• Unhappy customers have no voice. There's no mechanism to capture their frustration if they never write back.
• The team stays blind to the AI's mistakes. If nobody escalates, nobody reviews the conversation, and the same errors repeat, chat after chat, with no human ever catching them.
• Dashboards create false confidence. Product and leadership teams make calls (on investment, roadmap, staffing) based on a number that doesn't reflect what's really happening.
• There's no feedback loop. Without distinguishing "actually resolved" from "the customer gave up," there's no way to know which AI responses need training, fixing, or pulling altogether.
• Trust erodes before anyone notices. By the time the problem shows up in NPS or in cancellations, it's already a business problem, not just a product one.

Getting a clearer read on AI performance with Zentesimal

Resolution isn't a binary state. It's a spectrum, and treating it that way is what lets a team act before losing the customer, not after.

There are different ways to understand this and track it properly. At Zentesimal we measure four states, not two:

• Confirmed satisfaction. The customer explicitly said, within the conversation, that their problem was resolved.
• Inferred satisfaction. The AI, reading tone, context, and how the conversation wrapped up, estimates the customer was satisfied even without an explicit confirmation.
• Uncertain. The conversation ends with no clear signal either way, no satisfaction, no frustration. On its own, this category is already more honest than any "not escalated."
• Escalated. The case goes to a human, with the full conversation context attached.

This level of detail changes what a team can actually do. It's no longer about chasing one number that goes up or down; it's about seeing, precisely, where the AI is failing, where it's genuinely working, and where you simply don't know yet. And that visibility is what lets you improve it.

Across Zentesimal accounts, the uncertain bucket sits at about 15% of conversations on average, against an average 78% resolution rate, while user-confirmed satisfaction typically runs 15–20%. That's exactly the kind of blind spot a binary "escalated / not escalated" metric would have missed entirely.

Instead of a dashboard that only says "resolved or not," teams using Zentesimal can see exactly how many customers left without saying a word, which conversations need a second look, and where the AI is building real trust instead of just silence. That visibility is what lets you fix the AI before an unhappy customer turns into a silent cancellation.

See how Zentesimal measures real customer satisfaction