In traditional call centers, quality assurance is done by sampling: a reviewer listens to a small share of calls, scores them against a form, gives feedback. That method was reasonable for human agents because there was no alternative.
Working with a voice agent removes that constraint. If a transcript is produced for every call, every call can be scored. Instead of a one percent sample, one hundred percent review is possible — and that changes how quality is managed.
Why sampling is not enough
Errors in agent conversations are not randomly distributed. They cluster around a particular question type, a particular knowledge base gap, or a particular hour. A small sample misses that clustering.
Also, the agent's behavior is consistent: a mistake it makes in one call it makes in every similar call. That sounds like bad news but is actually good news — fix one error and thousands of calls improve at once. But to find it you have to look at all of them.
What to score
Scoring criteria vary by business, but five headings apply in almost every deployment.
1. Accuracy. Was the information the agent gave correct? This is the most critical criterion and the hardest to automate. The practical approach: automatically compare the prices, dates, durations and conditions the agent stated against their counterparts in the knowledge base. Every call that does not match goes to human review.
2. Resolution. Did the customer's business get done? Information given without action taken does not count as resolution. The most reliable way to measure it is to check whether the post-call action occurred: was an appointment created, was the order updated, was a ticket opened?
3. Handoff behavior. Did it hand off where it should, and did it hand off unnecessarily? Both error rates should be tracked separately.
4. Conversation quality. Did it interrupt, ramble, or repeat itself? Measurable from the transcript: the agent's word count, turn count, and repetition of the same sentence.
5. Compliance. Was the recording notice given, did it state its identity, did it collect data it should not? These should be pass-or-fail, not scored.
How to build the workflow
A working review workflow has three layers.
Layer 1 — automated rule checks. Every call passes through a rule set: is the recording notice present, did a forbidden phrase appear, was a duration threshold exceeded, did a handoff rule fail to fire when it should have? This layer is cheap and applies to every call.
Layer 2 — automated evaluation. The transcript is scored against your defined criteria. The thing to watch here: the evaluating system can be wrong too. So treat the scores not as absolute truth but as a ranking tool — route low-scoring calls to human review.
Layer 3 — human review. A set number of calls per week is listened to by a person. The selection should not be random: prioritize low scores, calls that ended in handoff, long calls, and calls with signals of customer dissatisfaction.
Do not remove the third layer entirely. Automated review cannot see problems outside its criteria; a human ear discovers new problem types.
Connecting findings to fixes
The value of review is measured by how fast a finding becomes a fix. The loop that works:
- A weekly findings list is produced.
- Each finding is assigned to one of three categories: knowledge base gap, prompt problem, missing rule.
- The fix is made according to the category.
- The following week, the rate of that finding type is checked.
Step two is where this loop most often breaks. The tendency to solve every problem by adding a line to the prompt eventually produces a bloated, self-contradicting instruction set. Do not try to close a knowledge base gap with a prompt.
The top-level numbers to track
The numbers reported to management should be kept simple:
- Resolution rate. What share of calls finished the job?
- Handoff rate and late handoff rate.
- Accuracy violations. Calls contradicting the knowledge base. The target is zero.
- Compliance violations. Missing recording notice or identity statement. The target is zero.
- Average call duration and turn count. Tracked as a trend.
Those five are enough to show the health of a voice agent operation. More than that produces reports, not decisions.
Three traps
Turning the score into the goal. It is easy to loosen criteria so the quality score goes up. The score is an indicator, not a target.
Only looking at bad calls. Good calls carry information too: which question types the agent genuinely handles well, which flows run cleanly. That guides you when expanding the agent's scope.
Limiting review to the agent. What happens after a handoff, on the human side, must be reviewed too. If the agent handed off correctly but the team handled the conversation badly, the outcome for the customer is the same.
You can see how conversation data is reported on the analytics and insights page.