Quality monitoring · Quality Assurance (QA)
What is quality monitoring and how do you assess a conversation fairly?
Quality monitoring is reading conversations yourself instead of waiting for customer scores. It's the only measure showing whether the answer was also correct.
In short
Quality monitoring, or quality assurance, is systematically reading and assessing a sample of customer conversations against fixed criteria. It complements CSAT precisely where that score is blind: a customer can be satisfied with a friendly answer that was factually wrong, and you only see that difference by reading the conversation itself.
Why CSAT isn't enough here
A customer assesses whether they felt helped, not whether the answer was correct. An agent who politely quotes the wrong return period gets a 5 and hands you an argument two weeks later. In the satisfaction score that conversation is a success.
That's why quality monitoring exists alongside customer scores rather than instead of them. The two measure different things, and where they diverge is usually the most interesting information.
What do you assess on?
An assessment form with fifteen points stops being filled in after three weeks. Four or five criteria applied consistently deliver more than an elaborate list that stalls halfway.
- Was the answer correct?
- The only criterion that's objectively assessable, and the one missing from most forms. Test against your knowledge base, not against the assessor's opinion.
- Was it complete?
- Was the question the customer asked answered, including the part they asked between the lines. This is where most follow-up questions come from.
- Did the tone match the situation?
- Not whether it was friendly, but whether it fitted. A cheerful standard line above a complaint about a damaged parcel doesn't fit.
- Was the next step clear?
- Does the customer know after this message what happens and when. This criterion alone prevents a large share of your chaser emails.
How do you stop it feeling like surveillance?
Quality monitoring reported only upwards reads as surveillance and produces agents who answer safely rather than well. The outcome should reach the agent themselves first, and only then reach a manager in aggregate.
What also helps is letting assessments run both ways. An agent who reads a colleague's conversation themselves sees within an hour why the criteria exist, and the conversations that stand out are usually exactly the ones a manager would miss.
What you do with AI conversations
Conversations handled by an AI agent belong in the same sample as agents'. In practice that rarely happens, and then an AI gets assessed on deflection while a human gets assessed on quality.
How often an agent adjusts an AI draft before it goes out is also the most usable quality measure that exists for an agent. That number is already there without anyone filling in a form, and it can be split by topic.
Where it goes wrong
Only reading conversations with a low CSAT
Then you miss exactly the category this instrument should find: the friendly answer that was factually wrong. Take a random sample alongside the outliers.
A form with too many criteria
Fifteen points per conversation stops happening after three weeks. Four criteria you sustain are worth more than a complete list that stalls.
Reporting the outcome only upwards
Without feedback to the agent it isn't a quality instrument but a performance file, and then nothing changes in the conversations.
Leaving AI conversations out of the sample
An AI that isn't reviewed can give a wrong answer on the same topic for months without anyone noticing.
What to do this week
Read ten random conversations from last week and assess on one question only: does this message say what happens now and when. You don't need more criteria for a first round. The share that fails that question is usually larger than expected, and it's the share causing your chaser emails.
What this looks like in a system
In Cuego every conversation shows what was sent, by whom or by the AI, and what draft preceded it. That makes it visible how often an agent adjusted an AI answer and on which topics, without anyone filling in a form. Conversations can be filtered by topic, channel and outcome, so a sample isn't made up of the most recent conversations but of the category you want to test.
AnalyticsRelated terms
CSAT
CSAT is the percentage of customers who rate a contact moment as positive. The score is easy to collect and easy to misread.
Read onAI agent
An AI agent understands a question in plain language, looks up information in your systems and carries out the action behind it. That's something other than a chatbot with choice buttons.
Read onCanned response
A canned response saves typing on questions that keep returning. It becomes harmful the moment it answers a question that was slightly different.
Read on