A “resolved” count can mean the person said the answer helped. It can also mean the conversation ended and nothing suggested otherwise. Find out which before reporting the number.
The resolved rate is high. Isn’t that the result?
That depends on how the tool decided each conversation was resolved. A rate made up mostly of assumed results says less than one the people confirmed.
1. Find out how your tool marks a conversation resolved.
Confirmed. The person answers a closing question and says the answer helped.
Assumed. The conversation meets the tool’s own conditions, without anyone confirming.
The conditions differ from tool to tool. Look them up for yours, and check whether the dashboard shows which kind each conversation was.
2. Read a sample of each kind.
If your role allows you to see the conversations, open some confirmed ones and some assumed ones. Read the question, the reply and what happened next. Note whether the two groups show different problems.
Practical check: In the assumed group, how many people got an answer to the question they asked?
3. Report the two kinds separately.
Give the dates covered and the count for each kind. For a sample, check whether the person got done what they came to do. List the conversations that weren’t resolved and the ones you couldn’t check.
A fictional monthly report.
- Conversations
- 500.
- Confirmed by the person
- 120.
- Assumed by the tool
- 290.
- Passed to a person
- 90.
- Sample checked
- 30 of the assumed conversations were read. In 21, the person got what they came for. In 9, the conversation ended before the question was answered.
Read more: How Do You Know an AI Agent Actually Finished the Task? · How to Tell Whether an HR Chatbot Is Working
The figures are fictional. Tools record results in different ways. Read conversations only if your role allows it. This is general guidance, not advice for your situation.