Start with one failed question. Then check the information, the access, the answer and the route to a person. A bad result deserves investigation, not dismissal, and not a defence of AI either.
A bad experience is worth investigating.
In one practitioner discussion, someone described poor experiences with a company-policy chatbot and felt a person would have done the job better. The comment doesn’t reveal why it failed, and better setup isn’t a guaranteed cure. The checks below are how you find out.
1. Keep the question. Record the gap.
Capture the question, the response, its references, the date and the employee group involved.
Define the expectation. Identify the approved source, and what a useful response should have included.
Reproduce carefully. Retest in the channel employees use, with authorized test access and protected records.
Practical check: Don’t share private employee conversations publicly.
2. Was the source right for this question?
Current. Is the guidance approved and in effect?
Applicable. Does it cover this site, this assignment and this employee group?
Complete. Does it actually resolve the question? Gaps and conflicts are for a policy owner to settle.
Practical check: An unclear policy is not a prompt-writing problem.
3. Could it find the right information?
Connected. Confirm the intended source is available and has finished processing or refreshing.
Authorized. Test with the access the intended employee has, not only the builder’s administrator account.
Reviewable. Check which source was retrieved, and whether the employee can open the reference.
Practical check: Don’t widen access simply to make an answer appear.
4. A citation is a clue, not proof of accuracy.
Meaning. Does the answer accurately reflect the passage it cites?
Context. Are important exceptions, dates or conditions missing?
Boundary. Does it separate process guidance from an individual decision or entitlement?
Practical check: A retrieved document can still be misinterpreted.
5. An unresolved question still needs an owner.
Offer help. Make the route to a person clear when the answer is missing or not enough.
Confirm the submission. Let the employee review what will be shared. Verify it arrived, and handle the case where it didn’t.
Check the result. Did the right person receive enough context? Did the employee get a useful next step?
Practical check: Silence after a reply doesn’t prove the question was resolved.
Fix, narrow, replace or stop.
Retest the failed case and the questions related to it. Count the checking, corrections and upkeep. Keep the service only where it proves useful.
Read more: Seven Things to Check Before You Trust AI in HR · How the HR Knowledge Agent Works