Why your chatbot makes things up

Give the chatbot a safe way to be uncertain

First define what counts as an unsupported claim for your business. A fabricated return policy is different from a harmless variation in wording. Gather examples of both supported answers and questions the chatbot should hand to a person. Check the source, date and product context for every critical claim. A request to “be accurate” does not supply missing facts or enforce business rules.

What to check

• Ask questions outside the knowledge base and check whether the chatbot admits the gap.
• Test outdated policies, conflicting sources and requests that imply a false premise.
• Check whether citations support the actual claim, rather than merely linking to a relevant-looking page.

What to change

Use authoritative sources for factual responses. Keep calculations and eligibility decisions in deterministic code when appropriate. Provide a clear response for missing or contradictory information and an escalation route that retains useful context. Restrict what the chatbot may promise about payments, refunds and other business commitments.

How to evaluate the result

Track unsupported claims by category and severity. Review both answered and declined questions: declining every request is safe in one sense but not useful. Measure how often an escalation helps the customer complete the task.

What the result does not establish

Grounding and evaluation can reduce certain errors, but they do not guarantee truth. Critical claims still need controls appropriate to their consequences, including human review where necessary.