Stopping AI hallucinations: grounding, guardrails, evaluation

An airline's chatbot invented a refund policy that did not exist, a customer relied on it, and a tribunal held the airline to what its bot had promised. That case is now the standard cautionary tale for a reason: it captures exactly what businesses fear about putting AI in front of customers. The AI does not just get things wrong. It gets them wrong confidently, in fluent, professional language that sounds exactly like a correct answer.

This is called hallucination, and it is not a bug that gets patched away. It is inherent to how language models work: they are built to produce plausible text, and a plausible falsehood is, to the model, indistinguishable from a plausible truth. You cannot eliminate it. You can engineer around it well enough to run reliably in production, and that engineering is precisely what separates a serious AI system from a demo. Here is how it is done.

Why business AI makes things up

A language model does not look up facts. It predicts likely text based on patterns in its training. Ask it something it was well-trained on and it is usually right. Ask it about your business, your policies, your pricing, your inventory, and it has no reliable knowledge, so it generates something that reads correctly and may be entirely invented. Worse, it has no built-in sense of its own uncertainty, so the made-up answer arrives with the same confident tone as a true one.

This is why a raw, ungrounded chatbot is dangerous for business use. It is not that it is often wrong. It is that when it is wrong, nothing signals it, and the language is persuasive.

The three-layer defence

Reliable production AI does not rely on one trick. It stacks three independent layers, so that a failure in one is caught by another.

Layer 1: Grounding, answer only from real sources

The single most important technique. Instead of letting the model answer from its training, you retrieve the relevant passage from your own content, hand it to the model, and instruct it to answer only from that passage, with a citation. This is retrieval-augmented generation (RAG).

The effect is transformative. Ask a grounded assistant about your return policy and it retrieves your actual return-policy text and answers from it, with a link. Ask it something not covered in your content and, done properly, it says it does not know rather than inventing. Grounding converts “the model's opinion” into “a quotation from your documents,” which is a completely different risk profile.

Grounding is only as good as the retrieval behind it. If the system fetches the wrong passage, the model faithfully answers from the wrong passage. Good retrieval, tuned and tested, is the unglamorous core of a trustworthy assistant.

Layer 2: Guardrails, boundaries on input and output

Grounding handles honest questions. Guardrails handle everything else: the customer trying to make your bot say something absurd, the question that strays into territory you do not want it in, the attempt to extract data it should not reveal.

• Input guardrails screen incoming messages for manipulation attempts and off-limits topics before the model ever sees them.
• Output validation checks the model's answer before it reaches the customer: does it cite a source, does it stay on topic, does it avoid making commitments the business has not authorised.
• Topic and scope limits keep a support assistant answering support questions, and politely declining to give legal, medical or financial advice it has no business giving.
• Data protection stops the assistant echoing back personal or sensitive information, which also keeps you on the right side of the DPDP Act.

Layer 3: Evaluation, prove it works and keep proving it

You cannot trust what you cannot measure. Evaluation means building a test suite of representative real questions, with known correct answers, and running the whole thing every time you change a prompt, swap a model or update the content.

This is the layer businesses skip, and it is the one that makes AI safe to change. Without it, every tweak is a gamble: you fix one thing and silently break three others, and your customers find the breakage before you do. With it, a change that regresses accuracy is caught before it ships, exactly like a test suite in ordinary software. The parallel to our security discipline is deliberate: reliability comes from a repeatable process, not from hoping.

The human escalation path

No matter how good the three layers are, the assistant will sometimes hit a question it cannot answer safely. The correct behaviour then is not to guess, it is to hand off to a human. A well-designed assistant knows the edge of its competence and routes the customer to a person rather than fabricating an answer to seem helpful. “Let me connect you to someone who can help with that” is a feature, not a failure.

What good looks like, as a buyer

You do not need to build any of this yourself, but you should recognise it when hiring. Ask any AI vendor:

• Where do the answers come from? The right answer involves your content and citations, not “the model knows.”
• What happens when it does not know? The right answer is “it says so and escalates,” not “it always answers.”
• How do you measure accuracy? The right answer is a concrete evaluation process, not “we tested it and it seemed good.”
• How do you stop it going off-topic or leaking data? The right answer describes actual guardrails.
• What do you monitor in production? The right answer means someone is watching real conversations, not deploying and forgetting.

A vendor who answers these crisply is doing the engineering. A vendor who waves them away with “the AI is very advanced now” is selling you the airline's chatbot.

How RoseLeap can help

Every assistant we build under our AI Solutions practice ships with all three layers: grounding with citations so answers reference your real content, guardrails on input and output, and an offline evaluation suite that runs on every change. Production deployments include monitoring for drift and a clear human escalation path.

This is exactly why we take on a limited number of AI engagements per quarter, the safety engineering is the work, and it does not compress. If you want AI in front of your customers that you can actually trust, tell us the use case on the contact page. For the bigger picture, see our overview of practical AI for small businesses.