An AI chatbot on your website: what works, where it goes wrong, and a better way

An AI chatbot is software that answers visitors' questions in conversation, these days usually with a large language model. More and more organizations put one on their website, and user frustration grows just as fast. This article lays out the figures and explains why we at vragen.ai took a different turn: AI search with sources cited, that says honestly when it can't find an answer.

Everyone wants an AI chatbot, and that makes sense

The promise is appealing: an instant answer, day and night, less pressure on your customer service or contact center. Organizations are embracing that promise en masse. More than 74% of companies now use chatbots in customer service. Users are open to it in principle too: 67% of consumers are willing to use an AI assistant for service questions, certainly when the alternative is a queue.

So the question isn't whether visitors want to be helped by AI. The question is whether the execution deserves that willingness. That is where it chafes.

The figures: frustration is the norm, not the exception

Put the research side by side and a strikingly consistent picture emerges:

• 77% of adults find customer service chatbots frustrating, and only 35% say a chatbot usually solves their problem efficiently (Ipsos).
• 64% of customers would prefer companies not to use AI for customer service at all, and only 14% of service questions are fully resolved through self-service (Gartner, 2024).
• More than two thirds of customers have had a bad chatbot experience. The two biggest pain points: the bot can't answer the question and doesn't understand the need (Verint).

Note what this actually says. The problem is rarely that people don't want AI. The problem is that the chatbot doesn't know the answer, won't admit it, and sends the visitor round in circles. Researchers at Berkeley describe how that experience produces more than frustration: it demonstrably leads to anger, aggression and customer loss. So every failed chatbot session costs more than one missed question: it damages the relationship.

The biggest risk: a chatbot that invents answers

Language models can hallucinate. They then formulate a convincing, fluent and completely incorrect answer. For any organization that wants to be trusted at its word, that is not a cosmetic problem: it goes straight to reliability and liability.

The best-known example is the Air Canada case. The chatbot on the website invented a refund policy for bereavement fares that did not exist. A passenger relied on it, didn't get his money back and went to court. The tribunal held Air Canada liable for negligent misrepresentation and dismissed the argument that the chatbot bore its own responsibility.

"While a chatbot has an interactive component, it is still just a part of Air Canada's website. It should be obvious to Air Canada that it is responsible for all the information on its website."

The lesson for any organization: you are responsible for what your AI says on your website. An invented answer about a permit, a reimbursement or a medical arrangement isn't a bug, it is a misleading statement with your logo above it.

The psychology of an honest "I don't know"

There is another reason to want an AI that knows its limits, and it is a human one. Research into trust in AI shows that people judge systems that show uncertainty more mildly than systems that get things wrong with confidence. That research lines up with an older psychological insight: we have more sympathy for those who underestimate themselves than for those who overestimate themselves.

On top of that comes a strong negativity bias. In a study among pharmacists, trust fell considerably harder after one wrong piece of AI advice than it rose after a correct one. Translated to your website: one confidently invented answer costs more trust than ten good answers build. A review study in Nature Humanities and Social Sciences Communications concludes accordingly that AI systems which show their uncertainty and are explainable are experienced as more trustworthy.

Everyone knows the experience of a so-called AI colleague who knows nothing and sends you off in the wrong direction with great confidence. For a search engine that says "I can't find this, please contact our team", we as people have far more tolerance. That is not a weakness of the system. It is exactly the behavior that keeps trust intact.

AI chatbot or AI search: choose based on the task

Does this mean an AI chatbot is always the wrong choice? No. It means you have to choose based on the task.

A conversation-led chatbot suits transactional flows: changing an order, booking an appointment, qualifying a lead. Platforms like Intercom, Zendesk and Watermelon are built for that and do it well, as long as the conversation stays within the scripted paths.

But plenty of organizations have a different problem. The answer is already on their website or in their knowledge base, the visitor just can't find it. That happens everywhere knowledge is the product: from municipalities and healthcare organizations to publishers, smaller companies and web shops with a well-stocked knowledge base. The question then isn't "how do I hold a conversation" but "how do I open up my own knowledge reliably". AI search fits better there: retrieval-augmented generation (RAG). The system first looks up the right information in your content and only then formulates an answer, with the source included. If the answer isn't there, it says so and refers the visitor to a person.

	AI chatbot	AI search (RAG)
Strong at	Scripted conversations and transactions	Answering questions from your own content
Source of answers	A language model, sometimes topped up with scripts	Only your own, controlled sources
When the answer is missing	Often formulates an answer anyway	Says "not found" and refers to a person
Verifiable	Limited, no source with the answer	Every answer with the source cited
Risk	Hallucination, blind spots in the script	As good as the content you provide

vragen.ai is such an AI search engine: built and hosted in the Netherlands, with the source cited with every answer and often live on your website or intranet the same day.

Where our own approach has limits

Comparing honestly also means being honest about the limits of our own approach.

• RAG is as good as your content. If the answer isn't anywhere on your website, our system can't find it either. For us, every unanswered question is a signal to improve content, not a reason to invent the answer.
• Caution has a price. We would rather say "not found" once too often than once too rarely. That sometimes costs an answer that a bit more nerve would have produced. We make that trade-off deliberately.
• Not every conversation belongs with AI. Debt counseling, objection procedures, bad news: where empathy and judgment are needed, a person has to hold the conversation. At most, AI can point the way to the right desk.
• The human channel stays open. AI search takes pressure off a contact center or support team, but doesn't replace it. A visitor who gets stuck must always be able to reach a person without detours.

Frequently asked questions about AI chatbots

What is the difference between an AI chatbot and AI search?

An AI chatbot holds a conversation and formulates answers freely using a language model. AI search first looks the answer up in the organization's own content and then formulates an answer with the source cited. If the answer isn't there, a good AI search system says so honestly.

Can an AI chatbot invent answers?

Yes. Language models can hallucinate: they then give a convincing but incorrect answer. Air Canada's chatbot invented a refund policy that did not exist. The tribunal held the company liable for it.

Is an organization liable for what its chatbot says?

In February 2024 the Civil Resolution Tribunal ruled that a chatbot is simply part of the website and that the organization is responsible for all information on it. Incorrect chatbot answers can therefore have legal and financial consequences.

What is a good alternative to an AI chatbot?

For organizations with a lot of knowledge on their website: an AI search engine running on their own content, with sources cited and an honest referral when the answer is missing. vragen.ai is such an AI search engine.

What is RAG (retrieval-augmented generation)?

RAG is a technique in which an AI first looks up the right information in controlled sources and only then formulates an answer. That keeps every answer traceable and stops the system inventing facts.

How quickly can AI search be on my website?

Often the same day. The crawler reads your existing website or knowledge base, and two lines of code put the widget on your site. See how it works or what it costs.

Sources

1. Ipsos, chatbot research among consumers, via Backlinko Chatbot Statistics
2. Gartner (2024), via California Management Review, Chatbot Frustration is Real
3. Verint (2024), via CX Dive
4. Zendesk (2025), via Pylon Customer Support Statistics
5. Moffatt v. Air Canada, Civil Resolution Tribunal BC (2024)
6. A Diachronic Perspective on User Trust in AI under Uncertainty (arXiv, 2023)
7. The Effects of Presenting AI Uncertainty Information on Pharmacists' Trust (JMIR, 2025)
8. Trust in AI: progress, challenges, and future directions (Nature HSSC, 2024)