Yes, AI chatbots can give wrong answers, and they often sound confident when they do. This is known as hallucination. The chatbot produces plausible but false information, outdated facts or invented sources. Errors can be reduced by limiting answers to trusted content and showing sources, and by handing uncertain questions to a person.
At a glance
- A Nature paper published on 22 April 2026 argues that models produce confident, plausible falsehoods and that evaluation methods which reward guessing help the problem persist.
- Which? (November 2025) tested six tools, ChatGPT, Gemini, AI Overview, Copilot, Meta AI and Perplexity, on everyday consumer questions and found inaccurate, unclear and risky advice.
- Common chatbot mistakes include misreading intent, hallucination, outdated knowledge, dropping context in longer chats, bias and ambiguous questions.
- Chati answers from a business’s own website, shows its sources and hands chats to a human team when unsure.
Can AI chatbots give wrong answers?
Yes. Chatbots can state false information fluently and confidently, and the problem affects even the most advanced models. If you run a business and are thinking about putting a chatbot in front of your customers, it is worth understanding this before you start.
The technical name for the problem is hallucination. In plain English, the chatbot makes something up that reads as if it were true. It might invent a fact, quote a policy that does not exist or point to a source that was never written. The tone stays calm and sure throughout, which is exactly what makes it risky.
A wrong answer is not the same as an unhelpful one
An unhelpful answer is vague or dodges the question. Sometimes it says “I’m not sure, let me get someone to help”. That is annoying at worst. A wrong answer is worse, because the customer believes it and acts on it. A guest turns up at the wrong time, a patient expects a price you do not charge, a client assumes a deadline has not passed. Telling the two apart matters, because a good chatbot should prefer being unhelpful and honest over being helpful and wrong.
General chatbots and website chatbots carry different risks
A general chatbot such as ChatGPT draws on a huge mix of text from across the internet. It can talk about almost anything, which also means it can drift into guesses about almost anything. A chatbot built to answer from one business’s own website has a much narrower job. It can still make mistakes, but there is far less room for it to wander. We come back to this in the sections below.
How current is the evidence?
It’s not an old worry that has since been fixed. Research and consumer investigations have pointed the same way, and confident errors are still a known feature of these tools.
Why do AI chatbots get things wrong?
Chatbots predict likely wording rather than look up verified facts, so they can produce text that sounds right but is not. Several other causes sit on top of that, and most of them are easy to picture once you see them laid out.
They predict the next word
A large language model is trained on vast amounts of text. It learns which words tend to follow which. When you ask a question, it builds an answer one piece at a time, choosing wording that looks most likely. It is not checking a database of facts as it goes. If the likely-sounding wording happens to be false, the chatbot has no built-in alarm that tells it so.
The Nature paper describes this tendency to produce confident, plausible falsehoods as a core issue with how these models work.
Testing rewards guessing
The same paper argues that the way models are evaluated helps the problem stick around. If a test gives credit for a right answer and nothing for “I don’t know”, then a model that guesses will score at least as well as one that admits it is unsure. Over time, models are shaped to guess. Think of a pupil in an exam who writes something down for every question because a blank earns nothing.
Outdated information
A model learns from text collected up to a certain point. Anything that changed afterwards, such as a new price, a new opening time or a new rule, may be missing. The chatbot can still describe the old version as though it were current.
Misread or ambiguous questions
People ask short, loose questions. “How much is it?” could mean a consultation, a full treatment or a monthly plan. When the chatbot picks the wrong meaning and answers with confidence, the customer gets a neat reply to a question they did not ask.
Invented sources, quotes and links
Ask a chatbot to back up a claim and it may produce a reference that looks perfect: a plausible title, a believable author, a tidy web address. The source may not exist. It’s one of the more awkward forms of hallucination because it looks like proof.
Context dropped in long conversations
In a long chat, a chatbot can lose track of details given earlier, or mix them up. A customer who said at the start that they were a new patient may later get an answer meant for an existing one.
What does the latest research say about chatbot accuracy?
None of it suggests the problem has gone away.
The Nature paper
The paper, published on 22 April 2026, looks at how large language models are judged. Its argument, put simply, is that the methods used to score these models reward them for giving an answer, even a guessed one, and so encourage hallucination. The result is a model that produces confident, plausible falsehoods rather than admitting uncertainty.
For a business, a warm, agreeable tone is good for customer service, but warmth must not come at the expense of accuracy. Agreeing with a customer’s wrong assumption is not helping them.
The developers admit it too
OpenAI, the company behind ChatGPT, has itself acknowledged that hallucinations remain a problem. So this is not only an outside criticism. The people building these tools say the same thing.
What we are not claiming
We are not quoting percentages here. Error rates vary by tool, topic and the way a question is asked, and we would rather say nothing than give you a made-up figure. The safe reading of the research is simple. Errors are real and they sound convincing. They have not been fully solved.
What are some examples of AI chatbots going wrong?
The clearest UK example is Which? testing six popular tools and finding risky advice on everyday consumer questions. It shows that the problem reaches ordinary people asking ordinary things.
The Which? test
In November 2025, Which? tested six tools: ChatGPT, Gemini, AI Overview, Copilot, Meta AI and Perplexity. It put everyday consumer questions to them and found advice that was inaccurate, unclear and in some cases risky. These are the best-known names in the field, not obscure experiments.
Fabricated references
Ask a general chatbot to find articles or studies on a topic and it may return titles that look real but are not. Anyone who copies those into a report, a website or a client letter risks passing on something that cannot be checked.
Wrong hours, prices and policies
For a small business, the most damaging errors tend to be dull ones. The wrong opening hours, an old price or a cancellation policy you stopped using last year can all slip out. Each one sounds minor. Each one can lead to a cross customer at the front desk.
Why this costs a small business
A wrong answer can cost you trust, and sometimes a booking. A customer who is told the wrong thing and then finds out may not come back, and may tell others. When your reputation rests on being reliable, the chatbot speaks in your name, so its mistakes become yours.
Why does a chatbot give different answers to the same question?
Chatbots generate each reply fresh, with some built-in randomness, so wording and sometimes facts can differ between attempts. Ask twice and you may not get the same reply twice.
Randomness and phrasing
The model picks among likely words rather than always the single most likely one. That keeps replies from sounding robotic. It also means a different path through the sentence on each attempt. Small changes in how you phrase a question can also push the answer in another direction.
Different earlier context
If two customers ask the same question after different earlier messages, the chatbot has different material to work from. One may get a correct answer and the other a muddled one.
Why consistency matters
Customers expect the same answer from your business whoever asks and whenever they ask. If your chatbot says one thing on Monday and another on Tuesday, you have a problem that is hard to spot. Limiting the chatbot to a fixed body of your own content narrows the range of things it can say, which makes its answers steadier.
Can a business chatbot be wrong too?
Yes, but the risk is lower when it only answers from your own content and cannot wander into guesses. Lower is not zero, so it helps to know where the remaining risk sits.
Where the risks remain
- Out-of-date pages. If your website still shows last year’s price, the chatbot will repeat it faithfully.
- Gaps in content. When a question is not covered anywhere on your site, a poorly set up chatbot may try to fill the gap with a guess.
- Ambiguous questions. A vague question can still be matched to the wrong page.
Where accuracy matters most
Some businesses work in fields such as dentistry, law, accountancy and veterinary care. In those fields a wrong answer matters more than it does for a shop selling mugs. A patient who misunderstands an aftercare instruction, or a client who thinks a legal deadline works differently, can be harmed by confident nonsense. These are the businesses that most need a chatbot that stays inside what they have actually said.
Trained on your data, integrated to fit
It is a sensible starting point, but the quality of the answers still depends on the quality of what it is given.
Your website has to be right
Your chatbot is only as accurate as the site it’s trained on. Treat your pages as the single source of truth. When a price, a policy or an opening time changes, change it on the website first, and the chatbot follows.
How can you reduce wrong answers from an AI chatbot?
Train it on your own verified content, show sources, set it to hand over when unsure, and review conversations regularly. Each step closes off a different route to error.
1. Limit answers to your own content
The biggest single step is to stop the chatbot drawing on anything but material you trust.
2. Show sources
When a chatbot shows where an answer came from, the customer can click through and check. It also keeps the chatbot honest, since every answer has to point back to something real. Chati shows its sources for exactly this reason.
3. Hand over to a person when unsure
No chatbot should be forced to answer everything. Chati hands chats to a human team when it is unsure, so a tricky or sensitive question lands with someone who can deal with it properly. A quick “let me pass you to a colleague” is far better than a confident error.
4. Review chat logs and fix gaps
Read what customers actually ask. Questions the chatbot struggled with show you where your website is thin or unclear. Add the missing answer to your site and the chatbot improves along with it.
5. Keep prices, hours and policies current
Set a regular reminder to check the facts that change most often. Opening times over bank holidays, price lists and booking rules are the usual culprits.
Why you can trust the advice
You can see how the pieces fit together on our how it works page.
How can customers check whether an AI answer is right?
Check the cited source, compare against an official page, and ask a person for anything involving money, health, legal or safety decisions. These habits apply to any chatbot, yours or anyone else’s.
A short checklist
- Look for sources. When the chatbot gives one, click through and read it. With none, treat the answer with more caution.
- Check the link is real. Invented references often lead nowhere or to an unrelated page.
- Cross-check. Compare the answer with the official website of the organisation concerned, or an independent source.
- Watch for confidence without detail. A smooth answer with no specifics is a warning sign.
- Ask for a person on important matters. Anything about money, health, legal rights or safety deserves a human check.
- Note the date. Ask yourself whether the information could have changed since it was written.
Make it easy for your own customers
On your own site, make checking effortless. Visible sources, a clear route to a real person and plain contact details all tell customers they are not stuck with a machine. Our industry guides show how this looks for dentists, solicitors, trades and more.
Would you like a chatbot that answers from your own website, shows its sources and passes the tricky questions to your team? Get in touch with Chati and we’ll talk it through with you.
Frequently asked questions
Does ChatGPT give wrong answers on purpose?
No. It does not set out to mislead. It predicts likely wording, and sometimes that wording is false. The result can look like a confident lie, but there is no intent behind it.
How often does ChatGPT give wrong answers?
There is no single reliable figure, as it depends on the topic and the question. The research we have cited, including the Nature paper, shows errors are a persistent feature, so it is wise to check anything important.
Why does ChatGPT say it can make mistakes?
Because it can. The developers acknowledge that hallucinations remain, and the warning is there to remind you to verify what you read.
Can a Google answer be wrong?
Yes. Which? included Google’s AI Overview among the six tools it tested in November 2025, and found inaccurate, unclear and risky advice from the group as a whole.
Can I trust an AI chatbot on my website with customer questions?
You can trust it more if it only answers from your own content, shows sources and hands over to a person when unsure. It still needs accurate website pages and occasional review of its chats.
What should a chatbot do when it does not know the answer?
It should say so plainly and pass the customer to a person, rather than guess. A short, honest handover protects both the customer and your reputation.
Who is responsible if a business chatbot gives a customer wrong information?
The chatbot speaks in your name, so customers will see the mistake as yours. For anything with legal or financial weight, take advice on your own position and make sure the chatbot hands those questions to your team.
Will AI chatbots stop making mistakes in future?
Nobody can promise that. Researchers are working on better ways to measure and reduce hallucination, but for now the practical approach is to limit what a chatbot can say, show its sources and keep a human close by.
