Chatbots Hallucinate Everything, Always

A Word that Misleads

Photo by droberson / https://pixabay.com/photos/fruit-apple-food-market-freshness-3210171/

Much of the popular coverage of AI these days focuses on hallucinations. This is generally understood to refer to things that chatbots (or LLMs) “make up.”

For example, lawyers in multiple cases have recently been admonished by judges for submitting chatbot-written materials that reference previous cases that don’t exist ¹ ². Anyone who’s tried to learn about a new topic with the help of a chatbot will have run into this: links that go to non-existent websites, or books that were never printed.

I think referring to these as “hallucinations” is a fascinating use of language to shape popular understanding of a technical problem. The term minimizes the public’s perception of how deeply this problem affects chatbots, and not coincidentally, helps protect the bottom lines of companies selling them. The language implies a distinction between common “reality-based” output and infrequent “hallucinated” output. But there is no difference. Chatbots always make up words without understanding or grounding in reality. Not just sometimes. It’s all they do.

It’s helpful to see where the term originated. In the early days of AI (you know, a few years ago), our training data was small enough that we knew what was in it. If I was creating a chatbot to run the stock at a small corner grocery, I might train it on lists of fruits and vegetables and other items. If I later asked it for what apples we have on-hand, it might say, “Red Delicious, Granny Smith, Pink Lady, Upright Harold, Honeycrisp, and Macintosh.” That’s fine, so... wait a second. “Upright Harold”? What? I go looking through the training data and that phrase doesn’t appear anywhere. It’s definitely not a variety of apple. So people gave this kind of thing the name hallucination, which was a nicely evocative metaphor.

But it hides the fact that everything that comes out of a chatbot is produced the same way. Given a sequence of words, the chatbot predicts the next word (or it chooses from a list of next words). There’s no conceptual underpinning, no idea of “reality” or “truth.” There is no mechanism that determines if the chosen word does or does not refer to something that exists in the world.

AI professionals are probably already mentally composing comments about how the word “hallucination,” used as a technical term, can refer to a specific, narrowly-defined idea. I don’t argue with that. My interest here is in how the word is frequently used in the popular press and marketing materials to imply that chatbots only make up stuff sometimes, and with effort we will reduce the frequency and severity of this made-up stuff. In actuality, chatbots make up stuff always.

Suppose that I told you, “Sometimes I eat pancakes.” That sounds reasonable, and you’d naturally assume from the language and context that I also eat other things. But suppose that I only eat pancakes. Ever. Nothing but. We shouldn’t think of pancakes as an occasional meal, but rather my complete diet. Remove them, and there’s nothing left.

I’m fascinated by how metaphors like this can take on lives of their own in the hands of clever marketers. It takes a special skill - and a special mindset - to torque a technical term into a misleading popular idea.

1. Linn F. Freedman, “Lawyers sanctioned for citing AI generated fake cases,” National Law Review, 2025. https://natlawreview.com/article/lawyers-sanctioned-citing-ai-generated-fake-cases

2. Jason Proctor, “B.C. Lawyer Reprimanded for citing fake cases invented by ChatGPT,” CBC News. https://www.cbc.ca/news/canada/british-columbia/lawyer-chatgpt-fake-precedent-1.7126393