AI is an indispensable part of my daily routine, yet it remains profoundly terrible at specific types of writing. For months, I have been trying to understand why a tool so deeply useful can simultaneously feel so hollow.
This issue is important to me, because I do think AI is incredibly useful. I use it almost every day. And I'm surrounded by people who have used AI to write documents like business reports, project plans, and technical documentation. But when it comes to a lot of the writing I do, I increasingly find AI wanting. I've been trying to understand why, and I haven't figured it out yet.
The problem, as I see it, is that AI creates what I call rhetorical hallucinations.
Most of us who work with AI know what a hallucination is: it's when the AI makes things up. Things like: "37% of all users prefer VIM" when no such data exists. I think of these types of hallucinations as factual hallucinations: hallucinations where the AI is making up facts that do not exist. A significant amount of thought and effort goes into preventing these sorts of factual hallucinations, because they can cause significant harm.
Rhetorical hallucinations, however, are more insidious. This type of hallucination occurs when the AI uses a rhetorical pattern that reads well but, when you dig deeper, is essentially meaningless.
I'll give an example. I was working with AI on an article. In that article, I used a simile of a ladder to convey different levels of trust. The idea is that each level of trust is like a rung on a ladder. You can't get to the next rung without traversing the one that preceded it. I was struggling with the paragraph, so I turned to AI to help. My tools happily generated a paragraph about the metaphor, ending with:
"Trust is like a ladder. And the ladder climbs itself."
When I read those sentences in the context of the paragraph, everything sounded good. Everything felt good. Then I stopped and reread those sentences and I realized: the content meant nothing. "Trust is like a ladder" is a metaphor. The following sentence, "And the ladder climbs itself," is meaningless. (Ladders don't climb.)
What the AI was trying to do was employ a very common rhetorical device: a chiasmus. A chiasmus is a rhetorical device in which words, grammatical structures, or concepts are repeated in reverse order. "Ask not what your country can do for you, but what you can do for your country" is one example of a chiasmus. When done correctly, it's a powerful tool that can make the reader or listener think. But employing this device, or any rhetorical device, requires a deep understanding both of language and how it relates to human psychology. It's a skill that makes great writers great.
AI does not have this skill.
The moment that you start writing something where your point is to convey an idea, or convince someone to take an action, AI falters. It can recognize rhetorical devices such as metaphors or tricolons (the rule of three), but it does not know how to apply them meaningfully.
What makes these rhetorical hallucinations so dangerous is that they are very easy to miss. These rhetorical devices exist because, over time, they have proven effective at being engaging, convincing, and impactful. These devices exploit a sort of glitch in human psychology: our brains naturally mistake a satisfying linguistic rhythm for logical truth. If a statement sounds perfectly balanced, we are predisposed to believe it — even when the words underneath are entirely vacant. It's only when we carefully read through the text, when we peel back the rhetorical device and try to understand the meaning of the words, that we find that a lot of what AI has written is empty and hollow.
When readers do find that these words are, in fact, meaningless, their reaction is swift. And well it should be — you have broken their trust. Worse, you have made them feel foolish. You have made what they thought was a convincing argument and they have discovered that you weren't saying anything at all. You have insulted their intelligence and they will remember that.
As we continue to use AI to write, it becomes more and more important that we have people who know what great writing is, who can catch rhetorical hallucinations and remove them. Or, perhaps more importantly, understand that AI cannot use rhetorical devices; these must be crafted by hand.
Because the ladder doesn't climb itself.