The chatbots that rely on generative artificial intelligence have their uses, just as Wikipedia does. But relying on the details that either tool provides will land you in trouble eventually. In the case of AI chatbots, you’ll be in trouble sooner.
The common term for chatbot error is “hallucination,” and that’s already a dangerous term. A chatbot is technology, not a person or a character or a pet. It can’t hallucinate; it can present false claims confidently and with an air of authority. The most recent chatbot models continue to reply to queries with errors; this is not a problem only with earlier versions.
Here’s an example based on characters from The West Wing TV series, broadcast from 1999-2006.
I asked ChatGPT-4 (specifically the GPT-4-turbo variant1) for its help in explaining how:
- President Josiah “Jed” Barlet , a college-educated man in his mid-fifties
- who has been married for 32 years (“Five Votes Down”, s1 e04)
- to a college-educated woman
- can have a twelve-year-old granddaughter (“Pilot”, s01 e01)
- when all the granddaughter’s potential mothers also received college educations (“20 Hours in L.A.”, s01 e16).
(Yes, it’s technically possible if you juggle real fast. That’s not what this post is about.)
I asked ChatGPT how old Bartlet was during the first season of the show. It told me that he’s 55. Not totally out of line; actor Martin Sheen, who played the part, was around 58 when the pilot was filmed. It’s when the chatbot tries to cite its source that things go bad.
This is confirmed in Season 1, Episode 9 (“The Short List”), when it’s mentioned that he is 55.
That response is precise but it is wrong. At no point is Barlet’s age mentioned in that episode2. Toby Ziegler has a line in “The Short List” about a different male character being 55 years old, and that’s as close as we get.
I saw the same problem magnified when I asked about the age of Barlet’s granddaughter Annie in a specific episode — the pilot. The line of dialog that ChatGPT cites is total nonsense:
“My granddaughter Annie had a school teacher who gave her a hard time for wearing a Star of David. Annie’s in the third grade.” So, Annie is in third grade, which would typically make her about 8 or 9 years old during the pilot episode.
- That line of dialog does not appear in the pilot episode.
- No similar line of dialog appears in a different episode.
- At no point in the pilot2 is Annie’s teacher mentioned…
- …nor even is Annie’s grade mentioned.
- Annie is not Jewish and there is no mention of her wearing any Jewish symbols for any other reason.
- At no point in the first four seasons of the series is there any mention of the Star of David.
- Annie’s age is specifically mentioned to be “all of 12.”
The point is not that you, a human being, can check on the content of these scripts online or that you can watch the scene on YouTube if you think the details are wrong. The key word in that sentence is “think.”
The point is that you need a human being’s brain to recognize when something doesn’t seem right. A knowledge of the topic is a prerequisite to something not seeming right. These are human capacities, not artificial ones. Intelligence is based on a lot more than details. Knowledge is based on more than intelligence.
And that is why technical writers have value, and why the first rule of the 5 duties of a technical writer is “You have to understand your topic.” You have to have the relevant knowledge in your head; there isn’t time to research every little detail. And with AI chatbots, you do have to fact-check an incredible amount.
A good technical writer not only feeds the development cycle faster than a chatbot, an experienced tech writer avoids costly errors. Accurate answers beat fast answers every time. That won’t change.
No matter how advanced the language model, a human brain is more complex. It can keep all the rules of good technical writing in mind at once. A chatbot cannot. And a technical writer’s brain can also, while keeping the rules in mind, keep the user in mind.
If you have someone who accepts that any claims are true because the claim includes details, there’s a frighteningly high chance that they are no more intelligent than a chatbot.