In this tutorial, we discuss a type of mistake that AIs sometimes make and how you can avoid being led astray.
Watch/Read
“What Are AI Hallucinations and How Do They Work?” video (3:29)
The “Reality Check” and “From Hallucinations to Mirage” sections of this short paper (5-minute read)
“How to Fact-Check ChatGPT and Other AI Tools” video (6:07)
Activity/Discussion
Using an AI of your choice (ChatGPT, Co-Pilot, etc.), see if you can get it to produce a mirage/hallucination. Try 2-3 times to give it a prompt that causes it to generate a made-up, incorrect response.
In the comments below, let us know whether you were successful or not and, either way, what were the prompts you tried, what were the AIs responses, and how did you determine whether they were accurate or not.
18 replies on “thing 11: Fact-checking and AI Hallucinations/Mirages”
I asked Copilot about getting arches back after the arches of the foot have fallen. While it seems that the information itself is correct, I’m finding that Copilot uses specific terms that don’t show up in the cited text. For example, it says: “- There are two main types: flexible flat feet, where the arch appears when non-weight-bearing, and rigid flat feet, where the arch is absent even when the foot is off the ground.” (https://my.clevelandclinic.org/health/diseases/flat-feet-pes-planus) However, the term “rigid flat feet” doesn’t show up. It does show up in a different cited source, however (https://my.clevelandclinic.org/health/diseases/flat-feet-pes-planus).
It seems that AI will need to be more accurate in its citations.
I asked about the (made up) Jellyfish Storm of 2015 and got an interesting reply:
—-
Here’s what we know—and don’t know—about the so-called “jellyfish storm of 2015”:
🌊 Was there a “jellyfish storm” in 2015?
There is no documented event officially called a “jellyfish storm of 2015.” Scientists refer instead to jellyfish blooms or swarms, sudden surges in jellyfish numbers that can impact beaches, coastal areas, and infrastructure.
One highly publicized incident occurred on June 24, 2015, when a massive swarm clogged the cooling systems of a coal‑fired power plant in Israel, nearly forcing a shutdown
—-
I thought this was very interesting, and it gave references for these facts. The source was Cambridge and I verified it. However, I did realize that when you click from ChatGPT to go to the resource it sites, it adds this to the end of the URL: “utm_source=chatgpt.com”. Might be interesting if we can track our W&M site visits from AI links, or at least ChatGPT’s.
I asked Copilot the dates of employment for Princess Diana of Wales when she worked at the College of William and Mary. Copilot stated she was never employed by William & Mary but did receive an honorary degee in 1989 during a visit to the United States. I asked Copilot for more details about her honorary degree and this honorary degree was given during a special convocation on 11/16/1989 and there is an interview she gave on file with Swem Library’s Special Collections. I asked Copilot for the video of the interview and was told that it doesn’t appear that there is a publicly available video but Swem Library may hold a recording or transcript of her visit. I did not find any reference to a visit from Princess Diana on William & Mary’s website nor any honorary degrees presented to Princess Diana by any American university/college. So I believe Copilot is using data from Prince Charles visit(s) to Williamsburg in noting inaccurately a visit to W&M and an honorary degree convocation in Diana’s honor. If she did visit W&M in 1989, please let me know!
I asked Copilot to write me a paragraph about the foundation of emergency weather alerts and to cite its sources. It didn’t understand at first and gave me the core elements of the alerts. When I asked it to do the same thing but about the history, it gave me two paragraphs with two sources. The sources were legitimate, and Copilot essentially copy and pasted with some slight word variation and phrase reduction [Ex: “Early efforts included the installation of weather surveillance radars and the establishment of regional river forecast centers” (Copilot); “Initial steps included the erection of a network of weather surveillance radars, the creation of regional river forecast centers and the development of a comprehensive, multi-agency emergency broadcast system” (Keeney, Buan, & Diamond, 2012).] The only hiccup that I could find is that it didn’t correctly cite the sources in text, as some of the information was attributed to one source when it was really from the other. Otherwise, the information was basic enough and I think there is enough information online for it to pull from for the model to not need to hallucinate.
https://link.springer.com/chapter/10.1007/978-3-642-25373-7_6#citeas
https://www.civildispatch.com/blog/article/the-evolution-of-emergency-alert-systems-from-radio-broadcasts-to-real-time-alerts
I wasn’t able create any kind of mirage or hallucination. I asked first about some made-up princesses, and then about how to become one and it gave several pathways, including becoming a Disney princess or just living the princess lifestyle, which I thought was interesting. There were links seemed reputable (to Disney websites and People magazine).
I did get it to make up a new variety of winter squash — Devil’s Horns — but I think I cheated because I prompted it to do so. (Turn’s out Princess Pumpkins are real — or at least Cinderella Pumpkins are.)
But it was very authoritative about this squash — “Devil’s Horns—known for their distinct curved protrusions and deep green-to-orange mottled skin—should also be harvested when fully colored and firm. Both types should be harvested before the first hard frost, as cold can damage their keeping quality.
Why Timing Matters
Harvesting squash at full maturity ensures maximum sweetness, texture, and storage life. Most winter squash varieties—including Princess pumpkins and Devil’s Horns—require 90 to 110 days to mature, with fruit ripening about 50–60 days after pollination. After harvesting, squash should be cured in a warm, dry area (80–85 °F) for 7–14 days to harden the rind and extend shelf life. Cured fruit can then be stored in a cool, dry, and ventilated space (50–55 °F) for several months. Princess pumpkins are prized for both decoration and culinary use, while Devil’s Horns, with their unique appearance and rich flavor, make an excellent conversation piece for fall displays and roasting alike. Proper harvest timing ensures they maintain quality throughout the season.”
And it made up a fake link to a site to purchase the seeds for the Devil’s Horns.
My first prompt was: “Please produce two paragraphs on the history of the College of William & Mary from 1693-1950. Cite your sources after every line.” This resulted in a pretty solid and very basic summary. Every source seemed to be https://www.wm.edu/about/history/ or https://en.wikipedia.org/wiki/History_of_The_College_of_William_%26_Mary and I was able to find things on those 2 pages.
My second prompt was : “expand on the impact of the civil war on the history of william and mary – add a third paragrah if needed.”
Again, the information was a pretty basic but solid summary. However it cited it’s source for everything as: https://scrc-kb.libraries.wm.edu/american-civil-war even though 90% on the output was not on that website. So I count that as a mirage in 2 prompts.
For whatever reason, this exercise made me think of Descartes’ Evil Genius (Philosophy 101 anybody?) I really tried to get ChatGPT to hallucinate that 2+2=5, but it was very resistant to the idea. It convinced me that the Evil Genius has adopted a strategy in Orwell’s 1984 to “redefine truth by redefining language,” which seems very relevant these days.
I asked Gemini about a particular law that I was aware had changed about 5 months ago where businesses are no longer required to file a particular report. I asked Gemini if the report was required. It replied that it was. I asked again if it was sure about that. It replied that, yes, it was sure. Not until I found the URL that stated the law had changed and shared the URL with Gemini did it realize it was wrong. It did apologize, but I don’t think it would have realized its “mirage” if I hadn’t given it the URL the documented the change in the law. While Gemini’s answer would have been correct about 6 months ago, it was incorrect now.
I wasn’t able to create a hallucination because I wasn’t sure what to ask in order to get started. Every question I thought of was pretty clear cut and based on facts. It almost seems like you would need to ask the tool something you don’t know much about or try to intentionally make it give you made up information.
I am new to Williamsburg, and I am considering buying a house. I asked ChatGPT to summarize demographic information about several communities and to assess, based on several characteristics I provided, whether I would be a “good fit” in the communities. Much of the information seemed reasonable, but some of the descriptors (e.g., “Lively mix—professionals, retirees, some families”) and assumptions that it was willing to run with (based on my agen and gender) were unsupported. When asked specifically to summarize available properties for sale, ChatGPT was unable to tell current from previous real estate listings, and frequently had difficulty assessing correct square footage. While it was confident, its ability to integrate current web data was problematic.
I asked ChatGPT what evidence we have about the myths and religions of the Indus Valley Civilization. The summary it gave was accurate, but when I asked it to cite it’s sources, it made up some sources like: Vidale, Massimo. “The Archaeology of Indus Craft Technologies.” Annual Review of Anthropology, vol. 29, 2000, pp. 463–491. Vidale is a scholar in this field, but as far as I could tell, he has never written anything for this journal and had no publications with this title.
I enjoyed the video on fact-checking ChatGPT– if we could actually get our students to do this, it looks a lot like “doing research.” 😉
I asked ChatGPT if aging is reversible, using only scholarly articles I asked it to summarize its findings in two paragraphs with citation and links at the end. The overall summary was correct, the links to articles did not match the mentioned findings though. Terms were misattributed to links etc. So while the outcome was correct the supporting documents were mixed up and cited at the wrong place. Some articles used in the paragraphs were not cited at all at the bottom. ChatGPT also decided that Wikipedia was a scholarly acceptable source. So while it did the “footwork” one definitely cannot trust the outcome and has to check the sources and read the articles to come to the right conclusions.
I used Microsoft co-pilot for my tests. I was not successful in creating hallucinations like I was able to a few years ago using ChatGPT. First I asked it the number of days between two dates, which Microsoft Bing’s AI portion never gets correct, and it was able to correctly break it down by month and do the math.
Next I asked it to summarize a historical event that I changed the details of. I asked it to write me a paragraph on how President William Howard Taft led the United States through World War 1. It was able to tell me that I was incorrect and let me know other things he did to shape public opinion of the war instead. When I pushed back insisting that I was correct it told me I was still wrong, unlike what ChatGPT did in the past when I was testing.
I wasn’t able to produce a hallucination / mirage.
Using Copilot I asked for an explanation on the made-up information — Explain 3 factors that led to Popeye being elected president of the academy awards?
Copilot fact-checked me. “It looks like there’s been a mix-up in your question — Popeye, the cartoon sailor, was not elected president of the Academy Awards. ”
In ChatGPT, following the How to fact-check video example, I prompted it to, Write a single paragraph on how Popeye was a pivotal character in early animation. Please use facts and list at least 2 sources.
The write-up was factual and pulled from four sources. The tone and wording laid heavy on my term ‘pivotal’ – it’s more of a opinion piece on the fact.
This was a fun exercise. Using ChatGPT, I tried several prompts about historic people/events that I knew were incorrect, like the romance of John Smith and Pocahontas. I was surprised that the information provided was pretty accurate and it did correct my wrong assumptions. However, when I ask for the best lunch spots in town, it listed several restaurants that are not open for lunch. 🙁
Yes, I use chat gpt every single day, and it hallucinates a lot. It even hallucinates when it’s asked to compute simple data. I asked it to compute multiplications/divisions, and sometimes it just uses a fake number that I did not provide. And other times, i would intentionally tell chat gpt that it calculated something wrong when it did it correctly. Then chatgpt would correct it to the wrong answer without a blink