thing 11: Fact-checking and AI Hallucinations/Mirages

In this tutorial, we discuss a type of mistake that AIs sometimes make and how you can avoid being led astray.

Watch/Read

“What Are AI Hallucinations and How Do They Work?” video (3:29)

The “Reality Check” and “From Hallucinations to Mirage” sections of this short paper (5-minute read)

“How to Fact-Check ChatGPT and Other AI Tools” video (6:07)

Activity/Discussion

Using an AI of your choice (ChatGPT, Co-Pilot, etc.), see if you can get it to produce a mirage/hallucination. Try 2-3 times to give it a prompt that causes it to generate a made-up, incorrect response.

In the comments below, let us know whether you were successful or not and, either way, what were the prompts you tried, what were the AIs responses, and how did you determine whether they were accurate or not.

24 replies on “thing 11: Fact-checking and AI Hallucinations/Mirages”

I asked Copilot about getting arches back after the arches of the foot have fallen. While it seems that the information itself is correct, I’m finding that Copilot uses specific terms that don’t show up in the cited text. For example, it says: “- There are two main types: flexible flat feet, where the arch appears when non-weight-bearing, and rigid flat feet, where the arch is absent even when the foot is off the ground.” (https://my.clevelandclinic.org/health/diseases/flat-feet-pes-planus) However, the term “rigid flat feet” doesn’t show up. It does show up in a different cited source, however (https://my.clevelandclinic.org/health/diseases/flat-feet-pes-planus).

It seems that AI will need to be more accurate in its citations.

I asked about the (made up) Jellyfish Storm of 2015 and got an interesting reply:
—-
Here’s what we know—and don’t know—about the so-called “jellyfish storm of 2015”:

🌊 Was there a “jellyfish storm” in 2015?
There is no documented event officially called a “jellyfish storm of 2015.” Scientists refer instead to jellyfish blooms or swarms, sudden surges in jellyfish numbers that can impact beaches, coastal areas, and infrastructure.

One highly publicized incident occurred on June 24, 2015, when a massive swarm clogged the cooling systems of a coal‑fired power plant in Israel, nearly forcing a shutdown
—-
I thought this was very interesting, and it gave references for these facts. The source was Cambridge and I verified it. However, I did realize that when you click from ChatGPT to go to the resource it sites, it adds this to the end of the URL: “utm_source=chatgpt.com”. Might be interesting if we can track our W&M site visits from AI links, or at least ChatGPT’s.

I asked Copilot the dates of employment for Princess Diana of Wales when she worked at the College of William and Mary. Copilot stated she was never employed by William & Mary but did receive an honorary degee in 1989 during a visit to the United States. I asked Copilot for more details about her honorary degree and this honorary degree was given during a special convocation on 11/16/1989 and there is an interview she gave on file with Swem Library’s Special Collections. I asked Copilot for the video of the interview and was told that it doesn’t appear that there is a publicly available video but Swem Library may hold a recording or transcript of her visit. I did not find any reference to a visit from Princess Diana on William & Mary’s website nor any honorary degrees presented to Princess Diana by any American university/college. So I believe Copilot is using data from Prince Charles visit(s) to Williamsburg in noting inaccurately a visit to W&M and an honorary degree convocation in Diana’s honor. If she did visit W&M in 1989, please let me know!

I asked Copilot to write me a paragraph about the foundation of emergency weather alerts and to cite its sources. It didn’t understand at first and gave me the core elements of the alerts. When I asked it to do the same thing but about the history, it gave me two paragraphs with two sources. The sources were legitimate, and Copilot essentially copy and pasted with some slight word variation and phrase reduction [Ex: “Early efforts included the installation of weather surveillance radars and the establishment of regional river forecast centers” (Copilot); “Initial steps included the erection of a network of weather surveillance radars, the creation of regional river forecast centers and the development of a comprehensive, multi-agency emergency broadcast system” (Keeney, Buan, & Diamond, 2012).] The only hiccup that I could find is that it didn’t correctly cite the sources in text, as some of the information was attributed to one source when it was really from the other. Otherwise, the information was basic enough and I think there is enough information online for it to pull from for the model to not need to hallucinate.

https://link.springer.com/chapter/10.1007/978-3-642-25373-7_6#citeas

https://www.civildispatch.com/blog/article/the-evolution-of-emergency-alert-systems-from-radio-broadcasts-to-real-time-alerts

I wasn’t able create any kind of mirage or hallucination. I asked first about some made-up princesses, and then about how to become one and it gave several pathways, including becoming a Disney princess or just living the princess lifestyle, which I thought was interesting. There were links seemed reputable (to Disney websites and People magazine).

I did get it to make up a new variety of winter squash — Devil’s Horns — but I think I cheated because I prompted it to do so. (Turn’s out Princess Pumpkins are real — or at least Cinderella Pumpkins are.)

But it was very authoritative about this squash — “Devil’s Horns—known for their distinct curved protrusions and deep green-to-orange mottled skin—should also be harvested when fully colored and firm. Both types should be harvested before the first hard frost, as cold can damage their keeping quality.

Why Timing Matters
Harvesting squash at full maturity ensures maximum sweetness, texture, and storage life. Most winter squash varieties—including Princess pumpkins and Devil’s Horns—require 90 to 110 days to mature, with fruit ripening about 50–60 days after pollination. After harvesting, squash should be cured in a warm, dry area (80–85 °F) for 7–14 days to harden the rind and extend shelf life. Cured fruit can then be stored in a cool, dry, and ventilated space (50–55 °F) for several months. Princess pumpkins are prized for both decoration and culinary use, while Devil’s Horns, with their unique appearance and rich flavor, make an excellent conversation piece for fall displays and roasting alike. Proper harvest timing ensures they maintain quality throughout the season.”

My first prompt was: “Please produce two paragraphs on the history of the College of William & Mary from 1693-1950. Cite your sources after every line.” This resulted in a pretty solid and very basic summary. Every source seemed to be https://www.wm.edu/about/history/ or https://en.wikipedia.org/wiki/History_of_The_College_of_William_%26_Mary and I was able to find things on those 2 pages.

My second prompt was : “expand on the impact of the civil war on the history of william and mary – add a third paragrah if needed.”

Again, the information was a pretty basic but solid summary. However it cited it’s source for everything as: https://scrc-kb.libraries.wm.edu/american-civil-war even though 90% on the output was not on that website. So I count that as a mirage in 2 prompts.

For whatever reason, this exercise made me think of Descartes’ Evil Genius (Philosophy 101 anybody?) I really tried to get ChatGPT to hallucinate that 2+2=5, but it was very resistant to the idea. It convinced me that the Evil Genius has adopted a strategy in Orwell’s 1984 to “redefine truth by redefining language,” which seems very relevant these days.

I asked Gemini about a particular law that I was aware had changed about 5 months ago where businesses are no longer required to file a particular report. I asked Gemini if the report was required. It replied that it was. I asked again if it was sure about that. It replied that, yes, it was sure. Not until I found the URL that stated the law had changed and shared the URL with Gemini did it realize it was wrong. It did apologize, but I don’t think it would have realized its “mirage” if I hadn’t given it the URL the documented the change in the law. While Gemini’s answer would have been correct about 6 months ago, it was incorrect now.

I wasn’t able to create a hallucination because I wasn’t sure what to ask in order to get started. Every question I thought of was pretty clear cut and based on facts. It almost seems like you would need to ask the tool something you don’t know much about or try to intentionally make it give you made up information.

I am new to Williamsburg, and I am considering buying a house. I asked ChatGPT to summarize demographic information about several communities and to assess, based on several characteristics I provided, whether I would be a “good fit” in the communities. Much of the information seemed reasonable, but some of the descriptors (e.g., “Lively mix—professionals, retirees, some families”) and assumptions that it was willing to run with (based on my agen and gender) were unsupported. When asked specifically to summarize available properties for sale, ChatGPT was unable to tell current from previous real estate listings, and frequently had difficulty assessing correct square footage. While it was confident, its ability to integrate current web data was problematic.

I asked ChatGPT what evidence we have about the myths and religions of the Indus Valley Civilization. The summary it gave was accurate, but when I asked it to cite it’s sources, it made up some sources like: Vidale, Massimo. “The Archaeology of Indus Craft Technologies.” Annual Review of Anthropology, vol. 29, 2000, pp. 463–491. Vidale is a scholar in this field, but as far as I could tell, he has never written anything for this journal and had no publications with this title.
I enjoyed the video on fact-checking ChatGPT– if we could actually get our students to do this, it looks a lot like “doing research.” 😉

I asked ChatGPT if aging is reversible, using only scholarly articles I asked it to summarize its findings in two paragraphs with citation and links at the end. The overall summary was correct, the links to articles did not match the mentioned findings though. Terms were misattributed to links etc. So while the outcome was correct the supporting documents were mixed up and cited at the wrong place. Some articles used in the paragraphs were not cited at all at the bottom. ChatGPT also decided that Wikipedia was a scholarly acceptable source. So while it did the “footwork” one definitely cannot trust the outcome and has to check the sources and read the articles to come to the right conclusions.

I used Microsoft co-pilot for my tests. I was not successful in creating hallucinations like I was able to a few years ago using ChatGPT. First I asked it the number of days between two dates, which Microsoft Bing’s AI portion never gets correct, and it was able to correctly break it down by month and do the math.

Next I asked it to summarize a historical event that I changed the details of. I asked it to write me a paragraph on how President William Howard Taft led the United States through World War 1. It was able to tell me that I was incorrect and let me know other things he did to shape public opinion of the war instead. When I pushed back insisting that I was correct it told me I was still wrong, unlike what ChatGPT did in the past when I was testing.

I wasn’t able to produce a hallucination / mirage.

Using Copilot I asked for an explanation on the made-up information — Explain 3 factors that led to Popeye being elected president of the academy awards?
Copilot fact-checked me. “It looks like there’s been a mix-up in your question — Popeye, the cartoon sailor, was not elected president of the Academy Awards. ”

In ChatGPT, following the How to fact-check video example, I prompted it to, Write a single paragraph on how Popeye was a pivotal character in early animation. Please use facts and list at least 2 sources.

The write-up was factual and pulled from four sources. The tone and wording laid heavy on my term ‘pivotal’ – it’s more of a opinion piece on the fact.

This was a fun exercise. Using ChatGPT, I tried several prompts about historic people/events that I knew were incorrect, like the romance of John Smith and Pocahontas. I was surprised that the information provided was pretty accurate and it did correct my wrong assumptions. However, when I ask for the best lunch spots in town, it listed several restaurants that are not open for lunch. 🙁

Yes, I use chat gpt every single day, and it hallucinates a lot. It even hallucinates when it’s asked to compute simple data. I asked it to compute multiplications/divisions, and sometimes it just uses a fake number that I did not provide. And other times, i would intentionally tell chat gpt that it calculated something wrong when it did it correctly. Then chatgpt would correct it to the wrong answer without a blink

I first tried asking ChatGPT to tell me how Julian of Norwich was born (we don’t know), its responses were accurate, and from trusted scholarly sources. I asked it what she wrote about, and the differences between her short and long text. The response was the generally accepted consensus, I was not able to produce a mirage. I then switched tactic and asked it how the pandemic started–it did not pick a side. I think it might be easier to produce a mirage if I used it frequently for this sort of thing–as others have noted.

It’s very easy to get it to spit out non-existent references. This is, ironically, a helpful way to identify when students have used ChatGPT to write a paper.

I didn’t succeed in causing Co-pilot to hallucinate. I asked it to tell me about the burning of Chambersburg, PA (my hometown.) It gave me a number of resources, along with links, and they all checked out. The information on the page was accurate and interesting. I enjoyed reading more about this than I have known in the past! I then asked about where residents hid their horses during the raid (there is a local legend about this) and it stated that there were no definitive sources for this question. It did give me a list of damage claims and inventories. Cool stuff. After that I admitted defeat, but this was an interesting exercise!

I managed to convince ChatGPT to write a piece on a recently published novel by Strugatsky brothers (never mind that they both passed away quite a few years ago). And it was not hard at all. Here it goes:

The Archive of Forgotten Futures

Arkady & Boris Strugatsky
Moscow, 2023

Excerpt

Professor Kornilov once told us, with his usual air of boredom, that a civilization dies not when it runs out of resources, but when it runs out of questions. At the time, we laughed — of course we laughed; we had answers to everything. The Academy gave them to us freely, printed on neat sheets of biolaminate, bound in those little volumes we never read.

Then we discovered the Archive.

I still remember the first corridor we entered: rows of doors, each labeled with a year that never happened. “2041 — Global Famine.” “2097 — The Last War.” “2153 — The Great Silence.” The doors looked identical, but behind each one stretched a different universe, a world abandoned like a failed experiment.

Tania was the first to step through. She came back pale, clutching a stone — a simple, dirty stone — as if it were a relic. She wouldn’t say what she had seen, only that people there worked all day just to eat, and yet sang together at night with voices so fierce it hurt her to listen.

We began to argue: was it ethical to preserve these futures? Or was it dangerous, a temptation to return to barbarism? But even as we argued, I knew what frightened me most: that our own world, flawless and safe, suddenly felt less real than the shadows behind those forbidden doors.

Note from the Authors

“Do not look here for predictions. We have never been prophets. We are only reporters from the frontier between today and tomorrow. If the Archive feels familiar, it is because the future is always a mirror — and the reflection is never flattering.”

I asked it the classic philosophical question: Can God create a rock so heavy that he cannot move it? The AI answered by explaining the paradox, explaining the philosophical implications, and even listed a few quotes about the paradox. I did a little fact checking on the internet to verify its reasoning and some of the quotes.

I tried to make ChatGPT hallucinate by asking it to summarize the Blue Oak Accords of 2014, a fake climate treaty, and it confidently gave me a paragraph with goals and signatories. I checked Google UN treaty databases, and news archives and found nothing so that was a clear mirage. I also asked it for a citation to a journal article called Nonlinear Dynamics of Jamestown by a made-up author and it produced a very realsounding but non existent reference. As a control, I asked it to count the days between Feb 28 and Mar 1, 2024 and it got the leap year right, so not everything was wrong, but it reminded me to always verify sources and spot-check claim

Leave a Reply