AI Hallucination Rates Reach Over 80% in Some Clinical Scenarios, Raising Concerns About Factual Accuracy

Recent observations by Dr. Bill Mitchell, a Fellow of BCS, the UK's Chartered Institute for Information Technology, highlight a significant challenge in the rapidly evolving field of artificial intelligence: the tendency of AI models to "embellish" or "hallucinate" information. Dr. Mitchell noted in a recent social media post, "> "Be very very careful working with AI. Recently when AI tells me something that seems outlandish, I will ask, is this story real or did you embellish it? Easily 80% of the time, it will say that there were some facts but it embellished the rest or that it was based upon fiction." This candid assessment underscores a growing concern regarding the factual reliability of AI-generated content.

AI hallucination refers to instances where large language models (LLMs) produce false or misleading information, presenting it as fact with high confidence. Studies indicate a wide range of hallucination rates, depending on the model and task. For example, a recent Nature study found that LLMs repeated or elaborated on fabricated clinical details in 50% to 82% of outputs, while another report showed Bard (now Gemini) hallucinated in 91.4% of cases when generating scientific references. Overall, 77% of businesses surveyed by Deloitte express concern over this issue.

The root causes of these AI "embellishments" are multi-faceted. They often stem from limitations in training data, which can be insufficient, low-quality, biased, or outdated, leading models to invent facts rather than admit ignorance. LLMs are also designed to generate fluent and coherent text, sometimes prioritizing linguistic plausibility over factual accuracy. Furthermore, internal inference dynamics and the practice of training models on synthetic (AI-generated) data can lead to a degradation of quality and diversity, potentially resulting in "model collapse" or "Habsburg AI."

The real-world implications of AI hallucination are substantial, impacting trust and potentially leading to legal and ethical dilemmas. Instances include chatbots fabricating legal citations, as seen in a U.S. court case where lawyers were sanctioned for relying on AI-generated, non-existent precedents. Similarly, Air Canada was held liable for a chatbot's incorrect policy information, highlighting the legal risks when AI systems provide false data. In academic research, AI has been observed to generate fabricated sources, posing challenges to scientific integrity.

In response, researchers and developers are actively pursuing various mitigation strategies. These include Retrieval Augmented Generation (RAG) systems, which ground AI responses in verified external knowledge bases, and advanced prompt engineering to guide models toward more accurate outputs. Efforts are also underway to develop mechanisms for AI models to communicate their uncertainty, allowing users to better gauge the reliability of the information received. Despite these advancements, the challenge of fully eliminating AI hallucination remains a complex and ongoing area of research.