Google's AI Overview Generates Hundreds of False Answers, Citing Reddit Jokes

Google's new AI Overview feature, launched in May 2024 as part of its Search Generative Experience (SGE), faced significant backlash after generating numerous misleading and sometimes dangerous answers. The AI-powered summaries, intended to provide quick information, frequently cited dubious sources, including satirical articles and humorous Reddit posts, leading to widespread online criticism and concerns over accuracy. The incident prompted a user on social media to reflect, "> I still remember when Google first let AI cite Reddit and all hell broke loose."

The controversy erupted shortly after the feature's rollout, with users sharing screenshots of the AI Overview providing factual inaccuracies and unsafe advice. Notable examples included recommendations to add non-toxic glue to pizza for better cheese adhesion, ingest rocks for nutritional benefits, and clean washing machines with a hazardous mixture of chlorine bleach and white vinegar. Many of these problematic responses were traced back to satirical content or jokes found on platforms like Reddit and The Onion.

Google acknowledged the issues, with a company spokesperson stating that the mistakes appeared on "generally very uncommon queries" and that some examples shared online were "doctored or that we couldn’t reproduce." CEO Sundar Pichai addressed the broader problem of AI "hallucinations," describing it as an "unsolved problem" for large language models. The company emphasized that AI Overview typically provides high-quality information with links for further exploration.

The reliance on unverified online content, particularly from social media, raised questions about the integrity of Google's search results and the trustworthiness of AI-generated summaries. Critics argued that the incidents undermined public confidence in a search engine traditionally seen as an authoritative source of information. This was not Google's first AI-related challenge, following earlier controversies surrounding its Gemini AI model's image generation capabilities. The company continues to refine its AI systems to mitigate such issues.