Google AI Overviews recommends adding glue to pizza — RAG hallucination at search scale
Date: 2024-05
Affected: Google Search — AI Overviews shown to billions of users globally
Attack vector: hallucination
Category: real-world
Severity: High

Google's AI Overviews feature, which provides AI-generated summaries at the top of search results, recommended adding non-toxic glue to pizza sauce to make cheese stick better. The source was a satirical Reddit comment from 11 years ago that the RAG system retrieved and presented as factual advice. Other AI Overviews hallucinations included recommending eating rocks for minerals and suggesting Barack Obama was Muslim.

Impact: Viral embarrassment; question of AI reliability at web scale; reduced AI Overview visibility; Google added more guardrails

Mitigations:
Source quality scoring for RAG corpus — deprioritise unverified UGC
Factual consistency verification layer between retrieval and generation
Domain-specific safety checks for health, legal, and safety topics
Staged rollout with monitoring before global deployment

Architectural layers (MAESTRO):
label: Data Operations
notes: RAG corpus included satirical/unverified user-generated content without quality filtering
role: origin
label: Foundation Models
notes: Model failed to distinguish satirical from factual content in retrieved passages
role: propagation
label: Evaluation & Observability
notes: No content quality validation between retrieval and generation
role: blind-spot