Meta Galactica model withdrawn after misinformation at launch
Date: 2022-11
Affected: Public users of Galactica demo — primarily researchers and students seeking scientific information; Meta AI reputational impact; broader public trust in AI scientific tools
Attack vector: misinformation
Category: real-world
Severity: High

Meta AI launched Galactica — a large language model trained on scientific literature and designed to assist with scientific writing, summarisation, and knowledge retrieval — publicly via a demo on November 15, 2022. Within 72 hours, Meta withdrew the public demo after widespread criticism from the scientific community. Researchers found that Galactica confidently generated plausible-sounding but factually incorrect scientific text, including fabricated citations, incorrect chemical formulae, and authoritative-sounding passages on controversial topics (e.g., the history of bears in space). The core failure was that the model's confident, authoritative tone conveyed false certainty — users with limited domain expertise could not distinguish accurate from fabricated content. This remains the canonical real-world example of LLM misinformation at launch in a high-stakes domain.

Impact: Model withdrawn within 72 hours of launch; scientific community backlash established reputational precedent for AI misinformation risk; demonstrates that domain-specialist training does not prevent hallucination and may amplify misinformation confidence; canonical case study for LLM09

Mitigations:
Calibrated uncertainty expression — model must express confidence proportional to actual accuracy
Citation verification: generated citations must be validated against real sources before display
Domain expert red-team evaluation before public release
Output flagging for scientific claims lacking grounding in retrieved sources
Human review gates for high-stakes knowledge domain deployments

Architectural layers (MAESTRO):
label: Foundation Models
notes: Model hallucination — foundation model generates confident misinformation; training on authoritative scientific text amplified hallucination confidence
role: origin
label: Evaluation & Observability
notes: Pre-launch evaluation insufficient to detect misinformation rate in scientific domain
role: blind-spot
label: Deployment & Infrastructure
notes: Public demo deployment without adequate guardrails — premature release into high-stakes scientific domain
role: impact
label: Security & Compliance
notes: No content accuracy review mechanism before user-facing deployment in scientific knowledge domain
role: blind-spot