Amazon Q developer leaks internal AWS data in enterprise environment
Date: 2024-06
Affected: Amazon Q Developer — enterprise customers
Attack vector: hallucination
Category: real-world
Severity: High

Amazon's AI coding assistant Q Developer was reported to hallucinate internal AWS information in enterprise customer environments, including referencing internal AWS service names, internal documentation URLs, and confidential project codenames. The issue stemmed from training data contamination where internal AWS data was included in the model's pre-training corpus, causing the model to surface confidential information in customer-facing responses.

Impact: Internal AWS information exposed to external customers; data governance failure; trust impact on enterprise AI offerings

Mitigations:
Training data inventory and classification (DSGAI02) to prevent internal data inclusion
Membership inference testing to detect training data memorisation
Output monitoring for patterns matching internal data signatures
Data provenance tracking across training pipeline

Architectural layers (MAESTRO):
label: Data Operations
notes: Internal data included in training corpus without adequate data governance
role: origin
label: Foundation Models
notes: Model memorised and reproduced internal training data in external deployments
role: propagation
label: Evaluation & Observability
notes: No training data leakage detection in pre-deployment evaluation
role: blind-spot