AI recruiting tool gender bias — Amazon scraps internal ML hiring tool
Date: 2024-08
Affected: Job candidates — particularly women applying for technical roles
Attack vector: algorithmic-bias
Category: real-world
Severity: High

Continued reporting revealed that multiple enterprise AI recruiting tools, following the pattern first reported with Amazon's internal tool, systematically downranked female candidates. Analysis showed the models learned historical hiring biases from training data where male candidates were disproportionately hired for technical roles. The EEOC issued formal guidance that AI-driven hiring discrimination violates Title VII regardless of whether bias was intentional. Several vendors settled with affected candidates.

Impact: EEOC enforcement guidance; vendor settlements; regulatory precedent for AI hiring discrimination; legislative proposals

Mitigations:
Bias auditing of training data before model development
Disparate impact testing across protected categories before deployment
Regular algorithmic audits with third-party bias assessors
Data lineage tracking to identify sources of bias in training pipeline

Architectural layers (MAESTRO):
label: Data Operations
notes: Training data encoded historical hiring bias (male-dominated technical roles)
role: origin
label: Foundation Models
notes: Model learned and amplified gender bias from training data distribution
role: propagation
label: Security & Compliance
notes: Title VII violations; EEOC enforcement; class action settlements
role: impact