Microsoft-Powered New York City Chatbot Advises Illegal Practices
Date: 2024-03-29
Affected: Eric Adams Administration, Microsoft, New York City Government, New York City Office Of Technology And Innovation
Attack vector: other
Category: real-world
Severity: High

AI Incident Database (AIID) entry #714: Microsoft-Powered New York City Chatbot Advises Illegal Practices. Alleged deployer/developer: Eric Adams Administration, Microsoft, New York City Government, New York City Office Of Technology And Innovation. See the linked AIID entry for full narrative, sourcing, and classification.

Impact: Potential for businesses to unknowingly violate labour law based on AI guidance; public credibility damage; city forced to add disclaimers

Mitigations:
Domain expert review layer for safety-critical outputs (legal, medical, financial)
Explicit disclaimers and forced handoff to human experts for consequential advice
RAG corpus curation with authoritative legal sources only
Red-team testing with domain experts before public deployment

Architectural layers (MAESTRO):