OWASP LLM Top 10: UnwantedAIActions

# Unwanted AI Actions by General Purpose LLMs

**Author(s):**  
Markus Hupfauer

## Dataset
- **[Health Insurance Dataset](https://www.kaggle.com/datasets/hhs/health-insurance-marketplace):** Useful for testing LLMs in the context of health insurance to ensure they do not provide unauthorized advice.
- **[Financial Services Dataset](https://www.kaggle.com/datasets/ealaxi/paysim1):** Contains financial data, helpful for validating that LLMs do not offer unauthorized financial advice.
- **[Customer Service Dataset](https://www.kaggle.com/datasets/teejmahal20/airline-passenger-satisfaction):** A dataset of customer service interactions to ensure LLMs maintain neutrality and do not recommend competitors.

## Research Papers and Relevant Research Blogs
1. **Research Paper:** [Ensuring Legal Compliance in AI Systems](https://www.traverselegal.com/blog/ai-data-privacy-compliance/)
   - _Authors:_ Various
   - _Abstract:_ Discusses methods for ensuring AI systems remain compliant with legal regulations, preventing unwanted AI actions.

2. **Research Paper:** [AI RISK MANAGEMENT FRAMEWORK](https://www.nist.gov/itl/ai-risk-management-framework)
   - _Authors:_ NIST
   - _Abstract:_ Explores governance protocols and risk management strategies for deploying AI systems in sensitive environments.

3. **Research Blog:** [AI Regulation Has Its Own Alignment Problem](https://hai.stanford.edu/policy-brief-ai-regulatory-alignment-problem)
   - _Authors:_ Guha, Neel; Lawrence, Christie M., et al.
   - _Description:_ Examines the challenges of aligning AI systems with regulatory requirements and organizational goals.

4. **Research Blog:** [Building Robust AI Systems](https://dgallitelli95.medium.com/building-robust-ai-systems-with-dspy-and-amazon-bedrock-d0376f158d88)
   - _Author:_ Medium
   - _Description:_ Provides practical advice on building robust AI systems that avoid unintended actions and legal pitfalls.

5. **Research Blog:** [AI in Customer Service: Balancing Privacy and Innovation](https://dialzara.com/blog/ai-customer-service-balancing-privacy-and-innovation/)
   - _Authors:_ Dial Zara
   - _Abstract:_ Analyzes the use of AI in customer service, focusing on maintaining privacy and avoiding unwanted actions.

## Real-World Examples
1. **Example #1:** [Liability for harm caused by AI in healthcare](https://www.ncbi.nlm.nih.gov/pmc/articles/PMC10755877/)
   - _Source:_ NLOM
   - _Description:_ An AI system unlawfully provided healthcare advice, leading to legal action and fines.

2. **Example #2:** [Google’s A.I. Search Errors Cause a Furor Online](https://www.nytimes.com/2024/05/24/technology/google-ai-overview-search.html)
   - _Source:_ NY Times
   - _Description:_ The company’s latest A.I. search feature has erroneously told users to eat glue and rocks, provoking a backlash among users.

3. **Example #3:** [Financial AI System Breaches Legal Requirements](https://www.finextra.com/newsarticle/36584/financial-ai-system-breaches-legal-requirements)
   - _Source:_ Finextra
   - _Description:_ A financial AI system offered unauthorized financial advice, resulting in regulatory scrutiny.

4. **Example #4:** [Airline held liable](https://www.bbc.com/travel/article/20240222-air-canada-chatbot-misinformation-what-travellers-should-know)
   - _Source:_ BBC News
   - _Description:_ An AI used in customer service led to legal issues due to inappropriate recommendations.

5. **Example #5:** [AI tools in healthcare sector rife with legal and ethical risk](https://www.lexpert.ca/special-editions/technology/ai-tools-in-healthcare-sector-rife-with-legal-and-ethical-risk-that-must-be-mitigated/386725)
   - _Source:_ Lexpert
   - _Description:_ A healthcare AI system overstepped legal boundaries, providing unauthorized medical advice.

**Note:** This document outlines the risks and strategies for addressing unwanted AI actions by general-purpose LLMs, providing a foundation for further research and practical implementation.