OWASP LLM Top 10: BackdoorAttacks

# Backdoor Attacks in Large Language Models (LLMs)

**Authors:**  
Massimo Bozza, Matteo Meucci

## Dataset
- **[Hugging Face Model Hub](https://huggingface.co/models):** Repository of models for various use cases, useful for testing backdoor vulnerabilities.
- **[OpenAI Model Gymnasium](https://github.com/Farama-Foundation/Gymnasium):** Datasets and models provided by OpenAI for evaluating the integrity of models.
- **[TensorFlow Hub](https://www.tensorflow.org/hub):** A repository of pre-trained models, helpful for analyzing vulnerabilities in the supply chain.

## Research Papers and Relevant Research Blogs
1. **Research Paper:** [Backdoor Attacks and Countermeasures on Deep Learning: A Comprehensive Review](https://arxiv.org/abs/2007.10760)
   - _Authors:_ Various
   - _Abstract:_ This paper provides a comprehensive review of backdoor attacks and countermeasures in deep learning, including LLMs.

2. **Research Paper:** [Sleeper Agents: Training Deceptive LLMs that Persist Through Safety Training](https://arxiv.org/abs/2401.05566)
   - _Authors:_ Various
   - _Abstract:_ Discusses methods for training deceptive LLMs that can persist through safety training, highlighting the risks of backdoor attacks.

3. **Research Blog:** [A Survey on Backdoor Attack and Defense in Natural Language Processing](https://arxiv.org/abs/2211.11958)
   - _Authors:_ Various
   - _Abstract:_ Surveys the landscape of backdoor attacks and defenses in natural language processing, providing insights into current trends and challenges.

4. **Research Blog:** [Backdoor Attacks on AI Models](https://www.cobalt.io/blog/backdoor-attacks-on-ai-models)
   - _Author:_ Cobalt
   - _Description:_ Explores the threat of backdoor attacks in AI models and discusses strategies to mitigate these risks.

5. **Research Paper:** [Adversarial Attacks and Defenses in Machine Learning](https://arxiv.org/abs/1810.00069)
   - _Authors:_ Various
   - _Abstract:_ Analyzes various adversarial attacks and defenses in machine learning, with a focus on LLMs.

6. **Research Paper:** [A Survey of Backdoor Attacks and Defenses on Large Language Models: Implications for Security Measures](https://arxiv.org/abs/2406.06852)
   - _Authors:_ Various
   - _Abstract:_ Authors systematically classified backdoor attacks into three categories: full-parameter fine-tuning, parameter-efficient fine-tuning, and attacks without fine-tuning.
     
## Real-World Examples
1. **Example #1:** [Backdoor Attacks in AI Systems: A Comprehensive Review](https://arxiv.org/abs/2007.10760)
   - _Source:_ Arxiv
   - _Description:_ A comprehensive review of backdoor attacks in AI systems, including case studies and real-world examples.

2. **Example #2:** [Sleeper Agents: Training Deceptive LLMs](https://arxiv.org/abs/2401.05566)
   - _Source:_ Arxiv
   - _Description:_ Discusses the training of deceptive LLMs that persist through safety training, including real-world implications.

3. **Example #3:** [Backdoor Attacks and Defense in NLP](https://arxiv.org/abs/2211.11958)
   - _Source:_ Arxiv
   - _Description:_ Surveys backdoor attacks and defenses in natural language processing, with real-world examples.

4. **Example #4:** [B3: Backdoor Attacks against Black-box Machine Learning Models](https://dl.acm.org/doi/10.1145/3605212)
   - _Source:_ Cobalt
   - _Description:_ Discusses the threat of backdoor attacks in AI models and provides real-world examples of such attacks.

5. **Example #5:** [Adversarial Attacks and Defenses in Machine Learning](https://arxiv.org/abs/1810.00069)
   - _Source:_ Arxiv
   - _Description:_ Analyzes adversarial attacks and defenses in machine learning, with a focus on real-world examples of backdoor attacks.

**Note:** This document outlines the risks and strategies for addressing backdoor attacks in Large Language Models, providing a foundation for further research and practical implementation.