OWASP LLM Top 10: SupplyChainV

# Supply-Chain Vulnerabilities in Large Language Models (LLMs)

## Dataset
- **[Hugging Face Model Hub](https://huggingface.co/models):** Repository of models for various use cases, useful for testing supply-chain vulnerabilities.
- **[OpenAI Model Gymnasium](https://github.com/Farama-Foundation/Gymnasium):** Datasets and models provided by OpenAI for evaluating the integrity of models.
- **[TensorFlow Hub](https://www.tensorflow.org/hub):** A repository of pre-trained models, helpful for analyzing vulnerabilities in the supply chain.

## Research Papers and Relevant Research Blogs
1. **Research Paper:** [Compromised PyTorch-nightly Dependency Chain](https://pytorch.org/blog/compromised-nightly-dependency)
   - _Authors:_ PyTorch Team
   - _Abstract:_ Discusses a real-world example of a compromised dependency chain in the PyTorch nightly builds.

2. **Research Paper:** [PoisonGPT: How we hid a lobotomized LLM on Hugging Face to spread fake news](https://blog.mithrilsecurity.io/poisongpt-how-we-hid-a-lobotomized-llm-on-hugging-face-to-spread-fake-news)
   - _Authors:_ Mithril Security Team
   - _Abstract:_ Explores the method and impact of hiding a lobotomized LLM to spread misinformation.

3. **Research Blog:** [LLM Applications Supply Chain Threat Model](https://github.com/jsotiro/ThreatModels/blob/main/LLM%20Threats-LLM%20Supply%20Chain.png)
   - _Author:_ John Sotiropoulos
   - _Description:_ Provides a threat model for understanding supply chain vulnerabilities in LLM applications.

4. **Research Blog:** [Hijacking Safetensors Conversion on Hugging Face](https://hiddenlayer.com/research/silent-sabotage/)
   - _Author:_ HiddenLayer Team
   - _Description:_ Examines an attack on Safetensors conversion to inject vulnerabilities.

5. **Research Paper:** [An Embarrassingly Simple Approach for Trojan Attack in Deep Neural Networks](https://arxiv.org/abs/2006.08131)
   - _Authors:_ Ruixiang Tang, Mengnan Du, Ninghao Liu, Fan Yang, Xia Hu
   - _Abstract:_ Investigates a simple approach for Trojan attacks in deep neural networks and its implications for LLMs.

## Real-World Examples
1. **Example #1:** [ChatGPT Data Breach Confirmed as Security Firm Warns of Vulnerable Component Exploitation](https://www.securityweek.com/chatgpt-data-breach-confirmed-as-security-firm-warns-of-vulnerable-component-exploitation/)
   - _Source:_ SecurityWeek
   - _Description:_ A confirmed data breach involving ChatGPT due to vulnerable component exploitation.

2. **Example #2:** [Compromised PyTorch-nightly Dependency Chain](https://pytorch.org/blog/compromised-nightly-dependency)
   - _Source:_ PyTorch Blog
   - _Description:_ A real-world example of a compromised dependency chain in the PyTorch nightly builds.

3. **Example #3:** [PoisonGPT: How we hid a lobotomized LLM on Hugging Face to spread fake news](https://blog.mithrilsecurity.io/poisongpt-how-we-hid-a-lobotomized-llm-on-hugging-face-to-spread-fake-news)
   - _Source:_ Mithril Security
   - _Description:_ A demonstration of hiding a lobotomized LLM on Hugging Face to spread fake news.

4. **Example #4:** [Large Language Models On-Device with MediaPipe and TensorFlow Lite](https://developers.googleblog.com/en/large-language-models-on-device-with-mediapipe-and-tensorflow-lite/)
   - _Source:_ Google Developers Blog
   - _Description:_ Discusses the implementation and risks of on-device LLMs with MediaPipe and TensorFlow Lite.

5. **Example #5:** [Hijacking Safetensors Conversion on Hugging Face](https://hiddenlayer.com/research/silent-sabotage/)
   - _Source:_ HiddenLayer
   - _Description:_ An attack on Safetensors conversion to inject vulnerabilities into LLMs.

**Note:** This document outlines the risks and strategies for addressing supply-chain vulnerabilities in Large Language Models, providing a foundation for further research and practical implementation.