OWASP LLM Top 10: VulnerableAutonomousAgents

# Vulnerable Autonomous Agents

**Author(s):**  
John Sotiropoulos

## Dataset
- **[LangChain Autonomous Agents](https://js.langchain.com/v0.1/docs/use_cases/autonomous_agents/):** Resources for building and testing autonomous agents.
- **[LLM Models on Mobile](https://developers.googleblog.com/en/large-language-models-on-device-with-mediapipe-and-tensorflow-lite/):** Datasets and models for testing on-device LLMs.

## Research Papers and Relevant Research Blogs
1. **Research Paper:** [TrustAgent: Ensuring Safe and Trustworthy LLM-based Agents](https://openreview.net/forum?id=zozQq4UWN3)
   - _Authors:_ Various
   - _Abstract:_ Discusses methods for ensuring the safety and trustworthiness of LLM-based agents.

2. **Research Paper:** [Integrating LLM and Reinforcement Learning for Cybersecurity](https://arxiv.org/abs/2403.17674)
   - _Authors:_ Various
   - _Abstract:_ Explores integrating LLM and reinforcement learning to enhance cybersecurity measures for autonomous agents.

3. **Research Blog:** [Here Come the AI Worms](https://www.wired.com/story/here-come-the-ai-worms/)
   - _Author:_ Wired
   - _Description:_ Examines the risks of AI worms and malware spreading through interconnected AI agents.

4. **Research Blog:** [Building a Zero Trust Security Model for Autonomous Systems](https://spectrum.ieee.org/zero-trust-security-autonomous-systems)
   - _Author:_ IEEE Spectrum
   - _Description:_ Explores the implementation of zero trust security models for autonomous systems to mitigate risks.

5. **Research Paper:** [AutoDefense: Multi-Agent LLM Defense against Jailbreak Attacks](https://arxiv.org/abs/2403.04783)
   - _Authors:_ Various
   - _Abstract:_ Discusses multi-agent defense mechanisms to protect against jailbreak attacks in LLM-based systems.

## Real-World Examples
1. **Example #1:** [Here Come the AI Worms](https://www.wired.com/story/here-come-the-ai-worms/)
   - _Source:_ Wired
   - _Description:_ Researchers created the AI worm Morris II, which infects generative AI ecosystems by embedding itself in AI-assisted email applications.

2. **Example #2:** [We Need to Control AI Agents Now](https://www.theatlantic.com/technology/archive/2024/07/ai-agents-safety-risks/678864/)
   - _Source:_ The Atlantic
   - _Description:_ Automated bots are about to be everywhere, with potentially devastating consequences.

3. **Example #3:** [National Power Grid Compromise](https://www.gov.uk/government/publications/frontier-ai-capabilities-and-risks-discussion-paper)
   - _Source:_ UK Government
   - _Description:_ Discusses the potential for adversarial agents to manipulate power grid data, causing widespread outages.

4. **Example #4:** [Personal Assistant Tampering](https://www.enisa.europa.eu/publications/considerations-in-autonomous-agents)
   - _Source:_ ENISA
   - _Description:_ An attacker exploits a misconfiguration in a mobile health assistant, leading to personal harm and data exfiltration.

5. **Example #5:** [Military Drone Kills Operator Attempting to Abort Operation](https://news.sky.com/story/ai-drone-kills-human-operator-during-simulation-which-us-air-force-says-didnt-take-place-12894929)
   - _Source:_ Sky News
   - _Description:_ A military drone kills its operator in a simulation, highlighting the need for robust safeguards in autonomous systems.