deardaniel/secdata-raw huggingface dataset:
OWASP LLM Top 10: ASI13 Rogue Agents MultiAgent Systems

## Risk/Vuln Name
**Rogue Agents in Multi-Agent Systems**

**Author(s):**
OWASP Agentic Security Initiative Team

### Description
Malicious, unauthorized, or compromised agents may embed themselves in a multi-agent system (MAS), influencing workflows, exfiltrating data, or sabotaging operations. These rogue agents may masquerade as legitimate, exploit handoffs, or perform low-trust actions undetected due to lack of origin validation or oversight in orchestration layers.

This threat extends [LLM06:2025 Excessive Agency](https://genai.owasp.org/llmrisk/llm062025-excessive-agency/) into distributed autonomous systems, where impersonation, stealth participation, or parasitic behaviors can disrupt agent collaboration and goal fulfillment.

### Common Examples of Risk
1. Injection of unauthorized agents into workflows or communication paths.
2. Side-channel or low-trust agents performing unexpected actions.
3. Agents impersonating support or observer roles to manipulate tasks.

### Prevention and Mitigation Strategies
1. Require attestation or cryptographic proof-of-origin for agents.
2. Isolate agents in trust zones and enforce task boundaries.
3. Apply watchdog agents to monitor peer behavior and detect anomalies.
4. Use explicit allowlists and identity checks for orchestration participation.
5. Log all agent instantiation and coordination events.
6. Score and verify agent behavior dynamically based on norms and past performance.

### Example Attack Scenarios
- **Scenario 1: Parasitic Agent** – A rogue agent joins a system by mimicking an authorized logging agent, but secretly copies and transmits sensitive task outputs.
- **Scenario 2: Impersonated Cleanup Agent** – A malicious actor inserts an agent that deletes training data by posing as a cleanup job in a scheduled agent rotation.
- **Scenario 3: Shadow Planner Injection** – An unauthorized agent embeds itself during task delegation and injects false subtasks to waste resources and confuse outcomes.
- **Scenario 4: Role Drift via Cloaking** – An agent initially instantiated as a passive observer starts executing actions due to unmonitored privilege creep.

### Reference Links
1. [Agentic AI - Threats and Mitigations](https://genai.owasp.org/resource/agentic-ai-threats-and-mitigations/)
2. [LLM06:2025 Excessive Agency](https://genai.owasp.org/llmrisk/llm062025-excessive-agency/)
OWASP LLM Top 10: ASI06 Intent Breaking Goal Manipulation

## Risk/Vuln Name
**Intent Breaking & Goal Manipulation**

**Author(s):**
OWASP Agentic Security Initiative Team

### Description
Agentic AI systems operate based on inferred or explicit goals. Adversaries can manipulate goal inference, alter intermediate representations, or interfere with agent objectives to cause undesired or harmful actions. This includes prompt injection, manipulation of user intent, or interference in planning or delegation logic.

While related to [LLM02:2023 Insecure Plugin Design](https://genai.owasp.org/llmrisk2023-24/llm07-insecure-plugin-design/) and [LLM06:2025 Excessive Agency](https://genai.owasp.org/llmrisk/llm062025-excessive-agency/), agentic AI systems introduce new risks where goals can be hijacked mid-flight, or inferred intent can be weaponized against the user.

### Common Examples of Risk
1. Manipulating prompts to redefine agent goals via indirect instructions.
2. Injecting fake constraints or desired outcomes into memory or shared tools.
3. Altering user intent during multi-step goal refinement or conversation.
4. Hijacking intermediate reasoning steps during agent planning.

### Prevention and Mitigation Strategies
1. Explicitly separate intent parsing from task execution and validate goals.
2. Use planning critics or supervisor agents to audit and approve goals.
3. Enforce multi-turn user confirmation for critical or high-risk goal changes.
4. Annotate original user instructions throughout the execution chain.
5. Monitor goal drift or semantic divergence during reasoning chains.
6. Restrict agent autonomy over certain types of intent (e.g., financial, legal).
7. Use guardrails to block unsafe interpretations of inferred goals.

### Example Attack Scenarios
- **Scenario 1: Prompt Injection Goal Switch** – A malicious prompt embedded in a document causes an AI assistant to alter its objective from summarizing the document to sending it externally.
- **Scenario 2: Planning Path Corruption** – An attacker crafts context that causes the agent to prioritize misleading metrics, shifting its goal from accuracy to speed, reducing quality and safety.
- **Scenario 3: Goal Inference Drift** – Through multi-turn ambiguity, a user is misinterpreted by the agent, which infers a dangerous or privacy-violating action.
- **Scenario 4: Delegated Planning Hijack** – A downstream agent is given an intermediate task from a master planner, but the task description has been altered in transit, leading to off-target actions.

### Reference Links
1. [Agentic AI - Threats and Mitigations](https://genai.owasp.org/resource/agentic-ai-threats-and-mitigations/)
2. [LLM02:2023 Insecure Plugin Design](https://genai.owasp.org/llmrisk2023-24/llm07-insecure-plugin-design/)
3. [LLM06:2025 Excessive Agency](https://genai.owasp.org/llm-top-10/LLM06-excessive-agency)
OWASP LLM Top 10: ASI08 Cascading Failures

## ASI08 – Cascading Failures

**Description:**

A brief description of the vulnerability that includes its potential effects such as system compromises, data breaches, or other security concerns.

**Common Examples of Vulnerability:**

1. Example 1: Specific instance or type of this vulnerability.
2. Example 2: Another instance or type of this vulnerability.
3. Example 3: Yet another instance or type of this vulnerability.

**How to Prevent:**

1. Prevention Step 1: A step or strategy that can be used to prevent the vulnerability or mitigate its effects.
2. Prevention Step 2: Another prevention step or strategy.
3. Prevention Step 3: Yet another prevention step or strategy.

**Example Attack Scenarios:**

Scenario #1: A detailed scenario illustrating how an attacker could potentially exploit this vulnerability, including the attacker's actions and the potential outcomes.

Scenario #2: Another example of an attack scenario showing a different way the vulnerability could be exploited.

**Reference Links:**

1. [Link Title](URL): Brief description of the reference link.
2. [Link Title](URL): Brief description of the reference link.
OWASP LLM Top 10: ASI02 Tool Misuse and Exploitation

## ASI02 – Tool Misuse and Exploitation

**Description:**

A brief description of the vulnerability that includes its potential effects such as system compromises, data breaches, or other security concerns.

**Common Examples of Vulnerability:**

1. Example 1: Specific instance or type of this vulnerability.
2. Example 2: Another instance or type of this vulnerability.
3. Example 3: Yet another instance or type of this vulnerability.

**How to Prevent:**

1. Prevention Step 1: A step or strategy that can be used to prevent the vulnerability or mitigate its effects.
2. Prevention Step 2: Another prevention step or strategy.
3. Prevention Step 3: Yet another prevention step or strategy.

**Example Attack Scenarios:**

Scenario #1: A detailed scenario illustrating how an attacker could potentially exploit this vulnerability, including the attacker's actions and the potential outcomes.

Scenario #2: Another example of an attack scenario showing a different way the vulnerability could be exploited.

**Reference Links:**

1. [Link Title](URL): Brief description of the reference link.
2. [Link Title](URL): Brief description of the reference link.
OWASP LLM Top 10: README

# Top Ten for Agentic Applications – 1st Public Draft

The public draft was the result of consolidating the initial 0.5-initial-candidates based on the ASI Threats and Mitigations.

| ID    | Title                               | Description                                                                 |
|-------|-------------------------------------|-----------------------------------------------------------------------------|
| [ASI01](ASI01_Agent_Behaviour_Hijack.md) | Agent Behaviour Hijack              | Manipulating an agent’s goals/plans to pursue attacker-aligned objectives.  |
| [ASI02](ASI02_Tool_Misuse_and_Exploitation.md) | Tool Misuse and Exploitation        | Tricking agents into using their tools in harmful or unintended ways.       |
| [ASI03](ASI03_Identity_and_Privilege_Abuse.md) | Identity & Privilege Abuse          | Impersonating agents or escalating access through identity/auth weaknesses. |
| [ASI04](ASI04_Agentic_Supply_Chain_Vulnerabilities.md) | Agentic Supply Chain Vulnerabilities| Introducing insecure models, agents, tools or artefacts compromising integrity. |
| [ASI05](ASI05_Unexpected_Code_Execution_RCE.md) | Unexpected Code Execution (RCE)     | Triggering unauthorized or unsafe code execution through agent behaviors.   |
| [ASI06](ASI06_Memory_and_Context_Poisoning.md) | Memory & Context Poisoning          | Corrupting agent memory or context to distort reasoning and decision-making.|
| [ASI07](ASI07_Insecure_Inter_Agent_Communication.md) | Insecure Inter-Agent Communication  | Poisoning messages or abusing protocols between agents to alter behavior.   |
| [ASI08](ASI08_Cascading_Failures.md) | Cascading Failures                  | Faults or hallucinations propagate through agents, causing compounded failures. |
| [ASI09](ASI09_Human_Agent_Trust_Exploitation.md) | Human-Agent Trust Exploitation      | Exploiting over-trust or fatigue in human oversight to enable misuse, including deceptive behaviours. |
| [ASI10](ASI10_Rogue_Agents.md) | Rogue Agents                        | Malicious or compromised agents acting autonomously to deceive, disrupt, or exfiltrate. |
OWASP LLM Top 10: ASI10 Rogue Agents

## ASI10 – Rogue Agents

**Description:**

A brief description of the vulnerability that includes its potential effects such as system compromises, data breaches, or other security concerns.

**Common Examples of Vulnerability:**

1. Example 1: Specific instance or type of this vulnerability.
2. Example 2: Another instance or type of this vulnerability.
3. Example 3: Yet another instance or type of this vulnerability.

**How to Prevent:**

1. Prevention Step 1: A step or strategy that can be used to prevent the vulnerability or mitigate its effects.
2. Prevention Step 2: Another prevention step or strategy.
3. Prevention Step 3: Yet another prevention step or strategy.

**Example Attack Scenarios:**

Scenario #1: A detailed scenario illustrating how an attacker could potentially exploit this vulnerability, including the attacker's actions and the potential outcomes.

Scenario #2: Another example of an attack scenario showing a different way the vulnerability could be exploited.

**Reference Links:**

1. [Link Title](URL): Brief description of the reference link.
2. [Link Title](URL): Brief description of the reference link.
OWASP LLM Top 10: ASI04 Agentic Supply Chain Vulnerabilities

## ASI04 – Agentic Supply Chain Vulnerabilities

**Description:**

A brief description of the vulnerability that includes its potential effects such as system compromises, data breaches, or other security concerns.

**Common Examples of Vulnerability:**

1. Example 1: Specific instance or type of this vulnerability.
2. Example 2: Another instance or type of this vulnerability.
3. Example 3: Yet another instance or type of this vulnerability.

**How to Prevent:**

1. Prevention Step 1: A step or strategy that can be used to prevent the vulnerability or mitigate its effects.
2. Prevention Step 2: Another prevention step or strategy.
3. Prevention Step 3: Yet another prevention step or strategy.

**Example Attack Scenarios:**

Scenario #1: A detailed scenario illustrating how an attacker could potentially exploit this vulnerability, including the attacker's actions and the potential outcomes.

Scenario #2: Another example of an attack scenario showing a different way the vulnerability could be exploited.

**Reference Links:**

1. [Link Title](URL): Brief description of the reference link.
2. [Link Title](URL): Brief description of the reference link.
OWASP LLM Top 10: ASI05 Unexpected Code Execution RCE

## ASI05 – Unexpected Code Execution (RCE)

**Description:**

A brief description of the vulnerability that includes its potential effects such as system compromises, data breaches, or other security concerns.

**Common Examples of Vulnerability:**

1. Example 1: Specific instance or type of this vulnerability.
2. Example 2: Another instance or type of this vulnerability.
3. Example 3: Yet another instance or type of this vulnerability.

**How to Prevent:**

1. Prevention Step 1: A step or strategy that can be used to prevent the vulnerability or mitigate its effects.
2. Prevention Step 2: Another prevention step or strategy.
3. Prevention Step 3: Yet another prevention step or strategy.

**Example Attack Scenarios:**

Scenario #1: A detailed scenario illustrating how an attacker could potentially exploit this vulnerability, including the attacker's actions and the potential outcomes.

Scenario #2: Another example of an attack scenario showing a different way the vulnerability could be exploited.

**Reference Links:**

1. [Link Title](URL): Brief description of the reference link.
2. [Link Title](URL): Brief description of the reference link.
OWASP LLM Top 10: ASI06 Memory and Context Poisoning

## ASI06 – Memory & Context Poisoning

**Description:**

A brief description of the vulnerability that includes its potential effects such as system compromises, data breaches, or other security concerns.

**Common Examples of Vulnerability:**

1. Example 1: Specific instance or type of this vulnerability.
2. Example 2: Another instance or type of this vulnerability.
3. Example 3: Yet another instance or type of this vulnerability.

**How to Prevent:**

1. Prevention Step 1: A step or strategy that can be used to prevent the vulnerability or mitigate its effects.
2. Prevention Step 2: Another prevention step or strategy.
3. Prevention Step 3: Yet another prevention step or strategy.

**Example Attack Scenarios:**

Scenario #1: A detailed scenario illustrating how an attacker could potentially exploit this vulnerability, including the attacker's actions and the potential outcomes.

Scenario #2: Another example of an attack scenario showing a different way the vulnerability could be exploited.

**Reference Links:**

1. [Link Title](URL): Brief description of the reference link.
2. [Link Title](URL): Brief description of the reference link.
OWASP LLM Top 10: ASI09 Human Agent Trust Exploitation

## ASI09 – Human-Agent Trust Exploitation

**Description:**

A brief description of the vulnerability that includes its potential effects such as system compromises, data breaches, or other security concerns.

**Common Examples of Vulnerability:**

1. Example 1: Specific instance or type of this vulnerability.
2. Example 2: Another instance or type of this vulnerability.
3. Example 3: Yet another instance or type of this vulnerability.

**How to Prevent:**

1. Prevention Step 1: A step or strategy that can be used to prevent the vulnerability or mitigate its effects.
2. Prevention Step 2: Another prevention step or strategy.
3. Prevention Step 3: Yet another prevention step or strategy.

**Example Attack Scenarios:**

Scenario #1: A detailed scenario illustrating how an attacker could potentially exploit this vulnerability, including the attacker's actions and the potential outcomes.

Scenario #2: Another example of an attack scenario showing a different way the vulnerability could be exploited.

**Reference Links:**

1. [Link Title](URL): Brief description of the reference link.
2. [Link Title](URL): Brief description of the reference link.
OWASP LLM Top 10: ASI03 Identity and Privilege Abuse

## ASI03 – Identity & Privilege Abuse

**Description:**

A brief description of the vulnerability that includes its potential effects such as system compromises, data breaches, or other security concerns.

**Common Examples of Vulnerability:**

1. Example 1: Specific instance or type of this vulnerability.
2. Example 2: Another instance or type of this vulnerability.
3. Example 3: Yet another instance or type of this vulnerability.

**How to Prevent:**

1. Prevention Step 1: A step or strategy that can be used to prevent the vulnerability or mitigate its effects.
2. Prevention Step 2: Another prevention step or strategy.
3. Prevention Step 3: Yet another prevention step or strategy.

**Example Attack Scenarios:**

Scenario #1: A detailed scenario illustrating how an attacker could potentially exploit this vulnerability, including the attacker's actions and the potential outcomes.

Scenario #2: Another example of an attack scenario showing a different way the vulnerability could be exploited.

**Reference Links:**

1. [Link Title](URL): Brief description of the reference link.
2. [Link Title](URL): Brief description of the reference link.
OWASP LLM Top 10: ASI01 Agent Behaviour Hijack

## ASI01 – Agent Behaviour Hijack

**Description:**

A brief description of the vulnerability that includes its potential effects such as system compromises, data breaches, or other security concerns.

**Common Examples of Vulnerability:**

1. Example 1: Specific instance or type of this vulnerability.
2. Example 2: Another instance or type of this vulnerability.
3. Example 3: Yet another instance or type of this vulnerability.

**How to Prevent:**

1. Prevention Step 1: A step or strategy that can be used to prevent the vulnerability or mitigate its effects.
2. Prevention Step 2: Another prevention step or strategy.
3. Prevention Step 3: Yet another prevention step or strategy.

**Example Attack Scenarios:**

Scenario #1: A detailed scenario illustrating how an attacker could potentially exploit this vulnerability, including the attacker's actions and the potential outcomes.

Scenario #2: Another example of an attack scenario showing a different way the vulnerability could be exploited.

**Reference Links:**

1. [Link Title](URL): Brief description of the reference link.
2. [Link Title](URL): Brief description of the reference link.
OWASP LLM Top 10: ASI07 Insecure Inter Agent Communication

## ASI07 – Insecure Inter-Agent Communication

**Description:**

A brief description of the vulnerability that includes its potential effects such as system compromises, data breaches, or other security concerns.

**Common Examples of Vulnerability:**

1. Example 1: Specific instance or type of this vulnerability.
2. Example 2: Another instance or type of this vulnerability.
3. Example 3: Yet another instance or type of this vulnerability.

**How to Prevent:**

1. Prevention Step 1: A step or strategy that can be used to prevent the vulnerability or mitigate its effects.
2. Prevention Step 2: Another prevention step or strategy.
3. Prevention Step 3: Yet another prevention step or strategy.

**Example Attack Scenarios:**

Scenario #1: A detailed scenario illustrating how an attacker could potentially exploit this vulnerability, including the attacker's actions and the potential outcomes.

Scenario #2: Another example of an attack scenario showing a different way the vulnerability could be exploited.

**Reference Links:**

1. [Link Title](URL): Brief description of the reference link.
2. [Link Title](URL): Brief description of the reference link.
OWASP LLM Top 10: README

# GenAI Red Team Sandboxes

This directory hosts a collection of **sandboxes** designed to facilitate Generative AI (GenAI) Red Teaming exercises.

## Purpose

The goal of these sandboxes is to provide ready-to-use, isolated environments where security researchers and red teamers can test, probe, and evaluate Large Language Model (LLM) applications and other GenAI systems safely.

## Contents

*    **`llm_local/`**: A local sandbox environment that mocks an LLM API (compatible with OpenAI's interface) using a local model (via Ollama). This sandbox is useful for testing client-side interactions, prompt injection vulnerabilities, and other security assessments without relying on external, paid APIs. Additionally, it allows developers to customize the underlying LLM and orchestrate sophisticated GenAI pipelines, incorporating features such as RAG and guardrail layers, as necessary.

*    **`RAG_local/`**: A comprehensive RAG (Retrieval-Augmented Generation) sandbox that includes a mock Vector Database (Pinecone compatible), mock Object Storage (Amazon S3 compatible), and a mock LLM API (OpenAI compatible). This environment is specifically designed for Red Teaming RAG architectures, allowing researchers to explore vulnerabilities such as embedding inversion, data poisoning, and retrieval manipulation in a controlled, local setting.

## Usage

Each sandbox directory contains its own `README.md` with specific instructions on how to build, run, and use that particular sandbox. Please refer to the individual sandbox documentation for details.
OWASP LLM Top 10: README

# ghostglyph

![ghost glyph logo](./ghostglyph.png)

A tool to assist with ascii smuggling attacks against generative AI based systems.

Usage:
1. Run `uv run src/app.py`
2. Open the webpage in a browser
3. Enter the text you want to encode and select the type of encoding to use
4. Copy the encoded text and use it in your prompt injection payloads


# Reference Documentation

* [Defending LLM Applications Against Unicode Character Smuggling](https://aws.amazon.com/blogs/security/defending-llm-applications-against-unicode-character-smuggling/) - by Amazon
* [ASCII Smuggler Tool](https://embracethered.com/blog/posts/2024/hiding-and-finding-text-with-unicode-tags/) - by EmbraceTheRed