OWASP Top 10 for
LLM Applications
VERSION 1.1
Published: October 16, 2023

| OWASP Top 10 for LLM Applications v1.1

OWASP Top 10 for LLM Applications
LLM01

LLM02

LLM03

LLM04

LLM05

Insecure Output

Handling

Training Data

Poisoning

Model Denial of Service

Supply Chain
Vulnerabilities

model (LLM) through crafty inputs,

This vulnerability occurs when an LLM

This occurs when LLM training data is

LLM application lifecycle can becausing unintended actions by the LLM.

output is accepted without scrutiny,

tampered, introducing vulnerabilities or

Direct injections overwrite system

exposing backend systems. Misusebiases that compromise security,

prompts, while indirect ones manipulate

may lead to severe consequences likeeffectiveness, or ethical behavior.

inputs from external sources.

XSS, CSRF, SSRF, privilege escalation, orSources include Common Crawl,

remote code execution.

WebText, OpenWebText, & books.

Attackers cause resource-heavy
operations on LLMs, leading to service
degradation or high costs. The
vulnerability is magnified due to theresource-intensive nature of LLMs andunpredictability of user inputs.

Prompt Injection
This manipulates a large language

LLM06

LLM07

LLM08

LLM09

compromised by vulnerable
components or services, leading tosecurity attacks. Using third-party
datasets, pre- trained models, andplugins can add vulnerabilities.

LLM10

Sensitive Information
Disclosure

Insecure Plugin

Design

Excessive Agency

Overreliance

Model Theft

LLM-based systems may undertake

Systems or people overly depending on

This involves unauthorized access,

LLMs may inadvertently reveal

LLM plugins can have insecure inputs

actions leading to unintended

LLMs without oversight may facecopying, or exfiltration of proprietaryconfidential data in its responses,

and insufficient access control. Thisconsequences. The issue arises frommisinformation, miscommunication,

LLM models. The impact includes

leading to unauthorized data access,lack of application control makes themexcessive functionality, permissions, orlegal issues, and security vulnerabilities

economic losses, compromised

privacy violations, and security

easier to exploit and can result inautonomy granted to the LLM-based

due to incorrect or inappropriate content

competitive advantage, and potential

breaches. It’s crucial to implement dataconsequences like remote code

systems.

generated by LLMs.

access to sensitive information.

sanitization and strict user policies to

execution.

mitigate this.

| OWASP Top 10 for LLM Applications v1.1

Data Flow Diagram
The diagram here presents a high levelarchitecture for a hypothetical large
language model application. 

Overlaid in the diagram are highlighted
areas of risk illustrating how the OWASPTop 10 for LLM Applications entries
intersect with the application flow.

| OWASP Top 10 for LLM Applications v1.1

LLM01

Prompt Injection
Attackers can manipulate LLMs throughcrafted inputs, causing it to execute theattacker's intentions. This can be done
directly by adversarially prompting thesystem prompt or indirectly throughmanipulated external inputs, potentiallyleading to data exfiltration, social
engineering, and other issues.

EXAMPLES
Direct Prompt Injection: Malicious user injects prompts to extract
sensitive information
Indirect Prompt Injection: Users request sensitive data via webpage
prompts
Scam Through Plugins: Websites exploit plugins for scams.
PREVENTION
Privilege Control: Limit LLM access and apply role-based permissions
Human Approval: Require user consent for privileged actions
Segregate Content: Separate untrusted content from user prompts
Trust Boundaries: Treat LLM as untrusted and visually highlight unreliable
responses.
ATTACK SCENARIOS
Chatbot Remote Execution: Injection leads to unauthorized access via
chatbot
Email Deletion: Indirect injection causes email deletion
Exfiltration via Image: Webpage prompts exfiltrate private data
Misleading Resume: LLM incorrectly endorses a candidate
Prompt Replay: Attacker replays system prompts for potential further
attacks.

| OWASP Top 10 for LLM Applications v1.1

LLM02

Insecure Output
Handling
Insecure Output Handling is a vulnerability
that arises when a downstream component
blindly accepts large language model (LLM)
output without proper scrutiny. This canlead to XSS and CSRF in web browsers aswell as SSRF, privilege escalation, or remotecode execution on backend systems.

EXAMPLES
Remote Code Execution: LLM output executed in system shell,leading to code execution
Cross-Site Scripting (XSS): LLM-generated JavaScript or Markdown
causes browser interpretation.
PREVENTION
Zero-Trust Approach: Treat LLM output like user input; validate andsanitize it properly
OWASP ASVS Guidelines: Follow OWASP's standards for input validationand sanitization
Output Encoding: Encode LLM output to prevent code execution in
JavaScript or Markdown.
ATTACK SCENARIOS
Chatbot Shutdown: LLM output shuts down a plugin due to a lack ofvalidation
Sensitive Data Capture: LLM captures and sends sensitive data to an
attacker-controlled server
Database Table Deletion: LLM crafts a destructive SQL query, potentially
deleting all tables
XSS Exploitation: LLM returns unsanitized JavaScript payload, leading to
XSS on the victim's browser.

| OWASP Top 10 for LLM Applications v1.1

LLM03

Training Data
Poisoning
Training Data Poisoning refers tomanipulating the data or fine-tuning process
to introduce vulnerabilities, backdoors or
biases that could compromise the model’ssecurity, effectiveness or ethical behavior.
This risks performance degradation,
downstream software exploitation andreputational damage.

EXAMPLES
Malicious Data Injection: Injecting falsified data during modeltraining
Biased Training Outputs: Model reflects inaccuracies from tainted
data
Content Injection: Malicious actors inject biased content intotraining.
PREVENTION
Supply Chain Verification: Verify external data sources and maintain "MLBOM" records
Legitimacy Verification: Ensure data legitimacy throughout training
stages
Use-Case Specific Training: Create separate models for different usecases.
ATTACK SCENARIOS
Misleading Outputs: LLM generates content that promotes bias or hate
Toxic Data Injection: Malicious users manipulate the model with biased
data
Malicious Document Injection: Competitors insert false data duringmodel training.

| OWASP Top 10 for LLM Applications v1.1

LLM04

Model Denial ofServiceModel Denial of Service occurs when an
attacker interacts with a Large Language
Model (LLM) in a way that consumes an
exceptionally high amount of resources.
This can result in a decline in the quality ofservice for them and other users, as well as
potentially incurring high resource costs.

EXAMPLES
High-Volume Queuing: Attackers overload LLM with resourceintensive tasks
Resource-Consuming Queries: Unusual queries strain system
resources
Continuous Input Overflow: Flooding LLM with excessive input
Repetitive Long Inputs: Repeated long queries exhaust resources
Recursive Context Expansion: Attackers exploit recursive behavior.
PREVENTION
Input Validation: Implement input validation and content filtering
Resource Caps: Limit resource use per request
API Rate Limits: Enforce rate limits for users or IP addresses
Queue Management: Control queued and total actions
Resource Monitoring: Continuously monitor resource usage.
ATTACK SCENARIOS
Resource Overuse: Attacker overloads a hosted model, impacting otherusers
Webpage Request Amplification: LLM tool consumes excessive
resources due to unexpected content
Input Flood: Overwhelm LLM with excessive input, causing slowdown
Sequential Input Drain: Attacker exhausts context window with sequential
inputs.

| OWASP Top 10 for LLM Applications v1.1

LLM05

EXA M PLES
Package Vulnerabilities: Using outdated components
Vulnerable Models: Risky pre-trained models for fine-tuning

Supply Chain
Vulnerabilities
Supply chain vulnerabilities in LLMs cancompromise training data, ML models, anddeployment platforms, causing biased
results, security breaches, or total system

Poisoned Data: Tainted crowd-sourced data
Outdated Models: Using unmaintained models
Unclear Terms: Data misuse due to unclear terms.

PREVENTI ON
Supplier Evaluation: Vet suppliers and policies
Plugin Testing: Use tested, trusted plugins
OWASP A06: Mitigate outdated component risks
Inventory Management: Maintain an up-to-date inventory
Security Measures: Sign models and code, apply anomaly detection, andmonitor.

failures. Such vulnerabilities can stem fromoutdated software, susceptible pre-trained

AT TACK SCEN A RI OS
Library Exploitation: Exploiting vulnerable Python libraries

models, poisoned training data, andinsecure plugin designs.

Scamming Plugin: Deploying a plugin for scams
Package Registry Attack: Tricking developers with a compromised
packageMisinformation Backdoor: Poisoning models for fake news
Data Poisoning: Poisoning datasets during fine-tuning.

| OWASP Top 10 for LLM Applications v1.1

LLM06

Sensitive Information
Disclosure
LLM applications can inadvertently disclosesensitive information, proprietaryalgorithms, or confidential data, leading tounauthorized access, intellectual property
theft, and privacy breaches. To mitigate
these risks, LLM applications should
employ data sanitization, implement
appropriate usage policies, and restrict thetypes of data returned by the LLM.

EXAMPLES
Incomplete Filtering: LLM responses may contain sensitive data
Overfitting: LLMs memorize sensitive data during training
Unintended Disclosure: Data leaks due to misinterpretation or lack ofscrubbing.
PREVENTION
Data Sanitization: Use scrubbing to prevent user data in training
Input Validation: Filter malicious inputs to avoid model poisoning
Fine-Tuning Caution: Be careful with sensitive data in model fine-tuning
Data Access Control: Limit external data source access.
ATTACK SCENARIOS
Unintentional Exposure: User A exposed to other user data
Filter Bypass: User A extracts PII by bypassing filters
Training Data Leak: Personal data leaks during training.

| OWASP Top 10 for LLM Applications v1.1

LLM07

Insecure Plugin
DesignPlugins can be prone to malicious requests
leading to harmful consequences like data
exfiltration, remote code execution, andprivilege escalation due to insufficient
access controls and improper input
validation. Developers must follow robust
security measures to prevent exploitation,
like strict parameterized inputs and secureaccess control guidelines.

EXAMPLES
Single Field Parameters: Plugins lack parameter separation
Configuration Strings: Configurations can override settings
Authentication Issues: Lack of specific plugin authorization
Raw SQL or Code: Unsafe acceptance of code or SQL.
PREVENTION
Parameter Control: Enforce type checks and use a validation layer
OWASP Guidance: Apply ASVS recommendations
Thorough Testing: Inspect and test with SAST, DAST, IAST
Least-Privilege: Follow ASVS Access Control Guidelines
Auth Identities: Use OAuth2 and API Keys for custom authorization
User Confirmation: Require manual authorization for sensitive actions.
ATTACK SCENARIOSURL Manipulation: Attackers inject content via manipulated URLs
Reconnaissance and Exploitation: Exploiting lack of validation for code
execution and data theft
Unauthorized Access: Accessing unauthorized data through parameter
manipulation
Repository Takeover: Exploiting insecure code management plugin forrepository takeover.

| OWASP Top 10 for LLM Applications v1.1

LLM08

Excessive Agency
Excessive Agency in LLM-based systems isa vulnerability caused by over-functionality,
excessive permissions, or too muchautonomy. To prevent this, developers needto limit plugin functionality, permissions,and autonomy to what's absolutely
necessary, track user authorization, require
human approval for all actions, andimplement authorization in downstream
systems.

EXAMPLES
Excessive Functionality: LLM agents have unnecessary functions, risking
misuseExcessive Permissions: Plugins may have excessive access to systems
Excessive Autonomy: LLMs lack human verification for high-impact
actions.
PREVENTION
Limit Plugin Functions: Allow only essential functions for LLM agents
Plugin Scope Control: Restrict functions within LLM plugins
Granular Functionality: Avoid open-ended functions; use specific plugins
Permissions Control: Limit permissions to the minimum required
User Authentication: Ensure actions are in the user's context
Human-in-the-Loop: Require human approval for actions
Downstream Authorization: Implement authorization in downstream
systems.
ATTACK SCENARIOSAn LLM-based personal assistant app with excessive permissions andautonomy is tricked by a malicious email into sending spam. This could be
prevented by limiting functionality, permissions, requiring user approval, or
implementing rate limiting.

| OWASP Top 10 for LLM Applications v1.1

LLM09

Overreliance
Overreliance on LLMs can lead to serious
consequences such as misinformation,legal issues, and security vulnerabilities.

It occurs when an LLM is trusted to makecritical decisions or generate content
without adequate oversight or validation.

EXAMPLES
Misleading Info: LLMs can provide misleading info without validation
Insecure Code: LLMs may suggest insecure code in software.
PREVENTION
Monitor and Validate: Regularly review LLM outputs with consistency
checks
Cross-Check: Verify LLM output with trusted sources
Fine-Tuning: Enhance LLM quality with task-specific fine-tuning
Auto Validation: Implement systems to verify output against known facts
Task Segmentation: Divide complex tasks to reduce risks
Risk Communication: Communicate LLM limitations
User-Friendly Interfaces: Create interfaces with content filters andwarnings
Secure Coding: Establish guidelines to prevent vulnerabilities.
ATTACK SCENARIOSDisinfo Spread: Malicious actors exploit LLM-reliant news organizations
Plagiarism: Unintentional plagiarism leads to copyright issues
Insecure Software: LLM suggestions introduce security vulnerabilities
Malicious Package: LLM suggests a non-existent code library.

| OWASP Top 10 for LLM Applications v1.1

LLM10

Model Theft
LLM model theft involves unauthorizedaccess to and exfiltration of LLM models,
risking economic loss, reputation damage,and unauthorized access to sensitive data.
Robust security measures are essential toprotect these models.

EXAMPLES
Vulnerability Exploitation: Unauthorized access due to security flaws
Central Model Registry: Centralized security for governanceInsider Threat: Risk of employee model leaks
Side-Channel Attack: Extraction of model details through sidetechniques.
PREVENTION & MITIGATION
Access Control and Authentication: Strong access controls andauthentication
Network Restrictions: Limit LLM access to resources and APIs
Monitoring and Auditing: Regular monitoring of access logs
MLOps Automation: Secure deployment with approval workflows.
ATTACK SCENARIOS
Model Theft: Unauthorized access and use for competition
Employee Leak: Exposure increases risks
Shadow Model Creation: Replicating models with queries
Side-Channel Attack: Extraction through side techniques.

| OWASP Top 10 for LLM Applications v1.1

Key Reference Links
Prompt Injection attack against LLM-integrated Applications: Cornell Universit

Tay Poisoning: MITR

Defending ChatGPT against Jailbreak Attack via Self-Reminder: Research Squar

Backdoor Attacks on Language Models: Can We Trust Our Model’s Weights?: Mediu

OpenAI Chat Markup Language: GitHu

Poisoning Language Models During Instruction Tuning: Cornell Universit

Not what you’ve signed up for: Compromising Real-World LLM-Integrated Applications with

ChatGPT Data Breach Confirmed as Security Firm Warns of Vulnerable Component

Indirect Prompt Injection: Cornell Universit

Exploitation: Security Wee

Threat Modeling LLM Applications: AI Villag

What Happens When an AI Company Falls Victim to a Software Supply Chain Vulnerability:

Safety Best Practices: OpenA

Security Boulevar

Arbitrary Code Execution: Sny

Plugin Review Process: OpenA

CS324 - Large Language Models: Stanford Universit

Compromised PyTorch-nightly dependency chain: PyTorch

How data poisoning attacks corrupt machine learning models: CSO Onlin
ML Supply Chain Compromise: MITRE