OWASP Top 10 for LLM
2023

owasp.org/www-project-top-10-for-large-language-model-applications

| OWASP Top 10 for LLM

Introduction
Welcome to the first iteration of the OWASP Top 10 for Large Language Models (LLMs) Applications.

This document marks an exciting new chapter in the ongoing efforts to enhance security in the rapidly evolving field of artificial intelligence.

A GROUNDBREAKING EFFORTThis initiative is the culmination of the tireless efforts of our first expert team, a diverse group of security
specialists, AI researchers, developers, and industry leaders. Since our inception just a month ago, our ranks
have swelled to over 370 members, with more than 100 experts actively contributing. This remarkable
growth is a testament to the gravity and immediacy of the challenge we face, and the dedication of thoseworking to meet it head on.
The heartiest of congratulations and deepest gratitude is due to our team. The knowledge, time, and passion
they have dedicated to this project have been invaluable. This endeavor wouldn't have been possible without
their profound insights and unwavering commitment.
OUR PURPOSE
The purpose of our group, as outlined in the OWASP Top 10 for LLM Applications Working Group Charter, is
to identify and highlight the top security and safety issues that developers and security teams must consider
when building applications leveraging Large Language Models (LLMs). Our objective is to provide clear,practical, and actionable guidance to enable these teams to proactively address potential vulnerabilities in
LLM-based applications.
The ultimate aim is to provide a robust foundation for the safe and secure utilization of LLMs across a widearray of scenarios, from small-scale individual projects to large-scale corporate and governmental
implementations. We firmly believe that by understanding and mitigating the top vulnerabilities associatedwith LLMs, we can contribute to a safer and more reliable digital environment for everyone.
ABOUT THIS VERSIONThis document, Version 0.5, serves as a crucial milestone in our ongoing journey. It encapsulates thecollective insights and understanding of our group, at this early stage, of the unique vulnerabilities inherent
to applications leveraging LLMs. It's important to note that this is not the final version of the OWASP Top 10
for LLMs. Instead, consider it a 'preview' of what's to come.
We are committed to refining, expanding, and deepening our work over the coming month as we work
towards the true Version 1.0. We invite you to explore this preliminary list, to share your feedback, and to join
us in our mission to create a safer, more secure future for AI.
Once again, we extend our heartfelt thanks to our expert team and the broader community for theircontinued support. Together, let's navigate the exciting and complex world of LLMs with an eye towards
security, safety, and inclusivity.
Steve Wilson

Project Lead, OWASP Top 10 for LLM AI Applications

Twitter: @virtualsteve

OWASP.org

2

| OWASP Top 10 for LLM

OWASP Top 10 for LLM
This is a draft list of important vulnerability types for Artificial Intelligence (AI) applications built on
Large Language Models (LLMs).

LLM01: Prompt Injections

LLM06: Permission Issues

Prompt Injection Vulnerabilities in LLMs involve crafty
inputs leading to undetected manipulations. The impact
ranges from data exposure to unauthorized actions,serving attacker's goals.

Lack of authorization tracking between plugins can enable
indirect prompt injection or malicious plugin usage,
leading to privilege escalation, confidentiality loss, andpotential remote code execution.

LLM02: Insecure Output Handling

LLM07: Data Leakagehese occur when plugins or apps accept LLM output
without scrutiny, potentially leading to XSS, CSRF, SSRF,
privilege escalation, remote code execution, and canenable agent hijacking attacks.

Data leakage in LLMs can expose sensitive information or
proprietary details, leading to privacy and security
breaches. Proper data sanitization, and clear terms of use
are crucial for prevention.

LLM03: Training Data Poisoning

LLM08: Excessive Agency

LLMs learn from diverse text but risk training datapoisoning, leading to user misinformation. Overreliance on
AI is a concern. Key data sources include Common Crawl,
WebText, OpenWebText, and books.

When LLMs interface with other systems, unrestrictedagency may lead to undesirable operations and actions.
Like web-apps, LLMs should not self-police; controls must
be embedded in APIs.

LLM04: Denial of Service

LLM09: Overreliance

An attacker interacts with an LLM in a way that isparticularly resource-consuming, causing quality ofservice to degrade for them and other users, or for highresource costs to be incurred.

Overreliance on LLMs can lead to misinformation or
inappropriate content due to "hallucinations." Without
proper oversight, this can result in legal issues andreputational damage.

LLM05: Supply Chain

LLM10: Insecure Plugins

LLM supply chains risk integrity due to vulnerabilities
leading to biases, security breaches, or system failures.
Issues arise from pre-trained models, crowdsourced data,and plugin extensions.

Plugins connecting LLMs to external resources can be
exploited if they accept free-form text inputs, enabling
malicious requests that could lead to undesired behaviors
or remote code execution.

TOWASP.org

3

| OWASP Top 10 for LLM

LLM01: Prompt Injection
First Published: July 1st, 2023

Prompt Injection Vulnerabilities occur when attackers manipulate a trusted large
language model (LLM) using crafted input prompts, via multiple channels. This
manipulation often goes undetected due to inherent trust in LLM's output.
Two types exist: Direct, where the attacker influences the LLM's input, andIndirect, where a 'poisoned' data source affects the LLM.
Outcomes can range from exposing sensitive information to influencing
decisions. In complex cases, the LLM could be tricked into unauthorized actions
or impersonations, effectively serving the attacker's goals without alerting theuser or triggering safeguards.

Common Vulnerabilities
Example 1: An attacker introduces a malicious prompt in a webpage, which isthen accessed by the LLM, leading to manipulated responses.
Example 2: The LLM is tricked into divulging sensitive information ormanipulating output to downstream systems.
Example 3: An attacker exploits the LLM's interaction with plugins, triggering
unauthorized actions like unauthorized purchases, undesired social media posts,
deleted emails, etc.

HOW TO PREVENT
Privilege Control: Limit the privileges of an LLM to the least necessary for its
functionality. Prevent the LLM from altering the user’s state without explicit

EXAMPLE ATTACK SCENARIOSScenario A: A user employs an
LLM to summarize a webpage
containing a hidden prompt
injection. This then causes the
LLM to solicit sensitive
information from the user andperform exfiltration via
JavaScript or Markdown.

Scenario B: A malicious user

uploads a resume with aprompt injection. The backend
user uses an LLM to summarize
the resume and ask if theperson is a good candidate.
Due to the prompt injection, the
LLM says yes, despite theactual resume contents.

Scenario C: A user enables aplugin linked to an e-commercesite. A rogue instruction
embedded on a visited website
exploits this plugin, leading tounauthorized purchases.

REFERENCE LINKS

approval.

ChatGPT Plugin Vulnerabilities - Chat

Enhanced Input Validation: Implement robust input validation and sanitization
methods to filter out potential malicious prompt inputs from untrusted sources.

ChatGPT Cross Plugin Request Forgery

Segregation and Control of External Content Interaction: Segregate untrusted
content from user prompts and control the interaction with external content,
especially with plugins that could result in irreversible actions or exposure of
personally identifiable information (PII).

Manage Trust: Establish trust boundaries between the LLM, external sources,and extensible functionality (e.g., plugins or downstream functions). Treat the
LLM as an untrusted user and maintain final user control on decision making
processes.

with Codand Prompt Injectio
Defending ChatGPT against Jailbreak
Attack via Self-Reminde
Prompt Injection attack against LLMintegrated Application
Inject My PDF: Prompt Injection foryour Resum
ChatML for OpenAI API Call
Not what you’ve signed up for:Compromising Real-World LLMIntegrated Applications with Indirect
Prompt Injectio
Threat Modeling LLM Application
AI Injections: Direct and Indirect
Prompt Injections and Their
Implications

OWASP.org

4

| OWASP Top 10 for LLM

LLM02: Insecure Output Handling

EXAMPLE ATTACK SCENARIOSFirst Published: July 1st, 2023

An application
utilizes an LLM plugin to
generate responses for achatbot feature. However, theapplication directly passes the
LLM-generated response into
an internal function responsible
for executing systemcommands without proper
validation. This allows an
attacker to manipulate the LLM
output to execute arbitrary
commands on the underlying
system, leading to unauthorizedaccess or unintended systemmodifications.
Scenario A:

An Insecure Output Handling vulnerability is a type of prompt injection
vulnerability that arises when a plugin or application blindly accepts large
language model (LLM) output without proper scrutiny and directly passes it tobackend, privileged, or client-side functions. Since LLM-generated content can becontrolled by prompt input, this behavior is akin to providing users indirectaccess to additional functionality.
Successful exploitation of an Insecure Output Handling vulnerability can result in
XSS and CSRF in web browsers as well as SSRF, privilege escalation, or remotecode execution on backend systems. The impact of this vulnerability increases
when the application allows LLM content to perform actions above the intended
user's privileges. Additionally, this can be combined with agent hijacking attacks
to allow an attacker privileged access into a target user's environment.
Common Vulnerabilities
Example 1: LLM output is entered directly into a backend function, resulting inremote code execution. Example 2: JavaScript or Markdown is generated by the
LLM and returned to a user. The code is then interpreted by the browser, resulting
in XSS
HOW TO PREVENT
Treat the model as any other user and apply proper input validation on responses
coming from the model to backend functions. Prevention Step 2: Likewise,
encode output coming from the model back to users to mitigate undesired
JavaScript or Markdown code interpretations.

A user utilizes a
website summarizer tool
powered by a LLM to generate aconcise summary of an article.
The website includes a prompt
injection instructing the LLM to
capture sensitive content fromeither the website or from theusers conversation. From there
the LLM can encode thesensitive data and send it out to
an attacker-controlled server
Scenario B:

C: A malicious user
instructs the LLM to return a
JavaScript payload back to auser, without sanitization
controls. This can occur either
through a sharing a prompt,
prompt injected website, or
chatbot that accepts prompts
from a GET request. The LLM
would then return theunsanitized XSS payload backto the user. Without additional
filters, outside of those
expected by the LLM itself, the
JavaScript would execute
within the users browser.
Scenario

REFERENCE LINKS
Arbitrary Code Execution- Snyk Vulnerabilit
ChatGPT Plugin Exploit Explained: From Prompt Injection to Accessing Private Dat
New prompt injection attack on ChatGPT web version. Markdown images can steal yourchat data
Don't blindly trust LLM responses. Threats to chatbots
Threat Modeling LLM Applications

OWASP.org

5

| OWASP Top 10 for LLM

LLM03: Training Data Poisioning

C

EXAMPLE ATTACK SENARIOSFirst Published: July 1st, 2023

The LLM
generative AI prompt output
can mislead users of theapplication which can lead tounbiased opinions, followings
or even worse, hate crimes etc.
Scenario A:

Large language models (LLMs) use diverse raw text to learn and generate
outputs. However, training data poisoning, where an attacker introduces
vulnerabilities, can compromise the model, exposing users to incorrect
information. The OWASP List for LLMs highlights the risk of overreliance on AI
content. Key data sources include Common Crawl, used for models like T5 andGPT-3; WebText and OpenWebText, containing public news and Wikipedia; andbooks, which make up 16% of GPT-3's training data.

Common Vulnerabilities

If the training data
is not correctly filtered, amalicious user of theapplication may try to influenceand in ect toxic data into themodel for it to adapt to theunbiased and false data.
Scenario B:

j

Example 1: A malicious actor, or a competitor brand intentionally createsinaccurate or malicious documents which are targeted at a model’s training data
The victim model trains using falsified information which is reflected in
outputs of generative AI prompts to it's consumers.
Example 2: In reverse, unintentionally, a model is trained using data which has
not been verified by its source, origin or content.
Example 3: The model itself has unrestricted access or inadequate sandboxing
to gather datasets to be used as training data which has negative influence on
outputs of generative AI prompts as well as loss of control from a management
perspective.

C A malicious actor,
or a competitor brandintentionally creates inaccurate
or malicious documents which
are targeted at a model’s
training data. The victim modeltrains using falsified
information which is reflectedin outputs of generative AI
prompts to it's consumers.
Scenario:V

HOW TO PRE

ENT

erify supply chain of the Training ata if sourced externally as well as
maintaining attestations, similar to S OM (Software ill of Materials)
methodology
Prevention Step 2 erify legitimacy of data sources and data within
Prevention Step 3 Craft different models via separate Training ata fordifferent use-cases to create a more granular and accurate generative AI
outputPrevention Step
nsure su cient sandboxing (LLM03 2023 - Inadequate
Sandboxing) is present to prevent the model from scraping unintended data
sourcesPrevention Step 5 se strict vetting or input filters for specific Training ata,
or categories of data sources to control volume of falsified data
Prevention Step 6 Implement dedicated LLM's to benchmark against
undesired consequences and train other LLM's using reinforcement learningtechniques
Optional Prevention Step Perform LLM-based red team exercises or LLM
vulnerability scanning into the testing phases of the LLM's lifecycle.
V

D

B

B

: V

:

D

4: E

ffi

: U

:

D

C

REFEREN

E LINKS

Stanford Resarch Pape
AI Hypocrisy: OpenAI, Google, andAnthropic train using others' content,
but wont let others use their conten
Inject My PDF: Prompt Injection foryour Resum
How data poisoning attacks corrupt
machine learning model
LLM10:2023 - Training Data Poisoning:

7:

OWASP.org

6

| OWASP Top 10 for LLM

LLM04: Denial of Service

EXAMPLE ATTACK SCENARIOSFirst Published: July 1st, 2023

An attacker interacts with an LLM in a way that is particularly resourceconsuming, causing quality of service to degrade for them and other users, or forhigh resource costs to be incurred.
Common Vulnerabilities
Example 1: Posing queries that lead to recurring resource usage through highvolume generation of tasks in a queue, e.g. with LangChain or AutoGPTExample 2: Sending queries that are unusually resource-consuming, perhaps
because they use unusual orthography or sequences
HOW TO PREVENTCap resource use per reques
Cap resource use per step, so that requests involving complex parts execute
more slowl
Limit the number of queued actions and the number of total actions in asystem reacting to LLM responses

Scenario A: An attacker
repeatedly sends multiple
requests to a hosted model thatare difficult and costly for it toprocess. Many resources areallocated, leading to worse
service for other users andincreased resource bills for the
host.
Scenario B: A piece of text on a
webpage is encountered while
an LLM-driven tool is collecting
information to respond to abenign query. That piece of text
leads to the tool making many
more web page requests. The
query ends up leading to large
amounts of resource
consumption, and receives aslow or even absent response,as do any other queries fromsimilar systems that end up
encountering the given piece oftext.

REFERENCE LINKS

LangChain max_iteration
Sponge Examples: Energy-Latency
Attacks on Neural Networks

OWASP.org

7

| OWASP Top 10 for LLM

LLM05: Supply Chain

EXAMPLE ATTACK SCENARIOSFirst Published: July 1st, 2023

The supply chain in LLMs can be vulnerable impacting the integrity of training
data, ML models, deployment platfoms and lead to biased outcomes, security
breaches, or complete system failures. Traditionally, vulnerabilities focused on
software components but is extended in AI because of the popularity of transfer
learning, re-use of pre-trained models, as well as crowdsourced data. In public
LLMs such as OpenGPT extension plugins are also an area susceptible to thisvulnerability.

Common Vulnerabilities
Example 1: Use of Vulnerable Model used for Transfer Learning
Example 2: Use of poisoned crowd-sourced data
Example 3: Tampered model or data by an out-sourcing supplier
Example 4: Traditional third-paty Package Vulnerabilities

HOW TO PREVENT
Careful Vetting of sources and supplier

Vulnerabilities scanning of components, not only when deploying to
production but before used in development and testing; model development
environment
Use own curated package repositories with vulnerability check
Code Signin
Avdersarial robustness tests on supplied models and data for tampering andpoisoning and throughout the MLOps pipelin
Implement adversarial robustness training to help detect extraction queries
Review and Monitor Supplier Security and Acces
Auditing

REFERENCE LINKS
ChatGPT Data Breach Confirmed as Security Firm Warns of Vulnerable Component
Exploitatio
What Happens When an AI Company Falls Victim to a Software Supply Chain Vulnerabilit
Plugin review proces
Compromised PyTorch-nightly dependency chai
Failure Modes in Machine Learnin
ML Supply Chain Compromis
Transferability in Machine Learning: from Phenomena to Black-Box Attacks using
Adversarial Sample
BadNets: Identifying Vulnerabilities in the Machine Learning Model Supply Chai
VirusTotal Poisoning

OWASP.org

Scenario A: An attacker

exploits a vulnerable package
to compromise a system. The
recent OpenAI breach, was due
to a vulnerable third party
package

Scenario B: An attacker

exploits a malicious orvulnerable ChatGPT plugin to
exfiltrate data, bypass
restriction,s execute code, spana user or produce malicious
content. Although this is a
supply-chain vulnerability,
because of the ChatGPT plugun
integration models it is also
covered in Insecure Plugin
Integration

Scenario C: An attacker

exploits the PyPi package
registry to trick model
developers to download acompromised package andexfiltrate data or escalate
privilege in a model
development environment.

Scenario D: An attacker

poisons or tampers a copy of
publicly available model prebuilt model (e.g. LlaMa ) to
create a backdoor or trojan
horse deploys it for others touse either directly or fortransfer learning

Scenario E: An attacker

poisons or tampers a copy of
publicly available data set (e.g.
Kaggle ) to help create abackdoor or trojan horse inother models.

8

| OWASP Top 10 for LLM

LLM06: Permission Issues

EXAMPLE ATTACK SCENARIOSFirst Published: July 1st, 2023

Authorization is not tracked between plugins, allowing a malicious actor to takeaction in the context of the LLM user via indirect prompt injection, use of
malicious plugins, or other methods. This can result in privilege escalation, lossof confidentiality, and even remote code execution, depending on available
plugins.

Scenario A: Indirect prompt

injection is used to induce anemail plugin to deliver thecontents of the current user's
inbox to a malicious URL via
POST request.
Scenario B: An attacker uses

Common Vulnerabilities
Example 1: Authentication is performed without explicit authorization to aparticular plugin.
Example 2: A plugin treats all LLM content as being created entirely by the user,
and performs any requested actions without requiring additional authorization.
Example 3: Plugins are chained together without considering the authorization of
one plugin to perform an action using another plugin.
HOW TO PREVENT
Require manual authorization of any action taken by sensitive plugins
Call no more than one plugin with each user input, resetting any plugin-supplied data between calls
Prevent sensitive plugins from being called after any other plugin
Perform taint tracing on all plugin content, ensuring that plugin is called with
an authorization level corresponding to the lowest authorization of any pluginthat has provided input to the LLM prompt.

OWASP.org

indirect prompt injection toabuse a Slack integration,
sending a Slack message to@everyone in all available
Slacks with an obscene anddefamatory comment.

REFERENCE LINKS
Plugin Vulnerabilities: Visit a Website
and Have Your Source Code Stole
ChatGPT Plugin Exploit Explained:
From Prompt Injection to Accessing
Private Data

9

| OWASP Top 10 for LLM

LLM07: Data Leakage

EXAMPLE ATTACK SCENARIOSFirst Published: July 1st, 2023

Data leakage occurs when an LLM accidentally reveals sensitive information,
proprietary algorithms, or other confidential details through its responses. Thiscan result in unauthorized access to sensitive data or intellectual property,
privacy violations, and other security breaches. It's important to note that usersof an LLM application should be aware of how to interact with an LLM and
identify the risks present on how they might unintentionally input sensitive data.
Vice versa, an LLM application should perform adequate data sanitization andscrubbing validation in aid to prevent user data from entering the training model
data. Furthermore, the company hosting the LLM should have appropriate Termsof User policies available to make users aware on how data is processed.
Common Vulnerabilities
Example 1: Incomplete or improper filtering of sensitive information in the LLM’s
responses.
Example 2: Overfitting or memorization of sensitive data in the LLM’s training
process.
Example 3: Unintended disclosure of confidential information due to LLM
misinterpretation, lack of data scrubbing methods or errors.

Scenario A: Unsuspecting
legitimate user A is exposed tocertain other user data via the
LLM when interacting with the
LLM application in a nonmalicious manner.
Scenario B: User A targets awell crafted set of prompts to
bypass input filters andsanitization from the LLM tocause it to reveal sensitive
information (I.E PII) about otherusers of the application.

Scenario C: Personal data suchas PII is leaked into the model

via training data due to either
negligence from the user
themselves, or the LLM
application. This case could
increase risk and probability of
scenario 1 or 2 above.

HOW TO PREVENT

Integrate adequate data sanitization and scrubbing techniques in aid toprevent user data from entering the training model data
Implement robust input validation and sanitization methods to identify andfilter out potential malicious inputs
Maintain ongoing supply chain mitigation of risk through techniques such as
SAST (Static Application Security Testing) and SBOM (Software Bill ofMaterials) attestations to identify and remediate vulnerabilities independencies for third-party software or packages
Implement dedicated LLM's to benchmark against undesired consequences
and train other LLM's using reinforcement learning techniques
Perform LLM-based red team exercises or LLM vulnerability scanning into thetesting phases of the LLM's lifecycle.

OWASP.org

REFERENCE LINKS

AI data leak crisis: New tool prevents
company secrets from being fed to
ChatGP
Lessons learned from ChatGPT’s
Samsung lea
Cohere - Terms Of Use

10

| OWASP Top 10 for LLM

LLM08: Excessive Agency

EXAMPLE ATTACK SCENARIOSFirst Published: July 1st, 2023

An LLM may be granted a degree of agency - the ability to interface with othersystems in order to undertake actions. Without restriction, any undesireable
operation of the LLM (regardless of the root casue; e.g., halucination, direct/
indirect prompt injection, or just poorly-engineered benign prompts, etc) may
result in undesireable actions being taken. Just like we never trust client-side
validation in web-apps, LLMs should not be trusted to self-police or self-restrict;controls should be embedded in the APIs of the systems being interefaced with.
Common Vulnerabilities
Example 1: Undesireable Actions Performed: The LLM triggers actions outside of
the LLM that are unintended or undesireable, leading to second orderconsequences on downstream systems and processes.
HOW TO PREVENT
Reduce the permissions granted to the LMM to the minimum necessary tolimit the scope of undesirable actions
Implement rate-limiting to reduce the number of undesirable actions
Utilize human-in-the-loop control to require a human to approve all actions
before they are taken.

OWASP.org

Scenario A: A personal
assistant LLM is granted
access to an individuals's
mailbox in order to summarise
the content of incoming emails.
The LMM is vulnerable to anindirect promot injection attack,whereby a maliciously-crafted
incoming email tricks the LLM
into sending spam messages
from the user's mailbox. This
could be avoided by only
granting the LLM read-onlyaccess to the mailbox (not theability to send messages), or byrequiring the user to manuallyreview and hit 'send' on every
mail drafted by the LLM.
Alternatively The damage
caused could be reduced by
implementing rate limiting on
the mail-sending interface.
Scenario B: A customer service
LLM has an interface to apayments system to provide
service credits or refunds tocustomers in the case ofcomplaints. The system prompt
instructs the LLM to limit
refunds to no more than one
month's subscription fee,however a malicious customer
engineers a direct prompt
injection attack to convince theLMM to issue a refund of 100
years of fees. This could be
avoided by implementing the'one month max' limit within therefund API, rather than relying
on the LMM to honour the limit
in it's system prompt.

11

| OWASP Top 10 for LLM

LLM09: Overreliance

EXAMPLE ATTACK SCENARIOSFirst Published: July 1st, 2023

Overreliance on LLMs is a security vulnerability that arises when systems
excessively depend on LLMs for decision-making or content generation without
adequate oversight, validation mechanisms, or risk communication. LLMs, whilecapable of generating creative and informative content, are also susceptible to"hallucinations," producing content that is factually incorrect, nonsensical, or
inappropriate. These hallucinations can lead to misinformation,
miscommunication, potential legal issues, and damage to a company's
reputation if unchecked.
Common Vulnerabilities

Factually Incorrect Information: An LLM provides incorrect information as a
response, leading to misinformation. For example, an LLM may inaccurately
describe historical events, resulting in misleading outputs.
Nonsensical Outputs: An LLM generates grammatically correct but logicallyincoherent or nonsensical text. For instance, the LLM might generate a poem or astory that doesn't make logical sense.
Source Conflation: LLM conflates information from different sources, creating
misleading content. It might combine historical data with current events in anincorrect manner.
Overindulgence: LLM might generate an output that could incorrectly be seen as
disclosure of confidential information.
Inadequate Risk Communication: Tech companies fail to adequately
communicate the inherent risks of using LLMs to the end users, leading topotential negative consequences.

Scenario A: A corporation uses
an LLM to generate customerfacing content. Due to ahallucination, the LLM
generates incorrect information
about a product, leading tocustomer confusion, potential
loss of sales, and damage tothe company's reputation.
Scenario B: An LLM is used in anews organization to assist in
generating news articles. The
LLM conflates information fromdifferent sources and produces
an article with misleading
information, leading to thedissemination of
misinformation and potential
legal consequences.
Scenario C: A user
communicates with an AI
chatbot based on an LLM. The
user, unaware of the limitations
and risks of the AI, acts on
harmful content generated by
the model due to the lack ofeffective risk communicationfrom the tech company.

HOW TO PREVENT
Continuous Monitoring: Regularly monitor and review the outputs of the LLM
to ensure they are factual, coherent, and appropriate. Use manual reviews or
automated tools for larger scale applications
Fact Checking: Verify the accuracy of information provided by LLMs before
using it for decision-making, information dissemination, or other critical
functions
Model Tuning: Tune your LLM to reduce the likelihood of hallucinations.
Techniques can include prompt engineering, parameter efficient tuning (PET),
and full model tuning
Set Up Validation Mechanisms: Implement automatic validation mechanisms
to check the generated output against known facts or data
Improve Risk Communication: Follow best practices from risk
communication literature and lessons from other sectors to facilitate
dialogue with users, establish actionable risk communication, and measure
the effectiveness of risk communications on an ongoing basis.

OWASP.org

REFERENCE LINKSHow Should Companies Communicate
the Risks of Large Language Models to
Users
Understanding LLM Hallucinations

12

| OWASP Top 10 for LLM

LLM10: Insecure Plugins

EXAMPLE ATTACK SCENARIOSFirst Published: July 1st, 2023

A plugin designed to connect an LLM to some external resource accepts freeform text as an input instead of parameterized and type-checked inputs. This
allows a potential attacker significant latitude to construct a malicious request to
the plugin that could result in a wide range of undesired behaviors, up to andincluding remote code execution.

Common Vulnerabilities
A plugin designed to call a specific API hosted at a specific endpoint accepts astring containing the entire URL to be retrieved, instead of query parameters to beinserted into the URL.
A plugin designed to look up information from a SQL database accepts a raw
SQL query rather than paramters to be inserted into a fully parameterized query.

HOW TO PREVENT
Plugin calls should be strictly parameterized wherever possible, including typeand range checks on input
When freeform input must be accepted, it should be carefully inspected toensure that no potentially harmful methods are being called
The plugin should be designed from a least-privilege perspective, exposing as
little functionality as possible while still performing its desired function.

OWASP.org

Scenario: A plugin prompt
provides a base URL and
instructs the LLM to combine
the URL with a query to obtainweather forecasts in response
to user requests. The resulting
URL is then accessed and theresults shown to the user. A
malicious user crafts a request
such that a URL pointing to adomain they control, and not
the URL hosting the weather
service API, is accessed,
allowing the malicious user toobtain the IP address of theplugin for further
reconnaisance, as well as toinject their own text into the
LLM system via their domain,
potentially granting themfurther access to downstream
plugins.

13

| OWASP Top 10 for LLM

Core Team & Contributors
Core Team Members are listed in Blue

Adrian Culley

Trellix

John Sotiropoulos

Ads Dawson

CohereJoshua Nussbau

Kaino

s

m

Alexander Zai

Kai Greshake

Amay Trivedi

Kelvin Lo

w

Aigos

Andrea Succi

Ken AroraF

Ken Huang

DistributedApps.ai

Andrew Amaro

Andy Dyrcz

Klavan Security Group

Linkfire

Andy Smith

Ananda Krishna

Ashish Rajan

Astra Security

Cloud Security Podcast

Autumn Moulder

Bilal Siddiqui

Trustwave

5

Larry Carson

U of Washington IT U of Copenhage

Leonardo Shikida

IB

Lior Drihe

HackerOne

Mike Finch

HackerOne

Nathan Hamie

Brodie McRaeAWS

Nir Pazl

Dan Frommer

Otto Sulin

David RowePatrick Biyaga

Emanuel Valente

Emmanuel Guilherme JuniorKudelski Security

Nordic Venture FamilyoThenavig

Priyadharshini Parthasarathy

Coalfire

AWARE7

Rahul Zhade

Git u

iFood

Rachit Sood

McMaster University

Reza Rashidi

 H b

HADESSRich Harang

Eugene Neelou

Eugene Tawiah

M

Manjesh S

AVID

Dr. Matteo Große-KampmannnmBrian Pendleton

David Taylor,

Leon DerczynskioComplex Technologies

Santosh Kumar

Cisc

Guillaume Ehinger

GoogleSarah Thornton

Red atGavin Klondike

AI Village

Steve Wilson

H

Contrast Security

Vandana Verma Sehga

Itamar Golan

James Rabe

IriusRisk

Vinay Vishwanatha

Jason Haddix

Buddobot

Vladimir Fedoto

Jason Ross

Salesforce

Vishwas Manra

Jeff Williams

Contrast Security

Jinson Varghese Behanan

OWASP.org

v

Will Chilcutt

l

Snyk

r

Sprinkl

M

EPA

l

Yahoo

Astra Security

14