The product has a component that relies on a
generative AI/ML model configured with inference parameters that
produce an unacceptably high rate of erroneous or unexpected
outputs.

Generative AI/ML models, such as those used for text
generation, image synthesis, and other creative tasks, rely on
inference parameters that control model behavior, such as
temperature, Top P, and Top K. These parameters affect the
model's internal decision-making processes, learning rate, and
probability distributions. Incorrect settings can lead to
unusual behavior such as text "hallucinations," unrealistic
images, or failure to converge during training. The impact of
such misconfigurations can compromise the integrity of the
application. If the results are used in security-critical
operations or decisions, then this could violate the intended
security policy, i.e., introduce a vulnerability.

Primary

Build and Compilation
During model training, hyperparameters may be set
without adequate validation or understanding of their
impact.

Installation
During deployment, model parameters may be adjusted
to optimize performance without comprehensive
testing.

Patching and Maintenance
Updates or modifications may be made to the model
that alter its behavior without thorough
re-evaluation.

Integrity
Other
Varies by Context
Unexpected State
The product can generate inaccurate, misleading, or
nonsensical information.

Other
Alter Execution Logic
Unexpected State
Varies by Context
If outputs are used in critical decision-making
processes, errors could be propagated to other systems or
components.

Automated Dynamic Analysis
Manipulate inference parameters and perform
comparative evaluation to assess the impact of selected
values. Build a suite of systems using targeted tools that
detect problems such as prompt injection (CWE-1427) and
other problems. Consider statistically measuring token
distribution to see if it is consistent with expected
results.
Moderate
Given the large variety of outcomes,
it can be difficult to design testing to be comprehensive
enough, and there is still a risk of unpredictable
behavior.

Manual Dynamic Analysis
Manipulate inference parameters and perform
comparative evaluation to assess the impact of selected
values. Build a suite of systems using targeted tools that
detect problems such as prompt injection (CWE-1427) and
other problems. Consider statistically measuring token
distribution to see if it is consistent with expected
results.
Moderate
Given the large variety of outcomes,
it can be difficult to design testing to be comprehensive
enough, and there is still a risk of unpredictable
behavior.

Implementation
System Configuration
Operation
Develop and adhere to robust parameter tuning
processes that include extensive testing and
validation.

Implementation
System Configuration
Operation
Implement feedback mechanisms to continuously
assess and adjust model performance.

Documentation
Provide comprehensive documentation and
guidelines for parameter settings to ensure consistent and
accurate model behavior.

Assume the product offers an
LLM-based AI coding assistant to help users to write code
as part of an Integrated Development Environment (IDE).
Assume the model has been trained on real-world code, and
the model behaves normally under its default settings.
Suppose there is a default temperature of 1, with a range
of temperature values from 0 (most deterministic) to
2.
Consider the following configuration.

{

"model": "my-coding-model",
"context_window": 8192,
"max_output_tokens": 4096,
"temperature", 1.5,
...

}

The problem is that the configuration contains a temperature
hyperparameter that is higher than the default. This significantly
increases the likelihood that the LLM will suggest a package that did
not exist at training time, a behavior sometimes referred to as
"package hallucination." Note that other possible behaviors could
arise from higher temperature, not just package hallucination.
An adversary could anticipate which package names could be generated
and create a malicious package. For example, it has been observed that
the same LLM might hallucinate the same package regularly. Any code
that is generated by the LLM, when run by the user, would download and
execute the malicious package. This is similar to typosquatting.
The risk could be reduced by lowering the temperature so that it
reduces the unpredictable outputs and has a better chance of staying
more in line with the training data. If the temperature is set too
low, then some of the power of the model will be lost, and it may be
less capable of producing solutions for rarely-encountered problems
that are not reflected in the training data. However, if the
temperature is not set low enough, the risk of hallucinating package
names may still be too high. Unfortunately, the "best" temperature
cannot be determined a priori, and sufficient empirical testing is
needed.

{

...
"temperature", 0.2,
...

}

In addition to more restrictive temperature settings, consider adding
guardrails that test that independently verify any referenced package
to ensure that it exists, is not obsolete, and comes from a trusted
party.
Note that reducing temperature does not entirely eliminate the risk of
package hallucination. Even with very low temperatures or other
settings, there is still a small chance that a non-existent package
name will be generated.

Allowed
This CWE entry is at the Base level of abstraction, which is a preferred level of abstraction for mapping to the root causes of vulnerabilities.
Carefully read both the name and description to ensure that this mapping is an appropriate fit. Do not try to 'force' a mapping to a lower-level Base/Variant simply to comply with this preferred level of abstraction.

This weakness might be under-reported as of CWE 4.18,
since there are no clear observed examples in
CVE. However, inference parameters may be the root cause
for various vulnerabilities - or important factors - but
the vulnerability reports may concentrate more on the
negative impact (e.g. code execution) or the weaknesses
that the insecure settings contribute to. Alternately,
dynamic techniques might not reveal the root cause if the
researcher does not have access to the underlying source
code and environment.

Lily Wong
MITRE
2024-06-28
4.18
2025-09-09

AI WG "New Entry" subgroup
2025-02-28
4.18
2025-09-09
Participated in regular meetings from February to August 2025 to develop and refine most elements of this entry.