deardaniel/secdata-raw huggingface dataset:
CWE-163: Improper Neutralization of Multiple Trailing Special Elements

Description:
The product receives input from an upstream component, but it does not neutralize or incorrectly neutralizes multiple trailing special elements that could be interpreted in unexpected ways when they are sent to a downstream component.

Extended Description:
As data is parsed, improperly handled multiple trailing special elements may cause the process to take unexpected actions that result in an attack.

Affected languages: Not Language-Specific
Affected technologies: Not Technology-Specific
Parent weaknesses: CWE-162

Consequences:
  - Integrity: Unexpected State

Mitigations:
  - Developers should anticipate that multiple trailing special elements will be injected/removed/manipulated in the input vectors of their product. Use an appropriate combination of denylists and allowlists to ensure only valid, expected and appropriate input is processed by the system.
  - [Implementation] Assume all input is malicious. Use an "accept known good" input validation strategy, i.e., use a list of acceptable inputs that strictly conform to specifications. Reject any input that does not strictly conform to specifications, or transform it into something that does. 
                   When pe
  - [Implementation] While it is risky to use dynamically-generated query strings, code, or commands that mix control and data together, sometimes it may be unavoidable. Properly quote arguments and escape any special characters within those arguments. The most conservative approach is to escape or filter all characters
  - [Implementation] Inputs should be decoded and canonicalized to the application's current internal representation before being validated (CWE-180). Make sure that the application does not decode the same input twice (CWE-174). Such errors could be used to bypass allowlist validation schemes by introducing dangerous i

Real-world examples:
  - CVE-2002-1078: Directory listings in web server using multiple trailing slash
  - CVE-2004-0281: Multiple trailing dot allows directory listing
CWE-164: Improper Neutralization of Internal Special Elements

Description:
The product receives input from an upstream component, but it does not neutralize or incorrectly neutralizes internal special elements that could be interpreted in unexpected ways when they are sent to a downstream component.

Extended Description:
As data is parsed, improperly handled internal special elements may cause the process to take unexpected actions that result in an attack.

Affected languages: Not Language-Specific
Parent weaknesses: CWE-138

Consequences:
  - Integrity: Unexpected State

Mitigations:
  - Developers should anticipate that internal special elements will be injected/removed/manipulated in the input vectors of their product. Use an appropriate combination of denylists and allowlists to ensure only valid, expected and appropriate input is processed by the system.
  - [Implementation] Assume all input is malicious. Use an "accept known good" input validation strategy, i.e., use a list of acceptable inputs that strictly conform to specifications. Reject any input that does not strictly conform to specifications, or transform it into something that does. 
                   When pe
  - [Implementation] While it is risky to use dynamically-generated query strings, code, or commands that mix control and data together, sometimes it may be unavoidable. Properly quote arguments and escape any special characters within those arguments. The most conservative approach is to escape or filter all characters
  - [Implementation] Inputs should be decoded and canonicalized to the application's current internal representation before being validated (CWE-180). Make sure that the application does not decode the same input twice (CWE-174). Such errors could be used to bypass allowlist validation schemes by introducing dangerous i
CWE-165: Improper Neutralization of Multiple Internal Special Elements

Description:
The product receives input from an upstream component, but it does not neutralize or incorrectly neutralizes multiple internal special elements that could be interpreted in unexpected ways when they are sent to a downstream component.

Extended Description:
As data is parsed, improperly handled multiple internal special elements may cause the process to take unexpected actions that result in an attack.

Affected languages: Not Language-Specific
Parent weaknesses: CWE-164

Consequences:
  - Integrity: Unexpected State

Mitigations:
  - Developers should anticipate that multiple internal special elements will be injected/removed/manipulated in the input vectors of their product. Use an appropriate combination of denylists and allowlists to ensure only valid, expected and appropriate input is processed by the system.
  - [Implementation] Assume all input is malicious. Use an "accept known good" input validation strategy, i.e., use a list of acceptable inputs that strictly conform to specifications. Reject any input that does not strictly conform to specifications, or transform it into something that does. 
                   When pe
  - [Implementation] While it is risky to use dynamically-generated query strings, code, or commands that mix control and data together, sometimes it may be unavoidable. Properly quote arguments and escape any special characters within those arguments. The most conservative approach is to escape or filter all characters
  - [Implementation] Inputs should be decoded and canonicalized to the application's current internal representation before being validated (CWE-180). Make sure that the application does not decode the same input twice (CWE-174). Such errors could be used to bypass allowlist validation schemes by introducing dangerous i
CWE-166: Improper Handling of Missing Special Element

Description:
The product receives input from an upstream component, but it does not handle or incorrectly handles when an expected special element is missing.

Affected languages: Not Language-Specific
Parent weaknesses: CWE-159, CWE-228

Consequences:
  - Availability: DoS: Crash, Exit, or Restart

Mitigations:
  - Developers should anticipate that special elements will be removed in the input vectors of their product. Use an appropriate combination of denylists and allowlists to ensure only valid, expected and appropriate input is processed by the system.
  - [Implementation] Assume all input is malicious. Use an "accept known good" input validation strategy, i.e., use a list of acceptable inputs that strictly conform to specifications. Reject any input that does not strictly conform to specifications, or transform it into something that does. 
                   When pe
  - [Implementation] Inputs should be decoded and canonicalized to the application's current internal representation before being validated (CWE-180). Make sure that the application does not decode the same input twice (CWE-174). Such errors could be used to bypass allowlist validation schemes by introducing dangerous i

Real-world examples:
  - CVE-2002-1362: Crash via message type without separator character
  - CVE-2002-0729: Missing special character (separator) causes crash
  - CVE-2002-1532: HTTP GET without \r\n\r\n CRLF sequences causes product to wait indefinitely and prevents other users from accessing it
CWE-167: Improper Handling of Additional Special Element

Description:
The product receives input from an upstream component, but it does not handle or incorrectly handles when an additional unexpected special element is provided.

Affected languages: Not Language-Specific
Parent weaknesses: CWE-159, CWE-228

Consequences:
  - Integrity: Unexpected State

Mitigations:
  - Developers should anticipate that extra special elements will be injected in the input vectors of their product. Use an appropriate combination of denylists and allowlists to ensure only valid, expected and appropriate input is processed by the system.
  - [Implementation] Assume all input is malicious. Use an "accept known good" input validation strategy, i.e., use a list of acceptable inputs that strictly conform to specifications. Reject any input that does not strictly conform to specifications, or transform it into something that does. 
                   When pe
  - [Implementation] While it is risky to use dynamically-generated query strings, code, or commands that mix control and data together, sometimes it may be unavoidable. Properly quote arguments and escape any special characters within those arguments. The most conservative approach is to escape or filter all characters
  - [Implementation] Inputs should be decoded and canonicalized to the application's current internal representation before being validated (CWE-180). Make sure that the application does not decode the same input twice (CWE-174). Such errors could be used to bypass allowlist validation schemes by introducing dangerous i

Real-world examples:
  - CVE-2000-0116: Extra "<" in front of SCRIPT tag bypasses XSS prevention.
  - CVE-2001-1157: Extra "<" in front of SCRIPT tag.
  - CVE-2002-2086: "<script" - probably a cleansing error
CWE-168: Improper Handling of Inconsistent Special Elements

Description:
The product does not properly handle input in which an inconsistency exists between two or more special characters or reserved words.

Extended Description:
An example of this problem would be if paired characters appear in the wrong order, or if the special characters are not properly nested.

Affected languages: Not Language-Specific
Parent weaknesses: CWE-159, CWE-228

Consequences:
  - Availability: DoS: Crash, Exit, or Restart

Mitigations:
  - Developers should anticipate that inconsistent special elements will be injected/manipulated in the input vectors of their product. Use an appropriate combination of denylists and allowlists to ensure only valid, expected and appropriate input is processed by the system.
  - [Implementation] Assume all input is malicious. Use an "accept known good" input validation strategy, i.e., use a list of acceptable inputs that strictly conform to specifications. Reject any input that does not strictly conform to specifications, or transform it into something that does. 
                   When pe
  - [Implementation] Inputs should be decoded and canonicalized to the application's current internal representation before being validated (CWE-180). Make sure that the application does not decode the same input twice (CWE-174). Such errors could be used to bypass allowlist validation schemes by introducing dangerous i
CWE-172: Encoding Error

Description:
The product does not properly encode or decode the data, resulting in unexpected values.

Affected languages: Not Language-Specific
Parent weaknesses: CWE-707

Consequences:
  - Integrity: Unexpected State

Mitigations:
  - [Implementation] Assume all input is malicious. Use an "accept known good" input validation strategy, i.e., use a list of acceptable inputs that strictly conform to specifications. Reject any input that does not strictly conform to specifications, or transform it into something that does. 
                   When pe
  - [Implementation] While it is risky to use dynamically-generated query strings, code, or commands that mix control and data together, sometimes it may be unavoidable. Properly quote arguments and escape any special characters within those arguments. The most conservative approach is to escape or filter all characters
  - [Implementation] Inputs should be decoded and canonicalized to the application's current internal representation before being validated (CWE-180). Make sure that the application does not decode the same input twice (CWE-174). Such errors could be used to bypass allowlist validation schemes by introducing dangerous i

Real-world examples:
  - CVE-2004-1315: Forum software improperly URL decodes the highlight parameter when extracting text to highlight, which allows remote attackers to execute arbitrary PHP code by double-encoding the highlight value so t
  - CVE-2004-1939: XSS protection mechanism attempts to remove "/" that could be used to close tags, but it can be bypassed using double encoded slashes (%252F)
  - CVE-2001-0709: Server allows a remote attacker to obtain source code of ASP files via a URL encoded with Unicode.
  - CVE-2005-2256: Hex-encoded path traversal variants - "%2e%2e", "%2e%2e%2f", "%5c%2e%2e"
CWE-173: Improper Handling of Alternate Encoding

Description:
The product does not properly handle when an input uses an alternate encoding that is valid for the control sphere to which the input is being sent.

Affected languages: Not Language-Specific
Parent weaknesses: CWE-172

Consequences:
  - Access Control: Bypass Protection Mechanism

Mitigations:
  - [Architecture and Design] Avoid making decisions based on names of resources (e.g. files) if those resources can have alternate names.
  - [Implementation] Assume all input is malicious. Use an "accept known good" input validation strategy, i.e., use a list of acceptable inputs that strictly conform to specifications. Reject any input that does not strictly conform to specifications, or transform it into something that does. 
                   When pe
  - [Implementation] Use and specify an output encoding that can be handled by the downstream component that is reading the output. Common encodings include ISO-8859-1, UTF-7, and UTF-8. When an encoding is not specified, a downstream component may choose a different encoding, either by assuming a default encoding or au
  - [Implementation] Inputs should be decoded and canonicalized to the application's current internal representation before being validated (CWE-180). Make sure that the application does not decode the same input twice (CWE-174). Such errors could be used to bypass allowlist validation schemes by introducing dangerous i
CWE-174: Double Decoding of the Same Data

Description:
The product decodes the same input twice, which can limit the effectiveness of any protection mechanism that occurs in between the decoding operations.

Affected languages: Not Language-Specific
Parent weaknesses: CWE-172, CWE-675

Consequences:
  - Access Control: Bypass Protection Mechanism

Mitigations:
  - [Architecture and Design] Avoid making decisions based on names of resources (e.g. files) if those resources can have alternate names.
  - [Implementation] Assume all input is malicious. Use an "accept known good" input validation strategy, i.e., use a list of acceptable inputs that strictly conform to specifications. Reject any input that does not strictly conform to specifications, or transform it into something that does. 
                   When pe
  - [Implementation] Use and specify an output encoding that can be handled by the downstream component that is reading the output. Common encodings include ISO-8859-1, UTF-7, and UTF-8. When an encoding is not specified, a downstream component may choose a different encoding, either by assuming a default encoding or au
  - [Implementation] Inputs should be decoded and canonicalized to the application's current internal representation before being validated (CWE-180). Make sure that the application does not decode the same input twice (CWE-174). Such errors could be used to bypass allowlist validation schemes by introducing dangerous i

Real-world examples:
  - CVE-2004-1315: Forum software improperly URL decodes the highlight parameter when extracting text to highlight, which allows remote attackers to execute arbitrary PHP code by double-encoding the highlight value so t
  - CVE-2004-1939: XSS protection mechanism attempts to remove "/" that could be used to close tags, but it can be bypassed using double encoded slashes (%252F)
  - CVE-2001-0333: Directory traversal using double encoding.
  - CVE-2004-1938: "%2527" (double-encoded single quote) used in SQL injection.
  - CVE-2005-1945: Double hex-encoded data.
CWE-175: Improper Handling of Mixed Encoding

Description:
The product does not properly handle when the same input uses several different (mixed) encodings.

Affected languages: Not Language-Specific
Parent weaknesses: CWE-172

Consequences:
  - Integrity: Unexpected State

Mitigations:
  - [Architecture and Design] Avoid making decisions based on names of resources (e.g. files) if those resources can have alternate names.
  - [Implementation] Assume all input is malicious. Use an "accept known good" input validation strategy, i.e., use a list of acceptable inputs that strictly conform to specifications. Reject any input that does not strictly conform to specifications, or transform it into something that does. 
                   When pe
  - [Implementation] Use and specify an output encoding that can be handled by the downstream component that is reading the output. Common encodings include ISO-8859-1, UTF-7, and UTF-8. When an encoding is not specified, a downstream component may choose a different encoding, either by assuming a default encoding or au
  - [Implementation] Inputs should be decoded and canonicalized to the application's current internal representation before being validated (CWE-180). Make sure that the application does not decode the same input twice (CWE-174). Such errors could be used to bypass allowlist validation schemes by introducing dangerous i
CWE-176: Improper Handling of Unicode Encoding

Description:
The product does not properly handle when an input contains Unicode encoding.

Affected languages: Not Language-Specific
Affected technologies: Not Technology-Specific
Parent weaknesses: CWE-172

Consequences:
  - Integrity: Unexpected State

Mitigations:
  - [Architecture and Design] Avoid making decisions based on names of resources (e.g. files) if those resources can have alternate names.
  - [Implementation] Assume all input is malicious. Use an "accept known good" input validation strategy, i.e., use a list of acceptable inputs that strictly conform to specifications. Reject any input that does not strictly conform to specifications, or transform it into something that does. 
                   When pe
  - [Implementation] Inputs should be decoded and canonicalized to the application's current internal representation before being validated (CWE-180). Make sure that the application does not decode the same input twice (CWE-174). Such errors could be used to bypass allowlist validation schemes by introducing dangerous i

Demonstrative examples:
Windows provides the MultiByteToWideChar(), WideCharToMultiByte(), UnicodeToBytes(), and BytesToUnicode() functions to convert between arbitrary multibyte (usually ANSI) character strings and Unicode (wide character) strings. The size arguments to these functions are specified in different units, (one in bytes, the other in characters) making their use prone to error.
In a multibyte character string, each character occupies a varying number of bytes, and therefore the size of such strings is most easily specified as a total number of bytes. In Unicode, however, characters are always a fixed size, and string lengths are typically given by the number of characters they contain. Mistakenly specifying the wrong units in a size argument can lead to a buffer overflow.
void getUserInfo(char *user

Real-world examples:
  - CVE-2000-0884: Server allows remote attackers to read documents outside of the web root, and possibly execute arbitrary commands, via malformed URLs that contain Unicode encoded characters.
  - CVE-2001-0709: Server allows a remote attacker to obtain source code of ASP files via a URL encoded with Unicode.
  - CVE-2001-0669: Overlaps interaction error.
CWE-177: Improper Handling of URL Encoding (Hex Encoding)

Description:
The product does not properly handle when all or part of an input has been URL encoded.

Affected languages: Not Language-Specific
Parent weaknesses: CWE-172

Consequences:
  - Integrity: Unexpected State

Mitigations:
  - [Architecture and Design] Avoid making decisions based on names of resources (e.g. files) if those resources can have alternate names.
  - [Implementation] Assume all input is malicious. Use an "accept known good" input validation strategy, i.e., use a list of acceptable inputs that strictly conform to specifications. Reject any input that does not strictly conform to specifications, or transform it into something that does. 
                   When pe
  - [Implementation] Inputs should be decoded and canonicalized to the application's current internal representation before being validated (CWE-180). Make sure that the application does not decode the same input twice (CWE-174). Such errors could be used to bypass allowlist validation schemes by introducing dangerous i

Real-world examples:
  - CVE-2000-0900: Hex-encoded path traversal variants - "%2e%2e", "%2e%2e%2f", "%5c%2e%2e"
  - CVE-2005-2256: Hex-encoded path traversal variants - "%2e%2e", "%2e%2e%2f", "%5c%2e%2e"
  - CVE-2004-2121: Hex-encoded path traversal variants - "%2e%2e", "%2e%2e%2f", "%5c%2e%2e"
  - CVE-2004-0280: "%20" (encoded space)
  - CVE-2003-0424: "%20" (encoded space)
CWE-178: Improper Handling of Case Sensitivity

Description:
The product does not properly account for differences in case sensitivity when accessing or determining the properties of a resource, leading to inconsistent results.

Extended Description:
Improperly handled case sensitive data can lead to several possible consequences, including: 
                
                   case-insensitive passwords reducing the size of the key space, making brute force attacks easier 
                   bypassing filters or access controls using alternate names 
                   multiple interpretation errors using alternate names.

Affected languages: Not Language-Specific
Affected technologies: Not Technology-Specific
Parent weaknesses: CWE-706, CWE-706

Consequences:
  - Access Control: Bypass Protection Mechanism

Mitigations:
  - [Architecture and Design] Avoid making decisions based on names of resources (e.g. files) if those resources can have alternate names.
  - [Implementation] Assume all input is malicious. Use an "accept known good" input validation strategy, i.e., use a list of acceptable inputs that strictly conform to specifications. Reject any input that does not strictly conform to specifications, or transform it into something that does. 
                   When pe
  - [Implementation] Inputs should be decoded and canonicalized to the application's current internal representation before being validated (CWE-180). Make sure that the application does not decode the same input twice (CWE-174). Such errors could be used to bypass allowlist validation schemes by introducing dangerous i

Demonstrative examples:
In the following example, an XSS neutralization method intends to replace script tags in user-supplied input with a safe equivalent:
The code only works when the "script" tag is in all lower-case, forming an incomplete denylist (CWE-184). Equivalent tags such as "SCRIPT" or "ScRiPt" will not be neutralized by this method, allowing an XSS attack.
public String preventXSS(String input, String mask) { return input.replaceAll("script", mask); }

Real-world examples:
  - CVE-2000-0499: Application server allows attackers to bypass execution of a jsp page and read the source code using an upper case JSP extension in the request.
  - CVE-2000-0497: The server is case sensitive, so filetype handlers treat .jsp and .JSP as different extensions. JSP source code may be read because .JSP defaults to the filetype "text".
  - CVE-2000-0498: The server is case sensitive, so filetype handlers treat .jsp and .JSP as different extensions. JSP source code may be read because .JSP defaults to the filetype "text".
  - CVE-2001-0766: A URL that contains some characters whose case is not matched by the server's filters may bypass access restrictions because the case-insensitive file system will then handle the request after it bypa
  - CVE-2001-0795: Server allows remote attackers to obtain source code of CGI scripts via URLs that contain MS-DOS conventions such as (1) upper case letters or (2) 8.3 file names.