deardaniel/secdata-raw huggingface dataset:
OWASP LLM Top 10: README

# Step 3: Data Validation and Quality Control Methodology

## Overview
The intent of this phase is to establish a robust Data Validation and Quality Control (QC) framework for assessing and ensuring the integrity and accuracy of the research data on the OWASP Top 10 for LLM AI Applications. This step is crucial for maintaining the reliability of our findings and ensuring that the data reflects real-world vulnerabilities accurately.

## Objectives
- **Define Key Performance Indicators (KPIs)**: Establish both qualitative and quantitative measures that will guide the validation process.
- **Develop Validation Tools**: Create scripts and utilize automated tools to systematically verify the data against our KPIs.
- **Ensure Data Quality**: Implement a QC process to identify and rectify any inconsistencies, inaccuracies, or biases in the dataset.

## What to Expect with the Codes
The provided Python scripts serve as a baseline for conducting data validation and quality control. These scripts are designed to be adaptable to various datasets and environments, depending on the specific needs of the research. Below is a brief overview of what to expect:

### Validation Scripts
- **Data Consistency Checks**: Scripts to verify that the data is consistent across different sources and timeframes.
- **Accuracy Tests**: Tools to compare sampled data against trusted benchmarks or manual checks to assess accuracy.
- **Completeness Checks**: Automated checks to ensure that the dataset is complete and all expected data points are present.

### Quality Control Tools
- **Automated Anomaly Detection**: Scripts to identify outliers or anomalies in the data that may indicate errors or inconsistencies.
- **Bias Detection**: Tools to assess the dataset for any potential biases that could affect the research outcomes.

### Adapting the Code
It is expected that the provided scripts will need to be adapted for each specific research environment and dataset. This adaptation may involve:

- **Parameter Tuning**: Adjusting thresholds, weights, or other parameters within the scripts to better fit the specific data characteristics.
- **Custom Checks**: Adding or modifying checks and validations to address unique aspects of the data or research objectives.
- **Integration with Other Tools**: Modifying the scripts to work seamlessly with other tools or platforms used in the research process.

## Getting Started
To begin using the validation and QC scripts, please ensure you have the following prerequisites installed:

- Python 3.8 or higher
- Necessary Python libraries as listed in `each md file`

Follow the setup instructions in `SETUP.md` to configure your environment and adapt the scripts to your dataset. For detailed documentation on each script and how to customize it, refer to the `md` files.

## Contribution
Your contributions are welcome! If you have suggestions for improving the validation and QC methodology, please open an issue or submit a pull request with your proposed changes. For more information on contributing, please see `CONTRIBUTING.md`.

Thank you for your interest in improving the security and reliability of LLM AI applications.
OWASP LLM Top 10: ASVS

```python
# Python Data Validation Script for LLM Vulnerabilities (LLM01 - LLM10)

import re
from cryptography.fernet import Fernet
import ratelimit
import requests

# Example key, generate your own using Fernet.generate_key()
key = b'your_encryption_key_here'
cipher_suite = Fernet(key)

# LLM01 & LLM02: Input Validation and Secure Output Handling
def validate_and_encode_input(input_data):
    """Validate and sanitize input data, then return encoded for secure handling."""
    # Simple validation example, adapt regex to your needs
    if re.match("^[a-zA-Z0-9 ]*$", input_data):
        # Securely encode output to prevent injection attacks
        encoded_data = cipher_suite.encrypt(input_data.encode('utf-8'))
        return encoded_data
    else:
        raise ValueError("Invalid input")

# LLM03: Secure Training Data
def encrypt_training_data(training_data):
    """Encrypt training data to ensure integrity."""
    encrypted_data = cipher_suite.encrypt(training_data.encode('utf-8'))
    return encrypted_data

# LLM04: Implement Rate Limiting to prevent DoS
@ratelimit.limits(calls=100, period=ratelimit.HOUR)
def process_request(request_data):
    """Process incoming request with rate limiting to prevent DoS attacks."""
    # Simulate request processing
    return "Request processed"

# LLM05, LLM06, LLM07, LLM08, LLM09, LLM10 are more conceptual and require organizational
# and architectural measures, including secure plugin design, API security,
# data protection methods, and more, which are beyond the scope of a simple script.
# These require comprehensive approaches involving multiple systems and practices.


### Automated Validation Tools

For automating validation and ensuring adherence to ASVS standards, consider integrating the following tools into your development and deployment pipelines:

- **ZAP (Zed Attack Proxy)**: For finding vulnerabilities in web applications.
- **SonarQube**: For continuous inspection of code quality to detect bugs, vulnerabilities, and code smells in your code.
- **OWASP Dependency-Check**: For detecting publicly disclosed vulnerabilities in project dependencies.
- **Bandit**: For finding common security issues in Python code.
- **cryptography**: Python library for encryption and decryption to secure data, as shown in the script.
- **Ratelimit**: Python library to implement rate limiting, as demonstrated in the script.

These tools can automate aspects of security validation and complement the script's functionalities, focusing on specific LLM vulnerabilities and general application security concerns outlined in the ASVS.
OWASP LLM Top 10: OPENCRE

```python
import re

# Function to read markdown content from a file
def read_markdown_file(file_path):
    with open(file_path, 'r', encoding='utf-8') as file:
        return file.read()

# Function to validate the presence of required sections in the markdown content
def validate_opencre_markdown(content):
    # Define the regex pattern for required sections (LLM01 through LLM10)
    section_pattern = re.compile(r'### LLM0[1-9]:|### LLM10:')
    
    # Find all matches
    matches = section_pattern.findall(content)
    
    # Count of expected sections
    expected_sections = 10
    found_sections = len(matches)
    
    # Check if all expected sections are found
    if found_sections == expected_sections:
        print("Success: All expected sections (LLM01 through LLM10) were found.")
    else:
        missing_count = expected_sections - found_sections
        print(f"Warning: Expected {expected_sections} sections, found {found_sections}.")
        print(f"{missing_count} sections are missing or not correctly labeled.")

# Example usage
markdown_file_path = 'opencre_mapping.md' # Replace with your actual file path
markdown_content = read_markdown_file(markdown_file_path)
validate_opencre_markdown(markdown_content)



### Note on Automated Validation Tools
For validating structured data derived from the markdown content or other aspects of your project where structured data validation is applicable, tools like Great Expectations, Pydantic, or Cerberus can be highly effective. They require structured data inputs, so additional preprocessing would be necessary to convert markdown sections into a format these tools can work with, such as JSON or a Python dictionary.

Great Expectations is particularly well-suited for complex validation scenarios and can integrate with data pipelines for continuous data quality checks. Pydantic excels in data parsing and validation through Python type annotations, offering a robust solution for ensuring data types and formats meet expected criteria. Cerberus provides a flexible and lightweight option for validating Python data structures against a customizable schema, useful for a variety of validation tasks.
OWASP LLM Top 10: SETUP

# Setup Instructions

This document provides detailed instructions for setting up your environment to use the data validation and quality control scripts for the OWASP Top 10 for LLM AI Applications project. Follow these steps to prepare your system for running the provided Python scripts.

## Prerequisites

Ensure you have the following installed on your system before proceeding:

- Python 3.8 or higher: [Download Python](https://www.python.org/downloads/)
- pip: Should be installed with Python 3.4 or higher

## Installation

1. **Clone the Repository**

   First, clone the project repository to your local machine using Git:

2. **Create a Virtual Environment**

Navigate to the project directory and create a virtual environment:


3. **Activate the Virtual Environment**

- On Windows:

  ```
  .\venv\Scripts\activate
  ```

- On Unix or MacOS:

  ```
  source venv/bin/activate
  ```

4. **Install Required Libraries**

Install all the necessary Python libraries as specified in `requirements.txt`:


## Adapting the Scripts

To adapt the validation and quality control scripts to your specific dataset and environment, follow these guidelines:

- Review each script in the `src` directory to understand its functionality.
- Modify parameters, thresholds, and checks within each script as needed to suit your data characteristics.
- For adding custom validation checks, refer to the script documentation in the `docs` folder for guidance on extending script capabilities.

## Running the Scripts

To run a specific script, use the following command from the project root directory:


Replace `yourscriptname.py` with the name of the script you wish to run. Ensure your dataset files are placed in the designated input directories as per the script documentation.

## Troubleshooting

If you encounter any issues during setup or while running the scripts, please open an issue on the project GitHub page.

Thank you for contributing to the security and reliability of LLM AI applications.
OWASP LLM Top 10: BSIMM

```python
# Python Data Validation Script Example

import re
from cryptography.fernet import Fernet
import tensorflow_data_validation as tfdv
import pandas as pd

# Generate a key for encryption/decryption
# In practice, store this key securely
key = Fernet.generate_key()
cipher_suite = Fernet(key)

# Example data validators
def validate_prompt(prompt):
    """Simple validation to avoid prompt injection."""
    if re.search(r"[^\w\s]", prompt):
        raise ValueError("Invalid characters in prompt.")
    return prompt

def sanitize_output(output):
    """Basic output sanitization to prevent insecure data exposure."""
    sanitized_output = re.sub(r"[^\w\s]", "", output)
    return sanitized_output

def validate_training_data(training_data_file):
    """Check integrity of training data using TensorFlow Data Validation."""
    stats = tfdv.generate_statistics_from_csv(training_data_file)
    anomalies = tfdv.validate_statistics(stats, tfdv.load_schema_text('schema.pbtxt'))
    tfdv.display_anomalies(anomalies)

def encrypt_sensitive_data(data):
    """Encrypt sensitive data."""
    if isinstance(data, str):
        data = data.encode()
    encrypted_data = cipher_suite.encrypt(data)
    return encrypted_data

def decrypt_sensitive_data(encrypted_data):
    """Decrypt sensitive data."""
    decrypted_data = cipher_suite.decrypt(encrypted_data)
    return decrypted_data.decode()

# Example usage
if __name__ == "__main__":
    # Validate and sanitize inputs/outputs
    prompt = validate_prompt("Example prompt with valid characters")
    output = sanitize_output("Example output with <tags> and special characters!")
    
    # Validate training data
    validate_training_data("training_data.csv")
    
    # Encrypt and decrypt sensitive information
    sensitive_info = "Sensitive data example"
    encrypted_info = encrypt_sensitive_data(sensitive_info)
    decrypted_info = decrypt_sensitive_data(encrypted_info)
    
    print(f"Original: {sensitive_info}, Encrypted: {encrypted_info}, Decrypted: {decrypted_info}")


Remember, the effectiveness of security measures greatly depends on the specific context and how they're implemented within your overall security strategy. Continuously update and refine your validation techniques to adapt to new vulnerabilities and threats.
OWASP LLM Top 10: README

# OWASP TOP 10 for LLM AI Apps - Data Support

This repository provides the necessary data support gathered for the project **OWASP TOP 10 for LLM AI Apps**. The data included here aims to assist developers, researchers, and security professionals in understanding and mitigating risks associated with Large Language Model (LLM) applications.

## Data and Resources

This repository contains various datasets, research papers, and real-world examples that support the identification and management of security risks in LLM AI applications. The resources are organized according to different security concerns and vulnerabilities relevant to LLMs.

## Disclaimer

Please note that running tests with datasets on LLMs may incur costs, including but not limited to computational resources and storage. Users are responsible for managing and covering any incurred costs.

Weblinks provided in this repository were verified at the time of creation. However, they may change over time. Users should verify the validity of these links and the fonts for their specific needs.

## Usage

This repository is intended for use by developers, researchers, and security professionals who are working on securing LLM applications. It is part of an open-source project and contributions are welcome. Please follow the contribution guidelines if you wish to add to this repository.

## Contribution Guidelines

- Fork the repository.
- Create a new branch for your changes.
- Make your changes and commit them with clear and concise messages.
- Submit a pull request for review.

## License

This project is licensed under the MIT License. See the LICENSE file for details.

## Contact

For any questions or further information, please contact the project maintainers.

---

**Note:** This project is part of an open-source initiative aimed at enhancing the security of LLM AI applications. Your contributions and feedback are highly appreciated.
OWASP LLM Top 10: ISO20547 4 2020

# OWASP Top 10 for LLMs Mapped to ISO/IEC 20547-4:2020

This document outlines the application of [ISO/IEC 20547-4:2020](https://www.iso.org/standard/72089.html), focusing on security and privacy in big data systems, to address the [OWASP Top 10 for Large Language Model Applications](https://owasp.org/www-project-top-10-for-large-language-model-applications/#). The standard provides a comprehensive framework for managing security and privacy risks in big data architectures, which can be adapted to the specific challenges posed by LLMs.

## Security and Privacy Considerations for LLM Vulnerabilities

Mapping each LLM vulnerability to relevant aspects of ISO/IEC 20547-4:2020:

## LLM01: Prompt Injection

- **Data Processing Security**: Implement secure processing controls to validate and sanitize inputs, preventing prompt injection vulnerabilities.

## LLM02: Insecure Output Handling

- **Data Management and Privacy**: Apply principles of data minimization and secure data handling to manage outputs securely, ensuring they are properly encoded and sanitized.

## LLM03: Training Data Poisoning

- **Data Source Security**: Ensure the integrity and security of data sources for training LLMs, including mechanisms for validating and vetting training data.

## LLM04: Model Denial of Service

- **System and Infrastructure Security**: Design robust infrastructure capable of handling high loads and resisting denial of service attacks through rate limiting and resource allocation.

## LLM05: Supply-Chain Vulnerabilities

- **Data Storage and Infrastructure Security**: Assess and secure the supply chain for software and data components, including third-party services and libraries used by LLMs.

## LLM06: Sensitive Information Disclosure

- **Privacy Protection**: Implement strong data protection measures, including encryption and access controls, to prevent unauthorized disclosure of sensitive information.

## LLM07: Insecure Plugin Design

- **System and Application Security**: Ensure plugins or extensions for LLMs are developed with security in mind, including regular security assessments and updates.

## LLM08: Excessive Agency

- **Data Processing and Management**: Clearly define and enforce boundaries for LLM decision-making capabilities, ensuring they align with security and privacy requirements.

## LLM09: Overreliance

- **Human Factor and Training**: Address the risks associated with overreliance on LLM technologies through user education and awareness programs.

## LLM10: Model Theft

- **Intellectual Property Protection**: Secure LLM models as valuable intellectual property, applying appropriate security measures to prevent unauthorized access and theft.

Note: Implementing the guidelines of ISO/IEC 20547-4:2020 for LLM technologies requires a comprehensive approach, taking into account the unique characteristics and risks associated with large-scale data processing systems. Organizations should conduct regular security and privacy assessments to identify and mitigate potential vulnerabilities in their LLM implementations.
OWASP LLM Top 10: STRIDE

# OWASP Top 10 for LLMs Mapped to STRIDE

This document maps the [OWASP Top 10 for Large Language Model Applications](https://owasp.org/www-project-top-10-for-large-language-model-applications/#) to the [STRIDE (Spoofing, Tampering, Repudiation, Information Disclosure, Denial of Service (DoS), Elevation of Privilege)](https://learn.microsoft.com/en-us/previous-versions/commerce-server/ee823878(v=cs.20)?redirectedfrom=MSDN)  threat model categories. STRIDE helps in identifying and categorizing common security threats into six types: Spoofing, Tampering, Repudiation, Information Disclosure, Denial of Service, and Elevation of Privileges.

## LLM01: Prompt Injection

- **STRIDE Category**: Spoofing
- **Description**: Prompt injection can allow an attacker to masquerade as a legitimate user, manipulating the LLM to generate desired outputs.

## LLM02: Insecure Output Handling

- **STRIDE Category**: Tampering/Information Disclosure
- **Description**: Insecure handling of outputs can lead to tampering or unintended information disclosure through manipulated outputs.

## LLM03: Training Data Poisoning

- **STRIDE Category**: Tampering
- **Description**: Poisoning the training data can tamper with the model's learning process, affecting its outputs and decision-making capabilities.

## LLM04: Model Denial of Service

- **STRIDE Category**: Denial of Service
- **Description**: Overloading the model with complex inputs or exploiting vulnerabilities to consume resources can deny service to legitimate users.

## LLM05: Supply-Chain Vulnerabilities

- **STRIDE Category**: Tampering
- **Description**: Exploiting vulnerabilities in the supply chain can lead to tampering with the model or its environment.

## LLM06: Sensitive Information Disclosure

- **STRIDE Category**: Information Disclosure
- **Description**: This vulnerability can lead to the unintended release of sensitive or confidential information.

## LLM07: Insecure Plugin Design

- **STRIDE Category**: Tampering/Elevation of Privilege
- **Description**: Insecure plugins can be exploited to tamper with the model or elevate privileges within the system.

## LLM08: Excessive Agency

- **STRIDE Category**: Elevation of Privilege
- **Description**: Giving the model excessive decision-making capabilities could inadvertently elevate its privileges beyond intended limits.

## LLM09: Overreliance

- **STRIDE Category**: Repudiation
- **Description**: Overreliance on LLMs without proper oversight or auditing mechanisms can lead to situations where actions cannot be adequately attributed or denied.

## LLM10: Model Theft

- **STRIDE Category**: Information Disclosure/Tampering
- **Description**: Stealing a model can lead to information disclosure about its training data or algorithms and could allow for tampering with the model itself.

Note: The mappings provided are interpretations based on the nature of each vulnerability and how they might be exploited according to the STRIDE model. This exercise highlights the importance of considering various types of threats when assessing the security of LLMs.
OWASP LLM Top 10: CVE CWE

# Mapping OWASP Top 10 for LLMs to CVEs and CWEs

This document maps the [OWASP Top 10 for Large Language Model Applications](https://owasp.org/www-project-top-10-for-large-language-model-applications/#) to related Common Vulnerabilities and Exposures ([CVEs](https://cve.mitre.org/)) and Common Weakness Enumeration ([CWEs](https://cwe.mitre.org/)). Given the novel nature of LLMs, direct CVE matches were not found at this stage, but relevant CWEs can provide insights into the types of weaknesses these vulnerabilities may exploit.

## LLM01: Prompt Injection

- **[CWE-77](https://cwe.mitre.org/data/definitions/77.html)**: Improper Neutralization of Special Elements used in a Command ('Command Injection')
- **[CWE-94](https://cwe.mitre.org/data/definitions/94.html)**: Improper Control of Generation of Code ('Code Injection')

## LLM02: Insecure Output Handling

- **[CWE-79](https://cwe.mitre.org/data/definitions/79.html)**: Improper Neutralization of Input During Web Page Generation ('Cross-site Scripting')
- **[CWE-116](https://cwe.mitre.org/data/definitions/116.html)**: Improper Encoding or Escaping of Output

## LLM03: Training Data Poisoning

- **[CWE-506](https://cwe.mitre.org/data/definitions/506.html)**: Embedded Malicious Code
- **[CWE-915](https://cwe.mitre.org/data/definitions/915.html)**: Improperly Controlled Modification of Dynamically-Determined Object Attributes

## LLM04: Model Denial of Service

- **[CWE-400](https://cwe.mitre.org/data/definitions/400.html)**: Uncontrolled Resource Consumption

## LLM05: Supply-Chain Vulnerabilities

- **[CWE-829](https://cwe.mitre.org/data/definitions/829.html)**: Inclusion of Functionality from Untrusted Control Sphere
- **[CWE-937](https://cwe.mitre.org/data/definitions/937.html)**: Using Components with Known Vulnerabilities

## LLM06: Sensitive Information Disclosure

- **[CWE-200](https://cwe.mitre.org/data/definitions/200.html)**: Exposure of Sensitive Information to an Unauthorized Actor

## LLM07: Insecure Plugin Design

- **[CWE-749](https://cwe.mitre.org/data/definitions/749.html)**: Exposed Dangerous Method or Function
- **[CWE-1203](https://cwe.mitre.org/data/definitions/1203.html)**: Insecure Direct Object References

## LLM08: Excessive Agency

- **[CWE-807](https://cwe.mitre.org/data/definitions/807.html)**: Reliance on Untrusted Inputs in a Security Decision
- No direct CVE mapping available.

## LLM09: Overreliance

- **[CWE-1048](https://cwe.mitre.org/data/definitions/1048.html)**: Software Reliance on Single Factor Authentication in a Security Decision
- No direct CVE mapping available.

## LLM10: Model Theft

- **[CWE-494](https://cwe.mitre.org/data/definitions/494.html)**: Download of Code Without Integrity Check
- **[CWE-1241](https://cwe.mitre.org/data/definitions/1241.html)**: Improper Protection of Sensitive Information During Manufacturing or Distribution

Note: Identifying specific CVE entries for LLM vulnerabilities is challenging due to the specificity of CVEs to software products or systems. However, the listed CWE entries provide a framework for understanding the types of weaknesses these vulnerabilities might exploit.