semvec/pypi-packages huggingface dataset:
privatetestsetgenerationforLLMeval 2.3.1
A package for generating evaluation set for LLM-based chatbots in a diverse and private manner
Requires Python: >=3.6

# LLM Test Set Generation Framework

This framework automates the creation of synthetic question-answer datasets for LLM evaluation from PDF sources.  
It extracts real-world domain text, ensures topic diversity, removes sensitive data (PII), and generates synthetic Q&A pairs.

---

##  Features

- **PDF Ingestion**: Extracts clean sentences from any PDF.
- **Embeddings**: Uses Azure OpenAI embeddings to represent sentences numerically.
- **Topic Diversification**: Clusters sentences into diverse topics using KMeans.
- **Privacy Filtering**: Detects and removes sensitive information (PII) with Presidio.
- **Synthetic QA Generation**: Generates rich, varied Q&A pairs via GPT-4 from sanitized text.
- **Modular Pipeline**: Built as a flexible multi-agent graph using LangGraph.

---

##  How It Works

Each step is an autonomous "agent" in a pipeline:

- **Diversity Agent**: Clusters data into meaningful topics.
- **Privacy Agent**: Analyzes clusters and flags sensitive content.
- **Synthetic Data Generator**: Creates QA samples while explaining coverage strategies.

## Usage 


To use the tool, follow these steps:

 ```bash
   pip install privatetestsetgenerationforLLMeval

   privatetestsetgeneration



```

  
   
## Contributing

Contributions to improve the tool are welcome! Feel free to open issues for bugs or feature requests, or submit pull requests for enhancements.



## Acknowledgements

This project utilizes various libraries, including LangChain for document processing and Presidio for PII detection and anonymization.



Clone the repository:
Install dependencies:
pip install -r requirements.txt
Create a .env file with the following variables:
AZURE_OPENAI_KEY=<your_azure_openai_api_key>
AZURE_OPENAI_API_VERSION=2023-12-01-preview
AZURE_ENDPOINT=https://<your_azure_resource>.openai.azure.com/
EMBEDDING_DEPLOYMENT_NAME=text-embedding-ada-002
GPT_4_MODEL_DEPLOYMENT_NAME=gpt-4



Run the main pipeline:

# to complete

To provide a path to a PDF.
To select the number of synthetic Q&A pairs to generate.
The pipeline will:

Process your document.
Create a topic-augmented, privacy-safe synthetic dataset.
Save a .csv file (e.g., qa_pairs10.csv) containing the generated Q&A pairs.


📄 Example Output


Question	Answer
What are the primary goals of digital transformation?	To leverage technology to optimize business processes, enhance customer experiences, and create new revenue streams.
How can data privacy be ensured when migrating to the cloud?	Through encryption, compliance audits, access control, and anonymization of sensitive data.
microhapdb 0.12
Portable database of microhaplotype marker and allele frequency data

# MicroHapDB

[![MicroHapDB build status][cibadge]](https://github.com/bioforensics/MicroHapDB/actions)
[![Install with bioconda][condabadge]](http://bioconda.github.io/recipes/microhapdb/README.html)
[![BSD licensed][licensebadge]](https://github.com/bioforensics/MicroHapDB/blob/master/LICENSE.txt)

NBFAC, 2018-2025
https://github.com/bioforensics/microhapdb

**MicroHapDB** is a comprehensive catalog of human microhaplotype variation.
The database integrates marker and population frequency data from numerous published research articles.
Allele frequency estimates and allelic diversity statistics ($A_e$) are computed for 26 global populations so that markers can be ranked and evaluated for various applications.

MicroHapDB is managed as a community resource requiring minimal infrastructure to maintain: the entire contents of the database are distributed in plain text with each copy of MicroHapDB.
The primary interface to MicroHapDB is through the console command line, which provides methods for browsing, searching, and filtering the database contents.
Full programmatic access is available via a Python API.
Alternatively, users can access the database tables directly using a spreadsheet program such as Microsoft Excel.

[![Binder](https://mybinder.org/badge_logo.svg)](https://mybinder.org/v2/gh/bioforensics/MicroHapDB/master?labpath=binder%2Fdemo_v0.11.ipynb)


## Installation

See [this page](https://microhapdb.readthedocs.io/en/latest/install.html) for complete installation instructions.
If this isn't your first rodeo, the commands below are provided as a quick reference.

```
conda install -c bioconda microhapdb pytest
pytest --pyargs microhapdb --doctest-modules
```

## Usage

MicroHapDB provides several methods to access the contents of a locally installed database.
The [MicroHapDB documentation](https://microhapdb.readthedocs.io/) includes a ["Getting started" guide](https://microhapdb.readthedocs.io/en/latest/starting.html) as well as a comprehensive reference for running MicroHapDB on the command line.


## Citation

If you use this database, please cite our work.

> Standage DS,  Mitchell RN (2020) MicroHapDB: A Portable and Extensible Database of All Published Microhaplotype Marker and Frequency Data. *Frontiers in Genetics* 11:781, [doi:10.3389/fgene.2020.00781](https://doi.org/10.3389/fgene.2020.00781).

MicroHapDB was created and is maintained by the Bioinformatics Group at the National Bioforensic Anaylsis Center (NBFAC).

Additional references are available [on this page](https://microhapdb.readthedocs.io/en/latest/citations.html).


[cibadge]: https://github.com/bioforensics/MicroHapDB/actions/workflows/cibuild.yml/badge.svg?branch=master
[pypibadge]: https://img.shields.io/pypi/v/microhapdb.svg
[condabadge]: https://img.shields.io/badge/install%20with-bioconda-brightgreen.svg
[licensebadge]: https://img.shields.io/badge/license-BSD-blue.svg