test-RAG-X 0.2.1
This library is to search the best parameters across different steps of the RAG process.
License: MIT


---

# RAG-X Library

## Overview

RAG-X is a comprehensive library designed to optimize Retrieval-Augmented Generation (RAG) processes. It provides a suite of tools to automatically determine the best parameters for processing specific documents. This includes selecting appropriate chunking techniques, embedding models, vector databases, and Language Model (LLM) configurations.

## Flow Chart

![Flow Chart](pictures/structure.png)

### Key Features:
- **Adaptive Chunking:** Incorporates four advanced text chunking methodologies to enhance the handling of diverse document structures.
  - Specific Text Splitting
  - Recursive Text Splitting
  - Sentence Window Splitting
  - Semantic Window Splitting
- **Expandability:** Future versions will introduce additional chunking strategies and enhancements based on user feedback and ongoing research.
- **Compatibility:** Designed to seamlessly integrate with a wide range of embedding models and vector databases.

## Getting Started

### Installation

To get started, install the test_RAG_X library using the following command:

```bash
pip install test-RAG-X
```

To verify the installation and view library details, execute:

```bash
pip show test_RAG_X
```

### Setting Up Your Environment

Before diving into the functionality of test-RAG-X, ensure that your environment variables are properly configured with your OpenAI API key and your Hugging Face token:

```python
import os

os.environ['OPENAI_API_KEY'] = "YOUR_OPENAI_API_KEY"

```
** Note :- API Key from Free tier OpenAI account is not supported. **  
## Usage

The following steps guide you through the process of utilizing the test-RAG-X library to optimize your RAG parameters:

```python
import hagrid as hg

# Specify the path to your PDF document
file_path = "PATH_TO_YOUR_PDF_FILE"

# Initialize the RAG-X instance
model = hg.ChunkEvaluator(file_path)

# Generate the optimal RAG parameters for your document
score_card = model.evaluate_parameters()

# Output the results
print(score_card)
```


## Set parameters for evaluation

If you wish to analyse the performance of your parameters, you can pass the parameters as below:
```python
kwarg = {
        'number_of_questions': 5, # Number of questions used to evaluate the process: type(int)
        'chunk_size': 250, # Chunk size: type(int)
        'chunk_overlap': 0, # Chunk overlap size: type(int)
        'separator': '',  # Separator to be used for chunking if any, type(str)
        'strip_whitespace': False, # Strip white space, type(bool)
        'sentence_buffer_window': 3, # Sentence Buffer window, type(int) 
        'sentence_cutoff_percentile': 80, # Sentence chunk split percentile for spliting context, type(int), range(1,100)
        }

# Specify the path to your PDF document
file_path = "PATH_TO_YOUR_PDF_FILE"

# Initialize the RAG-X instance
model = hg.ChunkEvaluator(file_path, **kwargs)

# Generate the optimal RAG parameters for your document
score_card = model.evaluate_parameters()

# Output the results, output will be a pandas dataframe
print(score_card)
```

## Output
![output_image](pictures/result.png)

## Contribution
We are open for contributions and any feedback from the users. Feel free to contact us.

## Contact Us:
If you wish to integarte GenAI into your company, please contact at ...