semvec/pypi-packages huggingface dataset:
contract-analyzer 0.2.0
A RAG system for contract analysis
Requires Python: >=3.7

# Contract Analyzer

This package provides a RAG system for contract analysis using the Meta-Llama-3-8B-Instruct model.

## Installation

You can install the package using pip:

```
pip install contract-analyzer
```

## Usage

Here's a basic example of how to use the ContractAnalysisRAG system:

```python
from huggingface_hub import login
from contract_analyzer import ContractAnalysisRAG

# Authenticate with Hugging Face
token = "your_huggingface_token_here"
model_name = "meta-llama/Meta-Llama-3-8B-Instruct"
login(token=token)

# Initialize the RAG system
rag_system = ContractAnalysisRAG(
    max_chunk_size=100,
    overlap=50,
    min_chunk_size=50,
    top_k=3,
    use_semantic_search=True,
    model_id=model_name,
    max_new_tokens=512,
    temperature=0.7
)

# Add your PDF documents
pdf_paths = ['path/to/your/contract.pdf']
rag_system.add_pdf_documents(pdf_paths)

# Example query
query = "Who is the employee named in the contract?"

# Retrieve the most relevant chunk and generate a response
most_related_chunk = rag_system.retrieve_and_print(query)
if most_related_chunk:
    response = rag_system.generate(query, [most_related_chunk])
    print(f"Query: {query}")
    print(f"Generated Response: {response}")
else:
    print("No relevant information found.")
```

Note: Replace 'your_huggingface_token_here' with your actual Hugging Face token to access the Meta-Llama-3-8B-Instruct model.

## Features

- PDF document processing
- Semantic search for relevant information retrieval
- Integration with Meta-Llama-3-8B-Instruct model for response generation
- Customizable chunking and retrieval parameters

## Requirements

- Python 3.7+
- Hugging Face account with access to the Meta-Llama-3-8B-Instruct model
sendalert 0.2.0
A simple Python package for sending alerts using SendAlert.io
Requires Python: >=3.6

# SendAlert.io Python Package

A simple Python package for sending alerts using SendAlert.io.

## Installation

```sh
pip install sendalert
```

## Usage

First set your API key:

```sh
export SENDALERT_API_KEY=your_api_key_here
export SENDALERT_PROJECT=your_project_name
```

Now you can send:

```python
from sendalert import sendalert

sendalert("message")
```

### Alternative: set your API key and project in your code

```python
import os
from sendalert import sendalert

os.environ["SENDALERT_API_KEY"] = "your_api_key_here"
os.environ["SENDALERT_PROJECT"] = "your_project_name"

sendalert("message")
```
XGRCpy 1.1.4
A package to decrypt and view XGRC table data
Requires Python: >=3.6

# XGRCpy

XGRCPy is a Python package that allows users to interact with an XGRC software service to retrieve encrypted data and decrypt it for further processing. It includes methods for key generation, decryption, and interacting with remote APIs.

## Installation

To install XGRCpy, ensure you have Python installed (version 3.6 or higher), then use pip to install the package:

```bash
pip install XGRCpy

## Usage

After installation, you can use the XGRCpy package to interact with the XGRC software service. Below is an example of how to use the XGRCViewTable function to fetch and decrypt a table from the service.

### Example Code
```python
from XGRCpy import XGRCViewTable

# User credentials for accessing the service
username = "your_username"
password = "your_password"
table = "desired_table"
api_key = "your_api_key"

# Fetching and decrypting the table
response = XGRCViewTable(username, password, table, api_key)

#OR

# Fetching and decrypting the SP
response = XGRCSPView(username, password, SPName, api_key)

# Display the decrypted content
print(response)



###PowerBI Example
#Returnes a pandas DataFrame to be used with PowerBI 
result = XGRCViewTablePowerBI(Table, API_UserName, API_Password, API_Key)
if result is not None:
   print("Decrypted DataFrame:\n", result)
    
else:
    print("Failed to decrypt data.")
gen-dedup 0.0.2
Generative deduplication
Requires Python: >=3.8

# Generative Deduplication (Findings of EMNLP 2024)

Revisiting data deuplication, we propose a fresh paradigm for semantic deduplication.

- Arxiv: https://arxiv.org/abs/2401.05883

![](./assets/framework.png)


**Core Idea:**

- Generative language models possess powerful language understanding capabilities. We use it for semantic deduplication.
- There are two crucial stages in generative deduplication:
  - **Memory stage**: The model learns the relationship between context and corresponding keywords. Semantically duplicate contexts establish stronger connections than non-duplicate ones in one-epoch training.
  $$g(y|context)$$
  - **Inference stage**: During inference, we use the trained generative model to generate keywords from the given context. If the generated keywords match the target keywords, we classify the data as duplicate.
  $$g(context) == y?$$


## Installation

```bash
python -m pip install gen-dedup
```

## Usage

```python
from datasets import load_dataset
from keybert import KeyBERT
from gen_dedup import GenDedup

# 1. Load dataset
ds = load_dataset('cardiffnlp/tweet_eval', 'hate', split='train')
ds = ds.select_columns(['text'])
ds = ds.rename_column('text', 'sentence')

# 2. Generate keywords with KeyBERT. Other keyword extraction models can also be used.
keybert = KeyBERT()
# Here, we generate two keywords.
max_label_words = 2
ds = ds.map(lambda x: {
    'labels': " ".join([k[0] for k in keybert.extract_keywords(x['sentence'].lower())[:max_label_words]]),
    'sentence': x['sentence'].lower()})

# 3. Fit the generative model to learn g(y|X)
gd = GenDedup('google/flan-t5-small')
gd.fit(ds, output_dir='./hate-dedup')

# 4. Inference as Deduplication. Check whether g(X) = y
gd.deduplicate('./hate-dedup', max_label_words=max_label_words)
```

The trained model, duplicate data, and non-duplicate data will be saved in the `./hate-dedup` directory.


## Citation

```bibtex
@article{li2024generative,
  title={Generative Deduplication For Socia Media Data Selection},
  author={Li, Xianming and Li, Jing},
  journal={arXiv preprint arXiv:2401.05883},
  year={2024}
}
```