semvec/pypi-packages huggingface dataset:
opendatagen 0.0.35
Data preparation system to build controllable AI system

# ⬜️ Open Datagen ⬜️

**Open Datagen** is a Data Preparation Tool designed to build Controllable AI Systems

It offers improvements for:

**RAG**: Generate large Q&A datasets to improve your Retrieval strategies.

**Evals**: Create unique, “unseen” datasets to robustly test your models and avoid overfitting.

**Fine-Tuning**: Produce large, low-bias, and high-quality datasets to get better models after the fine-tuning process.

**Guardrails**: Generate red teaming datasets to strengthen the security and robustness of your Generative AI applications against attack.

## Additional Features

- Use external sources to generate high-quality synthetic data (Local files, Hugging Face datasets and Internet)

- Data anonymization 

- Open-source model support + local inference

- Decontamination

- Tree of thought

- Multimodality (Text, Audio and Image)

- (SOON) No-code dataset generation

## Installation

```bash
pip install --upgrade opendatagen
```

### Setting up your API keys

```bash
export OPENAI_API_KEY='your_openai_api_key' #(using openai>=1.2)
export MISTRAL_API_KEY='your_mistral_api_key'
export TOGETHER_API_KEY='your_together_api_key'
export ANYSCALE_API_KEY='your_anyscale_api_key'
export ELEVENLABS_API_KEY='your_elevenlabs_api_key'
export SERPLY_API_KEY='your_serply_api_key' #Google Search API 
```

## Usage

Example: Generate a low-biased FAQ dataset based on Wikipedia content

```python
from opendatagen.template import TemplateManager
from opendatagen.data_generator import DataGenerator

output_path = "opendatagen.csv"
template_name = "opendatagen"
manager = TemplateManager(template_file_path="faq_wikipedia.json")
template = manager.get_template(template_name=template_name)

if template:
    
    generator = DataGenerator(template=template)
    
    data, data_decontaminated = generator.generate_data(output_path=output_path, output_decontaminated_path=None)
    
```

where faq_wikipedia.json is [here](opendatagen/examples/faq_wikipedia.json)

## Contribution

We welcome contributions to Open Datagen! Whether you're looking to fix bugs, add templates, new features, or improve documentation, your help is greatly appreciated.

## Acknowledgements

We would like to express our gratitude to the following open source projects and individuals that have inspired and helped us:

- **Textbooks are all you need** ([Read the paper](https://arxiv.org/abs/2306.11644)) 

- **Evol-Instruct Paper** ([Read the paper](https://arxiv.org/abs/2306.08568)) by [WizardLM_AI](https://twitter.com/WizardLM_AI)

- **Textbook Generation** by [VikParuchuri](https://github.com/VikParuchuri/textbook_quality)

## Connect

If you need help for your Generative AI strategy, implementation, and infrastructure, reach us on

Linkedin: [@Thomas](https://linkedin.com/in/thomasdordonne).
Twitter: [@thoddnn](https://twitter.com/thoddnn).
mswp 1.0.0
A decorator that automatically clears temporary local variables upon function execution, effectively preventing clutter and mitigating memory leaks.
Keywords: decorator,memory,memory-leaks,performance,variables,security,scalling

# mswp, a decorator that clears temporary local variables upon function execution, preventing clutter and mitigating memory leaks
[![Python](https://img.shields.io/pypi/pyversions/tensorflow.svg)](https://badge.fury.io/py/tensorflow) [![License](https://img.shields.io/badge/License-Apache_2.0-blue.svg)](https://opensource.org/licenses/Apache-2.0) ![Maintainer](https://img.shields.io/badge/maintainer-@louisbrulenaudet-blue)

mswp is a Python decorator designed to enhance code cleanliness and mitigate potential memory leaks by automatically clearing temporary local variables after function execution.

## Overview

Python functions often create temporary variables for intermediate computations, which can clutter the local namespace and potentially lead to memory leaks if not properly managed. mswp addresses this issue by providing a simple yet powerful decorator that automatically cleans up temporary local variables, ensuring an efficient execution environment.

## Installation
You can install mswp via pip:

```bash
pip3 install mswp
```
## Usage
  
To use mswp, simply decorate your functions with @mswp. This ensures that any temporary local variables used within the function are cleared upon completion, promoting code cleanliness and efficient memory usage.

```python
from mswp import clear

@clear
def my_function():
	temp_var = 10
	# Your code here
	return result
```
In this example, my_function is decorated with @clear, ensuring that any temporary local variables used within it are cleared upon completion.

### Parameters
`func`: The function to be decorated.

### Returns
`function`: The decorated function.

## Features
- Automatic Cleanup: mswp automatically clears temporary local variables after function execution, preventing clutter and potential memory leaks.

- Simplified Syntax: With a simple decorator, mswp streamlines code maintenance and promotes readability.

- Error Handling: mswp provides robust error handling to ensure reliable execution even in the presence of exceptions.

## Citing this project
If you use this code in your research, please use the following BibTeX entry.

```BibTeX
@misc{louisbrulenaudet2023,
	author = {Louis Brulé Naudet},
	title = {mswp, a decorator that automatically clears temporary local variables upon function execution, effectively preventing clutter and mitigating memory leaks},
	howpublished = {\url{https://github.com/louisbrulenaudet/mswp}},
	year = {2024}
}
```
## Feedback
If you have any feedback, please reach out at [louisbrulenaudet@icloud.com](mailto:louisbrulenaudet@icloud.com).
gpnam 0.1.0
GPNAM - an interpretable Gaussian Process Neural Additive Model.
Requires Python: >=3.7

# GPNAM: Gaussian Process Neural Additive Models

![The framework of GPNAM](https://raw.githubusercontent.com/Wei2624/GPNAM/main/imgs/framework.jpg)
*The framework of GPNAM. `$z_s, c_s$` and the sinusoidal function are predefined from the paper and do not require training. The only trainable parameter is `W` that maps to the output of each shape function.*

This package implements the paper titled Gaussian Process Neural Additive Models that appears at AAAI 2024. The paper is available on [Arxiv](https://arxiv.org/abs/2402.12518).

Basically, the GPNAM constructs a Neural Additive Model (NAM) by a GP with Random Fourier Features as the shape function for each input feature, which leads to a convex optimization with a significant reduction in trainable parameters. 

## Sklearn interface

You can install the package by:
```
pip install gpnam
```

Then, you can run the model simply by:
```python
from gpnam.sklearn import GPNAM

"""
input_dim: the dimensions of input data
problem: type of the task, 'classification' or 'regression'
"""
gpnam = GPNAM(input_dim, problem)
gpnam.fit(X, y)

y_pred = gpnam.predict(X_test)
```

## Citation
If you find this repo useful, please consider citing our paper:
```
@inproceedings{,
  title={Gaussian Process Neural Additive Models},
  author={Wei Zhang and Brian Barr and John Paisley},
  booktitle={AAAI Conference on Artificial Intelligence},
  year={2024}
}
```
tweezepy 1.2.7
Tweezepy
Requires Python: >=3.6

# Tweezepy
[![DOI](https://zenodo.org/badge/261266475.svg)](https://zenodo.org/badge/latestdoi/261266475)


This is [Tweezepy](https://github.com/ianlmorgan/tweezepy), a Python package for calibrating forces in single-molecule force spectroscopy video-tracking experiments using the power spectral density (PSD) and Allan variance (AV).

## Documentation
Read the documentation for [Tweezepy](https://tweezepy.readthedocs.io/).

## How to install
The simplest method of installing the `Tweezepy` package is via the [Python Package Index](https://packaging.python.org/glossary/#term-python-package-index-pypi) (PyPI). To install from PyPI, you will need to be able to run python from the command line and make sure you have [pip](https://packaging.python.org/key_projects/#pip) available.

Install from PyPI:

    pip install tweezepy
An alternative method to install `Tweezepy` is with setuptools.  Clone the repository onto a local machine, then navigate to the directory.

Using setuptools:
    
    cd path/to/tweezepy

    python setup.py install
    
## Contents
The `Tweezepy` package includes the following modules:
* 'smmcalibration' - classes for calibration methods using the PSD and AV
* 'expressions' - functions with closed-form expressions for thermal motion in the PSD and AV
* 'MLE' - classes for maximum likelihood estimation (MLE) and Monte Carlo Markov chain (MCMC) sampling
* 'allanvar' - tools for calculating the AV and equivalent degrees of freedom
* 'simulations' - tools to simulate bead thermal motion