semvec/pypi-packages huggingface dataset:
yDataPrep 0.0.1.2.1
prepare your dataset for finetuning LLMs

# Dataset Preparation for Transformers Fine-tuning


[![License](https://img.shields.io/badge/license-MIT-blue.svg)](https://opensource.org/licenses/MIT)

The **Dataset Prep Transformers** package simplifies the process of preparing datasets for fine-tuning or training various large language models available in the Hugging Face Transformers library. Whether you're using a model from the Hugging Face repository or have your own dataset, this package streamlines the data integration for a seamless training experience.

## Features

- Easily integrate your dataset with Hugging Face Transformers models for training or fine-tuning.
- Specify the model repository ID and dataset from the Hugging Face library to automatically fetch and configure the data.
- Seamlessly incorporate your custom dataset by providing it as input to the package.
- new: custom map function => map_function = your_function (having the facility of tokenization)

## Installation

You can install the package using pip:

```bash
pip install yDataPrep