semvec/pypi-packages huggingface dataset:
preprocessinglib 0.1
A comprehensive data preprocessing library for Python
Requires Python: >=3.6

PreprocessingLib
PreprocessingLib is a Python library designed to facilitate data preprocessing steps. It provides various classes and functions to automate the process of cleaning, transforming, and engineering features in datasets.

Features
1. Missing Value Handling
Detect missing values in a dataset.
Fill missing values using mean, median, or a constant value.
Remove rows or columns with missing values.
2. Feature Engineering
Create new features based on existing ones.
3. Date and Time Handling
Extract features like year, month, day, and day of the week from datetime columns.
4. Data Type Conversion
Convert columns to numeric or categorical data types.
5. Categorical Encoding
Perform one-hot encoding or label encoding on categorical variables.
6. Outlier Handling
Detect outliers in numerical data.
Handle outliers by removing or replacing them.
7. Data Scaling
Standardize or normalize numerical data.
8. Text Cleaning
Clean text data by removing punctuation, stop words, and lemmatizing words.
Installation
You can install PreprocessingLib using pip:

pip install preprocessinglib
Usage
Here's how you can use PreprocessingLib in your Python projects:

from mypreprocessinglib import FeatureEngineer, MissingValueHandler, DateTimeHandler, DataTypeConverter, CategoricalEncoder, OutlierHandler, Scaler, TextCleaner
import pandas as pd

# Load sample dataset
data = pd.read_csv("sample_dataset.csv")

# Example usage of preprocessing functions
missing_handler = MissingValueHandler()
filled_data = missing_handler.fill_missing_values(data)

data_with_new_features = FeatureEngineer.create_new_features(data, column1='Column1', column2='Column2')

date_with_features = DateTimeHandler.extract_date_features(data, column='DateColumn')

numeric_data = DataTypeConverter.convert_to_numeric(data, columns=['Column1', 'Column2'])

encoded_data = CategoricalEncoder.one_hot_encode(data, columns=['CategoricalColumn'])

outliers_removed_data = OutlierHandler.handle_outliers(data, method='drop')

scaled_data = Scaler.standardize_data(data)

cleaned_text = TextCleaner.clean_text("example text")
Testing
You can run the unit tests to ensure the proper functioning of the library:

python -m unittest test_data_preprocessing.py
Contributing
Contributions are welcome! If you find any issues or have suggestions for improvements, please feel free to open an issue or submit a pull request.

License
This project is licensed under the MIT License - see the LICENSE file for details.
remla24-team7-app 0.0.2
App for assigment 2 of REMLA
Keywords: frontend, backend

# app

This is the repository for the app of Assigment 2. 

The frontend is in the frontend folder and the backend is just `app.py`. 

Entering a URL and clicking "Predict" in the web frontend queries the `app-service` backend, which then communicates with `model-service` to retrieve the prediction. The prediciton is then shown on the website. Make sure to have `model-service` running and configured (`MODEL_SERVICE_URL`) for this backend to work.
pdf2anki 0.1.3
A Python package to create Anki cards from PDFs using OpenAI.

# pdf2anki

`pdf2anki` is a Python application that enables you to generate Anki flashcards questions from PDF files utilizing the power of OpenAI ChatGPT.

## Prerequisites

- Python 3.6 or higher
- Access to OpenAI API (ensure to set your API key as an environment variable named 'OPENAI_API_KEY')

## Installation

You can install `pdf2anki` from PyPI:

```bash
pip install pdf2anki
```

## Usage

Once installed, you can use the pdf2anki command to generate Anki cards from a directory of PDFs. The following arguments are needed:

```bash
pdf2anki --source-dir SOURCE_DIR --output-dir OUTPUT_DIR --model MODEL --temperature TEMPERATURE --max-tokens MAX_TOKENS
```

For example:

```bash
pdf2anki --source-dir ./pdfs --output-dir ./anki-cards --model gpt-4 --temperature 0.5 --max-tokens 1000
```

## For Developers
Developers can also use pdf2anki in their Python scripts as follows:

```bash
import os
from pdf2anki.main import read_pdf, create_anki_cards

#### Define the source PDF folder and the output folder for the Anki cards
source_dir = "path/to/pdf_folder"
output_dir = "path/to/output_folder"

#### Define the parameters for the OpenAI model
model = "gpt-4"
temperature = 0.5
max_tokens = 1000

#### Iterate over the PDF files in the source directory
for file_name in os.listdir(source_dir):
    if file_name.endswith(".pdf"):
        # Get the full path to the source PDF
        source_pdf = os.path.join(source_dir, file_name)

        # Read the PDF text
        pdf_text = read_pdf(source_pdf)

        # Create output file path
        output_file = os.path.join(output_dir, file_name.rsplit(".", 1)[0] + ".txt")

        # Create Anki cards from the PDF text
        create_anki_cards(pdf_text, output_file, model, temperature, max_tokens)
```

## License
This project is licensed under the terms of the MIT license. For more details, see the LICENSE file.

## References
- [pdf2anki package](https://pypi.org/project/pdf2anki/)
- [OpenAI API key](https://platform.openai.com/account/api-keys)