semvec/pypi-packages huggingface dataset:
niuload 0.2.4
A module for balanced loading of large language models
Requires Python: >=3.6

# NiuLoad

niuload is a Python module for balanced loading large language models across multiple GPUs(pipeline parallel).
This package can be used during training and inference.

## Report (Chinese)
More details can be found at:
https://zhuanlan.zhihu.com/p/792303768 

## Usage

niuload offers the following advanced features:

1. Customizable load distribution across each GPU. If not specified, the model will be evenly distributed across all GPUs by default.

```python
from niuload import balanced_load
model = balanced_load("openai-community/gpt2", ratio=["0.5,1,1"], num_devices=4)
```
This example allows you to split the model across four GPUs, with GPU 0 only having half the load compared to the others.

2. Specify which GPUs to use for model splitting.

```python
from niuload import balanced_load
model = balanced_load("openai-community/gpt2", device_idx=[1,2,5,7], num_devices=4)
```
3. Support for training. Models automatically split by Hugging Face's `device_map='auto'` do not always support training due to a current bug in accelerate. Specifically, when model.device is not on device 0, certain models can encounter errors. We currently force embeddings to be on GPU 0 to avoid this issue.

4. Support for splitting models that are not supported by Hugging Face, but this requires some additional adaptation. We are working on reorganizing the code structure soon and welcome community contributions to the project.


## Benchmark
For reproducing our results in report, you can checkout scripts under /benchmark.

## FQA
1. Q: Why do I see the model being loaded twice?

    A: To better analyze the model's parameter structure, we first load the model onto a meta device. This process is very fast because the initialized tensors are random and meaningless. After obtaining the device_map, we then actually load the model.
SciLite 0.0.1
Sci-Lite is Light version of Supervised Machine Learning Model
Requires Python: >=3.6

# **SciLite**


**SciLite** provides a streamlined solution for evaluating various supervised machine learning algorithms, simplifying the data scientist's workflow by automating model integration and evaluation. With just a few lines of code, SciLite enables quick access to essential machine learning evaluations, saving valuable time and effort.

### Installation

To install the package, run:

'''bash
pip install SciLite
'''

### Usage

Basic example of how to use the package in Python:

'''bash
from SciLite.Single_Output.Regression import Regression as reg

# Evaluate model performance
result = reg(X_train, X_test, y_train, y_test)
print(result)

'''

### Features

- **One-Click Model Integration:** Supports easy integration and evaluation of multiple supervised machine learning models, whether for regression or classification.
- **Efficient Model Evaluation:** Simplifies machine learning model evaluations with standardized output, aiding in faster analysis and comparisons.

### Contributing

Contributions to SciLite are warmly welcomed! If you'd like to contribute, please fork the repository and submit a pull request or open an issue to discuss improvements.

### Contact
For any questions or issues, feel free to reach out to:

Author: Ayush Jain Sparsh
Email: ayushjainsparsh2004.ajs@gmail.com
imagepig 1.0.7
Package to use Image Pig, the API for AI images
Requires Python: >=3.8

# imagepig
[Python package](https://pypi.org/project/imagepig/) for [Image Pig](https://imagepig.com/), the API for AI images.

## Installation

```
pip install imagepig
```

## Example of usage

```python
from imagepig import ImagePig

# create instance of API (put here your actual API key)
imagepig = ImagePig("your-api-key")

# call the API with a prompt to generate an image
result = imagepig.xl("cute piglet running on a green garden")

# save image to a file
result.save("cute-piglet.jpeg")

# or access image data (bytes)
result.data

# or access image as an object (needs to have the Pillow package installed)
result.image
```

## Contact us
Something does not work as expected? Feel free to [send us a message](https://imagepig.com/contact/), we are here for you.
oss2-uploader 0.1.0
Uploader files in a folder to Aliyun OSS
Requires Python: >=3.8

### Uploader all files in a folder to Aliyun OSS

Must load `OSS_ACCESS_KEY_ID` and `OSS_ACCESS_KEY_SECRET` into environment variables first.

## Install

`pip install oss2-uploader`

## Usage

only executable under `if __name__ == "__main__":`

```
if __name__ == "__main__":

    from oss2_uploader import folder_uploader_sync

    folder_uploader_sync(
        os.path.join(os.path.expanduser("~"), "Documents"),
        "bucket_name",
        "oss-ap-southeast-1.aliyuncs.com",
    )
```