Text Generation
Transformers
llm-rs
ggml
Inference Endpoints
Edit model card

GGML converted versions of BigScience's BloomZ models

Description

We present BLOOMZ & mT0, a family of models capable of following human instructions in dozens of languages zero-shot. We finetune BLOOM & mT5 pretrained multilingual language models on our crosslingual task mixture (xP3) and find the resulting models capable of crosslingual generalization to unseen tasks & languages.

Intended use

We recommend using the model to perform tasks expressed in natural language. For example, given the prompt "Translate to English: Je t’aime.", the model will most likely answer "I love you.". Some prompt ideas from our paper:

  • 一个传奇的开端,一个不灭的神话,这不仅仅是一部电影,而是作为一个走进新时代的标签,永远彪炳史册。你认为这句话的立场是赞扬、中立还是批评?
  • Suggest at least five related search terms to "Mạng neural nhân tạo".
  • Write a fairy tale about a troll saving a princess from a dangerous dragon. The fairy tale is a masterpiece that has achieved praise worldwide and its moral is "Heroes Come in All Shapes and Sizes". Story (in Spanish):
  • Explain in a sentence in Telugu what is backpropagation in neural networks.

Converted Models

Usage

Python via llm-rs:

Installation

Via pip: pip install llm-rs

Run inference

from llm_rs import AutoModel

#Load the model, define any model you like from the list above as the `model_file`
model = AutoModel.from_pretrained("rustformers/bloomz-ggml",model_file="bloomz-3b-q4_0-ggjt.bin")

#Generate
print(model.generate("The meaning of life is"))

Rust via Rustformers/llm:

Installation

git clone --recurse-submodules https://github.com/rustformers/llm.git
cd llm
cargo build --release

Run inference

cargo run --release -- bloom infer -m path/to/model.bin  -p "Tell me how cool the Rust programming language is:"
Downloads last month
57
Inference Examples
This model does not have enough activity to be deployed to Inference API (serverless) yet. Increase its social visibility and check back later, or deploy to Inference Endpoints (dedicated) instead.

Dataset used to train rustformers/bloomz-ggml