File size: 7,180 Bytes
1f0e430 |
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 |
---
datasets:
- bigscience/xP3
license: bigscience-bloom-rail-1.0
language:
- ak
- ar
- as
- bm
- bn
- ca
- code
- en
- es
- eu
- fon
- fr
- gu
- hi
- id
- ig
- ki
- kn
- lg
- ln
- ml
- mr
- ne
- nso
- ny
- or
- pa
- pt
- rn
- rw
- sn
- st
- sw
- ta
- te
- tn
- ts
- tum
- tw
- ur
- vi
- wo
- xh
- yo
- zh
- zu
programming_language:
- C
- C++
- C#
- Go
- Java
- JavaScript
- Lua
- PHP
- Python
- Ruby
- Rust
- Scala
- TypeScript
tags:
- llm-rs
- ggml
pipeline_tag: text-generation
---
# GGML covnerted Models of [BigScience](https://huggingface.co/bigscience)'s Bloom models
## Description
> We present BLOOMZ & mT0, a family of models capable of following human instructions in dozens of languages zero-shot. We finetune BLOOM & mT5 pretrained multilingual language models on our crosslingual task mixture (xP3) and find the resulting models capable of crosslingual generalization to unseen tasks & languages.
- **Repository:** [bigscience-workshop/xmtf](https://github.com/bigscience-workshop/xmtf)
- **Paper:** [Crosslingual Generalization through Multitask Finetuning](https://arxiv.org/abs/2211.01786)
- **Point of Contact:** [Niklas Muennighoff](mailto:niklas@hf.co)
- **Languages:** Refer to [bloom](https://huggingface.co/bigscience/bloom) for pretraining & [xP3](https://huggingface.co/datasets/bigscience/xP3) for finetuning language proportions. It understands both pretraining & finetuning languages.
### Intended use
We recommend using the model to perform tasks expressed in natural language. For example, given the prompt "*Translate to English: Je t’aime.*", the model will most likely answer "*I love you.*". Some prompt ideas from our paper:
- 一个传奇的开端，一个不灭的神话，这不仅仅是一部电影，而是作为一个走进新时代的标签，永远彪炳史册。你认为这句话的立场是赞扬、中立还是批评?
- Suggest at least five related search terms to "Mạng neural nhân tạo".
- Write a fairy tale about a troll saving a princess from a dangerous dragon. The fairy tale is a masterpiece that has achieved praise worldwide and its moral is "Heroes Come in All Shapes and Sizes". Story (in Spanish):
- Explain in a sentence in Telugu what is backpropagation in neural networks.
## Converted Models
| Name | Based on | Type | Container | GGML Version |
|:----------------------------------------------------------------------------------------------------------------|:------------------------------------------------------------------------|:-------|:------------|:---------------|
| [bloomz-1b1-f16.bin](https://huggingface.co/rustformers/bloomz-ggml/blob/main/bloomz-1b1-f16.bin) | [bigscience/bloomz-1b1](https://huggingface.co/bigscience/bloomz-1b1) | F16 | GGML | V3 |
| [bloomz-1b1-q4_0.bin](https://huggingface.co/rustformers/bloomz-ggml/blob/main/bloomz-1b1-q4_0.bin) | [bigscience/bloomz-1b1](https://huggingface.co/bigscience/bloomz-1b1) | Q4_0 | GGML | V3 |
| [bloomz-1b1-q4_0-ggjt.bin](https://huggingface.co/rustformers/bloomz-ggml/blob/main/bloomz-1b1-q4_0-ggjt.bin) | [bigscience/bloomz-1b1](https://huggingface.co/bigscience/bloomz-1b1) | Q4_0 | GGJT | V3 |
| [bloomz-1b1-q5_1-ggjt.bin](https://huggingface.co/rustformers/bloomz-ggml/blob/main/bloomz-1b1-q5_1-ggjt.bin) | [bigscience/bloomz-1b1](https://huggingface.co/bigscience/bloomz-1b1) | Q5_1 | GGJT | V3 |
| [bloomz-1b7-f16.bin](https://huggingface.co/rustformers/bloomz-ggml/blob/main/bloomz-1b7-f16.bin) | [bigscience/bloomz-1b7](https://huggingface.co/bigscience/bloomz-1b7) | F16 | GGML | V3 |
| [bloomz-1b7-q4_0.bin](https://huggingface.co/rustformers/bloomz-ggml/blob/main/bloomz-1b7-q4_0.bin) | [bigscience/bloomz-1b7](https://huggingface.co/bigscience/bloomz-1b7) | Q4_0 | GGML | V3 |
| [bloomz-1b7-q4_0-ggjt.bin](https://huggingface.co/rustformers/bloomz-ggml/blob/main/bloomz-1b7-q4_0-ggjt.bin) | [bigscience/bloomz-1b7](https://huggingface.co/bigscience/bloomz-1b7) | Q4_0 | GGJT | V3 |
| [bloomz-1b7-q5_1-ggjt.bin](https://huggingface.co/rustformers/bloomz-ggml/blob/main/bloomz-1b7-q5_1-ggjt.bin) | [bigscience/bloomz-1b7](https://huggingface.co/bigscience/bloomz-1b7) | Q5_1 | GGJT | V3 |
| [bloomz-3b-f16.bin](https://huggingface.co/rustformers/bloomz-ggml/blob/main/bloomz-3b-f16.bin) | [bigscience/bloomz-3b](https://huggingface.co/bigscience/bloomz-3b) | F16 | GGML | V3 |
| [bloomz-3b-q4_0.bin](https://huggingface.co/rustformers/bloomz-ggml/blob/main/bloomz-3b-q4_0.bin) | [bigscience/bloomz-3b](https://huggingface.co/bigscience/bloomz-3b) | Q4_0 | GGML | V3 |
| [bloomz-3b-q4_0-ggjt.bin](https://huggingface.co/rustformers/bloomz-ggml/blob/main/bloomz-3b-q4_0-ggjt.bin) | [bigscience/bloomz-3b](https://huggingface.co/bigscience/bloomz-3b) | Q4_0 | GGJT | V3 |
| [bloomz-3b-q5_1-ggjt.bin](https://huggingface.co/rustformers/bloomz-ggml/blob/main/bloomz-3b-q5_1-ggjt.bin) | [bigscience/bloomz-3b](https://huggingface.co/bigscience/bloomz-3b) | Q5_1 | GGJT | V3 |
| [bloomz-560m-f16.bin](https://huggingface.co/rustformers/bloomz-ggml/blob/main/bloomz-560m-f16.bin) | [bigscience/bloomz-560m](https://huggingface.co/bigscience/bloomz-560m) | F16 | GGML | V3 |
| [bloomz-560m-q4_0.bin](https://huggingface.co/rustformers/bloomz-ggml/blob/main/bloomz-560m-q4_0.bin) | [bigscience/bloomz-560m](https://huggingface.co/bigscience/bloomz-560m) | Q4_0 | GGML | V3 |
| [bloomz-560m-q4_0-ggjt.bin](https://huggingface.co/rustformers/bloomz-ggml/blob/main/bloomz-560m-q4_0-ggjt.bin) | [bigscience/bloomz-560m](https://huggingface.co/bigscience/bloomz-560m) | Q4_0 | GGJT | V3 |
| [bloomz-560m-q5_1-ggjt.bin](https://huggingface.co/rustformers/bloomz-ggml/blob/main/bloomz-560m-q5_1-ggjt.bin) | [bigscience/bloomz-560m](https://huggingface.co/bigscience/bloomz-560m) | Q5_1 | GGJT | V3 |
## Usage
### Python via [llm-rs](https://github.com/LLukas22/llm-rs-python):
#### Installation
Via pip: `pip install llm-rs`
#### Run inference
```python
from llm_rs import AutoModel
#Load the model, define any model you like from the list above as the `model_file`
model = AutoModel.from_pretrained("rustformers/bloomz-ggml",model_file="bloomz-3b-q4_0-ggjt.bin")
#Generate
print(model.generate("The meaning of life is"))
```
### Rust via [Rustformers/llm](https://github.com/rustformers/llm):
#### Installation
```
git clone --recurse-submodules https://github.com/rustformers/llm.git
cd llm
cargo build --release
```
#### Run inference
```
cargo run --release -- bloom infer -m path/to/model.bin -p "Tell me how cool the Rust programming language is:"
``` |