--- datasets: - bigscience/xP3 license: bigscience-bloom-rail-1.0 language: - ak - ar - as - bm - bn - ca - code - en - es - eu - fon - fr - gu - hi - id - ig - ki - kn - lg - ln - ml - mr - ne - nso - ny - or - pa - pt - rn - rw - sn - st - sw - ta - te - tn - ts - tum - tw - ur - vi - wo - xh - yo - zh - zu programming_language: - C - C++ - C# - Go - Java - JavaScript - Lua - PHP - Python - Ruby - Rust - Scala - TypeScript tags: - llm-rs - ggml pipeline_tag: text-generation --- # GGML converted versions of [BigScience](https://huggingface.co/bigscience)'s BloomZ models ## Description > We present BLOOMZ & mT0, a family of models capable of following human instructions in dozens of languages zero-shot. We finetune BLOOM & mT5 pretrained multilingual language models on our crosslingual task mixture (xP3) and find the resulting models capable of crosslingual generalization to unseen tasks & languages. - **Repository:** [bigscience-workshop/xmtf](https://github.com/bigscience-workshop/xmtf) - **Paper:** [Crosslingual Generalization through Multitask Finetuning](https://arxiv.org/abs/2211.01786) - **Point of Contact:** [Niklas Muennighoff](mailto:niklas@hf.co) - **Languages:** Refer to [bloom](https://huggingface.co/bigscience/bloom) for pretraining & [xP3](https://huggingface.co/datasets/bigscience/xP3) for finetuning language proportions. It understands both pretraining & finetuning languages. ### Intended use We recommend using the model to perform tasks expressed in natural language. For example, given the prompt "*Translate to English: Je t’aime.*", the model will most likely answer "*I love you.*". Some prompt ideas from our paper: - 一个传奇的开端，一个不灭的神话，这不仅仅是一部电影，而是作为一个走进新时代的标签，永远彪炳史册。你认为这句话的立场是赞扬、中立还是批评? - Suggest at least five related search terms to "Mạng neural nhân tạo". - Write a fairy tale about a troll saving a princess from a dangerous dragon. The fairy tale is a masterpiece that has achieved praise worldwide and its moral is "Heroes Come in All Shapes and Sizes". Story (in Spanish): - Explain in a sentence in Telugu what is backpropagation in neural networks. ## Converted Models | Name | Based on | Type | Container | GGML Version | |:----------------------------------------------------------------------------------------------------------------|:------------------------------------------------------------------------|:-------|:------------|:---------------| | [bloomz-1b1-f16.bin](https://huggingface.co/rustformers/bloomz-ggml/blob/main/bloomz-1b1-f16.bin) | [bigscience/bloomz-1b1](https://huggingface.co/bigscience/bloomz-1b1) | F16 | GGML | V3 | | [bloomz-1b1-q4_0.bin](https://huggingface.co/rustformers/bloomz-ggml/blob/main/bloomz-1b1-q4_0.bin) | [bigscience/bloomz-1b1](https://huggingface.co/bigscience/bloomz-1b1) | Q4_0 | GGML | V3 | | [bloomz-1b1-q4_0-ggjt.bin](https://huggingface.co/rustformers/bloomz-ggml/blob/main/bloomz-1b1-q4_0-ggjt.bin) | [bigscience/bloomz-1b1](https://huggingface.co/bigscience/bloomz-1b1) | Q4_0 | GGJT | V3 | | [bloomz-1b1-q5_1.bin](https://huggingface.co/rustformers/bloomz-ggml/blob/main/bloomz-1b1-q5_1.bin) | [bigscience/bloomz-1b1](https://huggingface.co/bigscience/bloomz-1b1) | Q5_1 | GGML | V3 | | [bloomz-1b1-q5_1-ggjt.bin](https://huggingface.co/rustformers/bloomz-ggml/blob/main/bloomz-1b1-q5_1-ggjt.bin) | [bigscience/bloomz-1b1](https://huggingface.co/bigscience/bloomz-1b1) | Q5_1 | GGJT | V3 | | [bloomz-1b7-f16.bin](https://huggingface.co/rustformers/bloomz-ggml/blob/main/bloomz-1b7-f16.bin) | [bigscience/bloomz-1b7](https://huggingface.co/bigscience/bloomz-1b7) | F16 | GGML | V3 | | [bloomz-1b7-q4_0.bin](https://huggingface.co/rustformers/bloomz-ggml/blob/main/bloomz-1b7-q4_0.bin) | [bigscience/bloomz-1b7](https://huggingface.co/bigscience/bloomz-1b7) | Q4_0 | GGML | V3 | | [bloomz-1b7-q4_0-ggjt.bin](https://huggingface.co/rustformers/bloomz-ggml/blob/main/bloomz-1b7-q4_0-ggjt.bin) | [bigscience/bloomz-1b7](https://huggingface.co/bigscience/bloomz-1b7) | Q4_0 | GGJT | V3 | | [bloomz-1b7-q5_1.bin](https://huggingface.co/rustformers/bloomz-ggml/blob/main/bloomz-1b7-q5_1.bin) | [bigscience/bloomz-1b7](https://huggingface.co/bigscience/bloomz-1b7) | Q5_1 | GGML | V3 | | [bloomz-1b7-q5_1-ggjt.bin](https://huggingface.co/rustformers/bloomz-ggml/blob/main/bloomz-1b7-q5_1-ggjt.bin) | [bigscience/bloomz-1b7](https://huggingface.co/bigscience/bloomz-1b7) | Q5_1 | GGJT | V3 | | [bloomz-3b-f16.bin](https://huggingface.co/rustformers/bloomz-ggml/blob/main/bloomz-3b-f16.bin) | [bigscience/bloomz-3b](https://huggingface.co/bigscience/bloomz-3b) | F16 | GGML | V3 | | [bloomz-3b-q4_0.bin](https://huggingface.co/rustformers/bloomz-ggml/blob/main/bloomz-3b-q4_0.bin) | [bigscience/bloomz-3b](https://huggingface.co/bigscience/bloomz-3b) | Q4_0 | GGML | V3 | | [bloomz-3b-q4_0-ggjt.bin](https://huggingface.co/rustformers/bloomz-ggml/blob/main/bloomz-3b-q4_0-ggjt.bin) | [bigscience/bloomz-3b](https://huggingface.co/bigscience/bloomz-3b) | Q4_0 | GGJT | V3 | | [bloomz-3b-q5_1.bin](https://huggingface.co/rustformers/bloomz-ggml/blob/main/bloomz-3b-q5_1.bin) | [bigscience/bloomz-3b](https://huggingface.co/bigscience/bloomz-3b) | Q5_1 | GGML | V3 | | [bloomz-3b-q5_1-ggjt.bin](https://huggingface.co/rustformers/bloomz-ggml/blob/main/bloomz-3b-q5_1-ggjt.bin) | [bigscience/bloomz-3b](https://huggingface.co/bigscience/bloomz-3b) | Q5_1 | GGJT | V3 | | [bloomz-560m-f16.bin](https://huggingface.co/rustformers/bloomz-ggml/blob/main/bloomz-560m-f16.bin) | [bigscience/bloomz-560m](https://huggingface.co/bigscience/bloomz-560m) | F16 | GGML | V3 | | [bloomz-560m-q4_0.bin](https://huggingface.co/rustformers/bloomz-ggml/blob/main/bloomz-560m-q4_0.bin) | [bigscience/bloomz-560m](https://huggingface.co/bigscience/bloomz-560m) | Q4_0 | GGML | V3 | | [bloomz-560m-q4_0-ggjt.bin](https://huggingface.co/rustformers/bloomz-ggml/blob/main/bloomz-560m-q4_0-ggjt.bin) | [bigscience/bloomz-560m](https://huggingface.co/bigscience/bloomz-560m) | Q4_0 | GGJT | V3 | | [bloomz-560m-q5_1.bin](https://huggingface.co/rustformers/bloomz-ggml/blob/main/bloomz-560m-q5_1.bin) | [bigscience/bloomz-560m](https://huggingface.co/bigscience/bloomz-560m) | Q5_1 | GGML | V3 | | [bloomz-560m-q5_1-ggjt.bin](https://huggingface.co/rustformers/bloomz-ggml/blob/main/bloomz-560m-q5_1-ggjt.bin) | [bigscience/bloomz-560m](https://huggingface.co/bigscience/bloomz-560m) | Q5_1 | GGJT | V3 | ## Usage ### Python via [llm-rs](https://github.com/LLukas22/llm-rs-python): #### Installation Via pip: `pip install llm-rs` #### Run inference ```python from llm_rs import AutoModel #Load the model, define any model you like from the list above as the `model_file` model = AutoModel.from_pretrained("rustformers/bloomz-ggml",model_file="bloomz-3b-q4_0-ggjt.bin") #Generate print(model.generate("The meaning of life is")) ``` ### Rust via [Rustformers/llm](https://github.com/rustformers/llm): #### Installation ``` git clone --recurse-submodules https://github.com/rustformers/llm.git cd llm cargo build --release ``` #### Run inference ``` cargo run --release -- bloom infer -m path/to/model.bin -p "Tell me how cool the Rust programming language is:" ```