| # BLOOMZ, a version for Petals | |
| This model is a version of [bigscience/bloomz](https://huggingface.co/bigscience/bloomz) | |
| post-processed to be run at home using the [Petals](https://github.com/bigscience-workshop/petals#readme) swarm. | |
| Please check out: | |
| - The [original model card](https://huggingface.co/bigscience/bloomz) | |
| to learn about the model's capabilities, specifications, and terms of use. | |
| - The [Petals repository](https://github.com/bigscience-workshop/petals#readme) | |
| to learn how to install Petals and run this model over the Petals swarm. | |
| We provide minimal code examples below. | |
| ## Using the model | |
| ```python | |
| from petals import DistributedBloomForCausalLM | |
| model = DistributedBloomForCausalLM.from_pretrained("bigscience/bloomz-petals") | |
| # Embeddings & prompts are on your device, BLOOM blocks are distributed across the Internet | |
| inputs = tokenizer("A cat sat", return_tensors="pt")["input_ids"] | |
| outputs = model.generate(inputs, max_new_tokens=5) | |
| print(tokenizer.decode(outputs[0])) # A cat sat on a mat... | |
| ``` | |
| ## Serving the model blocks | |
| ```bash | |
| python -m petals.cli.run_server bigscience/bloomz-petals | |
| ``` |