| license: cc-by-sa-4.0 | |
| language: | |
| - en | |
| tags: | |
| - llm-rs | |
| - ggml | |
| pipeline_tag: text-generation | |
| # GGML converted version of [StabilityAI](https://huggingface.co/stabilityai)'s StableLM models | |
| ## Description | |
| `StableLM-Base-Alpha` is a suite of 3B and 7B parameter decoder-only language models pre-trained on a diverse collection of English and Code datasets with a sequence length of 4096 to push beyond the context window limitations of existing open-source language models. | |
| ## Converted Models | |
| $MODELS$ | |
| ## Usage | |
| ### Python via [llm-rs](https://github.com/LLukas22/llm-rs-python): | |
| #### Installation | |
| Via pip: `pip install llm-rs` | |
| #### Run inference | |
| ```python | |
| from llm_rs import AutoModel | |
| #Load the model, define any model you like from the list above as the `model_file` | |
| model = AutoModel.from_pretrained("rustformers/stablelm-ggml",model_file="stablelm-base-alpha-3b-q4_0-ggjt.bin") | |
| #Generate | |
| print(model.generate("The meaning of life is")) | |
| ``` | |
| ### Rust via [Rustformers/llm](https://github.com/rustformers/llm): | |
| #### Installation | |
| ``` | |
| git clone --recurse-submodules https://github.com/rustformers/llm.git | |
| cd llm | |
| cargo build --release | |
| ``` | |
| #### Run inference | |
| ``` | |
| cargo run --release -- gptneox infer -m path/to/model.bin -p "Tell me how cool the Rust programming language is:" | |
| ``` |