LLukas22 commited on
Commit
c3e8713
1 Parent(s): 541dd70

Generated README.md

Browse files
Files changed (1) hide show
  1. README.md +68 -0
README.md ADDED
@@ -0,0 +1,68 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ ---
2
+ license: cc-by-sa-4.0
3
+ language:
4
+ - en
5
+ tags:
6
+ - llm-rs
7
+ - ggml
8
+ pipeline_tag: text-generation
9
+ ---
10
+
11
+ # GGML converted version of [StabilityAI](https://huggingface.co/stabilityai)'s StableLM models
12
+
13
+ ## Description
14
+
15
+ `StableLM-Base-Alpha` is a suite of 3B and 7B parameter decoder-only language models pre-trained on a diverse collection of English and Code datasets with a sequence length of 4096 to push beyond the context window limitations of existing open-source language models.
16
+
17
+
18
+ ## Converted Models
19
+ | Name | Based on | Type | Container | GGML Version |
20
+ |:------------------------------------------------------------------------------------------------------------------------------------------|:--------------------------------------------------------------------------------------------------|:-------|:------------|:---------------|
21
+ | [stablelm-base-alpha-3b-f16.bin](https://huggingface.co/rustformers/stablelm-ggml/blob/main/stablelm-base-alpha-3b-f16.bin) | [stabilityai/stablelm-base-alpha-3b](https://huggingface.co/stabilityai/stablelm-base-alpha-3b) | F16 | GGML | V3 |
22
+ | [stablelm-base-alpha-3b-q4_0.bin](https://huggingface.co/rustformers/stablelm-ggml/blob/main/stablelm-base-alpha-3b-q4_0.bin) | [stabilityai/stablelm-base-alpha-3b](https://huggingface.co/stabilityai/stablelm-base-alpha-3b) | Q4_0 | GGML | V3 |
23
+ | [stablelm-base-alpha-3b-q4_0-ggjt.bin](https://huggingface.co/rustformers/stablelm-ggml/blob/main/stablelm-base-alpha-3b-q4_0-ggjt.bin) | [stabilityai/stablelm-base-alpha-3b](https://huggingface.co/stabilityai/stablelm-base-alpha-3b) | Q4_0 | GGJT | V3 |
24
+ | [stablelm-base-alpha-3b-q5_1-ggjt.bin](https://huggingface.co/rustformers/stablelm-ggml/blob/main/stablelm-base-alpha-3b-q5_1-ggjt.bin) | [stabilityai/stablelm-base-alpha-3b](https://huggingface.co/stabilityai/stablelm-base-alpha-3b) | Q5_1 | GGJT | V3 |
25
+ | [stablelm-base-alpha-7b-f16.bin](https://huggingface.co/rustformers/stablelm-ggml/blob/main/stablelm-base-alpha-7b-f16.bin) | [stabilityai/stablelm-base-alpha-7b](https://huggingface.co/stabilityai/stablelm-base-alpha-7b) | F16 | GGML | V3 |
26
+ | [stablelm-base-alpha-7b-q4_0.bin](https://huggingface.co/rustformers/stablelm-ggml/blob/main/stablelm-base-alpha-7b-q4_0.bin) | [stabilityai/stablelm-base-alpha-7b](https://huggingface.co/stabilityai/stablelm-base-alpha-7b) | Q4_0 | GGML | V3 |
27
+ | [stablelm-base-alpha-7b-q4_0-ggjt.bin](https://huggingface.co/rustformers/stablelm-ggml/blob/main/stablelm-base-alpha-7b-q4_0-ggjt.bin) | [stabilityai/stablelm-base-alpha-7b](https://huggingface.co/stabilityai/stablelm-base-alpha-7b) | Q4_0 | GGJT | V3 |
28
+ | [stablelm-base-alpha-7b-q5_1-ggjt.bin](https://huggingface.co/rustformers/stablelm-ggml/blob/main/stablelm-base-alpha-7b-q5_1-ggjt.bin) | [stabilityai/stablelm-base-alpha-7b](https://huggingface.co/stabilityai/stablelm-base-alpha-7b) | Q5_1 | GGJT | V3 |
29
+ | [stablelm-tuned-alpha-3b-f16.bin](https://huggingface.co/rustformers/stablelm-ggml/blob/main/stablelm-tuned-alpha-3b-f16.bin) | [stabilityai/stablelm-tuned-alpha-3b](https://huggingface.co/stabilityai/stablelm-tuned-alpha-3b) | F16 | GGML | V3 |
30
+ | [stablelm-tuned-alpha-3b-q4_0.bin](https://huggingface.co/rustformers/stablelm-ggml/blob/main/stablelm-tuned-alpha-3b-q4_0.bin) | [stabilityai/stablelm-tuned-alpha-3b](https://huggingface.co/stabilityai/stablelm-tuned-alpha-3b) | Q4_0 | GGML | V3 |
31
+ | [stablelm-tuned-alpha-3b-q4_0-ggjt.bin](https://huggingface.co/rustformers/stablelm-ggml/blob/main/stablelm-tuned-alpha-3b-q4_0-ggjt.bin) | [stabilityai/stablelm-tuned-alpha-3b](https://huggingface.co/stabilityai/stablelm-tuned-alpha-3b) | Q4_0 | GGJT | V3 |
32
+ | [stablelm-tuned-alpha-3b-q5_1-ggjt.bin](https://huggingface.co/rustformers/stablelm-ggml/blob/main/stablelm-tuned-alpha-3b-q5_1-ggjt.bin) | [stabilityai/stablelm-tuned-alpha-3b](https://huggingface.co/stabilityai/stablelm-tuned-alpha-3b) | Q5_1 | GGJT | V3 |
33
+ | [stablelm-tuned-alpha-7b-f16.bin](https://huggingface.co/rustformers/stablelm-ggml/blob/main/stablelm-tuned-alpha-7b-f16.bin) | [stabilityai/stablelm-tuned-alpha-7b](https://huggingface.co/stabilityai/stablelm-tuned-alpha-7b) | F16 | GGML | V3 |
34
+ | [stablelm-tuned-alpha-7b-q4_0.bin](https://huggingface.co/rustformers/stablelm-ggml/blob/main/stablelm-tuned-alpha-7b-q4_0.bin) | [stabilityai/stablelm-tuned-alpha-7b](https://huggingface.co/stabilityai/stablelm-tuned-alpha-7b) | Q4_0 | GGML | V3 |
35
+ | [stablelm-tuned-alpha-7b-q4_0-ggjt.bin](https://huggingface.co/rustformers/stablelm-ggml/blob/main/stablelm-tuned-alpha-7b-q4_0-ggjt.bin) | [stabilityai/stablelm-tuned-alpha-7b](https://huggingface.co/stabilityai/stablelm-tuned-alpha-7b) | Q4_0 | GGJT | V3 |
36
+ | [stablelm-tuned-alpha-7b-q5_1-ggjt.bin](https://huggingface.co/rustformers/stablelm-ggml/blob/main/stablelm-tuned-alpha-7b-q5_1-ggjt.bin) | [stabilityai/stablelm-tuned-alpha-7b](https://huggingface.co/stabilityai/stablelm-tuned-alpha-7b) | Q5_1 | GGJT | V3 |
37
+
38
+ ## Usage
39
+
40
+ ### Python via [llm-rs](https://github.com/LLukas22/llm-rs-python):
41
+
42
+ #### Installation
43
+ Via pip: `pip install llm-rs`
44
+
45
+ #### Run inference
46
+ ```python
47
+ from llm_rs import AutoModel
48
+
49
+ #Load the model, define any model you like from the list above as the `model_file`
50
+ model = AutoModel.from_pretrained("rustformers/stablelm-ggml",model_file="stablelm-base-alpha-3b-q4_0-ggjt.bin")
51
+
52
+ #Generate
53
+ print(model.generate("The meaning of life is"))
54
+ ```
55
+
56
+ ### Rust via [Rustformers/llm](https://github.com/rustformers/llm):
57
+
58
+ #### Installation
59
+ ```
60
+ git clone --recurse-submodules https://github.com/rustformers/llm.git
61
+ cd llm
62
+ cargo build --release
63
+ ```
64
+
65
+ #### Run inference
66
+ ```
67
+ cargo run --release -- gptneox infer -m path/to/model.bin -p "Tell me how cool the Rust programming language is:"
68
+ ```