Hugging Face's logo

rustformers
/
redpajama-3b-ggml

Text Generation
Transformers
English
llm-rs
ggml
Inference Endpoints
Model card Files Community
LLukas22 commited on
Commit
0e0aeaf
•
1 Parent(s): 0cf84ac

Generated README.md

Browse files
Files changed (1) hide show
  1. README.md +18 -14
README.md CHANGED
@@ -19,17 +19,20 @@ The training was done on 3,072 V100 GPUs provided as part of the INCITE 2023 pro
19
 
20
  ## Converted Models:
21
 
22
- | Name | Based on | Type | Container |
23
- |-|-|-|-|
24
- | [RedPajama-INCITE-Base-3B-v1-f16.bin](https://huggingface.co/Rustformers/redpajama-ggml/blob/main/RedPajama-INCITE-Base-3B-v1-f16.bin) | [RedPajama-INCITE-Base](https://huggingface.co/togethercomputer/RedPajama-INCITE-Base-3B-v1) | fp16 | GGML |
25
- | [RedPajama-INCITE-Base-3B-v1-q4_0-ggjt.bin](https://huggingface.co/Rustformers/redpajama-ggml/blob/main/RedPajama-INCITE-Base-3B-v1-q4_0-ggjt.bin) | [RedPajama-INCITE-Base](https://huggingface.co/togethercomputer/RedPajama-INCITE-Base-3B-v1) | int4 | GGJT |
26
- | [RedPajama-INCITE-Base-3B-v1-q4_0.bin](https://huggingface.co/Rustformers/redpajama-ggml/blob/main/RedPajama-INCITE-Base-3B-v1-q4_0.bin) | [RedPajama-INCITE-Base](https://huggingface.co/togethercomputer/RedPajama-INCITE-Base-3B-v1) | int4 | GGML |
27
- | [RedPajama-INCITE-Chat-3B-v1-f16.bin](https://huggingface.co/Rustformers/redpajama-ggml/blob/main/RedPajama-INCITE-Chat-3B-v1-f16.bin) | [RedPajama-INCITE-Chat](https://huggingface.co/togethercomputer/RedPajama-INCITE-Chat-3B-v1) | fp16 | GGML |
28
- | [RedPajama-INCITE-Chat-3B-v1-q4_0-ggjt.bin](https://huggingface.co/Rustformers/redpajama-ggml/blob/main/RedPajama-INCITE-Chat-3B-v1-q4_0-ggjt.bin) | [RedPajama-INCITE-Chat](https://huggingface.co/togethercomputer/RedPajama-INCITE-Chat-3B-v1) | int4 | GGJT |
29
- | [RedPajama-INCITE-Chat-3B-v1-q4_0.bin](https://huggingface.co/Rustformers/redpajama-ggml/blob/main/RedPajama-INCITE-Chat-3B-v1-q4_0.bin) | [RedPajama-INCITE-Chat](https://huggingface.co/togethercomputer/RedPajama-INCITE-Chat-3B-v1) | int4 | GGML |
30
- | [RedPajama-INCITE-Instruct-3B-v1-f16.bin](https://huggingface.co/Rustformers/redpajama-ggml/blob/main/RedPajama-INCITE-Instruct-3B-v1-f16.bin) | [RedPajama-INCITE-Instruct](https://huggingface.co/togethercomputer/RedPajama-INCITE-Instruct-3B-v1) | fp16 | GGML |
31
- | [RedPajama-INCITE-Instruct-3B-v1-q4_0-ggjt.bin](https://huggingface.co/Rustformers/redpajama-ggml/blob/main/RedPajama-INCITE-Instruct-3B-v1-q4_0-ggjt.bin) | [RedPajama-INCITE-Instruct](https://huggingface.co/togethercomputer/RedPajama-INCITE-Instruct-3B-v1) | int4 | GGJT |
32
- | [RedPajama-INCITE-Instruct-3B-v1-q4_0.bin](https://huggingface.co/Rustformers/redpajama-ggml/blob/main/RedPajama-INCITE-Instruct-3B-v1-q4_0.bin) | [RedPajama-INCITE-Instruct](https://huggingface.co/togethercomputer/RedPajama-INCITE-Instruct-3B-v1) | int4 | GGML |
 
 
 
33
 
34
  ## Usage
35
 
@@ -43,7 +46,7 @@ Via pip: `pip install llm-rs`
43
  from llm_rs import AutoModel
44
 
45
  #Load the model, define any model you like from the list above as the `model_file`
46
- model = AutoModel.from_pretrained("Rustformers/redpajama-ggml",model_file="RedPajama-INCITE-Base-3B-v1-q4_0-ggjt.bin")
47
 
48
  #Generate
49
  print(model.generate("The meaning of life is"))
@@ -53,11 +56,12 @@ print(model.generate("The meaning of life is"))
53
 
54
  #### Installation
55
  ```
56
- git clone --recurse-submodules git@github.com:rustformers/llm.git
 
57
  cargo build --release
58
  ```
59
 
60
  #### Run inference
61
  ```
62
  cargo run --release -- gptneox infer -m path/to/model.bin -p "Tell me how cool the Rust programming language is:"
63
- ```
 
19
 
20
  ## Converted Models:
21
 
22
+ | Name | Based on | Type | Container | GGML Version |
23
+ |:-----------------------------------------------------------------------------------------------------------------------------------------------------------|:----------------------------------------------------------------------------------------------------------------------------|:-------|:------------|:---------------|
24
+ | [RedPajama-INCITE-Base-3B-v1-f16.bin](https://huggingface.co/rustformers/redpajama-ggml/blob/main/RedPajama-INCITE-Base-3B-v1-f16.bin) | [togethercomputer/RedPajama-INCITE-Base-3B-v1](https://huggingface.co/togethercomputer/RedPajama-INCITE-Base-3B-v1) | F16 | GGML | V3 |
25
+ | [RedPajama-INCITE-Base-3B-v1-q4_0.bin](https://huggingface.co/rustformers/redpajama-ggml/blob/main/RedPajama-INCITE-Base-3B-v1-q4_0.bin) | [togethercomputer/RedPajama-INCITE-Base-3B-v1](https://huggingface.co/togethercomputer/RedPajama-INCITE-Base-3B-v1) | Q4_0 | GGML | V3 |
26
+ | [RedPajama-INCITE-Base-3B-v1-q4_0-ggjt.bin](https://huggingface.co/rustformers/redpajama-ggml/blob/main/RedPajama-INCITE-Base-3B-v1-q4_0-ggjt.bin) | [togethercomputer/RedPajama-INCITE-Base-3B-v1](https://huggingface.co/togethercomputer/RedPajama-INCITE-Base-3B-v1) | Q4_0 | GGJT | V3 |
27
+ | [RedPajama-INCITE-Base-3B-v1-q5_1-ggjt.bin](https://huggingface.co/rustformers/redpajama-ggml/blob/main/RedPajama-INCITE-Base-3B-v1-q5_1-ggjt.bin) | [togethercomputer/RedPajama-INCITE-Base-3B-v1](https://huggingface.co/togethercomputer/RedPajama-INCITE-Base-3B-v1) | Q5_1 | GGJT | V3 |
28
+ | [RedPajama-INCITE-Chat-3B-v1-f16.bin](https://huggingface.co/rustformers/redpajama-ggml/blob/main/RedPajama-INCITE-Chat-3B-v1-f16.bin) | [togethercomputer/RedPajama-INCITE-Chat-3B-v1](https://huggingface.co/togethercomputer/RedPajama-INCITE-Chat-3B-v1) | F16 | GGML | V3 |
29
+ | [RedPajama-INCITE-Chat-3B-v1-q4_0.bin](https://huggingface.co/rustformers/redpajama-ggml/blob/main/RedPajama-INCITE-Chat-3B-v1-q4_0.bin) | [togethercomputer/RedPajama-INCITE-Chat-3B-v1](https://huggingface.co/togethercomputer/RedPajama-INCITE-Chat-3B-v1) | Q4_0 | GGML | V3 |
30
+ | [RedPajama-INCITE-Chat-3B-v1-q4_0-ggjt.bin](https://huggingface.co/rustformers/redpajama-ggml/blob/main/RedPajama-INCITE-Chat-3B-v1-q4_0-ggjt.bin) | [togethercomputer/RedPajama-INCITE-Chat-3B-v1](https://huggingface.co/togethercomputer/RedPajama-INCITE-Chat-3B-v1) | Q4_0 | GGJT | V3 |
31
+ | [RedPajama-INCITE-Chat-3B-v1-q5_1-ggjt.bin](https://huggingface.co/rustformers/redpajama-ggml/blob/main/RedPajama-INCITE-Chat-3B-v1-q5_1-ggjt.bin) | [togethercomputer/RedPajama-INCITE-Chat-3B-v1](https://huggingface.co/togethercomputer/RedPajama-INCITE-Chat-3B-v1) | Q5_1 | GGJT | V3 |
32
+ | [RedPajama-INCITE-Instruct-3B-v1-f16.bin](https://huggingface.co/rustformers/redpajama-ggml/blob/main/RedPajama-INCITE-Instruct-3B-v1-f16.bin) | [togethercomputer/RedPajama-INCITE-Instruct-3B-v1](https://huggingface.co/togethercomputer/RedPajama-INCITE-Instruct-3B-v1) | F16 | GGML | V3 |
33
+ | [RedPajama-INCITE-Instruct-3B-v1-q4_0.bin](https://huggingface.co/rustformers/redpajama-ggml/blob/main/RedPajama-INCITE-Instruct-3B-v1-q4_0.bin) | [togethercomputer/RedPajama-INCITE-Instruct-3B-v1](https://huggingface.co/togethercomputer/RedPajama-INCITE-Instruct-3B-v1) | Q4_0 | GGML | V3 |
34
+ | [RedPajama-INCITE-Instruct-3B-v1-q4_0-ggjt.bin](https://huggingface.co/rustformers/redpajama-ggml/blob/main/RedPajama-INCITE-Instruct-3B-v1-q4_0-ggjt.bin) | [togethercomputer/RedPajama-INCITE-Instruct-3B-v1](https://huggingface.co/togethercomputer/RedPajama-INCITE-Instruct-3B-v1) | Q4_0 | GGJT | V3 |
35
+ | [RedPajama-INCITE-Instruct-3B-v1-q5_1-ggjt.bin](https://huggingface.co/rustformers/redpajama-ggml/blob/main/RedPajama-INCITE-Instruct-3B-v1-q5_1-ggjt.bin) | [togethercomputer/RedPajama-INCITE-Instruct-3B-v1](https://huggingface.co/togethercomputer/RedPajama-INCITE-Instruct-3B-v1) | Q5_1 | GGJT | V3 |
36
 
37
  ## Usage
38
 
 
46
  from llm_rs import AutoModel
47
 
48
  #Load the model, define any model you like from the list above as the `model_file`
49
+ model = AutoModel.from_pretrained("rustformers/redpajama-ggml",model_file="RedPajama-INCITE-Base-3B-v1-q4_0-ggjt.bin")
50
 
51
  #Generate
52
  print(model.generate("The meaning of life is"))
 
56
 
57
  #### Installation
58
  ```
59
+ git clone --recurse-submodules https://github.com/rustformers/llm.git
60
+ cd llm
61
  cargo build --release
62
  ```
63
 
64
  #### Run inference
65
  ```
66
  cargo run --release -- gptneox infer -m path/to/model.bin -p "Tell me how cool the Rust programming language is:"
67
+ ```