Hugging Face's logo

rustformers
/
redpajama-3b-ggml

Text Generation
Transformers
English
llm-rs
ggml
Inference Endpoints
Model card Files Community
LLukas22 commited on
Commit
ef3021e
·
1 Parent(s): d4f0ef4

Generated README.md

Browse files
Files changed (1) hide show
  1. README.md +22 -15
README.md CHANGED
@@ -19,20 +19,20 @@ The training was done on 3,072 V100 GPUs provided as part of the INCITE 2023 pro
19
 
20
  ## Converted Models:
21
 
22
- | Name | Based on | Type | Container | GGML Version |
23
- |:-----------------------------------------------------------------------------------------------------------------------------------------------------------|:----------------------------------------------------------------------------------------------------------------------------|:-------|:------------|:---------------|
24
- | [RedPajama-INCITE-Base-3B-v1-f16.bin](https://huggingface.co/rustformers/redpajama-ggml/blob/main/RedPajama-INCITE-Base-3B-v1-f16.bin) | [togethercomputer/RedPajama-INCITE-Base-3B-v1](https://huggingface.co/togethercomputer/RedPajama-INCITE-Base-3B-v1) | F16 | GGML | V3 |
25
- | [RedPajama-INCITE-Base-3B-v1-q4_0.bin](https://huggingface.co/rustformers/redpajama-ggml/blob/main/RedPajama-INCITE-Base-3B-v1-q4_0.bin) | [togethercomputer/RedPajama-INCITE-Base-3B-v1](https://huggingface.co/togethercomputer/RedPajama-INCITE-Base-3B-v1) | Q4_0 | GGML | V3 |
26
- | [RedPajama-INCITE-Base-3B-v1-q4_0-ggjt.bin](https://huggingface.co/rustformers/redpajama-ggml/blob/main/RedPajama-INCITE-Base-3B-v1-q4_0-ggjt.bin) | [togethercomputer/RedPajama-INCITE-Base-3B-v1](https://huggingface.co/togethercomputer/RedPajama-INCITE-Base-3B-v1) | Q4_0 | GGJT | V3 |
27
- | [RedPajama-INCITE-Base-3B-v1-q5_1-ggjt.bin](https://huggingface.co/rustformers/redpajama-ggml/blob/main/RedPajama-INCITE-Base-3B-v1-q5_1-ggjt.bin) | [togethercomputer/RedPajama-INCITE-Base-3B-v1](https://huggingface.co/togethercomputer/RedPajama-INCITE-Base-3B-v1) | Q5_1 | GGJT | V3 |
28
- | [RedPajama-INCITE-Chat-3B-v1-f16.bin](https://huggingface.co/rustformers/redpajama-ggml/blob/main/RedPajama-INCITE-Chat-3B-v1-f16.bin) | [togethercomputer/RedPajama-INCITE-Chat-3B-v1](https://huggingface.co/togethercomputer/RedPajama-INCITE-Chat-3B-v1) | F16 | GGML | V3 |
29
- | [RedPajama-INCITE-Chat-3B-v1-q4_0.bin](https://huggingface.co/rustformers/redpajama-ggml/blob/main/RedPajama-INCITE-Chat-3B-v1-q4_0.bin) | [togethercomputer/RedPajama-INCITE-Chat-3B-v1](https://huggingface.co/togethercomputer/RedPajama-INCITE-Chat-3B-v1) | Q4_0 | GGML | V3 |
30
- | [RedPajama-INCITE-Chat-3B-v1-q4_0-ggjt.bin](https://huggingface.co/rustformers/redpajama-ggml/blob/main/RedPajama-INCITE-Chat-3B-v1-q4_0-ggjt.bin) | [togethercomputer/RedPajama-INCITE-Chat-3B-v1](https://huggingface.co/togethercomputer/RedPajama-INCITE-Chat-3B-v1) | Q4_0 | GGJT | V3 |
31
- | [RedPajama-INCITE-Chat-3B-v1-q5_1-ggjt.bin](https://huggingface.co/rustformers/redpajama-ggml/blob/main/RedPajama-INCITE-Chat-3B-v1-q5_1-ggjt.bin) | [togethercomputer/RedPajama-INCITE-Chat-3B-v1](https://huggingface.co/togethercomputer/RedPajama-INCITE-Chat-3B-v1) | Q5_1 | GGJT | V3 |
32
- | [RedPajama-INCITE-Instruct-3B-v1-f16.bin](https://huggingface.co/rustformers/redpajama-ggml/blob/main/RedPajama-INCITE-Instruct-3B-v1-f16.bin) | [togethercomputer/RedPajama-INCITE-Instruct-3B-v1](https://huggingface.co/togethercomputer/RedPajama-INCITE-Instruct-3B-v1) | F16 | GGML | V3 |
33
- | [RedPajama-INCITE-Instruct-3B-v1-q4_0.bin](https://huggingface.co/rustformers/redpajama-ggml/blob/main/RedPajama-INCITE-Instruct-3B-v1-q4_0.bin) | [togethercomputer/RedPajama-INCITE-Instruct-3B-v1](https://huggingface.co/togethercomputer/RedPajama-INCITE-Instruct-3B-v1) | Q4_0 | GGML | V3 |
34
- | [RedPajama-INCITE-Instruct-3B-v1-q4_0-ggjt.bin](https://huggingface.co/rustformers/redpajama-ggml/blob/main/RedPajama-INCITE-Instruct-3B-v1-q4_0-ggjt.bin) | [togethercomputer/RedPajama-INCITE-Instruct-3B-v1](https://huggingface.co/togethercomputer/RedPajama-INCITE-Instruct-3B-v1) | Q4_0 | GGJT | V3 |
35
- | [RedPajama-INCITE-Instruct-3B-v1-q5_1-ggjt.bin](https://huggingface.co/rustformers/redpajama-ggml/blob/main/RedPajama-INCITE-Instruct-3B-v1-q5_1-ggjt.bin) | [togethercomputer/RedPajama-INCITE-Instruct-3B-v1](https://huggingface.co/togethercomputer/RedPajama-INCITE-Instruct-3B-v1) | Q5_1 | GGJT | V3 |
36
 
37
  ## Usage
38
 
@@ -46,11 +46,18 @@ Via pip: `pip install llm-rs`
46
  from llm_rs import AutoModel
47
 
48
  #Load the model, define any model you like from the list above as the `model_file`
49
- model = AutoModel.from_pretrained("rustformers/redpajama-ggml",model_file="RedPajama-INCITE-Base-3B-v1-q4_0-ggjt.bin")
50
 
51
  #Generate
52
  print(model.generate("The meaning of life is"))
53
  ```
 
 
 
 
 
 
 
54
 
55
  ### Rust via [Rustformers/llm](https://github.com/rustformers/llm):
56
 
 
19
 
20
  ## Converted Models:
21
 
22
+ | Name | Based on | Type | Container | GGML Version |
23
+ |:--------------------------------------------------------------------------------------------------------------------------------------------------------------|:----------------------------------------------------------------------------------------------------------------------------|:-------|:------------|:---------------|
24
+ | [RedPajama-INCITE-Base-3B-v1-f16.bin](https://huggingface.co/rustformers/redpajama-3b-ggml/blob/main/RedPajama-INCITE-Base-3B-v1-f16.bin) | [togethercomputer/RedPajama-INCITE-Base-3B-v1](https://huggingface.co/togethercomputer/RedPajama-INCITE-Base-3B-v1) | F16 | GGML | V3 |
25
+ | [RedPajama-INCITE-Base-3B-v1-q4_0.bin](https://huggingface.co/rustformers/redpajama-3b-ggml/blob/main/RedPajama-INCITE-Base-3B-v1-q4_0.bin) | [togethercomputer/RedPajama-INCITE-Base-3B-v1](https://huggingface.co/togethercomputer/RedPajama-INCITE-Base-3B-v1) | Q4_0 | GGML | V3 |
26
+ | [RedPajama-INCITE-Base-3B-v1-q4_0-ggjt.bin](https://huggingface.co/rustformers/redpajama-3b-ggml/blob/main/RedPajama-INCITE-Base-3B-v1-q4_0-ggjt.bin) | [togethercomputer/RedPajama-INCITE-Base-3B-v1](https://huggingface.co/togethercomputer/RedPajama-INCITE-Base-3B-v1) | Q4_0 | GGJT | V3 |
27
+ | [RedPajama-INCITE-Base-3B-v1-q5_1-ggjt.bin](https://huggingface.co/rustformers/redpajama-3b-ggml/blob/main/RedPajama-INCITE-Base-3B-v1-q5_1-ggjt.bin) | [togethercomputer/RedPajama-INCITE-Base-3B-v1](https://huggingface.co/togethercomputer/RedPajama-INCITE-Base-3B-v1) | Q5_1 | GGJT | V3 |
28
+ | [RedPajama-INCITE-Chat-3B-v1-f16.bin](https://huggingface.co/rustformers/redpajama-3b-ggml/blob/main/RedPajama-INCITE-Chat-3B-v1-f16.bin) | [togethercomputer/RedPajama-INCITE-Chat-3B-v1](https://huggingface.co/togethercomputer/RedPajama-INCITE-Chat-3B-v1) | F16 | GGML | V3 |
29
+ | [RedPajama-INCITE-Chat-3B-v1-q4_0.bin](https://huggingface.co/rustformers/redpajama-3b-ggml/blob/main/RedPajama-INCITE-Chat-3B-v1-q4_0.bin) | [togethercomputer/RedPajama-INCITE-Chat-3B-v1](https://huggingface.co/togethercomputer/RedPajama-INCITE-Chat-3B-v1) | Q4_0 | GGML | V3 |
30
+ | [RedPajama-INCITE-Chat-3B-v1-q4_0-ggjt.bin](https://huggingface.co/rustformers/redpajama-3b-ggml/blob/main/RedPajama-INCITE-Chat-3B-v1-q4_0-ggjt.bin) | [togethercomputer/RedPajama-INCITE-Chat-3B-v1](https://huggingface.co/togethercomputer/RedPajama-INCITE-Chat-3B-v1) | Q4_0 | GGJT | V3 |
31
+ | [RedPajama-INCITE-Chat-3B-v1-q5_1-ggjt.bin](https://huggingface.co/rustformers/redpajama-3b-ggml/blob/main/RedPajama-INCITE-Chat-3B-v1-q5_1-ggjt.bin) | [togethercomputer/RedPajama-INCITE-Chat-3B-v1](https://huggingface.co/togethercomputer/RedPajama-INCITE-Chat-3B-v1) | Q5_1 | GGJT | V3 |
32
+ | [RedPajama-INCITE-Instruct-3B-v1-f16.bin](https://huggingface.co/rustformers/redpajama-3b-ggml/blob/main/RedPajama-INCITE-Instruct-3B-v1-f16.bin) | [togethercomputer/RedPajama-INCITE-Instruct-3B-v1](https://huggingface.co/togethercomputer/RedPajama-INCITE-Instruct-3B-v1) | F16 | GGML | V3 |
33
+ | [RedPajama-INCITE-Instruct-3B-v1-q4_0.bin](https://huggingface.co/rustformers/redpajama-3b-ggml/blob/main/RedPajama-INCITE-Instruct-3B-v1-q4_0.bin) | [togethercomputer/RedPajama-INCITE-Instruct-3B-v1](https://huggingface.co/togethercomputer/RedPajama-INCITE-Instruct-3B-v1) | Q4_0 | GGML | V3 |
34
+ | [RedPajama-INCITE-Instruct-3B-v1-q4_0-ggjt.bin](https://huggingface.co/rustformers/redpajama-3b-ggml/blob/main/RedPajama-INCITE-Instruct-3B-v1-q4_0-ggjt.bin) | [togethercomputer/RedPajama-INCITE-Instruct-3B-v1](https://huggingface.co/togethercomputer/RedPajama-INCITE-Instruct-3B-v1) | Q4_0 | GGJT | V3 |
35
+ | [RedPajama-INCITE-Instruct-3B-v1-q5_1-ggjt.bin](https://huggingface.co/rustformers/redpajama-3b-ggml/blob/main/RedPajama-INCITE-Instruct-3B-v1-q5_1-ggjt.bin) | [togethercomputer/RedPajama-INCITE-Instruct-3B-v1](https://huggingface.co/togethercomputer/RedPajama-INCITE-Instruct-3B-v1) | Q5_1 | GGJT | V3 |
36
 
37
  ## Usage
38
 
 
46
  from llm_rs import AutoModel
47
 
48
  #Load the model, define any model you like from the list above as the `model_file`
49
+ model = AutoModel.from_pretrained("rustformers/redpajama-3b-ggml",model_file="RedPajama-INCITE-Base-3B-v1-q4_0-ggjt.bin")
50
 
51
  #Generate
52
  print(model.generate("The meaning of life is"))
53
  ```
54
+ ### Using [local.ai](https://github.com/louisgv/local.ai) GUI
55
+
56
+ #### Installation
57
+ Download the installer at [www.localai.app](https://www.localai.app/).
58
+
59
+ #### Running Inference
60
+ Download your preferred model and place it in the "models" directory. Subsequently, you can start a chat session with your model directly from the interface.
61
 
62
  ### Rust via [Rustformers/llm](https://github.com/rustformers/llm):
63