rustformers
/

redpajama-3b-ggml

@@ -19,20 +19,20 @@ The training was done on 3,072 V100 GPUs provided as part of the INCITE 2023 pro
 ## Converted Models:
-| Name                                                                                                                                                       | Based on                                                                                                                    | Type   | Container   | GGML Version   |
-|:-----------------------------------------------------------------------------------------------------------------------------------------------------------|:----------------------------------------------------------------------------------------------------------------------------|:-------|:------------|:---------------|
-| [RedPajama-INCITE-Base-3B-v1-f16.bin](https://huggingface.co/rustformers/redpajama-ggml/blob/main/RedPajama-INCITE-Base-3B-v1-f16.bin)                     | [togethercomputer/RedPajama-INCITE-Base-3B-v1](https://huggingface.co/togethercomputer/RedPajama-INCITE-Base-3B-v1)         | F16    | GGML        | V3             |
-| [RedPajama-INCITE-Base-3B-v1-q4_0.bin](https://huggingface.co/rustformers/redpajama-ggml/blob/main/RedPajama-INCITE-Base-3B-v1-q4_0.bin)                   | [togethercomputer/RedPajama-INCITE-Base-3B-v1](https://huggingface.co/togethercomputer/RedPajama-INCITE-Base-3B-v1)         | Q4_0   | GGML        | V3             |
-| [RedPajama-INCITE-Base-3B-v1-q4_0-ggjt.bin](https://huggingface.co/rustformers/redpajama-ggml/blob/main/RedPajama-INCITE-Base-3B-v1-q4_0-ggjt.bin)         | [togethercomputer/RedPajama-INCITE-Base-3B-v1](https://huggingface.co/togethercomputer/RedPajama-INCITE-Base-3B-v1)         | Q4_0   | GGJT        | V3             |
-| [RedPajama-INCITE-Base-3B-v1-q5_1-ggjt.bin](https://huggingface.co/rustformers/redpajama-ggml/blob/main/RedPajama-INCITE-Base-3B-v1-q5_1-ggjt.bin)         | [togethercomputer/RedPajama-INCITE-Base-3B-v1](https://huggingface.co/togethercomputer/RedPajama-INCITE-Base-3B-v1)         | Q5_1   | GGJT        | V3             |
-| [RedPajama-INCITE-Chat-3B-v1-f16.bin](https://huggingface.co/rustformers/redpajama-ggml/blob/main/RedPajama-INCITE-Chat-3B-v1-f16.bin)                     | [togethercomputer/RedPajama-INCITE-Chat-3B-v1](https://huggingface.co/togethercomputer/RedPajama-INCITE-Chat-3B-v1)         | F16    | GGML        | V3             |
-| [RedPajama-INCITE-Chat-3B-v1-q4_0.bin](https://huggingface.co/rustformers/redpajama-ggml/blob/main/RedPajama-INCITE-Chat-3B-v1-q4_0.bin)                   | [togethercomputer/RedPajama-INCITE-Chat-3B-v1](https://huggingface.co/togethercomputer/RedPajama-INCITE-Chat-3B-v1)         | Q4_0   | GGML        | V3             |
-| [RedPajama-INCITE-Chat-3B-v1-q4_0-ggjt.bin](https://huggingface.co/rustformers/redpajama-ggml/blob/main/RedPajama-INCITE-Chat-3B-v1-q4_0-ggjt.bin)         | [togethercomputer/RedPajama-INCITE-Chat-3B-v1](https://huggingface.co/togethercomputer/RedPajama-INCITE-Chat-3B-v1)         | Q4_0   | GGJT        | V3             |
-| [RedPajama-INCITE-Chat-3B-v1-q5_1-ggjt.bin](https://huggingface.co/rustformers/redpajama-ggml/blob/main/RedPajama-INCITE-Chat-3B-v1-q5_1-ggjt.bin)         | [togethercomputer/RedPajama-INCITE-Chat-3B-v1](https://huggingface.co/togethercomputer/RedPajama-INCITE-Chat-3B-v1)         | Q5_1   | GGJT        | V3             |
-| [RedPajama-INCITE-Instruct-3B-v1-f16.bin](https://huggingface.co/rustformers/redpajama-ggml/blob/main/RedPajama-INCITE-Instruct-3B-v1-f16.bin)             | [togethercomputer/RedPajama-INCITE-Instruct-3B-v1](https://huggingface.co/togethercomputer/RedPajama-INCITE-Instruct-3B-v1) | F16    | GGML        | V3             |
-| [RedPajama-INCITE-Instruct-3B-v1-q4_0.bin](https://huggingface.co/rustformers/redpajama-ggml/blob/main/RedPajama-INCITE-Instruct-3B-v1-q4_0.bin)           | [togethercomputer/RedPajama-INCITE-Instruct-3B-v1](https://huggingface.co/togethercomputer/RedPajama-INCITE-Instruct-3B-v1) | Q4_0   | GGML        | V3             |
-| [RedPajama-INCITE-Instruct-3B-v1-q4_0-ggjt.bin](https://huggingface.co/rustformers/redpajama-ggml/blob/main/RedPajama-INCITE-Instruct-3B-v1-q4_0-ggjt.bin) | [togethercomputer/RedPajama-INCITE-Instruct-3B-v1](https://huggingface.co/togethercomputer/RedPajama-INCITE-Instruct-3B-v1) | Q4_0   | GGJT        | V3             |
-| [RedPajama-INCITE-Instruct-3B-v1-q5_1-ggjt.bin](https://huggingface.co/rustformers/redpajama-ggml/blob/main/RedPajama-INCITE-Instruct-3B-v1-q5_1-ggjt.bin) | [togethercomputer/RedPajama-INCITE-Instruct-3B-v1](https://huggingface.co/togethercomputer/RedPajama-INCITE-Instruct-3B-v1) | Q5_1   | GGJT        | V3             |
 ## Usage
@@ -46,11 +46,18 @@ Via pip: `pip install llm-rs`
 from llm_rs import AutoModel
 #Load the model, define any model you like from the list above as the `model_file`
-model = AutoModel.from_pretrained("rustformers/redpajama-ggml",model_file="RedPajama-INCITE-Base-3B-v1-q4_0-ggjt.bin")
 #Generate
 print(model.generate("The meaning of life is"))
 ```
 ### Rust via [Rustformers/llm](https://github.com/rustformers/llm):

 ## Converted Models:
+| Name                                                                                                                                                          | Based on                                                                                                                    | Type   | Container   | GGML Version   |
+|:--------------------------------------------------------------------------------------------------------------------------------------------------------------|:----------------------------------------------------------------------------------------------------------------------------|:-------|:------------|:---------------|
+| [RedPajama-INCITE-Base-3B-v1-f16.bin](https://huggingface.co/rustformers/redpajama-3b-ggml/blob/main/RedPajama-INCITE-Base-3B-v1-f16.bin)                     | [togethercomputer/RedPajama-INCITE-Base-3B-v1](https://huggingface.co/togethercomputer/RedPajama-INCITE-Base-3B-v1)         | F16    | GGML        | V3             |
+| [RedPajama-INCITE-Base-3B-v1-q4_0.bin](https://huggingface.co/rustformers/redpajama-3b-ggml/blob/main/RedPajama-INCITE-Base-3B-v1-q4_0.bin)                   | [togethercomputer/RedPajama-INCITE-Base-3B-v1](https://huggingface.co/togethercomputer/RedPajama-INCITE-Base-3B-v1)         | Q4_0   | GGML        | V3             |
+| [RedPajama-INCITE-Base-3B-v1-q4_0-ggjt.bin](https://huggingface.co/rustformers/redpajama-3b-ggml/blob/main/RedPajama-INCITE-Base-3B-v1-q4_0-ggjt.bin)         | [togethercomputer/RedPajama-INCITE-Base-3B-v1](https://huggingface.co/togethercomputer/RedPajama-INCITE-Base-3B-v1)         | Q4_0   | GGJT        | V3             |
+| [RedPajama-INCITE-Base-3B-v1-q5_1-ggjt.bin](https://huggingface.co/rustformers/redpajama-3b-ggml/blob/main/RedPajama-INCITE-Base-3B-v1-q5_1-ggjt.bin)         | [togethercomputer/RedPajama-INCITE-Base-3B-v1](https://huggingface.co/togethercomputer/RedPajama-INCITE-Base-3B-v1)         | Q5_1   | GGJT        | V3             |
+| [RedPajama-INCITE-Chat-3B-v1-f16.bin](https://huggingface.co/rustformers/redpajama-3b-ggml/blob/main/RedPajama-INCITE-Chat-3B-v1-f16.bin)                     | [togethercomputer/RedPajama-INCITE-Chat-3B-v1](https://huggingface.co/togethercomputer/RedPajama-INCITE-Chat-3B-v1)         | F16    | GGML        | V3             |
+| [RedPajama-INCITE-Chat-3B-v1-q4_0.bin](https://huggingface.co/rustformers/redpajama-3b-ggml/blob/main/RedPajama-INCITE-Chat-3B-v1-q4_0.bin)                   | [togethercomputer/RedPajama-INCITE-Chat-3B-v1](https://huggingface.co/togethercomputer/RedPajama-INCITE-Chat-3B-v1)         | Q4_0   | GGML        | V3             |
+| [RedPajama-INCITE-Chat-3B-v1-q4_0-ggjt.bin](https://huggingface.co/rustformers/redpajama-3b-ggml/blob/main/RedPajama-INCITE-Chat-3B-v1-q4_0-ggjt.bin)         | [togethercomputer/RedPajama-INCITE-Chat-3B-v1](https://huggingface.co/togethercomputer/RedPajama-INCITE-Chat-3B-v1)         | Q4_0   | GGJT        | V3             |
+| [RedPajama-INCITE-Chat-3B-v1-q5_1-ggjt.bin](https://huggingface.co/rustformers/redpajama-3b-ggml/blob/main/RedPajama-INCITE-Chat-3B-v1-q5_1-ggjt.bin)         | [togethercomputer/RedPajama-INCITE-Chat-3B-v1](https://huggingface.co/togethercomputer/RedPajama-INCITE-Chat-3B-v1)         | Q5_1   | GGJT        | V3             |
+| [RedPajama-INCITE-Instruct-3B-v1-f16.bin](https://huggingface.co/rustformers/redpajama-3b-ggml/blob/main/RedPajama-INCITE-Instruct-3B-v1-f16.bin)             | [togethercomputer/RedPajama-INCITE-Instruct-3B-v1](https://huggingface.co/togethercomputer/RedPajama-INCITE-Instruct-3B-v1) | F16    | GGML        | V3             |
+| [RedPajama-INCITE-Instruct-3B-v1-q4_0.bin](https://huggingface.co/rustformers/redpajama-3b-ggml/blob/main/RedPajama-INCITE-Instruct-3B-v1-q4_0.bin)           | [togethercomputer/RedPajama-INCITE-Instruct-3B-v1](https://huggingface.co/togethercomputer/RedPajama-INCITE-Instruct-3B-v1) | Q4_0   | GGML        | V3             |
+| [RedPajama-INCITE-Instruct-3B-v1-q4_0-ggjt.bin](https://huggingface.co/rustformers/redpajama-3b-ggml/blob/main/RedPajama-INCITE-Instruct-3B-v1-q4_0-ggjt.bin) | [togethercomputer/RedPajama-INCITE-Instruct-3B-v1](https://huggingface.co/togethercomputer/RedPajama-INCITE-Instruct-3B-v1) | Q4_0   | GGJT        | V3             |
+| [RedPajama-INCITE-Instruct-3B-v1-q5_1-ggjt.bin](https://huggingface.co/rustformers/redpajama-3b-ggml/blob/main/RedPajama-INCITE-Instruct-3B-v1-q5_1-ggjt.bin) | [togethercomputer/RedPajama-INCITE-Instruct-3B-v1](https://huggingface.co/togethercomputer/RedPajama-INCITE-Instruct-3B-v1) | Q5_1   | GGJT        | V3             |
 ## Usage
 from llm_rs import AutoModel
 #Load the model, define any model you like from the list above as the `model_file`
+model = AutoModel.from_pretrained("rustformers/redpajama-3b-ggml",model_file="RedPajama-INCITE-Base-3B-v1-q4_0-ggjt.bin")
 #Generate
 print(model.generate("The meaning of life is"))
 ```
+### Using [local.ai](https://github.com/louisgv/local.ai) GUI
+#### Installation
+Download the installer at [www.localai.app](https://www.localai.app/).
+#### Running Inference
+Download your preferred model and place it in the "models" directory. Subsequently, you can start a chat session with your model directly from the interface.
 ### Rust via [Rustformers/llm](https://github.com/rustformers/llm):