File size: 6,925 Bytes
6ec78d1 dccd36b 6ec78d1 56317e1 ef3021e dccd36b ef3021e dccd36b ef3021e dccd36b 0e0aeaf dccd36b 0e0aeaf |
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 |
---
tags:
- llm-rs
- ggml
pipeline_tag: text-generation
license: apache-2.0
language:
- en
---
# GGML converted versions of [Together](https://huggingface.co/togethercomputer)'s RedPajama models
## Description
RedPajama-INCITE-Base-3B-v1 was developed by Together and leaders from the open-source AI community including Ontocord.ai, ETH DS3Lab, AAI CERC, Université de Montréal, MILA - Québec AI Institute, Stanford Center for Research on Foundation Models (CRFM), Stanford Hazy Research research group and LAION.
The training was done on 3,072 V100 GPUs provided as part of the INCITE 2023 project on Scalable Foundation Models for Transferrable Generalist AI, awarded to MILA, LAION, and EleutherAI in fall 2022, with support from the Oak Ridge Leadership Computing Facility (OLCF) and INCITE program.
- Base Model: [RedPajama-INCITE-Base-3B-v1](https://huggingface.co/togethercomputer/RedPajama-INCITE-Base-3B-v1)
- Instruction-tuned Version: [RedPajama-INCITE-Instruct-3B-v1](https://huggingface.co/togethercomputer/RedPajama-INCITE-Instruct-3B-v1)
- Chat Version: [RedPajama-INCITE-Chat-3B-v1](https://huggingface.co/togethercomputer/RedPajama-INCITE-Chat-3B-v1)
## Converted Models:
| Name | Based on | Type | Container | GGML Version |
|:--------------------------------------------------------------------------------------------------------------------------------------------------------------|:----------------------------------------------------------------------------------------------------------------------------|:-------|:------------|:---------------|
| [RedPajama-INCITE-Base-3B-v1-f16.bin](https://huggingface.co/rustformers/redpajama-3b-ggml/blob/main/RedPajama-INCITE-Base-3B-v1-f16.bin) | [togethercomputer/RedPajama-INCITE-Base-3B-v1](https://huggingface.co/togethercomputer/RedPajama-INCITE-Base-3B-v1) | F16 | GGML | V3 |
| [RedPajama-INCITE-Base-3B-v1-q4_0.bin](https://huggingface.co/rustformers/redpajama-3b-ggml/blob/main/RedPajama-INCITE-Base-3B-v1-q4_0.bin) | [togethercomputer/RedPajama-INCITE-Base-3B-v1](https://huggingface.co/togethercomputer/RedPajama-INCITE-Base-3B-v1) | Q4_0 | GGML | V3 |
| [RedPajama-INCITE-Base-3B-v1-q4_0-ggjt.bin](https://huggingface.co/rustformers/redpajama-3b-ggml/blob/main/RedPajama-INCITE-Base-3B-v1-q4_0-ggjt.bin) | [togethercomputer/RedPajama-INCITE-Base-3B-v1](https://huggingface.co/togethercomputer/RedPajama-INCITE-Base-3B-v1) | Q4_0 | GGJT | V3 |
| [RedPajama-INCITE-Base-3B-v1-q5_1-ggjt.bin](https://huggingface.co/rustformers/redpajama-3b-ggml/blob/main/RedPajama-INCITE-Base-3B-v1-q5_1-ggjt.bin) | [togethercomputer/RedPajama-INCITE-Base-3B-v1](https://huggingface.co/togethercomputer/RedPajama-INCITE-Base-3B-v1) | Q5_1 | GGJT | V3 |
| [RedPajama-INCITE-Chat-3B-v1-f16.bin](https://huggingface.co/rustformers/redpajama-3b-ggml/blob/main/RedPajama-INCITE-Chat-3B-v1-f16.bin) | [togethercomputer/RedPajama-INCITE-Chat-3B-v1](https://huggingface.co/togethercomputer/RedPajama-INCITE-Chat-3B-v1) | F16 | GGML | V3 |
| [RedPajama-INCITE-Chat-3B-v1-q4_0.bin](https://huggingface.co/rustformers/redpajama-3b-ggml/blob/main/RedPajama-INCITE-Chat-3B-v1-q4_0.bin) | [togethercomputer/RedPajama-INCITE-Chat-3B-v1](https://huggingface.co/togethercomputer/RedPajama-INCITE-Chat-3B-v1) | Q4_0 | GGML | V3 |
| [RedPajama-INCITE-Chat-3B-v1-q4_0-ggjt.bin](https://huggingface.co/rustformers/redpajama-3b-ggml/blob/main/RedPajama-INCITE-Chat-3B-v1-q4_0-ggjt.bin) | [togethercomputer/RedPajama-INCITE-Chat-3B-v1](https://huggingface.co/togethercomputer/RedPajama-INCITE-Chat-3B-v1) | Q4_0 | GGJT | V3 |
| [RedPajama-INCITE-Chat-3B-v1-q5_1-ggjt.bin](https://huggingface.co/rustformers/redpajama-3b-ggml/blob/main/RedPajama-INCITE-Chat-3B-v1-q5_1-ggjt.bin) | [togethercomputer/RedPajama-INCITE-Chat-3B-v1](https://huggingface.co/togethercomputer/RedPajama-INCITE-Chat-3B-v1) | Q5_1 | GGJT | V3 |
| [RedPajama-INCITE-Instruct-3B-v1-f16.bin](https://huggingface.co/rustformers/redpajama-3b-ggml/blob/main/RedPajama-INCITE-Instruct-3B-v1-f16.bin) | [togethercomputer/RedPajama-INCITE-Instruct-3B-v1](https://huggingface.co/togethercomputer/RedPajama-INCITE-Instruct-3B-v1) | F16 | GGML | V3 |
| [RedPajama-INCITE-Instruct-3B-v1-q4_0.bin](https://huggingface.co/rustformers/redpajama-3b-ggml/blob/main/RedPajama-INCITE-Instruct-3B-v1-q4_0.bin) | [togethercomputer/RedPajama-INCITE-Instruct-3B-v1](https://huggingface.co/togethercomputer/RedPajama-INCITE-Instruct-3B-v1) | Q4_0 | GGML | V3 |
| [RedPajama-INCITE-Instruct-3B-v1-q4_0-ggjt.bin](https://huggingface.co/rustformers/redpajama-3b-ggml/blob/main/RedPajama-INCITE-Instruct-3B-v1-q4_0-ggjt.bin) | [togethercomputer/RedPajama-INCITE-Instruct-3B-v1](https://huggingface.co/togethercomputer/RedPajama-INCITE-Instruct-3B-v1) | Q4_0 | GGJT | V3 |
| [RedPajama-INCITE-Instruct-3B-v1-q5_1-ggjt.bin](https://huggingface.co/rustformers/redpajama-3b-ggml/blob/main/RedPajama-INCITE-Instruct-3B-v1-q5_1-ggjt.bin) | [togethercomputer/RedPajama-INCITE-Instruct-3B-v1](https://huggingface.co/togethercomputer/RedPajama-INCITE-Instruct-3B-v1) | Q5_1 | GGJT | V3 |
## Usage
### Python via [llm-rs](https://github.com/LLukas22/llm-rs-python):
#### Installation
Via pip: `pip install llm-rs`
#### Run inference
```python
from llm_rs import AutoModel
#Load the model, define any model you like from the list above as the `model_file`
model = AutoModel.from_pretrained("rustformers/redpajama-3b-ggml",model_file="RedPajama-INCITE-Base-3B-v1-q4_0-ggjt.bin")
#Generate
print(model.generate("The meaning of life is"))
```
### Using [local.ai](https://github.com/louisgv/local.ai) GUI
#### Installation
Download the installer at [www.localai.app](https://www.localai.app/).
#### Running Inference
Download your preferred model and place it in the "models" directory. Subsequently, you can start a chat session with your model directly from the interface.
### Rust via [Rustformers/llm](https://github.com/rustformers/llm):
#### Installation
```
git clone --recurse-submodules https://github.com/rustformers/llm.git
cd llm
cargo build --release
```
#### Run inference
```
cargo run --release -- gptneox infer -m path/to/model.bin -p "Tell me how cool the Rust programming language is:"
``` |