File size: 5,222 Bytes
6f22822 2d4e83e e3c738c 49942b2 e3c738c 6f22822 e3c738c a129834 e3c738c 70ddb55 e3c738c a1e7543 e3c738c ff0df0b e3c738c e17464f e3c738c e17464f 5da2abd e3c738c |
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170 171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 |
---
license: [llama2, other]
datasets:
- cerebras/SlimPajama-627B
- Open-Orca/OpenOrca
language:
- en
tags:
- Deci AI
- DeciLM
- Instruction
model-index:
- name: DeciLM 6B
results:
- task:
type: text-generation
dataset:
type: ai2/arc
name: ai2_arc
metrics:
- name: ARC Challenge
type: ARC Challenge
value: 43.43
verified: false
- task:
type: text-generation
dataset:
type: ai2/arc
name: ai2_arc
metrics:
- name: ARC Easy
type: ARC Easy
value: 70.58
verified: false
- task:
type: text-generation
dataset:
type: boolq
name: boolq
metrics:
- name: BoolQ
type: BoolQ
value: 77.34
verified: false
- task:
type: text-generation
dataset:
type: hellaswag
name: hellaswag
metrics:
- name: HellaSwag
type: HellaSwag
value: 74.57
verified: false
- task:
type: text-generation
dataset:
type: LAMBDA
name: OpenAI LAMBDA
metrics:
- name: LAMBDA
type: LAMBDA
value: 70.1
verified: false
- task:
type: text-generation
dataset:
type: OpenBookQA
name: openbookqa
metrics:
- name: OpenBookQA
type: OpenBookQA
value: 33
verified: false
- task:
type: text-generation
dataset:
type: PIQA
name: piqa
metrics:
- name: PIQA
type: PIQA
value: 77.52
verified: false
- task:
type: text-generation
dataset:
type: truthful_qa
name: truthful_qa
metrics:
- name: TruthfulQA
type: TruthfulQA
value: 43.89
verified: false
- task:
type: text-generation
dataset:
type: winogrande
name: winogrande
metrics:
- name: Winogrande
type: Winogrande
value: 67.64
verified: false
---
# DeciLM 6B-Instruct
DeciLM 6B-Instruct is a model for short-form instruction following. It is built by LoRA fine-tuning [DeciLM 6B](https://huggingface.co/Deci/DeciLM-6b) on a subset of the [OpenOrca dataset](https://huggingface.co/datasets/Open-Orca/OpenOrca).
- **Developed by:** Deci
- **Model type:** DeciLM is an auto-regressive language model using an optimized transformer decoder architecture that includes variable Grouped-Query Attention.
- **Language(s) (NLP):** English
- **License:** [Llama 2 Community License Agreement](https://huggingface.co/Deci/DeciLM-6b-instruct/blob/main/LICENSE.md) with an extention of Deci regarding hosting service providers.
### Model Sources
- **Paper:** [DeciLM 6B Technical Blog](https://deci.ai/blog/decilm-15-times-faster-than-llama2-nas-generated-llm-with-variable-gqa/)
- **Demo:** [DeciLM 6B-Instruct Demo](https://huggingface.co/spaces/Deci/DeciLM-6b-instruct)
- **Notebook:** [DeciLM 6B-Instruct Notebook](https://bit.ly/decilm-instruct-nb)
## Uses
The model is intended for commercial and research use in English and can be fine-tuned for use in other languages.
## How to Get Started with the Model
Use the code below to get started with the model.
```bibtex
# pip install -q transformers
import torch
from transformers import AutoModelForCausalLM, AutoTokenizer
checkpoint = "Deci/DeciLM-6b-instruct"
device = "cuda" # for GPU usage or "cpu" for CPU usage
tokenizer = AutoTokenizer.from_pretrained(checkpoint)
model = AutoModelForCausalLM.from_pretrained(checkpoint, torch_dtype=torch.bfloat16, trust_remote_code=True).to(device)
inputs = tokenizer.encode("How do I make french toast? Think through it step by step", return_tensors="pt").to(device)
outputs = model.generate(inputs, max_new_tokens=100, do_sample=True, top_p=0.95)
print(tokenizer.decode(outputs[0]))
```
## Training Details
DeciLM 6B underwent training utilizing the SlimPijamas dataset, leveraging advanced proprietary methodologies allowing for fast training. DeciLM 6B was further finetuned on a subset of the OpenOrca dataset, giving rise to DeciLM-6B-Instruct.
## Evaluation
Below are DeciLM's 6B-instruct evaluation results.
| Average | ARC Challenge* | ARC Easy* | BoolQ | HellaSwag* | LAMBDA OpenAI | OpenBookQA | PIQA | TruthfulQA | Winogrande |
|:----------|:----------|:----------|:----------|:----------|:----------|:----------|:----------|:----------|:----------|
| 62.01 | 44.43 | 70.58 | 77.34 | 74.57 | 70.1 | 33 | 77.52 |43.89 | 67.64 |
Accuracy-norm score*
## Runtime Benchmarks
|Inference Tool/Hardware | A10 (tokens/sec) |
|:----------|:----------|
| PyTorch | 652.49 |
| Infery LLM | 2,029.6 |
- Throughput (tokens/sec) - Measured with optimal batch - PyTorch BS 64, Infery LLM BS 128
- In order to replicate the results of the PyTorch benchmark, use this [code example](https://huggingface.co/Deci/DeciLM-6b-instruct/blob/main/hf_benchmark_example.py)
## Disclaimer
DeciLM 6B-Instruct has not been aligned for safety or trained using RLHF.
## How to Cite
Please cite this model using this format.
```bibtex
@misc{DeciFoundationModels,
title = {DeciLM 6B Instruct},
author = {DeciAI Research Team},
year = {2023}
url={[https://huggingface.co/Deci/DeciLM-6b-instruct](https://huggingface.co/Deci/DeciLM-6b-instruct)},
}
``` |