File size: 8,958 Bytes
5dd3c8f
8cf2540
 
 
 
 
167fa29
 
 
 
 
 
 
 
 
b522390
167fa29
 
 
 
 
 
5dd3c8f
37978d4
5dd3c8f
167fa29
5dd3c8f
87b82d9
5dd3c8f
167fa29
5dd3c8f
85ca782
167fa29
5dd3c8f
167fa29
 
5dd3c8f
37550b5
167fa29
37550b5
167fa29
 
 
5dd3c8f
37550b5
167fa29
 
 
 
 
5dd3c8f
167fa29
 
37978d4
 
167fa29
 
37550b5
5dd3c8f
37978d4
167fa29
5dd3c8f
37978d4
7441303
37550b5
7441303
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
37978d4
7441303
 
87b15b2
 
 
 
 
7441303
87b15b2
7441303
 
 
 
 
 
 
 
 
 
37550b5
7441303
37978d4
4a006f5
 
 
 
 
 
 
 
 
 
 
7441303
 
 
167fa29
5dd3c8f
167fa29
37978d4
167fa29
b522390
167fa29
 
87b82d9
167fa29
 
5dd3c8f
167fa29
5dd3c8f
b522390
167fa29
 
 
 
 
 
 
 
5dd3c8f
b522390
 
 
 
 
 
5dd3c8f
167fa29
5dd3c8f
167fa29
5d3a767
 
 
 
 
 
 
 
5dd3c8f
 
 
167fa29
5d3a767
 
 
 
 
 
 
 
 
 
 
 
 
5dd3c8f
 
167fa29
5dd3c8f
b522390
37978d4
 
5dd3c8f
167fa29
5dd3c8f
167fa29
5dd3c8f
167fa29
5dd3c8f
167fa29
5dd3c8f
b522390
 
 
 
167fa29
 
5dd3c8f
 
167fa29
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
---
language:
- en
license: other
tags:
- causal-lm
datasets:
- HuggingFaceH4/ultrachat_200k
- allenai/ultrafeedback_binarized_cleaned
- meta-math/MetaMathQA
- WizardLM/WizardLM_evol_instruct_V2_196k
- openchat/openchat_sharegpt4_dataset
- LDJnr/Capybara
- Intel/orca_dpo_pairs
- hkust-nlp/deita-10k-v0
- Anthropic/hh-rlhf
extra_gated_fields:
  Name: text
  Email: text
  Country: text
  Organization or Affiliation: text
  I ALLOW Stability AI to email me about new model releases: checkbox
---
# `StableLM 2 12B Chat`

## Model Description

`Stable LM 2 12B Chat` is a 12 billion parameter instruction tuned language model trained on a mix of publicly available datasets and synthetic datasets, utilizing [Direct Preference Optimization (DPO)](https://arxiv.org/abs/2305.18290).

## Usage

`StableLM 2 12B Chat` uses the following instruction ChatML format.
This format is also available through the tokenizer's `apply_chat_template` method:

```python
from transformers import AutoModelForCausalLM, AutoTokenizer

tokenizer = AutoTokenizer.from_pretrained('stabilityai/stablelm-2-12b-chat')
model = AutoModelForCausalLM.from_pretrained(
    'stabilityai/stablelm-2-12b-chat',
    device_map="auto",
    trust_remote_code=True,
)

prompt = [{'role': 'user', 'content': 'How to combine multiple rows of data into one row of data in Excel?'}]
inputs = tokenizer.apply_chat_template(
    prompt,
    add_generation_prompt=True,
    return_tensors='pt'
)

tokens = model.generate(
    inputs.to(model.device),
    max_new_tokens=100,
    temperature=0.7,
    do_sample=True
)
output = tokenizer.decode(tokens[:, inputs.shape[-1]:][0], skip_special_tokens=False)

print(output)
```

StableLM 2 12B Chat also supports function call usage this is an example how you can use it:
```python
system_prompt = """\
You are a helpful assistant with access to the following functions. You must use them if required -\n
[
  {
    "type": "function",
    "function": {
      "name": "TextToImage",
      "description": "This function able to creating, drawing, or illustrating an image from a text prompt.",
      "parameters": {
        "type": "object",
        "properties": {
          "prompt": {
            "type": "string",
            "description": "The description of image that user wanto to create."
          }
        },
        "required": [
          "prompt"
        ]
      }
    }
  }
]
"""
messages = [
    {'role': 'system', 'content': system_prompt},
    {'role': "user", 'content': "Help me to generate a picture of Eiffel Tower in the night!"}
]

inputs = tokenizer.apply_chat_template(
    messages,
    add_generation_prompt=True,
    return_tensors='pt'
)

tokens = model.generate(
    inputs.to(model.device),
    max_new_tokens=1024,
    temperature=0.5,
    do_sample=True
)
output = tokenizer.decode(tokens[:, inputs.shape[-1]:][0], skip_special_tokens=True)

print(output)
"""
[
  {
    "name": "TextToImage",
    "arguments": {
      "prompt": "Eiffel Tower in the night"
    }
  }
]
"""

```


## Model Details

* **Developed by**: [Stability AI](https://stability.ai/)
* **Model type**: `StableLM 2 12B Chat` model is an auto-regressive language model based on the transformer decoder architecture.
* **Language(s)**: English
TODO: Check if we want to keep paper link since it's not mentioned in that paper.
* **Paper**: [Stable LM 2 Chat Technical Report](https://drive.google.com/file/d/1JYJHszhS8EFChTbNAf8xmqhKjogWRrQF/view?usp=sharing)
* **Library**: [Alignment Handbook](https://github.com/huggingface/alignment-handbook.git)
* **Finetuned from model**: 
* **License**: [StabilityAI Non-Commercial Research Community License](https://huggingface.co/stabilityai/stablelm-2-zephyr-1_6b/blob/main/LICENSE). If you want to use this model for your commercial products or purposes, please contact us [here](https://stability.ai/contact) to learn more.
* **Contact**: For questions and comments about the model, please email `lm@stability.ai`

### Training Dataset

The dataset is comprised of a mixture of open datasets large-scale datasets available on the [HuggingFace Hub](https://huggingface.co/datasets) as well as an internal safety dataset:
1. SFT Datasets
- HuggingFaceH4/ultrachat_200k
- meta-math/MetaMathQA
- WizardLM/WizardLM_evol_instruct_V2_196k
- Open-Orca/SlimOrca
- openchat/openchat_sharegpt4_dataset
- LDJnr/Capybara
- hkust-nlp/deita-10k-v0

2. Safety Datasets:
- Anthropic/hh-rlhf
- Internal Safety Dataset

3. Preference Datasets:


## Performance

### MT-Bench
| Model                                 | Parameters | MT Bench (Inflection-corrected) |
|---------------------------------------|------------|---------------------------------|
| mistralai/Mixtral-8x7B-Instruct-v0.1 | 13B/47B    | 8.48 ± 0.06                    |
| stabilityai/stablelm-2-12b-chat       | 12B        | 8.15 ± 0.08                    |
| Qwen/Qwen1.5-14B-Chat                 | 14B        | 7.95 ± 0.10                    |
| HuggingFaceH4/zephyr-7b-gemma-v0.1    | 8.5B       | 7.82 ± 0.03                    |
| mistralai/Mistral-7B-Instruct-v0.2    | 7B         | 7.48 ± 0.02                    |
| meta-llama/Llama-2-70b-chat-hf        | 70B        | 7.29 ± 0.05                    |



### OpenLLM Leaderboard
| Model                                  | Parameters | Average | ARC Challenge (25-shot) | HellaSwag (10-shot) | MMLU (5-shot) | TruthfulQA (0-shot) | Winogrande (5-shot) | GSM8K (5-shot) |
| -------------------------------------- | ---------- | ------- | ---------------------- | ------------------- | ------------- | ------------------- | ------------------- | -------------- |
| mistralai/Mixtral-8x7B-Instruct-v0.1  | 13B/47B    | 72.71   | 70.14                  | 87.55               | 71.40         | 64.98               | 81.06               | 61.11          |
| stabilityai/stablelm-2-12b-chat        | 12B        | 68.45   | 65.02                  | 86.06               | 61.14         | 62.00               | 78.77               | 57.70          |
| Qwen/Qwen1.5-14B                       | 14B        | 66.70   | 56.57                  | 81.08               | 69.36         | 52.06               | 73.48               | 67.63          |
| mistralai/Mistral-7B-Instruct-v0.2     | 7B         | 65.71   | 63.14                  | 84.88               | 60.78         | 60.26               | 77.19               | 40.03          |
| HuggingFaceH4/zephyr-7b-gemma-v0.1     | 8.5B       | 62.41   | 58.45                  | 83.48               | 60.68         | 52.07               | 74.19               | 45.56          |
| Qwen/Qwen1.5-14B-Chat                  | 14B        | 62.37   | 58.79                  | 82.33               | 68.52         | 60.38               | 73.32               | 30.86          |
| google/gemma-7b                        | 8.5B       | 63.75   | 61.09                  | 82.20               | 64.56         | 44.79               | 79.01               | 50.87          |
| stabilityai/stablelm-2-12b             | 12B        | 63.53   | 58.45                  | 84.33               | 62.09         | 48.16               | 78.10               | 56.03          |
| mistralai/Mistral-7B-v0.1              | 7B         | 60.97   | 59.98                  | 83.31               | 64.16         | 42.15               | 78.37               | 37.83          |
| meta-llama/Llama-2-13b-hf              | 13B        | 55.69   | 59.39                  | 82.13               | 55.77         | 37.38               | 76.64               | 22.82          |
| meta-llama/Llama-2-13b-chat-hf         | 13B        | 54.92   | 59.04                  | 81.94               | 54.64         | 41.12               | 74.51               | 15.24          |


### Training Infrastructure

TODO: Fix this
* **Hardware**: `StableLM 2 12B Chat` was trained on the Stability AI cluster across 8 nodes with 8 A100 80GBs GPUs for each nodes.
* **Code Base**: We use our internal script for SFT training and [HuggingFace Alignment Handbook](https://github.com/huggingface/alignment-handbook) for DPO training.

## Use and Limitations

### Intended Use

The model is intended to be used in chat-like applications. Developers must evaluate the model for safety performance in their specific use case. Read more about [safety and limitations](#limitations-and-bias) below.

### Limitations and Bias

TODO: Do we need or have a standard template to throw in here now?

We strongly recommend pairing this model with an input and output classifier to prevent harmful responses.
Using this model will require guardrails around your inputs and outputs to ensure that any outputs returned are not hallucinations.
Additionally, as each use case is unique, we recommend running your own suite of tests to ensure proper performance of this model.
Finally, do not use the models if they are unsuitable for your application, or for any applications that may cause deliberate or unintentional harm to others.


## How to Cite