Update README.md
Browse files
README.md
CHANGED
|
@@ -124,6 +124,40 @@ tokens = model.generate(
|
|
| 124 |
output = tokenizer.batch_decode(tokens[:, inputs.input_ids.shape[-1]:], skip_special_tokens=False)[0]
|
| 125 |
```
|
| 126 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 127 |
|
| 128 |
## How to Cite
|
| 129 |
|
|
|
|
| 124 |
output = tokenizer.batch_decode(tokens[:, inputs.input_ids.shape[-1]:], skip_special_tokens=False)[0]
|
| 125 |
```
|
| 126 |
|
| 127 |
+
## Performance
|
| 128 |
+
### Multi-PL Benchmark:
|
| 129 |
+
| Model | Size | Avg | Python | C++ | JavaScript | Java | PHP | Rust |
|
| 130 |
+
|------------------------------|------|------|--------|------|------------|------|------|------|
|
| 131 |
+
| Codellama Instruct | 7B | 0.30 | 0.33 | 0.31 | 0.31 | 0.29 | 0.31 | 0.25 |
|
| 132 |
+
| Deepseek Instruct | 1.3B | 0.44 | 0.52 | **0.52** | 0.41 | **0.46** | 0.45 | 0.28 |
|
| 133 |
+
| Stable Code Instruct (SFT) | 3B | 0.44 | 0.55 | 0.45 | 0.42 | 0.42 | 0.44 | 0.32 |
|
| 134 |
+
| Stable Code Instruct (DPO) | 3B | **0.47** | **0.59** | 0.49 | **0.49** | 0.44 | **0.45** | **0.37** |
|
| 135 |
+
|
| 136 |
+
### MT-Bench Coding:
|
| 137 |
+
| Model | Size | Score |
|
| 138 |
+
|-----------------------------|------|-----------------|
|
| 139 |
+
| Stable Code Instruct (DPO) | 3B | 5.8 |
|
| 140 |
+
| Stable Code Instruct (SFT) | 3B | 5.5 |
|
| 141 |
+
| DeepSeek Coder | 1.3B | 4.6 |
|
| 142 |
+
| CodeLlama Instruct | 7B | 3.55 |
|
| 143 |
+
| DeepSeek Coder | 6.7B | 6.9 |
|
| 144 |
+
| StarChat2 | 15B | 5.7 |
|
| 145 |
+
|
| 146 |
+
|
| 147 |
+
|
| 148 |
+
## Model Details
|
| 149 |
+
|
| 150 |
+
* **Developed by**: [Stability AI](https://stability.ai/)
|
| 151 |
+
* **Model type**: `Stable Code Instruct 3B` model is an auto-regressive language model based on the transformer decoder architecture.
|
| 152 |
+
* **Language(s)**: English
|
| 153 |
+
* **Paper**: [Stable Code Technical Report](https://drive.google.com/file/d/1JYJHszhS8EFChTbNAf8xmqhKjogWRrQF/view?usp=sharing)
|
| 154 |
+
* **Library**: [Alignment Handbook](https://github.com/huggingface/alignment-handbook.git)
|
| 155 |
+
* **Finetuned from model**: [https://huggingface.co/stabilityai/stable-code-3b](https://huggingface.co/stabilityai/stable-code-3b)
|
| 156 |
+
* **License**: [StabilityAI Non-Commercial Research Community License](https://huggingface.co/stabilityai/stable-code-instruct-3b/blob/main/LICENSE). If you want to use this model for your commercial products or purposes, please contact us [here](https://stability.ai/contact) to learn more.
|
| 157 |
+
* **Contact**: For questions and comments about the model, please email `lm@stability.ai`
|
| 158 |
+
|
| 159 |
+
|
| 160 |
+
|
| 161 |
|
| 162 |
## How to Cite
|
| 163 |
|