Commit
•
63bc8ee
1
Parent(s):
8bf7bff
Update README.md
Browse files
README.md
CHANGED
|
@@ -37,6 +37,28 @@ Zephyr is a series of language models that are trained to act as helpful assista
|
|
| 37 |
- **Repository:** https://github.com/huggingface/alignment-handbook
|
| 38 |
- **Demo:** https://huggingface.co/spaces/HuggingFaceH4/zephyr-chat
|
| 39 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 40 |
## Intended uses & limitations
|
| 41 |
|
| 42 |
The model was initially fine-tuned on a filtered and preprocessed of the [`UltraChat`](https://huggingface.co/datasets/stingning/ultrachat) dataset, which contains a diverse range of synthetic dialogues generated by ChatGPT.
|
|
|
|
| 37 |
- **Repository:** https://github.com/huggingface/alignment-handbook
|
| 38 |
- **Demo:** https://huggingface.co/spaces/HuggingFaceH4/zephyr-chat
|
| 39 |
|
| 40 |
+
## Performance
|
| 41 |
+
|
| 42 |
+
|
| 43 |
+
|
| 44 |
+
| Model | Size | Align | MT-Bench (score) | AlpacaEval (win %) |
|
| 45 |
+
|-------------|-----|----|---------------|--------------|
|
| 46 |
+
| StableLM-Tuned-α | 7B| dSFT |2.75| -|
|
| 47 |
+
| MPT-Chat | 7B |dSFT |5.42| -|
|
| 48 |
+
| Xwin-LMv0.1 | 7B| dPPO| 6.19| 87.83|
|
| 49 |
+
| Mistral-Instructv0.1 | 7B| - | 6.84 |-|
|
| 50 |
+
| Zephyr-7b-α |7B| dDPO| 6.88| -|
|
| 51 |
+
| **Zephyr-7b-β** |7B| dDPO| 7.34| 90.60|
|
| 52 |
+
| Falcon-Instruct | 40B |dSFT |5.17 |45.71|
|
| 53 |
+
| Guanaco 65B | SFT |6.41| 71.80|
|
| 54 |
+
| Llama2-Chat | 70B |RLHF |6.86| 92.66|
|
| 55 |
+
| Vicuna v1.3 | 33B |dSFT |7.12 |88.99|
|
| 56 |
+
| WizardLM v1.0 | 70B |dSFT |7.71 |-|
|
| 57 |
+
| Xwin-LM v0.1 | 70B |dPPO |- |95.57|
|
| 58 |
+
| GPT-3.5-turbo | - |RLHF |7.94 |89.37|
|
| 59 |
+
| Claude 2 | - |RLHF |8.06| 91.36|
|
| 60 |
+
| GPT-4 | -| RLHF |8.99| 95.28|
|
| 61 |
+
|
| 62 |
## Intended uses & limitations
|
| 63 |
|
| 64 |
The model was initially fine-tuned on a filtered and preprocessed of the [`UltraChat`](https://huggingface.co/datasets/stingning/ultrachat) dataset, which contains a diverse range of synthetic dialogues generated by ChatGPT.
|