File size: 1,628 Bytes
27d4230 ceebb49 27d4230 44c9243 |
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 |
---
library_name: transformers
license: mit
datasets:
- HuggingFaceH4/ultrafeedback_binarized
language:
- en
---
# Zephyr-7B-DICE-Iter2
This model was developed using [Bootstrapping Language Models with DPO Implicit Rewards](https://arxiv.org/abs/2406.09760) (DICE) at iteration 2, based on the [HuggingFaceH4/zephyr-7b-beta](https://huggingface.co/HuggingFaceH4/zephyr-7b-beta) as the starting point.
## Links to Other Models
- [Zephyr-7B-DICE-Iter1](https://huggingface.co/sail/Zephyr-7B-DICE-Iter1)
- [Zephyr-7B-DICE-Iter2](https://huggingface.co/sail/Zephyr-7B-DICE-Iter2)
## Model Description
- Model type: A 7B parameter GPT-like model fine-tuned on synthetic datasets.
- Language(s) (NLP): Primarily English
- License: MIT
- Fine-tuned from model: HuggingFaceH4/zephyr-7b-beta
## [AlpacaEval Leaderboard Evaluation Results](https://tatsu-lab.github.io/alpaca_eval/)
| Model | LC. Win Rate | Win Rate |
|-------------------------------------------|:------------:|:--------:|
|[Zephyr-7b-beta](https://huggingface.co/HuggingFaceH4/zephyr-7b-beta) |12.69 |10.71
|[Zephyr-7B-DICE-Iter1](https://huggingface.co/sail/Zephyr-7B-DICE-Iter1) |19.03 |17.67
|[Zephyr-7B-DICE-Iter2](https://huggingface.co/sail/Zephyr-7B-DICE-Iter2) |**20.71** |**20.16**
## Citation
```bibtex
@article{chen2024bootstrapping,
title={Bootstrapping Language Models with DPO Implicit Rewards},
author={Chen, Changyu and Liu, Zichen and Du, Chao and Pang, Tianyu and Liu, Qian and Sinha, Arunesh and Varakantham, Pradeep and Lin, Min},
journal={arXiv preprint arXiv:2406.09760},
year={2024}
}
``` |