|
---
|
|
library_name: transformers
|
|
datasets:
|
|
- wzhouad/zephyr-ultrafeedback-hybrid
|
|
---
|
|
|
|
## Description |
|
|
|
mistral-7b-sft-beta model finetuned by hybrid WPO (GPT-4-turbo + on-policy sampling + Ultrafeedback). Details in [WPO: Enhancing RLHF with Weighted Preference Optimization](https://arxiv.org/abs/2406.11827). The training data is [wzhouad/zephyr-ultrafeedback-hybrid](https://huggingface.co/datasets/wzhouad/zephyr-ultrafeedback-hybrid). |
|
|
|
## License |
|
This model is licensed under the Zoom software license and is permitted for use only for noncommercial, educational, or academic research purposes. |
