---
  license: cc-by-nc-4.0
---



 * [This is DPO improved version of cloudyu/Mixtral_7Bx4_MOE_24B](https://huggingface.co/cloudyu/Mixtral_7Bx4_MOE_24B)

 * [DPO Trainer](https://huggingface.co/docs/trl/main/en/dpo_trainer)

* Metrics improved by DPO 
![Metrsc improment](dpo.jpg)
![Metrsc improment](dpo-metrics.jpg)