phi-2-dpo-pairs / README.md
rhysjones's picture
Update README.md
421d32c
---
license: mit
---
This is a test DPO finetune of Microsoft phi-2
Two DPO datasets are used. Training was for 1 epoch as a qlora with rank 64.
- [Intel/orca_dpo_pairs](https://huggingface.co/datasets/Intel/orca_dpo_pairs)
- [argilla/ultrafeedback-binarized-preferences-cleaned](https://huggingface.co/datasets/argilla/ultrafeedback-binarized-preferences-cleaned)
# Initial Evals
- ARC: 63.14
- TruthfulQA: 48.47