metadata
license: mit
Phi-2 Orange
A two-step finetune of Phi-2.
First using a collection of broad training data:
- Open-Orca/SlimOrca-Dedup
- migtissera/Synthia-v1.3
- LDJnr/Verified-Camel
- LDJnr/Pure-Dove
- LDJnr/Capybara
- meta-math/MetaMathQA
And then a DPO finetune using:
Initial Evals
- ARC: 62.29
- TruthfulQA: 49.85