teknium/OpenHermes-2.5-Mistral-7B
Text Generation
•
Updated
•
39k
•
818
Comparing DPO with IPO and KTO
A collection of chat models to explore the differences between three alignment techniques: DPO, IPO, and KTO.
Note The chat model we optimized with DPO, IPO, and KTO.
Intel/orca_dpo_pairs
Viewer • Updated • 12.9k • 1.4k • 291Note The AI feedback dataset we used to fine-tune OpenHermes-2.5 with DPO, IPO, and KTO.
-
trl-lib/OpenHermes-2-Mistral-7B-ipo-beta-0.1-steps-200
Updated • 6 -
trl-lib/OpenHermes-2-Mistral-7B-ipo-beta-0.2-steps-200
Updated • 6 -
trl-lib/OpenHermes-2-Mistral-7B-ipo-beta-0.3-steps-200
Updated -
trl-lib/OpenHermes-2-Mistral-7B-ipo-beta-0.4-steps-200
Updated • 4 -
trl-lib/OpenHermes-2-Mistral-7B-ipo-beta-0.5-steps-200
Updated • 5 -
trl-lib/OpenHermes-2-Mistral-7B-ipo-beta-0.6-steps-200
Updated • 6 -
trl-lib/OpenHermes-2-Mistral-7B-ipo-beta-0.7-steps-200
Updated • 4 -
trl-lib/OpenHermes-2-Mistral-7B-ipo-beta-0.8-steps-200
Updated • 5 -
trl-lib/OpenHermes-2-Mistral-7B-ipo-beta-0.9-steps-200
Updated • 4 -
trl-lib/OpenHermes-2-Mistral-7B-kto-beta-0.1-steps-200
Updated • 4 -
trl-lib/OpenHermes-2-Mistral-7B-kto-beta-0.2-steps-200
Updated • 5 -
trl-lib/OpenHermes-2-Mistral-7B-kto-beta-0.3-steps-200
Updated • 6 -
trl-lib/OpenHermes-2-Mistral-7B-kto-beta-0.4-steps-200
Updated • 6 -
trl-lib/OpenHermes-2-Mistral-7B-kto-beta-0.5-steps-200
Updated • 4 -
trl-lib/OpenHermes-2-Mistral-7B-kto-beta-0.6-steps-200
Updated • 4 -
trl-lib/OpenHermes-2-Mistral-7B-kto-beta-0.7-steps-200
Updated • 7 -
trl-lib/OpenHermes-2-Mistral-7B-kto-beta-0.8-steps-200
Updated • 7 -
trl-lib/OpenHermes-2-Mistral-7B-kto-beta-0.9-steps-200
Updated • 6 -
trl-lib/OpenHermes-2-Mistral-7B-sigmoid-beta-0.1-steps-200
Updated • 4 -
trl-lib/OpenHermes-2-Mistral-7B-sigmoid-beta-0.2-steps-200
Updated • 4 -
trl-lib/OpenHermes-2-Mistral-7B-sigmoid-beta-0.3-steps-200
Updated • 6 -
trl-lib/OpenHermes-2-Mistral-7B-sigmoid-beta-0.4-steps-200
Updated • 4 -
trl-lib/OpenHermes-2-Mistral-7B-sigmoid-beta-0.5-steps-200
Updated • 4 -
trl-lib/OpenHermes-2-Mistral-7B-sigmoid-beta-0.6-steps-200
Updated • 4 -
trl-lib/OpenHermes-2-Mistral-7B-sigmoid-beta-0.7-steps-200
Updated • 4 -
trl-lib/OpenHermes-2-Mistral-7B-sigmoid-beta-0.8-steps-200
Updated • 4 -
trl-lib/OpenHermes-2-Mistral-7B-sigmoid-beta-0.9-steps-200
Updated • 6 -
trl-lib/OpenHermes-2-Mistral-7B-sigmoid-beta-0.1-steps-800
Updated • 5 -
trl-lib/OpenHermes-2-Mistral-7B-sigmoid-beta-0.5-steps-800
Updated • 5 -
trl-lib/OpenHermes-2-Mistral-7B-sigmoid-beta-0.2-steps-800
Updated • 4 -
trl-lib/OpenHermes-2-Mistral-7B-sigmoid-beta-0.3-steps-800
Updated • 4 -
trl-lib/OpenHermes-2-Mistral-7B-sigmoid-beta-0.4-steps-800
Updated • 4 -
trl-lib/OpenHermes-2-Mistral-7B-sigmoid-beta-0.6-steps-800
Updated • 4 -
trl-lib/OpenHermes-2-Mistral-7B-sigmoid-beta-0.7-steps-800
Updated • 4 -
trl-lib/OpenHermes-2-Mistral-7B-sigmoid-beta-0.8-steps-800
Updated • 7 -
trl-lib/OpenHermes-2-Mistral-7B-sigmoid-beta-0.9-steps-800
Updated • 7 -
trl-lib/OpenHermes-2-Mistral-7B-kto-beta-0.1-steps-800
Updated • 4 -
trl-lib/OpenHermes-2-Mistral-7B-kto-beta-0.2-steps-800
Updated • 5 -
trl-lib/OpenHermes-2-Mistral-7B-kto-beta-0.3-steps-800
Updated • 4 -
trl-lib/OpenHermes-2-Mistral-7B-kto-beta-0.4-steps-800
Updated • 4 -
trl-lib/OpenHermes-2-Mistral-7B-kto-beta-0.5-steps-800
Updated • 6 -
trl-lib/OpenHermes-2-Mistral-7B-kto-beta-0.6-steps-800
Updated • 5 -
trl-lib/OpenHermes-2-Mistral-7B-kto-beta-0.7-steps-800
Updated • 4 -
trl-lib/OpenHermes-2-Mistral-7B-kto-beta-0.8-steps-800
Updated • 4 -
trl-lib/OpenHermes-2-Mistral-7B-kto-beta-0.9-steps-800
Updated • 4 -
trl-lib/OpenHermes-2-Mistral-7B-ipo-beta-0.1-steps-800
Updated • 8 -
trl-lib/OpenHermes-2-Mistral-7B-ipo-beta-0.2-steps-800
Updated • 4 -
trl-lib/OpenHermes-2-Mistral-7B-ipo-beta-0.4-steps-800
Updated • 6 -
trl-lib/OpenHermes-2-Mistral-7B-ipo-beta-0.3-steps-800
Updated • 4 -
trl-lib/OpenHermes-2-Mistral-7B-ipo-beta-0.5-steps-800
Updated • 5 -
trl-lib/OpenHermes-2-Mistral-7B-ipo-beta-0.6-steps-800
Updated • 5 -
trl-lib/OpenHermes-2-Mistral-7B-ipo-beta-0.7-steps-800
Updated • 6 -
trl-lib/OpenHermes-2-Mistral-7B-ipo-beta-0.8-steps-800
Updated • 4 -
trl-lib/OpenHermes-2-Mistral-7B-ipo-beta-0.9-steps-800
Updated • 4