argilla/ultrafeedback-binarized-preferences-cleaned-kto
Viewer
•
Updated
•
4
Preference Datasets for KTO
This collection contains a list of curated preference datasets for KTO fine-tuning for intent alignment of LLMs through signals.
Note KTO transformed version of "argilla/ultrafeedback-binarized-preferences-cleaned".
argilla/distilabel-intel-orca-kto
Viewer • Updated • 8 • 4Note KTO transformed version of "argilla/distilabel-intel-orca-dpo-pairs"
argilla/distilabel-capybara-kto-15k-binarized
Viewer • Updated • 116 • 4Note KTO transformed version of "argilla/distilabel-capybara-dpo-7k-binarized".
argilla/kto-mix-15k
Viewer • Updated • 10 • 8Note KTO transformed version of "argilla/dpo-mix-7k".
-
KTO: Model Alignment as Prospect Theoretic Optimization
Paper • 2402.01306 • Published • 11