Join the conversation

Join the community of Machine Learners and AI enthusiasts.

Sign Up
dvilasueroΒ 
posted an update Jan 31
Post
πŸš€ The Open Source AI community needs more open datasets for improving Open LLMs. Excited to share our new open dataset for boosting chat models:

πŸŽ‰ Welcome Distilabel Capybara DPO, a multi-turn, high-quality preference dataset.

argilla/distilabel-capybara-dpo-7k-binarized

Why?
Best closed chat models are built on top of multi-turn dialogue preference data. The OSS community lacks these datasets. This dataset is the first in the series to close this gap.

Is this dataset useful?
To test this dataset, we've built our virtual launching partner:

πŸŽ‰ Welcome CapybaraHermes, a preference tuned OpenHermes with increased second turn capabilities on MTBench

argilla/CapybaraHermes-2.5-Mistral-7B

As usual, models are the least important to us. We like to focus on the data. Our mission is to build and share high-quality datasets, sharing our methods in the open so the community can improve upon them.

That's why, we took some time to describe the full methodology on the dataset card, check it out and give us feedback! Data and methods are never perfect!

Finally, this is just a preview version and would love to collaborate with you to add more benchmarking results, what hyperparams work for DPO'ing models, what mix of datasets, etc.

Expect some more datasets in the coming weeks. Let's build the best data for AI, together.

Very cool!