Daniel Vila's picture

Daniel Vila PRO

dvilasuero

·

https://argilla.io

dvsrepo

AI & ML interests

RLHF, RLAIF, DPO, data, data, data

Articles

🦙⚗️ Using Llama3 and distilabel to build fine-tuning datasets

Data is better together

Organizations

dvilasuero's activity

upvoted a paper 4 days ago

RLHF Workflow: From Reward Modeling to Online RLHF

Paper • 2405.07863 • Published 5 days ago • 51

upvoted 2 articles 11 days ago

Article

Can we create pedagogically valuable multi-turn synthetic datasets from Cosmopedia?

By

•

11 days ago

• 6

Article

Data is better together

Mar 4

• 4

upvoted an article 15 days ago

Article

🧑‍⚖️ "Replacing Judges with Juries" using distilabel

By

•

15 days ago

• 14

upvoted an article 19 days ago

Article

⚗️ 🧑🏼‍🌾 Let's grow some Domain Specific Datasets together

By

•

19 days ago

• 25

upvoted a collection about 1 month ago

Zephyr ORPO

Models and datasets to align LLMs with Odds Ratio Preference Optimisation (ORPO). Recipes here: https://github.com/huggingface/alignment-handbook • 3 items • Updated Apr 12 • 14

upvoted a collection about 2 months ago

Creación de corpus en comunidad

Colección de esfuerzos colaborativos para crear corpus en español de calidad. Toda persona hispanohablante puede contribuir :) • 7 items • Updated 12 days ago • 5

upvoted a collection 2 months ago

DIBT Prompt collective SPIN

This collection contains resources related to the replication of SPIN with the dibt prompt collective dataset • 8 items • Updated Mar 12 • 7

upvoted 4 collections 4 months ago

Apple MLX-compatible 7B LLMs on the 🤗 Hub

This collection contains the model weights for 7B LLMs for Apple's MLX framework. Find more information at https://github.com/ml-explore/mlx • 8 items • Updated 11 days ago • 9

Datasets built with ⚗️ distilabel

This collection contains some datasets generated and/or labelled using https://github.com/argilla-io/distilabel • 5 items • Updated Apr 4 • 5

Preference Datasets for DPO

This collection contains a list of curated preference datasets for DPO fine-tuning for intent alignment of LLMs • 7 items • Updated Apr 4 • 19

Comparing DPO with IPO and KTO

A collection of chat models to explore the differences between three alignment techniques: DPO, IPO, and KTO. • 56 items • Updated Jan 9 • 31

upvoted a paper 5 months ago

Principled Instructions Are All You Need for Questioning LLaMA-1/2, GPT-3.5/4

Paper • 2312.16171 • Published Dec 26, 2023 • 30

upvoted 4 collections 5 months ago

Other Notus experiments

Contains some more experiments with Notus using different variants, different datasets, etc. • 2 items • Updated Dec 28, 2023 • 3

Datasets based on UltraFeedback

This collection contains some datasets created on top of UltraFeedback using Argilla for the dataset exploration and curation, sorted by release date. • 6 items • Updated Mar 19 • 10

Capybara

Un-aligned model for general use, leveraging Amplify-Instruct and novel quality curation techniques, made with a dataset of less than 20K examples. • 8 items • Updated Dec 3, 2023 • 22

Awesome feedback datasets

A curated list of datasets with human or AI feedback. Useful for training reward models or applying techniques like DPO. • 19 items • Updated Apr 12 • 53

upvoted a collection 6 months ago

Notus 7B v1

Notus 7B v1 models (DPO fine-tune of Zephyr SFT) and datasets used. More information at https://github.com/argilla-io/notus • 11 items • Updated Dec 28, 2023 • 17

upvoted a paper 6 months ago

Prometheus: Inducing Fine-grained Evaluation Capability in Language Models

Paper • 2310.08491 • Published Oct 12, 2023 • 49

upvoted a collection 6 months ago

OpenChat

OpenChat: Advancing Open-source Language Models with Mixed-Quality Data • 7 items • Updated Jan 10 • 31

upvoted a paper 8 months ago

Adapting Large Language Models via Reading Comprehension

Paper • 2309.09530 • Published Sep 18, 2023 • 69