mlabonne (Maxime Labonne)

upvoted a collection 11 days ago

📚 FineWeb-Edu

Collection

FineWeb-Edu datasets, classifier and ablation model • 5 items • Updated 21 days ago • 7

upvoted a paper 16 days ago

Magpie: Alignment Data Synthesis from Scratch by Prompting Aligned LLMs with Nothing

Paper • 2406.08464 • Published 20 days ago • 47

upvoted an article 29 days ago

Article

Uncensor any LLM with abliteration

By

•

20 days ago

• 226

upvoted an article about 1 month ago

Article

⚗️ 🔥 Building High-Quality Datasets with distilabel and Prometheus 2

By

•

29 days ago

• 21

upvoted 2 papers about 1 month ago

Zamba: A Compact 7B SSM Hybrid Model

Paper • 2405.16712 • Published May 26 • 19

Stacking Your Transformers: A Closer Look at Model Growth for Efficient LLM Pre-Training

Paper • 2405.15319 • Published May 24 • 23

upvoted 2 collections about 1 month ago

👿 Daredevil-8B

Collection

Fine-tuned abliterated merge of the best Llama 3 8B model. Highest MMLU score in its category. • 5 items • Updated May 28 • 5

🚀GGUF

Collection

Llama.cpp compatible models, can be used on CPUs and GPUs! • 676 items • Updated 5 days ago • 24

upvoted 2 papers about 2 months ago

Model Merging by Uncertainty-Based Gradient Matching

Paper • 2310.12808 • Published Oct 19, 2023 • 6

FP6-LLM: Efficiently Serving Large Language Models Through FP6-Centric Algorithm-System Co-Design

Paper • 2401.14112 • Published Jan 25 • 17

upvoted a paper 2 months ago

WildChat: 1M ChatGPT Interaction Logs in the Wild

Paper • 2405.01470 • Published May 2 • 57

upvoted an article 2 months ago

Article

🧑‍⚖️ "Replacing Judges with Juries" using distilabel

By

•

May 3

• 15

upvoted 3 papers 2 months ago

Self-Play Preference Optimization for Language Model Alignment

Paper • 2405.00675 • Published May 1 • 20

Prometheus 2: An Open Source Language Model Specialized in Evaluating Other Language Models

Paper • 2405.01535 • Published May 2 • 106

Replacing Judges with Juries: Evaluating LLM Generations with a Panel of Diverse Models

Paper • 2404.18796 • Published Apr 29 • 67

upvoted 3 articles 2 months ago

Article

StarCoder2-Instruct: Fully Transparent and Permissive Self-Alignment for Code Generation

Apr 29

• 70

Article

⚗️ 🧑🏼‍🌾 Let's grow some Domain Specific Datasets together

By

•

Apr 29

• 27

Article

🦙⚗️ Using Llama3 and distilabel to build fine-tuning datasets

By

•

29 days ago

• 64

upvoted a paper 2 months ago

Sailor: Open Language Models for South-East Asia

Paper • 2404.03608 • Published Apr 4 • 17

upvoted 3 articles 2 months ago

Article

Merge Large Language Models with mergekit

By

•

Jan 9

• 37

Article

Create Mixtures of Experts with MergeKit

By

•

Mar 28

• 9

Article

Fine-tune Llama 3 with ORPO

By

•

Apr 22

• 204

upvoted 2 articles 3 months ago

Article

Welcome Llama 3 - Meta's new open LLM

Apr 18

• 254

Article

Releasing Youtube-Commons: a massive open corpus for conversational and multimodal data

By

•

Apr 18

• 20

upvoted a collection 3 months ago

fuck quadratic attention

Collection

11 items • Updated Apr 24 • 20

upvoted 9 papers 3 months ago

JetMoE: Reaching Llama2 Performance with 0.1M Dollars

Paper • 2404.07413 • Published Apr 11 • 32

Direct Nash Optimization: Teaching Language Models to Self-Improve with General Preferences

Paper • 2404.03715 • Published Apr 4 • 58

upvoted 3 papers 4 months ago

Griffin: Mixing Gated Linear Recurrences with Local Attention for Efficient Language Models

Paper • 2402.19427 • Published Feb 29 • 50

The Era of 1-bit LLMs: All Large Language Models are in 1.58 Bits

Paper • 2402.17764 • Published Feb 27 • 574

FuseChat: Knowledge Fusion of Chat Models

Paper • 2402.16107 • Published Feb 25 • 36

upvoted 2 collections 4 months ago

Mamba

Collection

Mamba checkpoints compatible with transformers • 6 items • Updated Feb 19 • 2

Gemma release

Collection

Groups the Gemma models released by the Google team. • 40 items • Updated 6 days ago • 318

upvoted a paper 4 months ago

World Model on Million-Length Video And Language With RingAttention

Paper • 2402.08268 • Published Feb 13 • 35

upvoted 2 collections 5 months ago

👑 Monarch

Collection

Family of 7B models that combine excellent reasoning and conversational abilities. • 7 items • Updated May 27 • 9

🐶 Beagle

Collection

Merges done using mergekit and LazyMergekit: https://colab.research.google.com/drive/1obulZ1ROXHjYLn6PPZJwRR6GzgQogxxb#scrollTo=d5mYzDo1q96y • 8 items • Updated May 27 • 6

upvoted 3 papers 5 months ago

Hierarchical State Space Models for Continuous Sequence-to-Sequence Modeling

Paper • 2402.10211 • Published Feb 15 • 8

Mamba: Linear-Time Sequence Modeling with Selective State Spaces

Paper • 2312.00752 • Published Dec 1, 2023 • 132

StepCoder: Improve Code Generation with Reinforcement Learning from Compiler Feedback

Paper • 2402.01391 • Published Feb 2 • 41

upvoted a collection 5 months ago

Cultrix-Best-Models

Collection

11 items • Updated Feb 22 • 1

upvoted 3 papers 6 months ago

Self-Rewarding Language Models

Paper • 2401.10020 • Published Jan 18 • 135

Tuning Language Models by Proxy

Paper • 2401.08565 • Published Jan 16 • 19

Vision Mamba: Efficient Visual Representation Learning with Bidirectional State Space Model

Paper • 2401.09417 • Published Jan 17 • 52

upvoted a collection 6 months ago

MoEs papers reading list

Collection

44 items • Updated 20 days ago • 127

upvoted a paper 6 months ago

Quantum Denoising Diffusion Models

Paper • 2401.07049 • Published Jan 13 • 12

upvoted a collection 6 months ago

Hermes

Collection

Nous' Flagship LLM Series • 23 items • Updated 5 days ago • 92

upvoted 2 papers 6 months ago

DeepSeekMoE: Towards Ultimate Expert Specialization in Mixture-of-Experts Language Models

Paper • 2401.06066 • Published Jan 11 • 36

DocGraphLM: Documental Graph Language Model for Information Extraction

Paper • 2401.02823 • Published Jan 5 • 32

upvoted 2 collections 6 months ago

Preference Datasets for DPO

Collection

This collection contains a list of curated preference datasets for DPO fine-tuning for intent alignment of LLMs • 7 items • Updated Apr 4 • 20

Datasets built with ⚗️ distilabel

Collection

This collection contains some datasets generated and/or labelled using https://github.com/argilla-io/distilabel • 5 items • Updated Apr 4 • 5

upvoted a paper 6 months ago

Can Language Models Solve Graph Problems in Natural Language?

Paper • 2305.10037 • Published May 17, 2023 • 1

upvoted a collection 6 months ago

Comparing DPO with IPO and KTO

Collection

A collection of chat models to explore the differences between three alignment techniques: DPO, IPO, and KTO. • 56 items • Updated Jan 9 • 31

upvoted 2 papers 6 months ago

Self-Play Fine-Tuning Converts Weak Language Models to Strong Language Models

Paper • 2401.01335 • Published Jan 2 • 61

LLaMA Pro: Progressive LLaMA with Block Expansion

Paper • 2401.02415 • Published Jan 4 • 50

Maxime Labonne PRO

AI & ML interests

Articles

Uncensor any LLM with abliteration

Fine-tune Llama 3 with ORPO

Create Mixtures of Experts with MergeKit

Merge Large Language Models with mergekit

Organizations

mlabonne's activity

Uncensor any LLM with abliteration

⚗️ 🔥 Building High-Quality Datasets with distilabel and Prometheus 2

🧑‍⚖️ "Replacing Judges with Juries" using distilabel

StarCoder2-Instruct: Fully Transparent and Permissive Self-Alignment for Code Generation

⚗️ 🧑🏼‍🌾 Let's grow some Domain Specific Datasets together

🦙⚗️ Using Llama3 and distilabel to build fine-tuning datasets

Merge Large Language Models with mergekit

Create Mixtures of Experts with MergeKit

Fine-tune Llama 3 with ORPO

Welcome Llama 3 - Meta's new open LLM

Releasing Youtube-Commons: a massive open corpus for conversational and multimodal data