Leandro von Werra's picture

Leandro von Werra

lvwerra

·

https://github.com/lvwerra

AI & ML interests

NLP and RL

Recent Activity

reacted to lewtun's post with 🔥 12 days ago

This paper (https://huggingface.co/papers/2412.18925) has a really interesting recipe for inducing o1-like behaviour in Llama models: * Iteratively sample CoTs from the model, using a mix of different search strategies. This gives you something like Stream of Search via prompting. * Verify correctness of each CoT using GPT-4o (needed because exact match doesn't work well in medicine where there are lots of aliases) * Use GPT-4o to reformat the concatenated CoTs into a single stream that includes smooth transitions like "hmm, wait" etc that one sees in o1 * Use the resulting data for SFT & RL * Use sparse rewards from GPT-4o to guide RL training. They find RL gives an average ~3 point boost across medical benchmarks and SFT on this data already gives a strong improvement. Applying this strategy to other domains could be quite promising, provided the training data can be formulated with verifiable problems!

liked a Space 20 days ago

data-agents/jupyter-agent

updated a Space 22 days ago

data-agents/jupyter-agent

View all activity

Articles

LeMaterial: an open source initiative to accelerate materials discovery and research

CinePile 2.0 - making stronger datasets with adversarial refinement

FineVideo: behind the scenes

Fine-tuning LLMs to 1.58bit: extreme quantization made easy

A failed experiment: Infini-Attention, and why we should keep trying?

Llama 3.1 - 405B, 70B & 8B with multilinguality and long context

BigCodeBench: Benchmarking Large Language Models on Solving Practical and Challenging Programming Tasks

StarCoder2-Instruct: Fully Transparent and Permissive Self-Alignment for Code Generation

Welcome Llama 3 - Meta's new open LLM

StarCoder2 and The Stack v2

Constitutional AI with Open LLMs

Preference Tuning LLMs with Direct Preference Optimization Methods

Welcome Mixtral - a SOTA Mixture of Experts on Hugging Face

The N Implementation Details of RLHF with PPO

Finetune Stable Diffusion Models with DDPO via TRL

Spread Your Wings: Falcon 180B is here

Code Llama: Llama 2 learns to code

Fine-tune Llama 2 with DPO

The Falcon has landed in the Hugging Face ecosystem

Creating a Coding Assistant with StarCoder

StarCoder: A State-of-the-Art LLM for Code

StackLLaMA: A hands-on guide to train LLaMA with RLHF

Fine-tuning 20B LLMs with RLHF on a 24GB consumer GPU

Illustrating Reinforcement Learning from Human Feedback (RLHF)

Evaluating Language Model Bias with 🤗 Evaluate

Announcing Evaluation on the Hub

Organizations

Papers 12

arxiv:2410.24198

arxiv:2406.17557

arxiv:2405.18392

arxiv:2402.19173

spaces 21

Executor

3d Bench Viz

3d

Train LLMs

Text Source Viz

Harm Space

models 33

lvwerra/the-tokenizer-v1

Updated Feb 12, 2024 • 1

lvwerra/sc2

Updated Feb 11, 2024 • 2

lvwerra/starcoder-98k-no-regex-no-digits

Updated Sep 29, 2023

lvwerra/starcoder-393k

Updated Sep 28, 2023

lvwerra/starcoder-196k

Updated Sep 28, 2023

lvwerra/starcoder-98k

Updated Sep 27, 2023

lvwerra/starcoder-24k

Updated Sep 27, 2023

lvwerra/starcoder-12k

Updated Sep 27, 2023

lvwerra/starcoder-6k

Updated Sep 27, 2023

lvwerra/starcoderbase-gsm8k

Text Generation • Updated Aug 30, 2023 • 14

datasets 21

lvwerra/needle-llama3-16x524k

Viewer • Updated Apr 26, 2024 • 1.41k • 63 • 1

lvwerra/needle-llama3-16x65k

Viewer • Updated Apr 26, 2024 • 1.41k • 34 • 1

lvwerra/needle-llama3-16x8k

Viewer • Updated Apr 26, 2024 • 1.41k • 28 • 1

lvwerra/needle-llama3-16x512

Viewer • Updated Apr 26, 2024 • 1.41k • 32 • 1

lvwerra/admin

Viewer • Updated Mar 6, 2024 • 1 • 252

lvwerra/stack-exchange-paired

Viewer • Updated Mar 13, 2023 • 31.3M • 1.4k • 142

lvwerra/git-commits-clean

Updated Mar 2, 2023 • 2

lvwerra/changeit

Viewer • Updated Jan 8, 2023 • 31 • 83

lvwerra/code-ml

Viewer • Updated Jan 4, 2023 • 1.5k • 13

lvwerra/my_test_2

Viewer • Updated Mar 14, 2022 • 3 • 10