Loubna Ben Allal's picture

Loubna Ben Allal

loubnabnl

·

https://loubnabnl.github.io/

AI & ML interests

SmolLMs, ML for code, data

Recent Activity

updated a Space about 22 hours ago

HuggingFaceTB/README

reacted to anton-l's post with 🚀 2 days ago

Introducing 📐𝐅𝐢𝐧𝐞𝐌𝐚𝐭𝐡: the best public math pre-training dataset with 50B+ tokens! https://huggingface.co/datasets/HuggingFaceTB/finemath Math remains challenging for LLMs and by training on FineMath we see considerable gains over other math datasets, especially on GSM8K and MATH. We build the dataset by: 🛠️ carefully extracting math data from Common Crawl; 🔎 iteratively filtering and recalling high quality math pages using a classifier trained on synthetic annotations to identify math reasoning and deduction. We conducted a series of ablations comparing the performance of Llama-3.2-3B-Base after continued pre-training on FineMath and observe notable gains compared to the baseline model and other public math datasets. We hope this helps advance the performance of LLMs on math and reasoning! 🚀 We’re also releasing all the ablation models as well as the evaluation code. https://huggingface.co/collections/HuggingFaceTB/finemath-6763fb8f71b6439b653482c2

reacted to anton-l's post with 🔥 2 days ago

Introducing 📐𝐅𝐢𝐧𝐞𝐌𝐚𝐭𝐡: the best public math pre-training dataset with 50B+ tokens! https://huggingface.co/datasets/HuggingFaceTB/finemath Math remains challenging for LLMs and by training on FineMath we see considerable gains over other math datasets, especially on GSM8K and MATH. We build the dataset by: 🛠️ carefully extracting math data from Common Crawl; 🔎 iteratively filtering and recalling high quality math pages using a classifier trained on synthetic annotations to identify math reasoning and deduction. We conducted a series of ablations comparing the performance of Llama-3.2-3B-Base after continued pre-training on FineMath and observe notable gains compared to the baseline model and other public math datasets. We hope this helps advance the performance of LLMs on math and reasoning! 🚀 We’re also releasing all the ablation models as well as the evaluation code. https://huggingface.co/collections/HuggingFaceTB/finemath-6763fb8f71b6439b653482c2

View all activity

Articles

SmolLM - blazingly fast and remarkably powerful

CodeGemma - an official Google release for code LLMs

Cosmopedia: how to create large-scale synthetic data for pre-training Large Language Models

StarCoder2 and The Stack v2

Code Llama: Llama 2 learns to code

StarCoder: A State-of-the-Art LLM for Code

How to train a Language Model with Megatron-LM

Organizations

loubnabnl's activity

liked 2 models 4 days ago

SultanR/SmolTulu-1.7b-Instruct

Text Generation • Updated 5 days ago • 324 • 13

SultanR/SmolTulu-1.7b-Reinforced

Text Generation • Updated 5 days ago • 35 • 5

liked a Space 23 days ago

Github Issue Generator

liked a Space 24 days ago

SmolLM2 1.7B Instruct WebGPU

A blazingly fast & powerful AI chatbot that runs in-browser!

liked a dataset 25 days ago

ylecun/mnist

Viewer • Updated Aug 8 • 70k • 39.2k • 141

liked a Space 26 days ago

Running on CPU Upgrade

HuggingFace Trending Board

liked a model about 1 month ago

numind/NuExtract-1.5-smol

Text Generation • Updated Nov 18 • 1.15k • 53

liked a model 3 months ago

MBZUAI-Paris/Atlas-Chat-9B

Text Generation • Updated Oct 24 • 286 • 23

liked a dataset 3 months ago

argilla/FinePersonas-v0.1

Viewer • Updated 10 days ago • 42.1M • 6.03k • 365

liked a model 3 months ago

bigcode/starencoder

Updated May 10, 2023 • 44.5k • 50

liked a dataset 4 months ago

open-web-math/open-web-math

Viewer • Updated Oct 17, 2023 • 6.32M • 6.48k • 275

liked 4 Spaces 4 months ago

Spark on HF JupyterLab

JupyterLab Notebooks With PySpark Optimized for HF Datasets

SmolPilot

SmolLM 360M Instruct WebGPU

A blazingly fast and powerful AI chatbot that runs locally.

Instant SmolLM

Run SmolLM-360M-Instruct in realtime with MLC WebLLM

liked a Space 5 months ago

llm.c 1558M demo

liked a Space 6 months ago

Running on Zero

Live Portrait

Apply the motion of a video on a portrait

liked a dataset 6 months ago

bigcode/self-oss-instruct-sc2-exec-filter-50k

Viewer • Updated Nov 4 • 50.7k • 187 • 91

liked a Space 6 months ago

Open-LLM performances are plateauing, let’s make the leaderboard steep again

liked a model 6 months ago

Qwen/Qwen2-72B-Instruct

Text Generation • Updated Oct 8 • 40.8k • 689