Loubna Ben Allal's picture

Loubna Ben Allal

loubnabnl

·

https://loubnabnl.github.io/

AI & ML interests

SmolLMs, ML for code, data

Recent Activity

updated a Space about 21 hours ago

HuggingFaceTB/README

reacted to anton-l's post with 🚀 2 days ago

Introducing 📐𝐅𝐢𝐧𝐞𝐌𝐚𝐭𝐡: the best public math pre-training dataset with 50B+ tokens! https://huggingface.co/datasets/HuggingFaceTB/finemath Math remains challenging for LLMs and by training on FineMath we see considerable gains over other math datasets, especially on GSM8K and MATH. We build the dataset by: 🛠️ carefully extracting math data from Common Crawl; 🔎 iteratively filtering and recalling high quality math pages using a classifier trained on synthetic annotations to identify math reasoning and deduction. We conducted a series of ablations comparing the performance of Llama-3.2-3B-Base after continued pre-training on FineMath and observe notable gains compared to the baseline model and other public math datasets. We hope this helps advance the performance of LLMs on math and reasoning! 🚀 We’re also releasing all the ablation models as well as the evaluation code. https://huggingface.co/collections/HuggingFaceTB/finemath-6763fb8f71b6439b653482c2

reacted to anton-l's post with 🔥 2 days ago

Introducing 📐𝐅𝐢𝐧𝐞𝐌𝐚𝐭𝐡: the best public math pre-training dataset with 50B+ tokens! https://huggingface.co/datasets/HuggingFaceTB/finemath Math remains challenging for LLMs and by training on FineMath we see considerable gains over other math datasets, especially on GSM8K and MATH. We build the dataset by: 🛠️ carefully extracting math data from Common Crawl; 🔎 iteratively filtering and recalling high quality math pages using a classifier trained on synthetic annotations to identify math reasoning and deduction. We conducted a series of ablations comparing the performance of Llama-3.2-3B-Base after continued pre-training on FineMath and observe notable gains compared to the baseline model and other public math datasets. We hope this helps advance the performance of LLMs on math and reasoning! 🚀 We’re also releasing all the ablation models as well as the evaluation code. https://huggingface.co/collections/HuggingFaceTB/finemath-6763fb8f71b6439b653482c2

View all activity

Articles

SmolLM - blazingly fast and remarkably powerful

CodeGemma - an official Google release for code LLMs

Cosmopedia: how to create large-scale synthetic data for pre-training Large Language Models

StarCoder2 and The Stack v2

Code Llama: Llama 2 learns to code

StarCoder: A State-of-the-Art LLM for Code

How to train a Language Model with Megatron-LM

Organizations

loubnabnl's activity

New activity in HuggingFaceTB/SmolLM2-1.7B-Instruct 22 days ago

Performance on MATH dataset?

#3 opened about 2 months ago by

About evaluation

#7 opened about 2 months ago by

Multi-language support

#4 opened about 2 months ago by

GSM8K results replication

#9 opened about 2 months ago by

scale-up? or full datasets list?

#11 opened about 2 months ago by

New activity in HuggingFaceTB/SmolLM-135M 24 days ago

EOS Token

#18 opened 25 days ago by

New activity in HuggingFaceFW/fineweb-edu 26 days ago

Limitation - context length of the BERT classifier?

#17 opened 26 days ago by

New activity in HuggingFaceTB/SmolLM2-135M 28 days ago

About Instruction model evaluation

#1 opened about 2 months ago by

What MMLU (cloze) used for evaluation.

#2 opened about 1 month ago by

New activity in HuggingFaceTB/SmolLM2-135M-Instruct 28 days ago

add base model metadata

#4 opened about 1 month ago by

New activity in HuggingFaceTB/SmolLM2-360M 29 days ago

Reproducing Evaluation with lighteval

#1 opened about 1 month ago by

New activity in HuggingFaceTB/SmolLM2-360M-Instruct 29 days ago

finetuning

#2 opened about 2 months ago by

New activity in HuggingFaceTB/SmolLM2-1.7B-Instruct 29 days ago

What were the training datasets for this model?

#14 opened about 1 month ago by

New activity in HuggingFaceFW/fineweb-edu-classifier about 1 month ago

add dataset metadata

#8 opened about 1 month ago by

New activity in bigcode/bigcode-models-leaderboard about 1 month ago

[Community Submission] Model: Qwen/Qwen2.5-Coder-32B-Instruct, Username: ylfeng

#87 opened about 1 month ago by

[Community Submission] Model: Qwen/Qwen2.5-Coder-32B, Username: ylfeng

#88 opened about 1 month ago by

New activity in HuggingFaceTB/SmolLM2-1.7B-Instruct about 1 month ago

Request: DOI

#12 opened about 1 month ago by

Request: DOI

#13 opened about 1 month ago by

New activity in HuggingFaceTB/SmolLM2-1.7B-Instruct about 2 months ago

tools inventory

#2 opened about 2 months ago by

New activity in HuggingFaceTB/SmolLM2-135M-Instruct about 2 months ago

Upload ONNX weights

#1 opened about 2 months ago by