Elie Bakouch's picture

Elie Bakouch

eliebak

·

AI & ML interests

Training LLM's @ 🤗

Articles

SmolLM - blazingly fast and remarkably powerful

Organizations

eliebak's activity

upvoted a paper 3 days ago

To Code, or Not To Code? Exploring Impact of Code in Pre-training

Paper • 2408.10914 • Published 4 days ago • 35

upvoted 2 collections 7 days ago

💻 Local SmolLMs

SmolLM models in MLC, ONNX and GGUF format for local applications + in-browser demos • 14 items • Updated 4 days ago • 36

🪐 SmolLM

A series of smol LLMs: 135M, 360M and 1.7B. We release base and Instruct models as well as the training corpus and some WebGPU demos • 12 items • Updated 7 days ago • 154

upvoted an article 10 days ago

Article

A failed experiment: Infini-Attention, and why we should keep trying?

11 days ago

• 27

upvoted a paper 10 days ago

LongWriter: Unleashing 10,000+ Word Generation from Long Context LLMs

Paper • 2408.07055 • Published 11 days ago • 60

upvoted an article 11 days ago

Article

Introduction to ggml

12 days ago

• 77

upvoted a paper 13 days ago

Tree Attention: Topology-aware Decoding for Long-Context Attention on GPU clusters

Paper • 2408.04093 • Published 17 days ago • 4

upvoted an article 19 days ago

Article

The case for specialized pre-training: ultra-fast foundation models for dedicated tasks

By

•

20 days ago

• 24

upvoted a paper 23 days ago

The Llama 3 Herd of Models

Paper • 2407.21783 • Published 24 days ago • 99

upvoted a paper 24 days ago

Sailor: Open Language Models for South-East Asia

Paper • 2404.03608 • Published Apr 4 • 20

upvoted a paper 30 days ago

Data Mixture Inference: What do BPE Tokenizers Reveal about their Training Data?

Paper • 2407.16607 • Published Jul 23 • 21

upvoted 2 articles about 1 month ago

Article

LAVE: Zero-shot VQA Evaluation on Docmatix with LLMs - Do We Still Need Fine-Tuning?

about 1 month ago

• 17

Article

SmolLM - blazingly fast and remarkably powerful

Jul 16

• 226

upvoted a paper about 1 month ago

Qwen2 Technical Report

Paper • 2407.10671 • Published Jul 15 • 152

upvoted 3 papers about 2 months ago

MobileLLM: Optimizing Sub-billion Parameter Language Models for On-Device Use Cases

Paper • 2402.14905 • Published Feb 22 • 106

Let the Expert Stick to His Last: Expert-Specialized Fine-Tuning for Sparse Architectural Large Language Models

Paper • 2407.01906 • Published Jul 2 • 33

The FineWeb Datasets: Decanting the Web for the Finest Text Data at Scale

Paper • 2406.17557 • Published Jun 25 • 78

upvoted a collection 2 months ago

Nemotron 4 340B

Nemotron-4: open models for Synthetic Data Generation (SDG). Includes Base, Instruct, and Reward models. • 4 items • Updated Jul 17 • 154

upvoted a paper 3 months ago

Scaling Laws and Compute-Optimal Training Beyond Fixed Training Durations

Paper • 2405.18392 • Published May 28 • 12

upvoted 2 collections 4 months ago

🍷 FineWeb datasets

5 items • Updated Jun 26 • 15

[lecture artifacts] aligning open language models

artifacts referenced in the talk timeline! Slides: https://docs.google.com/presentation/d/1quMyI4BAx4rvcDfk8jjv063bmHg4RxZd9mhQloXpMn0/edit?usp=sharin • 63 items • Updated Apr 17 • 54

upvoted a paper 7 months ago

Secrets of RLHF in Large Language Models Part II: Reward Modeling

Paper • 2401.06080 • Published Jan 11 • 24