Models
Datasets
Spaces
Posts
Docs
Pricing
Log In
Sign Up

Collections

Discover the best community collections!

Collections including paper arxiv:2404.15045

MoEs papers reading list

Outrageously Large Neural Networks: The Sparsely-Gated Mixture-of-Experts Layer

Paper • 1701.06538 • Published Jan 23, 2017 • 4
Sparse Networks from Scratch: Faster Training without Losing Performance

Paper • 1907.04840 • Published Jul 10, 2019 • 3
ZeRO: Memory Optimizations Toward Training Trillion Parameter Models

Paper • 1910.02054 • Published Oct 4, 2019 • 3
A Mixture of h-1 Heads is Better than h Heads

Paper • 2005.06537 • Published May 13, 2020 • 2

about 17 hours ago

Self-Rewarding Language Models

Paper • 2401.10020 • Published Jan 18 • 135
Orion-14B: Open-source Multilingual Large Language Models

Paper • 2401.12246 • Published Jan 20 • 10
MambaByte: Token-free Selective State Space Model

Paper • 2401.13660 • Published Jan 24 • 47
MM-LLMs: Recent Advances in MultiModal Large Language Models

Paper • 2401.13601 • Published Jan 24 • 41

about 4 hours ago

MobileCLIP: Fast Image-Text Models through Multi-Modal Reinforced Training

Paper • 2311.17049 • Published Nov 28, 2023
DeepSeek-V2: A Strong, Economical, and Efficient Mixture-of-Experts Language Model

Paper • 2405.04434 • Published May 7 • 10
A Study of Autoregressive Decoders for Multi-Tasking in Computer Vision

Paper • 2303.17376 • Published Mar 30, 2023
Sigmoid Loss for Language Image Pre-Training

Paper • 2303.15343 • Published Mar 27, 2023 • 4

Smarter LLMs Research

RARR: Researching and Revising What Language Models Say, Using Language Models

Paper • 2210.08726 • Published Oct 17, 2022 • 1
Hypothesis Search: Inductive Reasoning with Language Models

Paper • 2309.05660 • Published Sep 11, 2023 • 1
In-context Learning and Induction Heads

Paper • 2209.11895 • Published Sep 24, 2022 • 2
ReAct: Synergizing Reasoning and Acting in Language Models

Paper • 2210.03629 • Published Oct 6, 2022 • 12

OpenELM: An Efficient Language Model Family with Open-source Training and Inference Framework

Paper • 2404.14619 • Published Apr 22 • 123
Multi-Head Mixture-of-Experts

Paper • 2404.15045 • Published Apr 23 • 55
Phi-3 Technical Report: A Highly Capable Language Model Locally on Your Phone

Paper • 2404.14219 • Published Apr 22 • 239
Learn Your Reference Model for Real Good Alignment

Paper • 2404.09656 • Published Apr 15 • 80

Multi-Head Mixture-of-Experts

Paper • 2404.15045 • Published Apr 23 • 55

Multi-Head Mixture-of-Experts

Paper • 2404.15045 • Published Apr 23 • 55

Papers - Pre-training - Text - Cross-lingual

Multi-Head Mixture-of-Experts

Paper • 2404.15045 • Published Apr 23 • 55

LM Architectures

about 16 hours ago

Megalodon: Efficient LLM Pretraining and Inference with Unlimited Context Length

Paper • 2404.08801 • Published Apr 12 • 62
RecurrentGemma: Moving Past Transformers for Efficient Open Language Models

Paper • 2404.07839 • Published Apr 11 • 40
Eagle and Finch: RWKV with Matrix-Valued States and Dynamic Recurrence

Paper • 2404.05892 • Published Apr 8 • 28
Mamba: Linear-Time Sequence Modeling with Selective State Spaces

Paper • 2312.00752 • Published Dec 1, 2023 • 132

Interesting Papers

Rho-1: Not All Tokens Are What You Need

Paper • 2404.07965 • Published Apr 11 • 80
LLM2Vec: Large Language Models Are Secretly Powerful Text Encoders

Paper • 2404.05961 • Published Apr 9 • 62
Compression Represents Intelligence Linearly

Paper • 2404.09937 • Published Apr 15 • 27
Multi-Head Mixture-of-Experts

Paper • 2404.15045 • Published Apr 23 • 55

Previous
1
2
3
Next

Company

© Hugging Face

TOS Privacy About Jobs

Website

Models Datasets Spaces Pricing Docs