MM1: Methods, Analysis & Insights from Multimodal LLM Pre-training
Paper
•
2403.09611
•
Published
•
122
Pierre Dulac
dulacp
AI & ML interests
None yet
Organizations
dulacp's activity
upvoted
a
paper
1 day ago
upvoted
a
paper
2 days ago
upvoted
2
papers
15 days ago
upvoted
an
article
24 days ago
Article
A Gentle Introduction to 8-bit Matrix Multiplication for transformers at scale using transformers, accelerate and bitsandbytes
•
23
upvoted
an
article
about 1 month ago
Article
Post-OCR-Correction: 1 billion words dataset of automated OCR correction by LLM
By
•
•
10
upvoted
7
papers
about 1 month ago
Make Your LLM Fully Utilize the Context
Paper
•
2404.16811
•
Published
•
52
SnapKV: LLM Knows What You are Looking for Before Generation
Paper
•
2404.14469
•
Published
•
23
Transformers Can Represent n-gram Language Models
Paper
•
2404.14994
•
Published
•
18
OpenELM: An Efficient Language Model Family with Open-source Training and Inference Framework
Paper
•
2404.14619
•
Published
•
122
Reuse Your Rewards: Reward Model Transfer for Zero-Shot Cross-Lingual Alignment
Paper
•
2404.12318
•
Published
•
14
Phi-3 Technical Report: A Highly Capable Language Model Locally on Your Phone
Paper
•
2404.14219
•
Published
•
239
RecurrentGemma: Moving Past Transformers for Efficient Open Language Models
Paper
•
2404.07839
•
Published
•
39
upvoted
a
paper
about 2 months ago
upvoted
an
article
about 2 months ago
Article
LoRA training scripts of the world, unite!
•
14
upvoted
2
papers
about 2 months ago
upvoted
5
papers
3 months ago
Zephyr: Direct Distillation of LM Alignment
Paper
•
2310.16944
•
Published
•
116
Yi: Open Foundation Models by 01.AI
Paper
•
2403.04652
•
Published
•
59
Gemini 1.5: Unlocking multimodal understanding across millions of tokens of context
Paper
•
2403.05530
•
Published
•
50
Stealing Part of a Production Language Model
Paper
•
2403.06634
•
Published
•
85
LM-Infinite: Simple On-the-Fly Length Generalization for Large Language Models
Paper
•
2308.16137
•
Published
•
38
upvoted
a
collection
3 months ago
upvoted
10
papers
3 months ago
YaRN: Efficient Context Window Extension of Large Language Models
Paper
•
2309.00071
•
Published
•
59
Linear Transformers with Learnable Kernel Functions are Better In-Context Models
Paper
•
2402.10644
•
Published
•
74
The Era of 1-bit LLMs: All Large Language Models are in 1.58 Bits
Paper
•
2402.17764
•
Published
•
567
Mixtures of Experts Unlock Parameter Scaling for Deep RL
Paper
•
2402.08609
•
Published
•
34
Synthetic Data (Almost) from Scratch: Generalized Instruction Tuning for Language Models
Paper
•
2402.13064
•
Published
•
45
In Search of Needles in a 10M Haystack: Recurrent Memory Finds What LLMs Miss
Paper
•
2402.10790
•
Published
•
39
Scaling Transformer to 1M tokens and beyond with RMT
Paper
•
2304.11062
•
Published
•
2
Recurrent Memory Transformer
Paper
•
2207.06881
•
Published
•
1
Transformers Can Achieve Length Generalization But Not Robustly
Paper
•
2402.09371
•
Published
•
12
Premise Order Matters in Reasoning with Large Language Models
Paper
•
2402.08939
•
Published
•
23
upvoted
25
papers
4 months ago
Aya Model: An Instruction Finetuned Open-Access Multilingual Language Model
Paper
•
2402.07827
•
Published
•
43
A Human-Inspired Reading Agent with Gist Memory of Very Long Contexts
Paper
•
2402.09727
•
Published
•
35
OpenMathInstruct-1: A 1.8 Million Math Instruction Tuning Dataset
Paper
•
2402.10176
•
Published
•
33
Data Engineering for Scaling Language Models to 128K Context
Paper
•
2402.10171
•
Published
•
18
Buffer Overflow in Mixture of Experts
Paper
•
2402.05526
•
Published
•
8
EVA-CLIP-18B: Scaling CLIP to 18 Billion Parameters
Paper
•
2402.04252
•
Published
•
21
Repeat After Me: Transformers are Better than State Space Models at Copying
Paper
•
2402.01032
•
Published
•
22
Learning Universal Predictors
Paper
•
2401.14953
•
Published
•
18
SliceGPT: Compress Large Language Models by Deleting Rows and Columns
Paper
•
2401.15024
•
Published
•
62
Chain-of-Table: Evolving Tables in the Reasoning Chain for Table Understanding
Paper
•
2401.04398
•
Published
•
18
LMDX: Language Model-based Document Information Extraction and Localization
Paper
•
2309.10952
•
Published
•
61
LLaMA Beyond English: An Empirical Study on Language Capability Transfer
Paper
•
2401.01055
•
Published
•
50
Self-Play Fine-Tuning Converts Weak Language Models to Strong Language Models
Paper
•
2401.01335
•
Published
•
61
DocLLM: A layout-aware generative language model for multimodal document understanding
Paper
•
2401.00908
•
Published
•
174
TinyLlama: An Open-Source Small Language Model
Paper
•
2401.02385
•
Published
•
81
Mixtral of Experts
Paper
•
2401.04088
•
Published
•
153
Vision Mamba: Efficient Visual Representation Learning with Bidirectional State Space Model
Paper
•
2401.09417
•
Published
•
51
Self-Rewarding Language Models
Paper
•
2401.10020
•
Published
•
135
Spotting LLMs With Binoculars: Zero-Shot Detection of Machine-Generated Text
Paper
•
2401.12070
•
Published
•
41
WebVoyager: Building an End-to-End Web Agent with Large Multimodal Models
Paper
•
2401.13919
•
Published
•
22
Unitxt: Flexible, Shareable and Reusable Data Preparation and Evaluation for Generative AI
Paper
•
2401.14019
•
Published
•
19
MambaByte: Token-free Selective State Space Model
Paper
•
2401.13660
•
Published
•
47
Lumiere: A Space-Time Diffusion Model for Video Generation
Paper
•
2401.12945
•
Published
•
82
Medusa: Simple LLM Inference Acceleration Framework with Multiple Decoding Heads
Paper
•
2401.10774
•
Published
•
50
VMamba: Visual State Space Model
Paper
•
2401.10166
•
Published
•
36
upvoted
2
papers
6 months ago