Rho-1: Not All Tokens Are What You Need
Paper
•
2404.07965
•
Published
•
80
LM Training
-
-
VASA-1: Lifelike Audio-Driven Talking Faces Generated in Real Time
Paper • 2404.10667 • Published • 12 -
Instruction-tuned Language Models are Better Knowledge Learners
Paper • 2402.12847 • Published • 25 -
DoRA: Weight-Decomposed Low-Rank Adaptation
Paper • 2402.09353 • Published • 21 -
QLoRA: Efficient Finetuning of Quantized LLMs
Paper • 2305.14314 • Published • 41 -
GaLore: Memory-Efficient LLM Training by Gradient Low-Rank Projection
Paper • 2403.03507 • Published • 176 -
Reverse Training to Nurse the Reversal Curse
Paper • 2403.13799 • Published • 12 -
SOLAR 10.7B: Scaling Large Language Models with Simple yet Effective Depth Up-Scaling
Paper • 2312.15166 • Published • 55 -
ReFT: Representation Finetuning for Language Models
Paper • 2404.03592 • Published • 74 -
Direct Nash Optimization: Teaching Language Models to Self-Improve with General Preferences
Paper • 2404.03715 • Published • 58 -
Learn Your Reference Model for Real Good Alignment
Paper • 2404.09656 • Published • 80 -
Megalodon: Efficient LLM Pretraining and Inference with Unlimited Context Length
Paper • 2404.08801 • Published • 62 -
Pre-training Small Base LMs with Fewer Tokens
Paper • 2404.08634 • Published • 32 -
JetMoE: Reaching Llama2 Performance with 0.1M Dollars
Paper • 2404.07413 • Published • 32 -
MiniCPM: Unveiling the Potential of Small Language Models with Scalable Training Strategies
Paper • 2404.06395 • Published • 18 -
SambaLingo: Teaching Large Language Models New Languages
Paper • 2404.05829 • Published • 12 -
Advancing LLM Reasoning Generalists with Preference Trees
Paper • 2404.02078 • Published • 41 -
Poro 34B and the Blessing of Multilinguality
Paper • 2404.01856 • Published • 12 -
Phi-3 Technical Report: A Highly Capable Language Model Locally on Your Phone
Paper • 2404.14219 • Published • 238 -
The Instruction Hierarchy: Training LLMs to Prioritize Privileged Instructions
Paper • 2404.13208 • Published • 37 -
Eagle and Finch: RWKV with Matrix-Valued States and Dynamic Recurrence
Paper • 2404.05892 • Published • 28 -
Mamba: Linear-Time Sequence Modeling with Selective State Spaces
Paper • 2312.00752 • Published • 131 -
OpenELM: An Efficient Language Model Family with Open-source Training and Inference Framework
Paper • 2404.14619 • Published • 122 -
Jamba: A Hybrid Transformer-Mamba Language Model
Paper • 2403.19887 • Published • 99 -
Make Your LLM Fully Utilize the Context
Paper • 2404.16811 • Published • 52 -
Tele-FLM Technical Report
Paper • 2404.16645 • Published • 17 -
PLLaVA : Parameter-free LLaVA Extension from Images to Videos for Video Dense Captioning
Paper • 2404.16994 • Published • 31 -
LoRA Land: 310 Fine-tuned LLMs that Rival GPT-4, A Technical Report
Paper • 2405.00732 • Published • 115 -
Iterative Reasoning Preference Optimization
Paper • 2404.19733 • Published • 41 -
What matters when building vision-language models?
Paper • 2405.02246 • Published • 87 -
MoRA: High-Rank Updating for Parameter-Efficient Fine-Tuning
Paper • 2405.12130 • Published • 41 -
Your Transformer is Secretly Linear
Paper • 2405.12250 • Published • 134