How to Train Data-Efficient LLMs
Paper
•
2402.09668
•
Published
•
40
Efficient LMs
-
-
SliceGPT: Compress Large Language Models by Deleting Rows and Columns
Paper • 2401.15024 • Published • 69 -
SHERL: Synthesizing High Accuracy and Efficient Memory for Resource-Limited Transfer Learning
Paper • 2407.07523 • Published • 4 -
Spectra: A Comprehensive Study of Ternary, Quantized, and FP16 Language Models
Paper • 2407.12327 • Published • 77 -
Compact Language Models via Pruning and Knowledge Distillation
Paper • 2407.14679 • Published • 38 -
DDK: Distilling Domain Knowledge for Efficient Large Language Models
Paper • 2407.16154 • Published • 21 -
Improving Text Embeddings for Smaller Language Models Using Contrastive Fine-tuning
Paper • 2408.00690 • Published • 22 -
MiniCPM-V: A GPT-4V Level MLLM on Your Phone
Paper • 2408.01800 • Published • 78 -
MobileQuant: Mobile-friendly Quantization for On-device Language Models
Paper • 2408.13933 • Published • 13 -
VPTQ: Extreme Low-bit Vector Post-Training Quantization for Large Language Models
Paper • 2409.17066 • Published • 27 -
Addition is All You Need for Energy-efficient Language Models
Paper • 2410.00907 • Published • 144 -
Selective Attention Improves Transformer
Paper • 2410.02703 • Published • 23