GQA: Training Generalized Multi-Query Transformer Models from Multi-Head Checkpoints
Paper
•
2305.13245
•
Published
•
5
Papers - Attention - Grouped-Query Attention (GQA)
Note Uptrained T5.1.1 checkpoints on 600 days TPUv3
-
Reka Core, Flash, and Edge: A Series of Powerful Multimodal Language Models
Paper • 2404.12387 • Published • 38 -
OpenELM: An Efficient Language Model Family with Open-source Training and Inference Framework
Paper • 2404.14619 • Published • 126 -
Qwen2 Technical Report
Paper • 2407.10671 • Published • 156