QMoE: Practical Sub-1-Bit Compression of Trillion-Parameter Models
Paper
•
2310.16795
•
Published
•
26
Papers: MoE/Ensemble
Papers related to Mixture of Experts topics.
-
-
Ensemble-Instruct: Generating Instruction-Tuning Data with a Heterogeneous Mixture of LMs
Paper • 2310.13961 • Published • 4 -
The Consensus Game: Language Model Generation via Equilibrium Search
Paper • 2310.09139 • Published • 12 -
Large Language Model Cascades with Mixture of Thoughts Representations for Cost-efficient Reasoning
Paper • 2310.03094 • Published • 12 -
EcoAssistant: Using LLM Assistant More Affordably and Accurately
Paper • 2310.03046 • Published • 5 -
Corex: Pushing the Boundaries of Complex Reasoning through Multi-Model Collaboration
Paper • 2310.00280 • Published • 3 -
LoRA ensembles for large language model fine-tuning
Paper • 2310.00035 • Published • 2 -
Pre-gated MoE: An Algorithm-System Co-Design for Fast and Scalable Mixture-of-Expert Inference
Paper • 2308.12066 • Published • 4 -
Towards MoE Deployment: Mitigating Inefficiencies in Mixture-of-Expert (MoE) Inference
Paper • 2303.06182 • Published • 1 -
SMILE: Scaling Mixture-of-Experts with Efficient Bi-level Routing
Paper • 2212.05191 • Published • 1 -
Cross-Domain Ensemble Distillation for Domain Generalization
Paper • 2211.14058 • Published • 1 -
Multi-Head Adapter Routing for Cross-Task Generalization
Paper • 2211.03831 • Published • 2 -
Mixtral of Experts
Paper • 2401.04088 • Published • 159 -
MoE-Mamba: Efficient Selective State Space Models with Mixture of Experts
Paper • 2401.04081 • Published • 71