GitHub repository: pjlab-sys4nlp/llama-moe
Primary language: Python
Description: ⛷️ LLaMA-MoE: Building Mixture-of-Experts from LLaMA with Continual Pre-training (EMNLP 2024)