GitHub repository: Agora-Lab-AI/HydraNet
Primary language: Shell
Created: 2024-12-04T05:56:44Z
Description: HydraNet is a state-of-the-art transformer architecture that combines Multi-Query Attention (MQA), Mixture of Experts (MoE), and continuous learning capabilities.