GitHub repository: shan1k13/dynamicK-mixture-of-experts
Description: A Pytorch implementation of Sparsely-Gated Mixture of Experts, for massively increasing the parameter count of language models