license: apache-2.0 | |
datasets: | |
- wikipedia | |
# Model details | |
`minimoe-6L-384H` distilled from `bert-large-uncased` on `Wikipedia`. | |
repository: [https://github.com/GeneZC/MiniMoE](https://github.com/GeneZC/MiniMoE) | |
arXiv: [https://arxiv.org/abs/2305.12129](https://arxiv.org/abs/2305.12129) |