---
license: apache-2.0
datasets:
- wikipedia
---

# Model details

`minimoe-3L-384H` distilled from `bert-base-uncased` on `Wikipedia`.

repository: [https://github.com/GeneZC/MiniMoE](https://github.com/GeneZC/MiniMoE)
arXiv: [https://arxiv.org/abs/2305.12129](https://arxiv.org/abs/2305.12129)