---
license: gpl-3.0
---

2expert 39layer experimental MoE for multilingual performance. 

Don't use this in production because it will very very soon get replaced , it's merely here for the benchmarks and it's actually bad at multilingual performance compared to previous experiment.