| license: gpl-3.0 | |
| 2expert 39layer experimental MoE for multilingual performance. | |
| Don't use this in production because it will very very soon get replaced , it's merely here for the benchmarks and it's actually bad at multilingual performance compared to previous experiment. |