Malaysian MaskLM
Collection
Trained on 17B tokens, 81GB of cleaned texts, able to understand standard Malay, local Malay, local Mandarin, Manglish, and local Tamil.
•
7 items
•
Updated
Pretrain BASE 512 masking context length DebertaV2 on Malaysian text
Special thanks to https://github.com/aisyahrzk for pretraining DebertaV2 Base.
WanDB at https://wandb.ai/aisyahrazak/deberta-base?nw=nwuseraisyahrazak
- Downloads last month
- 1
Unable to determine this model’s pipeline type. Check the
docs
.