mesolitica
/

malaysian-mistral-191M-MLM-512

Feature Extraction

text-generation-inference

text-embeddings-inference

Inference Endpoints

Model card Files Files and versions Community

Malaysian Mistral 191M on MLM task using 512 context length

Replicating https://github.com/McGill-NLP/llm2vec using https://huggingface.co/mesolitica/malaysian-mistral-191M-4096, done by https://github.com/aisyahrzk https://twitter.com/aisyahhhrzk

Source code at https://github.com/mesolitica/malaya/tree/master/session/llm2vec

WandB, https://wandb.ai/aisyahrazak/mistral-191M-mlm?nw=nwuseraisyahrazak

Downloads last month: 40

Safetensors

Model size

191M params

Tensor type

F32

·

Inference Providers NEW

Feature Extraction

This model is not currently available via any of the supported Inference Providers.

The model cannot be deployed to the HF Inference API: The HF Inference API does not support model that require custom code execution.

Space using mesolitica/malaysian-mistral-191M-MLM-512 1

Collections including mesolitica/malaysian-mistral-191M-MLM-512

Malaysian MaskLM

Trained on 17B tokens, 81GB of cleaned texts, able to understand standard Malay, local Malay, local Mandarin, Manglish, and local Tamil. • 7 items • Updated Dec 23, 2024

Malaysian LLM2Vec

Extending Malaysian CausalLM on non-causal masking training, https://arxiv.org/abs/2404.05961 • 5 items • Updated Dec 23, 2024