metadata
datasets:
- mesolitica/language-detection-dataset
fasttext-language-detection-ms-id
FastText model to classify standard Malay, local Malay, local Indonesian and others.
how to use
from huggingface_hub import hf_hub_download
import fasttext
filename = hf_hub_download(
repo_id="mesolitica/fasttext-language-detection-ms-id",
filename="fasttext.ftz"
)
lang_model = fasttext.load_model(filename)
lang_model.predict('hello name saye', k = 10)
Output,
(('__label__standard-malay',
'__label__other',
'__label__socialmedia-indonesian',
'__label__local-malay'),
array([8.16933990e-01, 1.58563450e-01, 2.42817383e-02, 2.62394664e-04]))