papluca

/

xlm-roberta-base-language-detection

like 299

Good sample code in this huggingface app

e.g.

model_ckpt = "papluca/xlm-roberta-base-language-detection"
pipe = pipeline("text-classification", model=model_ckpt)
preds = pipe(text, return_all_scores=True, truncation=True, max_length=128)

btw, i was getting a warning regarding the use of return_all_scores=True:

usr/local/lib/python3.9/dist-packages/transformers/pipelines/text_classification.py:105: UserWarning: return_all_scores is now deprecated, if want a similar functionality use top_k=None instead of return_all_scores=True or top_k=1 instead of return_all_scores=False.

so changed the last time to use top_k instead:

preds = langpipe(data_sample, top_k=3, truncation=True, max_length=128)

Sample code showing how to use the model was missing in fact—thanks for the feedback! It has now been added in a dedicated section.