|
--- |
|
library_name: transformers |
|
language: |
|
- udm |
|
--- |
|
|
|
# Zerpal-mBERT-tokenizer |
|
|
|
## How to use |
|
|
|
You can use this model directly with a pipeline for masked language modeling: |
|
|
|
```py |
|
from transformers import pipeline |
|
|
|
unmasker = pipeline('fill-mask', model='udmurtNLP/zerpal-mbert', tokenizer='udmurtNLP/zerpal-mbert-tokenizer') |
|
|
|
unmasker("Ӟечбур! Мынам нимы [MASK].") |
|
``` |
|
|
|
Here is how to use this model to get the features of a given text in PyTorch: |
|
```py |
|
from transformers import BertTokenizer, BertModel |
|
tokenizer = BertTokenizer.from_pretrained('udmurtNLP/zerpal-mbert-tokenizer') |
|
model = BertModel.from_pretrained("udmurtNLP/zerpal-mbert") |
|
text = "Яратон, яратон, мар меда сыӵе тон?" |
|
encoded_input = tokenizer(text, return_tensors='pt') |
|
output = model(**encoded_input) |
|
``` |
