|
--- |
|
inference: false |
|
language: |
|
- bg |
|
license: mit |
|
datasets: |
|
- oscar |
|
- chitanka |
|
- wikipedia |
|
tags: |
|
- torch |
|
--- |
|
|
|
# BERT BASE (cased) |
|
|
|
Pretrained model on Bulgarian language using a masked language modeling (MLM) objective. It was introduced in |
|
[this paper](https://arxiv.org/abs/1810.04805) and first released in |
|
[this repository](https://github.com/google-research/bert). This model is cased: it does make a difference |
|
between bulgarian and Bulgarian. |
|
|
|
## Model description |
|
|
|
The model was trained similarly to [RuBert](https://arxiv.org/pdf/1905.07213.pdf) wherein the Multilingual Bert was adapted for the Russian language. |
|
|
|
The training data was Bulgarian text from [OSCAR](https://oscar-corpus.com/post/oscar-2019/), [Chitanka](https://chitanka.info/) and [Wikipedia](https://bg.wikipedia.org/). |
|
|
|
### How to use |
|
|
|
Here is how to use this model in PyTorch: |
|
|
|
```python |
|
>>> from transformers import pipeline |
|
>>> |
|
>>> model = pipeline( |
|
>>> 'fill-mask', |
|
>>> model='rmihaylov/bert-base-bg', |
|
>>> tokenizer='rmihaylov/bert-base-bg', |
|
>>> device=0, |
|
>>> revision=None) |
|
>>> output = model("София е [MASK] на България.") |
|
>>> print(output) |
|
|
|
[{'score': 0.12665307521820068, |
|
'sequence': 'София е столица на България.', |
|
'token': 2659, |
|
'token_str': 'столица'}, |
|
{'score': 0.07470757514238358, |
|
'sequence': 'София е Перлата на България.', |
|
'token': 102146, |
|
'token_str': 'Перлата'}, |
|
{'score': 0.06786204129457474, |
|
'sequence': 'София е Столицата на България.', |
|
'token': 45495, |
|
'token_str': 'Столицата'}, |
|
{'score': 0.05533991754055023, |
|
'sequence': 'София е Столица на България.', |
|
'token': 100524, |
|
'token_str': 'Столица'}, |
|
{'score': 0.05485989898443222, |
|
'sequence': 'София е столицата на България.', |
|
'token': 2294, |
|
'token_str': 'столицата'}] |
|
``` |
