File size: 1,950 Bytes
37d9ca8 985c9aa 37d9ca8 5d8623a |
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 |
---
inference: false
language:
- bg
license: mit
datasets:
- oscar
- chitanka
- wikipedia
tags:
- torch
---
# BERT BASE (cased)
Pretrained model on Bulgarian language using a masked language modeling (MLM) objective. It was introduced in
[this paper](https://arxiv.org/abs/1810.04805) and first released in
[this repository](https://github.com/google-research/bert). This model is cased: it does make a difference
between bulgarian and Bulgarian.
## Model description
The model was trained similarly to [RuBert](https://arxiv.org/pdf/1905.07213.pdf) wherein the Multilingual Bert was adapted for the Russian language.
The training data was Bulgarian text from [OSCAR](https://oscar-corpus.com/post/oscar-2019/), [Chitanka](https://chitanka.info/) and [Wikipedia](https://bg.wikipedia.org/).
### How to use
Here is how to use this model in PyTorch:
```python
>>> from transformers import pipeline
>>>
>>> model = pipeline(
>>> 'fill-mask',
>>> model='rmihaylov/bert-base-bg',
>>> tokenizer='rmihaylov/bert-base-bg',
>>> device=0,
>>> revision=None)
>>> output = model("София е [MASK] на България.")
>>> print(output)
[{'score': 0.12665307521820068,
'sequence': 'София е столица на България.',
'token': 2659,
'token_str': 'столица'},
{'score': 0.07470757514238358,
'sequence': 'София е Перлата на България.',
'token': 102146,
'token_str': 'Перлата'},
{'score': 0.06786204129457474,
'sequence': 'София е Столицата на България.',
'token': 45495,
'token_str': 'Столицата'},
{'score': 0.05533991754055023,
'sequence': 'София е Столица на България.',
'token': 100524,
'token_str': 'Столица'},
{'score': 0.05485989898443222,
'sequence': 'София е столицата на България.',
'token': 2294,
'token_str': 'столицата'}]
``` |