How to use this model directly from the
from transformers import AutoTokenizer, AutoModelWithLMHead tokenizer = AutoTokenizer.from_pretrained("DeepPavlov/bert-base-bg-cs-pl-ru-cased") model = AutoModelWithLMHead.from_pretrained("DeepPavlov/bert-base-bg-cs-pl-ru-cased")
SlavicBERT (Slavic (bg, cs, pl, ru), cased, 12‑layer, 768‑hidden, 12‑heads, 180M parameters) was trained on Russian News and four Wikipedias: Bulgarian, Czech, Polish, and Russian. Subtoken vocabulary was built using this data. Multilingual BERT was used as an initialization for SlavicBERT.
: Arkhipov M., Trofimova M., Kuratov Y., Sorokin A. (2019). Tuning Multilingual Transformers for Language-Specific Named Entity Recognition. ACL anthology W19-3712.