---
language:
- uk
tags:
- ukrainian
widget:
 - text: "Тарас Шевченко – великий український <mask>."
license: mit
---
This is a smaller version of the [XLM-RoBERTa](https://huggingface.co/xlm-roberta-base) model with only Ukrainian and some English embeddings left. 

* The original model has 470M parameters, with 384M of them being input and output embeddings. 
* After shrinking the `sentencepiece` vocabulary from 250K to 31K (top 25K Ukrainian tokens and top English tokens) the number of model parameters reduced to 134M parameters, and model size reduced from 1GB to 400MB.