File size: 614 Bytes
6cf5dfd |
1 2 3 4 5 6 7 8 9 10 11 12 13 14 |
---
language:
- uk
tags:
- ukrainian
widget:
- text: "Тарас Шевченко – великий український <mask>."
license: mit
---
This is a smaller version of the [XLM-RoBERTa](https://huggingface.co/xlm-roberta-base) model with only Ukrainian and some English embeddings left.
* The original model has 470M parameters, with 384M of them being input and output embeddings.
* After shrinking the `sentencepiece` vocabulary from 250K to 31K (top 25K Ukrainian tokens and top English tokens) the number of model parameters reduced to 134M parameters, and model size reduced from 1GB to 400MB.
|