|
--- |
|
language: |
|
- sl |
|
|
|
license: cc-by-sa-4.0 |
|
--- |
|
|
|
# t5-sl-large |
|
t5-sl-large model is a Slovene T5 model. It has 24 encoder and 24 decoder layers, in total about 750 million parameters. |
|
It was trained for 3 epochs on the following corpora: |
|
|
|
## Corpora |
|
The following corpora were used for training the model: |
|
* Gigafida 2.0 |
|
* Kas 1.0 |
|
* Janes 1.0 (only Janes-news, Janes-forum, Janes-blog, Janes-wiki subcorpora) |
|
* Slovenian parliamentary corpus siParl 2.0 |
|
* slWaC |
|
|
|
## Evaluation |
|
The model is described in detail and evaluated in our paper ["*Sequence to sequence pretraining for a less-resourced Slovenian language*"](https://arxiv.org/abs/2207.13988) |
|
|
