|
--- |
|
license: apache-2.0 |
|
language: |
|
- yo |
|
--- |
|
|
|
|
|
# oyo-teams-base-discriminator |
|
|
|
OYO-TEAMS (or Oyo-dialect of Yoruba TEAMS) was created by pre-training a [TEAMS model based on ELECTRA architecture](https://aclanthology.org/2021.findings-acl.219/) on Yoruba language texts for about 100K steps. |
|
It was trained using ELECTRA-base architecture with [Tensorflow Model Garden](https://github.com/tensorflow/models/tree/master/official/projects) |
|
|
|
### Pre-training corpus |
|
A mix of WURA, Wikipedia and MT560 Yoruba data |
|
|
|
|
|
### Acknowledgment |
|
We thank [@stefan-it](https://github.com/stefan-it) for providing the pre-processing and pre-training scripts. Finally, we would like to thank Google Cloud for providing us access to TPU v3-8 through the free cloud credits. Model trained using flax, before converted to pytorch. |
|
|
|
|
|
### BibTeX entry and citation info. |