---
license: apache-2.0
language:
- yo
---


# oyo-teams-base-discriminator

OYO-TEAMS (or Oyo-dialect of Yoruba TEAMS) was created by pre-training a [TEAMS model based on ELECTRA architecture](https://aclanthology.org/2021.findings-acl.219/) on Yoruba language texts for about 100K steps. 
It was trained using ELECTRA-base architecture with [Tensorflow Model Garden](https://github.com/tensorflow/models/tree/master/official/projects)

### Pre-training corpus
A mix of WURA, Wikipedia and MT560 Yoruba data


### Acknowledgment
We thank [@stefan-it](https://github.com/stefan-it) for providing the pre-processing and pre-training scripts. Finally, we would like to thank Google Cloud for providing us access to TPU v3-8 through the free cloud credits. Model trained using flax, before converted to pytorch.


### BibTeX entry and citation info.