oyo-teams-base-discriminator

OYO-TEAMS (or Oyo-dialect of Yoruba TEAMS) was created by pre-training a TEAMS model based on ELECTRA architecture on Yoruba language texts for about 100K steps. It was trained using ELECTRA-base architecture with Tensorflow Model Garden

Pre-training corpus

A mix of WURA, Wikipedia and MT560 Yoruba data

Acknowledgment

We thank @stefan-it for providing the pre-processing and pre-training scripts. Finally, we would like to thank Google Cloud for providing us access to TPU v3-8 through the free cloud credits. Model trained using flax, before converted to pytorch.

BibTeX entry and citation info.