File size: 846 Bytes
6cb0d71 24ced61 6cb0d71 24ced61 e1e0e47 24ced61 |
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 |
---
license: apache-2.0
language:
- yo
---
# oyo-teams-base-discriminator
OYO-TEAMS (or Oyo-dialect of Yoruba TEAMS) was created by pre-training a [TEAMS model based on ELECTRA architecture](https://aclanthology.org/2021.findings-acl.219/) on Yoruba language texts for about 100K steps.
It was trained using ELECTRA-base architecture with [Tensorflow Model Garden](https://github.com/tensorflow/models/tree/master/official/projects)
### Pre-training corpus
A mix of WURA, Wikipedia and MT560 Yoruba data
### Acknowledgment
We thank [@stefan-it](https://github.com/stefan-it) for providing the pre-processing and pre-training scripts. Finally, we would like to thank Google Cloud for providing us access to TPU v3-8 through the free cloud credits. Model trained using flax, before converted to pytorch.
### BibTeX entry and citation info. |