language: no | |
thumbnail: https://i.imgur.com/QqSEC5I.png | |
# Norwegian Electra | |
![Image of norwegian electra](https://i.imgur.com/QqSEC5I.png) | |
Trained on Oscar + wikipedia + opensubtitles + some other data I had with the awesome power of TPUs(V3-8) | |
Use with caution. I have no downstream tasks in Norwegian to test on so I have no idea of its performance yet. | |
# Model | |
## Electra: Pre-training Text Encoders as Discriminators Rather Than Generators | |
Kevin Clark and Minh-Thang Luong and Quoc V. Le and Christopher D. Manning | |
- https://openreview.net/pdf?id=r1xMH1BtvB | |
- https://github.com/google-research/electra | |
# Acknowledgments | |
### TensorFlow Research Cloud | |
Research supported with Cloud TPUs from Google's TensorFlow Research Cloud (TFRC). Thanks for providing access to the TFRC ❤️ | |
- https://www.tensorflow.org/tfrc | |
#### OSCAR corpus | |
- https://oscar-corpus.com/ | |
#### OPUS | |
- http://opus.nlpl.eu/ | |
- http://www.opensubtitles.org/ | |