Norwegian Electra
Trained on Oscar + wikipedia + opensubtitles + some other data I had with the awesome power of TPUs(V3-8)
Use with caution. I have no downstream tasks in Norwegian to test on so I have no idea of its performance yet.
Model
Electra: Pre-training Text Encoders as Discriminators Rather Than Generators
Kevin Clark and Minh-Thang Luong and Quoc V. Le and Christopher D. Manning
- https://openreview.net/pdf?id=r1xMH1BtvB
- https://github.com/google-research/electra
Acknowledgments
TensorFlow Research Cloud
Research supported with Cloud TPUs from Google's TensorFlow Research Cloud (TFRC). Thanks for providing access to the TFRC ❤️ - https://www.tensorflow.org/tfrc
OSCAR corpus
- https://oscar-corpus.com/
OPUS
- http://opus.nlpl.eu/
- http://www.opensubtitles.org/
- Downloads last month
- 2,863
Unable to determine this model’s pipeline type.
Check the
docs 
.