Nordic Roberta Wikipedia

Description

Nord roberta model trainined on the swedish danish and norwegian wikipedia.

Evaluation

Evaluation on Named Entity recognition in Danish.

I finetuned each model on 3 epochs on DaNE, repeated it 5 times for each model, and calculated 95% confidence intervals for the means. Here are the results:

xlm-roberta-base : 88.01 +- 0.43

flax-community/nordic-roberta-wiki: 85.75 +- 0.69 (this model)

Maltehb/danish-bert-botxo: 85.38 +- 0.55

flax-community/roberta-base-danish: 80.14 +- 1.47

flax-community/roberta-base-scandinavian : 78.03 +- 3.02

Maltehb/-l-ctra-danish-electra-small-cased: 57.87 +- 3.19

NbAiLab/nb-bert-base : 30.24 +- 1.21

Randomly initialised RoBERTa model: 19.79 +- 2.00

Evaluation on Sentiment analysis in Dansish Here are the results on test set, where each model has been trained 5 times, and the “+-” refers to a 95% confidence interval of the mean score:

Maltehb/danish-bert-botxo: 65.19 +- 0.53

NbAiLab/nb-bert-base : 63.80 +- 0.77

xlm-roberta-base : 63.55 +- 1.59

flax-community/nordic-roberta-wiki : 56.46 +- 1.77

flax-community/roberta-base-danish : 54.73 +- 8.96

flax-community/roberta-base-scandinavian : 44.28 +- 9.21

Maltehb/-l-ctra-danish-electra-small-cased : 47.78 +- 12.65

Randomly initialised RoBERTa model: 36.96 +- 1.02

Maltehb/roberta-base-scandinavian : 33.65 +- 8.32

Model series

This model is part of a series of models training on TPU with Flax Jax during Huggingface Flax/Jax challenge.

Gpt models

Swedish Gpt

https://huggingface.co/birgermoell/swedish-gpt/

Swedish gpt wiki

https://huggingface.co/flax-community/swe-gpt-wiki

Nordic gpt wiki

https://huggingface.co/flax-community/nordic-gpt-wiki

Dansk gpt wiki

https://huggingface.co/flax-community/dansk-gpt-wiki

Norsk gpt wiki

https://huggingface.co/flax-community/norsk-gpt-wiki

Roberta models

Nordic Roberta Wiki

https://huggingface.co/flax-community/nordic-roberta-wiki

Swe Roberta Wiki Oscar

https://huggingface.co/flax-community/swe-roberta-wiki-oscar

Roberta Swedish Scandi

https://huggingface.co/birgermoell/roberta-swedish-scandi

Roberta Swedish

https://huggingface.co/birgermoell/roberta-swedish

Swedish T5 model

https://huggingface.co/birgermoell/t5-base-swedish

Downloads last month
500
Hosted inference API
Fill-Mask
Mask token: <mask>
This model can be loaded on the Inference API on-demand.