Edit model card

BERT Medium for Luxembourgish

Created from a dataset with 1M Luxembourgish sentences from Wikipedia. Corpus has approx. 16M words.

The MLM objective was trained. The BERT model has parameters L=8 and H=512. Vocabulary has 70K word pieces.

Final loss scores, after 3 epochs:

  • Final train loss: 4.230
  • Final train perplexity: 68.726
  • Final validation loss: 4.074
  • Final validation perplexity: 58.765
Downloads last month
165
Hosted inference API
Mask token: [MASK]
This model can be loaded on the Inference API on-demand.