raduion commited on
Commit
9774b72
·
1 Parent(s): 49d479e

Added model card

Browse files
Files changed (1) hide show
  1. README.md +15 -7
README.md CHANGED
@@ -1,12 +1,20 @@
1
  ## BERT Medium for Luxembourgish
2
 
3
  Created from a dataset with 1M Luxembourgish sentences from Wikipedia. Corpus has approx. 16M words.
4
- MLM objective was trained.
5
- The BERT model has parameters `L=8` and `H=512`.
6
- Vocabulary has 70K word pieces.
7
 
8
  Final loss scores, after 3 epochs:
9
- Final train loss: 4.230
10
- Final train perplexity: 68.726
11
- Final validation loss: 4.074
12
- Final validation perplexity: 58.765
 
 
 
 
 
 
 
 
 
 
1
  ## BERT Medium for Luxembourgish
2
 
3
  Created from a dataset with 1M Luxembourgish sentences from Wikipedia. Corpus has approx. 16M words.
4
+
5
+ The MLM objective was trained. The BERT model has parameters `L=8` and `H=512`. Vocabulary has 70K word pieces.
 
6
 
7
  Final loss scores, after 3 epochs:
8
+
9
+ - Final train loss: 4.230
10
+ - Final train perplexity: 68.726
11
+ - Final validation loss: 4.074
12
+ - Final validation perplexity: 58.765
13
+
14
+ ---
15
+ language:
16
+ - lu
17
+ tags:
18
+ - text
19
+ - MLM
20
+ ---