mrm8488 commited on
Commit
81f7312
1 Parent(s): a0db402

Create README.md

Browse files
Files changed (1) hide show
  1. README.md +38 -0
README.md ADDED
@@ -0,0 +1,38 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ ---
2
+ language: es
3
+ datasets:
4
+ - large_spanish_corpus
5
+ license: mit
6
+ ---
7
+
8
+ # ConvBERT base pre-trained on large_spanish_corpus
9
+
10
+ The ConvBERT architecture is presented in the ["ConvBERT: Improving BERT with Span-based Dynamic Convolution"](https://arxiv.org/abs/2008.02496) paper.
11
+
12
+ ## Metrics on evaluation set
13
+
14
+ ```
15
+ disc_accuracy = 0.9488542
16
+ disc_auc = 0.8833056
17
+ disc_loss = 0.15933733
18
+ disc_precision = 0.79224133
19
+ disc_recall = 0.27443287
20
+ global_step = 1000000
21
+ loss = 9.658503
22
+ masked_lm_accuracy = 0.6177698
23
+ masked_lm_loss = 1.7050561
24
+ sampled_masked_lm_accuracy = 0.5379228
25
+ ```
26
+
27
+ ## Usage
28
+
29
+ ```python
30
+ from transformers import AutoModel, AutoTokenizer
31
+ model_name = "mrm8488/convbert-base-spanish"
32
+ tokenizer = AutoTokenizer.from_pretrained(model_name)
33
+ model = AutoModel.from_pretrained(model_name)
34
+ ```
35
+
36
+ > Created by [Manuel Romero/@mrm8488](https://twitter.com/mrm8488) with the support of [Narrativa](https://www.narrativa.com/)
37
+
38
+ > Made with <span style="color: #e25555;">&hearts;</span> in Spain