julien-c HF staff commited on
Commit
4a78d82
1 Parent(s): 992920f

Migrate model card from transformers-repo

Browse files

Read announcement at https://discuss.huggingface.co/t/announcement-all-model-cards-will-be-migrated-to-hf-co-model-repos/2755
Original file history: https://github.com/huggingface/transformers/commits/master/model_cards/mrm8488/bert-base-spanish-wwm-cased-finetuned-spa-squad2-es/README.md

Files changed (1) hide show
  1. README.md +91 -0
README.md ADDED
@@ -0,0 +1,91 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ ---
2
+ language: es
3
+ thumbnail: https://i.imgur.com/jgBdimh.png
4
+ ---
5
+
6
+ # BETO (Spanish BERT) + Spanish SQuAD2.0
7
+
8
+ This model is provided by [BETO team](https://github.com/dccuchile/beto) and fine-tuned on [SQuAD-es-v2.0](https://github.com/ccasimiro88/TranslateAlignRetrieve) for **Q&A** downstream task.
9
+
10
+ ## Details of the language model('dccuchile/bert-base-spanish-wwm-cased')
11
+
12
+ Language model ([**'dccuchile/bert-base-spanish-wwm-cased'**](https://github.com/dccuchile/beto/blob/master/README.md)):
13
+
14
+ BETO is a [BERT model](https://github.com/google-research/bert) trained on a [big Spanish corpus](https://github.com/josecannete/spanish-corpora). BETO is of size similar to a BERT-Base and was trained with the Whole Word Masking technique. Below you find Tensorflow and Pytorch checkpoints for the uncased and cased versions, as well as some results for Spanish benchmarks comparing BETO with [Multilingual BERT](https://github.com/google-research/bert/blob/master/multilingual.md) as well as other (not BERT-based) models.
15
+
16
+ ## Details of the downstream task (Q&A) - Dataset
17
+ [SQuAD-es-v2.0](https://github.com/ccasimiro88/TranslateAlignRetrieve)
18
+
19
+ | Dataset | # Q&A |
20
+ | ---------------------- | ----- |
21
+ | SQuAD2.0 Train | 130 K |
22
+ | SQuAD2.0-es-v2.0 | 111 K |
23
+ | SQuAD2.0 Dev | 12 K |
24
+ | SQuAD-es-v2.0-small Dev| 69 K |
25
+
26
+ ## Model training
27
+
28
+ The model was trained on a Tesla P100 GPU and 25GB of RAM with the following command:
29
+
30
+ ```bash
31
+ export SQUAD_DIR=path/to/nl_squad
32
+ python transformers/examples/question-answering/run_squad.py \
33
+ --model_type bert \
34
+ --model_name_or_path dccuchile/bert-base-spanish-wwm-cased \
35
+ --do_train \
36
+ --do_eval \
37
+ --do_lower_case \
38
+ --train_file $SQUAD_DIR/train_nl-v2.0.json \
39
+ --predict_file $SQUAD_DIR/dev_nl-v2.0.json \
40
+ --per_gpu_train_batch_size 12 \
41
+ --learning_rate 3e-5 \
42
+ --num_train_epochs 2.0 \
43
+ --max_seq_length 384 \
44
+ --doc_stride 128 \
45
+ --output_dir /content/model_output \
46
+ --save_steps 5000 \
47
+ --threads 4 \
48
+ --version_2_with_negative
49
+ ```
50
+
51
+ ## Results:
52
+
53
+
54
+ | Metric | # Value |
55
+ | ---------------------- | ----- |
56
+ | **Exact** | **76.50**50 |
57
+ | **F1** | **86.07**81 |
58
+
59
+ ```json
60
+ {
61
+ "exact": 76.50501430594491,
62
+ "f1": 86.07818773108252,
63
+ "total": 69202,
64
+ "HasAns_exact": 67.93020719738277,
65
+ "HasAns_f1": 82.37912207996466,
66
+ "HasAns_total": 45850,
67
+ "NoAns_exact": 93.34104145255225,
68
+ "NoAns_f1": 93.34104145255225,
69
+ "NoAns_total": 23352,
70
+ "best_exact": 76.51223953064941,
71
+ "best_exact_thresh": 0.0,
72
+ "best_f1": 86.08541295578848,
73
+ "best_f1_thresh": 0.0
74
+ }
75
+ ```
76
+
77
+ ### Model in action (in a Colab Notebook)
78
+ <details>
79
+
80
+ 1. Set the context and ask some questions:
81
+
82
+ ![Set context and questions](https://media.giphy.com/media/mCIaBpfN0LQcuzkA2F/giphy.gif)
83
+
84
+ 2. Run predictions:
85
+
86
+ ![Run the model](https://media.giphy.com/media/WT453aptcbCP7hxWTZ/giphy.gif)
87
+ </details>
88
+
89
+ > Created by [Manuel Romero/@mrm8488](https://twitter.com/mrm8488)
90
+
91
+ > Made with <span style="color: #e25555;">&hearts;</span> in Spain