1
---
2
language:
3
  - multilingual
4
5
datasets:
6
  - squad
7
  - arcd
8
  - xquad
9
---
10
11
# Multilingual BERT fine-tuned on SQuADv1.1
12
13
[**WandB run link**](https://wandb.ai/salti/mBERT_QA/runs/wkqzhrp2)
14
15
**GPU**: Tesla P100-PCIE-16GB
16
17
## Training Arguments
18
19
```python
20
max_seq_length              = 512
21
doc_stride                  = 256
22
max_answer_length           = 64
23
bacth_size                  = 16
24
gradient_accumulation_steps = 2
25
learning_rate               = 5e-5
26
weight_decay                = 3e-7
27
num_train_epochs            = 3
28
warmup_ratio                = 0.1
29
fp16                        = True
30
fp16_opt_level              = "O1"
31
seed                        = 0
32
```
33
34
## Results
35
36
|   EM   |   F1   |
37
| :----: | :----: |
38
| 81.731 | 89.009 |
39
40
## Zero-shot performance
41
42
### on ARCD
43
44
|   EM   |   F1   |
45
| :----: | :----: |
46
| 20.655 | 48.051 |
47
48
### on XQuAD
49
50
|  Language  |   EM   |   F1   |
51
| :--------: | :----: | :----: |
52
|   Arabic   | 42.185 | 57.803 |
53
|  English   | 73.529 | 85.01  |
54
|   German   | 55.882 | 72.555 |
55
|   Greek    | 45.21  | 62.207 |
56
|  Spanish   | 58.067 | 76.406 |
57
|   Hindi    | 40.588 | 55.29  |
58
|  Russian   | 55.126 | 71.617 |
59
|    Thai    | 26.891 | 39.965 |
60
|  Turkish   | 34.874 | 51.138 |
61
| Vietnamese | 47.983 | 68.125 |
62
|  Chinese   | 47.395 | 58.928 |
63