MichelBartelsDeepset commited on
Commit
9fdd591
1 Parent(s): 26cd7db

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +6 -16
README.md CHANGED
@@ -1,7 +1,7 @@
1
  ---
2
  language: Multilingual
3
  datasets:
4
- - deepset/germanquad
5
  license: mit
6
  thumbnail: https://thumb.tildacdn.com/tild3433-3637-4830-a533-353833613061/-/resize/720x/-/format/webp/germanquad.jpg
7
  tags:
@@ -11,20 +11,14 @@ tags:
11
  ![bert_image](https://thumb.tildacdn.com/tild3433-3637-4830-a533-353833613061/-/resize/720x/-/format/webp/germanquad.jpg)
12
 
13
  ## Overview
14
- **Language model:** deepset/xlm-roberta-base-squad2-distilled
15
- **Language:** German
16
- **Training data:** GermanQuAD train set (~ 12MB)
17
- **Eval data:** GermanQuAD test set (~ 5MB)
18
  **Infrastructure**: 1x V100 GPU
19
  **Published**: Apr 21st, 2021
20
 
21
  ## Details
22
- - We trained a German question answering model with a gelectra-base model as its basis.
23
- - The dataset is GermanQuAD, a new, German language dataset, which we hand-annotated and published [online](https://deepset.ai/germanquad).
24
- - The training dataset is one-way annotated and contains 11518 questions and 11518 answers, while the test dataset is three-way annotated so that there are 2204 questions and with 2204·3−76 = 6536answers, because we removed 76 wrong answers.
25
- - In addition to the annotations in GermanQuAD, haystack's distillation feature was used for training. deepset/xlm-roberta-large-squad2 was used as the teacher model.
26
-
27
- See https://deepset.ai/germanquad for more details and dataset download in SQuAD format.
28
 
29
  ## Hyperparameters
30
  ```
@@ -38,11 +32,7 @@ temperature = 3
38
  distillation_loss_weight = 0.75
39
  ```
40
  ## Performance
41
- We evaluated the extractive question answering performance on the SQuAD v2 dev set.
42
- Model types and training data are included in the model name.
43
- For finetuning XLM-Roberta, we use the English SQuAD v2.0 dataset.
44
- The GELECTRA models are warm started on the German translation of SQuAD v1.1 and finetuned on \\\\germanquad.
45
- The human baseline was computed for the 3-way test set by taking one answer as prediction and the other two as ground truth.
46
  ```
47
  "exact": 79.8366040596311%
48
  "f1": 83.916407079888%
 
1
  ---
2
  language: Multilingual
3
  datasets:
4
+ - squad_v2
5
  license: mit
6
  thumbnail: https://thumb.tildacdn.com/tild3433-3637-4830-a533-353833613061/-/resize/720x/-/format/webp/germanquad.jpg
7
  tags:
 
11
  ![bert_image](https://thumb.tildacdn.com/tild3433-3637-4830-a533-353833613061/-/resize/720x/-/format/webp/germanquad.jpg)
12
 
13
  ## Overview
14
+ **Language model:** deepset/roberta-base-squad2-distilled
15
+ **Language:** Multilingual
16
+ **Training data:** SQuAD 2.0 training set
 
17
  **Infrastructure**: 1x V100 GPU
18
  **Published**: Apr 21st, 2021
19
 
20
  ## Details
21
+ - haystack's distillation feature was used for training. deepset/xlm-roberta-large-squad2 was used as the teacher model.
 
 
 
 
 
22
 
23
  ## Hyperparameters
24
  ```
 
32
  distillation_loss_weight = 0.75
33
  ```
34
  ## Performance
35
+ SQuAD v2 dev set:
 
 
 
 
36
  ```
37
  "exact": 79.8366040596311%
38
  "f1": 83.916407079888%