metadata

language: de, en

Bilingual English + German SQuAD2.0

We created German Squad 2.0 (deQuAD) and merged with SQuAD2.0 into an English and German training data for question answering. The bert-base-multilingual-cased is used to fine-tune bilingual QA downstream task.

Details of deQuAD 2.0

SQuAD2.0 was auto-translated into German. We hired professional editors to proofread the translated transcripts, correct mistakes and double check the answers to further polish the text and enhance annotation quality. The final German dataset contains 130k training and 11k test samples.

Evaluation on English SQuAD2.0

HasAns_exact = 85.79622132253711
HasAns_f1 = 90.92004586077663
HasAns_total = 5928
NoAns_exact = 94.76871320437343
NoAns_f1 = 94.76871320437343
NoAns_total = 5945
exact = 90.28889076054915
f1 = 92.84713483219753
total = 11873

Use Model in Pipeline

from transformers import pipeline

qa_pipeline = pipeline(
    "question-answering",
    model="deutsche-telekom/bert-multi-english-german-squad2",
    tokenizer="deutsche-telekom/bert-multi-english-german-squad2"
)

qa_pipeline({
    'context': " ",
    'question': " "})

Output:

{
  "score": 0.83,
  "start": 0, 
  "end": 9,
  "answer": " "
}