UKP-SQuARE/Extractive_MetaQA

datasets:

• squad
• newsqa
• hotpot_qa
• biu-nlp/qamr
• search_qa
• natural_questions
• trivia_qa
• duorc language:
• en metrics:
• squad

---

Model Card for Model ID

Checkpoint of MetaQA trained only on extractive QA datasets from MetaQA: Combining Expert Agents for Multi-Skill Question Answering (https://arxiv.org/abs/2112.01922)

Evaluation Results

{
    "SQuAD": {
        "exact_match": 86.73139158576052,
        "f1": 92.65156746563402
    },
    "NewsQA": {
        "exact_match": 55.84045584045584,
        "f1": 71.73547617592037
    },
    "HotpotQA": {
        "exact_match": 64.8135593220339,
        "f1": 79.61023604916922
    },
    "SearchQA": {
        "exact_match": 75.04122497055359,
        "f1": 81.37280639135817
    },
    "NaturalQuestionsShort": {
        "exact_match": 69.50763477718915,
        "f1": 81.30374741690376
    },
    "TriviaQA-web": {
        "exact_match": 77.18396711202466,
        "f1": 81.52989853015538
    },
    "QAMR": {
        "exact_match": 72.07531203723292,
        "f1": 83.9068616637681
    },
    "DuoRC": {
        "exact_match": 39.35626573106552,
        "f1": 51.033295034422466
    }
}