TASK DEFINITION: In this task, you will be presented with a context from an academic paper and a question separated with a 
. You have to answer the question based on the context.
PROBLEM: To tackle this issue, we investigate how to improve the generalization of BERT-based MCQA models with the constraint of limited training data using four representative MCQA datasets: DREAM, MCTest, TOEFL, and SemEval-2018 Task 11. 
 Question: What four representative datasets are used for bechmark?

SOLUTION: DREAM, MCTest, TOEFL, and SemEval-2018 Task 11

PROBLEM: In this paper, we present an extraction-then-synthesis framework for machine reading comprehension shown in Figure 1 , in which the answer is synthesized from the extraction results. We build an evidence extraction model to predict the most important sub-spans from the passages as evidence, and then develop an answer synthesis model which takes the evidence as additional features along with the question and passage to further elaborate the final answers. 
 Question: Which framework they propose in this paper?

SOLUTION:  extraction-then-synthesis framework

PROBLEM: Compared to using external models for confidence modeling, an advantage of the proposed method is that the base model does not change: the binary classification loss just provides additional supervision. For supervised open IE systems, the confidence score of an assertion is typically computed based on its extraction likelihood given by the model BIBREF3 , BIBREF5  We follow the evaluation metrics described by Stanovsky:2016:OIE2016: area under the precision-recall curve (AUC) and F1 score. 
 Question: How does this compare to traditional calibration methods like Platt Scaling?

SOLUTION:
No reliability diagrams are provided and no explicit comparison is made between confidence scores or methods.