TASK DEFINITION: In this task, you will be presented with a context from an academic paper and you have to write an answerable question based on the context. Your questions can be extractive, abstractive, or yes-no questions.
PROBLEM: In our experiments, we use as input the 2210 tokenized sentences of the Stanford Sentiment Treebank test set BIBREF2 , preprocessing them by lowercasing as was done in BIBREF8 .

SOLUTION: Which datasets are used for evaluation?

PROBLEM: The multi-task model outperforms the single-task model at all data sizes, and the relative performance increases as the size of the training data decreases. When only 200 sentences of training data are used, the performance of the multi-task model is about 60% better than the single-task model for both the Airbnb and Greyhound apps. The relative gain for the OpenTable app is 26%. The results differ between the tasks, but none have an overall benefit from the open vocabulary system.

SOLUTION: Does the performance increase using their method?

PROBLEM: The Bi-LSTM model consists of a Bi-LSTM layer followed by a linear layer to extract input features. The Bi-LSTM layer has a 300-dimensional hidden state for each direction. For the final classification, an additional linear layer is added to output predicted class distributions. The BERT model is obtained by fine-tuning the pre-trained BERT embeddings on NER data with an additional untrained CRF classifier. We fine-tuned all the parameters of BERT including that of the CRF end-to-end.

SOLUTION:
What classifiers were used in experiments?