instruction:
In this task, you will be presented with a context from an academic paper and a question based on the context. You have to classify the questions into "Extractive", "Abstractive", or "Yes-no" questions. Extractive questions can be answered by concatenating extracts taken from a context into a summary while answering abstractive questions involves paraphrasing the context using novel sentences. Yes-no question is a question whose expected answer is one of two choices, one that affirms the question and one that denies the question. Typically, the choices are either yes or no.
question:
Probing Methodology and Modeling ::: Task Definition and Modeling ::: Baselines and Sanity Checks.
When creating synthetic datasets, it is important to ensure that systematic biases, or annotation artifacts BIBREF41, are not introduced into the resulting probes and that the target datasets are sufficiently challenging (or good, in the sense of BIBREF42). To test for this, we use several of the MCQA baseline models first introduced in BIBREF0, which take inspiration from the LSTM-based models used in BIBREF43 for NLI and various partial-input baselines based on these models. 
 Question: How do they control for annotation artificats?
answer:
Extractive


question:
We thus adapt an off-the-shelf reward learning algorithm BIBREF7 to the supervised setting for automated data manipulation. 
 Question: What off-the-shelf reward learning algorithm from RL for joint data manipulation learning and model training is adapted?
answer:
Extractive


question:
Our architecture was designed to show that using a model that captures better information about the context and sequential nature of the text can outperform lexicon-based methods commonly used in the literature. 
 Question: What meaningful information does the GRU model capture, which traditional ML models do not?
answer:

Extractive