In this task, you will be presented with a context from an academic paper and a question based on the context. You have to classify the questions into "Extractive", "Abstractive", or "Yes-no" questions. Extractive questions can be answered by concatenating extracts taken from a context into a summary while answering abstractive questions involves paraphrasing the context using novel sentences. Yes-no question is a question whose expected answer is one of two choices, one that affirms the question and one that denies the question. Typically, the choices are either yes or no.

Tourist: I can't go straight any further.

Guide: ok. turn so that the theater is on your right.

Guide: then go straight

Tourist: That would be going back the way I came

Guide: yeah. I was looking at the wrong bank

Tourist: I'll notify when I am back at the brooks brothers, and the bank.

Tourist: ACTION:TURNRIGHT

Guide: make a right when the bank is on your left

Tourist: ACTION:FORWARD ACTION:FORWARD ACTION:TURNRIGHT

Tourist: Making the right at the bank.

Tourist: ACTION:FORWARD ACTION:FORWARD

Tourist: I can't go that way.

Tourist: ACTION:TURNLEFT

Tourist: Bank is ahead of me on the right

Tourist: ACTION:FORWARD ACTION:FORWARD ACTION:TURNLEFT

Guide: turn around on that intersection

Tourist: I can only go to the left or back the way I just came.

Tourist: ACTION:TURNLEFT

Guide: you're in the right place. do you see shops on the corners?

Guide: If you're on the corner with the bank, cross the street

Tourist: I'm back where I started by the shop and the bank.

Tourist: ACTION:TURNRIGHT 
 Question: What language do the agents talk in?
Abstractive

. Grammatically correct text is needed as the starting point for inserting artificial errors, and we used two different sources: 1) the corrected version of the same FCE training set on which the system is trained (450K tokens), and 2) example sentences extracted from the English Vocabulary Profile (270K tokens). 
 Question: Which languages are explored in this paper?
Extractive

We would expect that explanation performance should correlate with prediction performance. Since Possible-answer knowledge is primarily needed to decide if the net has enough information to answer the challenge question without guessing and relevant-variable knowledge is needed for the net to know what to query, we analyzed the network's performance on querying and answering separately. The memory network has particular difficulty learning to query relevant variables, reaching only about .5 accuracy when querying. At the same time, it learns to answer very well, reaching over .9 accuracy there. Since these two parts of the interaction are what we ask it to explain in the two modes, we find that the quality of the explanations strongly correlates with the quality of the algorithm executed by the network. 
 Question: How do they measure correlation between the prediction and explanation quality?

Abstractive