TASK DEFINITION: In this task, you will be presented with a context from an academic paper and a question based on the context. You have to classify the questions into "Extractive", "Abstractive", or "Yes-no" questions. Extractive questions can be answered by concatenating extracts taken from a context into a summary while answering abstractive questions involves paraphrasing the context using novel sentences. Yes-no question is a question whose expected answer is one of two choices, one that affirms the question and one that denies the question. Typically, the choices are either yes or no.
PROBLEM: We deployed the task on Amazon Mechanical Turk (AMT). To see how reasoning varies across workers, we hire 3 crowdworkers per one instance. We hire reliable crowdworkers with $\ge 5,000$ HITs experiences and an approval rate of $\ge $ 99.0%, and pay ¢20 as a reward per instance. 
 Question: Did they use any crowdsourcing platform?

SOLUTION: Yes-no

PROBLEM: The study of the mathematical structure of grammar has indicated that the fundamental things making up sentences are not the words, but some atomic grammatical types, such as the noun-type and the sentence-type BIBREF23 , BIBREF24 , BIBREF25 . The transitive verb-type is not an atomic grammatical type, but a composite made up of two noun-types and one sentence-type. Hence, particularly interesting here is that atomic doesn't really mean smallest...

On the other hand, just like in particle physics where we have particles and anti-particles, the atomic types include types as well as anti-types. But unlike in particle physics, there are two kinds of anti-types, namely left ones and right ones. This makes language even more non-commutative than quantum theory! 
 Question: Do they break down word meanings into elementary particles as in the standard model of quantum theory?

SOLUTION: Yes-no

PROBLEM: For instance, we observed that winter words were together and far away from summer ones. Week days were grouped as well and far from week-end days. 
 Question: What geometric properties do embeddings display?

SOLUTION:
Abstractive