In this task, you will be presented with a context from an academic paper and a question based on the context. You have to classify the questions into "Extractive", "Abstractive", or "Yes-no" questions. Extractive questions can be answered by concatenating extracts taken from a context into a summary while answering abstractive questions involves paraphrasing the context using novel sentences. Yes-no question is a question whose expected answer is one of two choices, one that affirms the question and one that denies the question. Typically, the choices are either yes or no.

Ex Input:
In HotpotQA, on average we can find 6 candidate chains (2-hop) in a instance, and the human labeled true reasoning chain is unique. A predicted chain is correct if the chain only contains all supporting passages (exact match of passages).

In MedHop, on average we can find 30 candidate chains (3-hop). For each candidate chain our human annotators labeled whether it is correct or not, and the correct reasoning chain is not unique. A predicted chain is correct if it is one of the chains that human labeled as correct.

The accuracy is defined as the ratio: The accuracy is defined as the ratio: 
 Question: What benchmarks are created?

Ex Output:
Abstractive


Ex Input:
We segment a hashtag into meaningful English phrases. In order to achieve this, we use a dictionary of English words. The sentiment of url: Since almost all the articles are written in well-formatted english, we analyze the sentiment of the first paragraph of the article using Standford Sentiment Analysis tool BIBREF4 .  
 Question: Do the authors report only on English language data?

Ex Output:
Yes-no


Ex Input:
Our Transformer Transducer model architecture has 18 audio and 2 label encoder layers. Every layer is identical for both audio and label encoders. The details of computations in a layer are shown in Figure FIGREF10 and Table TABREF11. All the models for experiments presented in this paper are trained on 8x8 TPU with a per-core batch size of 16 (effective batch size of 2048). 
 Question: Does model uses pretrained Transformer encoders?

Ex Output:

Yes-no