In this task, you will be presented with a context from an academic paper and a question based on the context. You have to classify the questions into "Extractive", "Abstractive", or "Yes-no" questions. Extractive questions can be answered by concatenating extracts taken from a context into a summary while answering abstractive questions involves paraphrasing the context using novel sentences. Yes-no question is a question whose expected answer is one of two choices, one that affirms the question and one that denies the question. Typically, the choices are either yes or no.

Ex Input:
We extract 5 Surface and Lexical features, namely sequence length in number of tokens, average word length, type-token ratio, and lexical to tokens ratio (ratio of adjectives, verbs, nouns, and adverbs to tokens). We extracted two features that use a learned representation: Firstly, we get a sentence embedding feature that is built by averaging the word embeddings of an input sentence. Secondly, we extract a fastText representation using the fastText library with the same parameters as reported in Joulin et al. joulin2016bag. 
 Question: Do they differentiate insights where they are dealing with learned or engineered representations?

Ex Output:
Yes-no


Ex Input:
SVM: We define 3 sets of features to characterize each question. The first is a simple bag-of-words set of features over the question (SVM-BOW), the second is bag-of-words features of the question as well as length of the question in words (SVM-BOW + LEN), and lastly we extract bag-of-words features, length of the question in words as well as part-of-speech tags for the question (SVM-BOW + LEN + POS). This results in vectors of 200, 201 and 228 dimensions respectively, which are provided to an SVM with a linear kernel. No-Answer Baseline (NA) : Most of the questions we receive are difficult to answer in a legally-sound way on the basis of information present in the privacy policy. We establish a simple baseline to quantify the effect of identifying every question as unanswerable. Word Count Baseline : To quantify the effect of using simple lexical matching to answer the questions, we retrieve the top candidate policy sentences for each question using a word count baseline BIBREF53, which counts the number of question words that also appear in a sentence. We include the top 2, 3 and 5 candidates as baselines. Human Performance: We pick each reference answer provided by an annotator, and compute the F1 with respect to the remaining references, as described in section 4.2.1. Each reference answer is treated as the prediction, and the remaining n-1 answers are treated as the gold reference. The average of the maximum F1 across all reference answers is computed as the human baseline. 
 Question: Were other baselines tested to compare with the neural baseline?

Ex Output:
Extractive


Ex Input:
Our system significantly out-performed the baseline on the joint task, as the baseline scored significantly lower for the gender task than for the variety task. 
 Question: On which task does do model do best?

Ex Output:

Abstractive