In this task, you will be presented with a context from an academic paper and a question based on the context. You have to classify the questions into "Extractive", "Abstractive", or "Yes-no" questions. Extractive questions can be answered by concatenating extracts taken from a context into a summary while answering abstractive questions involves paraphrasing the context using novel sentences. Yes-no question is a question whose expected answer is one of two choices, one that affirms the question and one that denies the question. Typically, the choices are either yes or no.

[EX Q]: For each qualifying diagnosis tweet we retrieve the timeline of the corresponding Twitter user using the Twitter user_timeline API endpoint . Subsequently, we remove all non-English tweets (Twitter API machine-detected“lang” field), all retweets, and tweets that contain “diagnos*” or “depress*”, but not a valid diagnosis statement. The resulting Depressed cohort contains 1,207 individuals and 1,759,644 tweets ranging from from May 2008 to September 2018. 
 Question: Do they report results only on English datasets?
[EX A]: Yes-no

[EX Q]: We use the BLEU BIBREF30 metric on the validation set for the VQG model training. BLEU is a measure of similitude between generated and target sequences of words, widely used in natural language processing. It assumes that valid generated responses have significant word overlap with the ground truth responses. We use it because in this case we have five different references for each of the generated questions. We obtain a BLEU score of 2.07.

Our chatbot model instead, only have one reference ground truth in training when generating a sequence of words. We considered that it was not a good metric to apply as in some occasions responses have the same meaning, but do not share any words in common. Thus, we save several models with different hyperparameters and at different number of training iterations and compare them using human evaluation, to chose the model that performs better in a conversation. 
 Question: How is performance of this system measured?
[EX A]: Abstractive

[EX Q]: Our approach improves LR by 5.17% (Accuracy) and 18.38% (AUC), and MLP by 10.71% (Accuracy) and 30.27% (AUC) on average. Such significant improvements clearly demonstrate that our approach is effective at improving model performance. 
 Question: How are the accuracy merits of the approach demonstrated?
[EX A]:
Extractive