Definition: In this task, you will be presented with a context from an academic paper and a question based on the context. You have to classify the questions into "Extractive", "Abstractive", or "Yes-no" questions. Extractive questions can be answered by concatenating extracts taken from a context into a summary while answering abstractive questions involves paraphrasing the context using novel sentences. Yes-no question is a question whose expected answer is one of two choices, one that affirms the question and one that denies the question. Typically, the choices are either yes or no.
Input: We use a standard set of supervised as well as unsupervised benchmark tasks from the literature to evaluate our trained models, following BIBREF16 Sentence embeddings are evaluated for various supervised classification tasks as follows. The predefined training split is used to tune the L2 penalty parameter using cross-validation and the accuracy and F1 scores are computed on the test set. We perform unsupervised evaluation of the learnt sentence embeddings using the sentence cosine similarity, on the STS 2014 BIBREF31 and SICK 2014 BIBREF32 datasets. These similarity scores are compared to the gold-standard human judgements using Pearson's INLINEFORM0 BIBREF33 and Spearman's INLINEFORM1 BIBREF34 correlation scores.
Question: What metric is used to measure performance?
Output:
Abstractive