In this task, you will be presented with a context from an academic paper and a question based on the context. You have to classify the questions into "Extractive", "Abstractive", or "Yes-no" questions. Extractive questions can be answered by concatenating extracts taken from a context into a summary while answering abstractive questions involves paraphrasing the context using novel sentences. Yes-no question is a question whose expected answer is one of two choices, one that affirms the question and one that denies the question. Typically, the choices are either yes or no.

We compare the SBERT sentence embeddings to other sentence embeddings methods on the following seven SentEval transfer tasks:

MR: Sentiment prediction for movie reviews snippets on a five start scale BIBREF25.

CR: Sentiment prediction of customer product reviews BIBREF26.

SUBJ: Subjectivity prediction of sentences from movie reviews and plot summaries BIBREF27.

MPQA: Phrase level opinion polarity classification from newswire BIBREF28.

SST: Stanford Sentiment Treebank with binary labels BIBREF29.

TREC: Fine grained question-type classification from TREC BIBREF30.

MRPC: Microsoft Research Paraphrase Corpus from parallel news sources BIBREF31. 
 Question: What transfer learning tasks are evaluated?
Extractive

Questions: We make use of the 7,787 science exam questions of the Aristo Reasoning Challenge (ARC) corpus BIBREF31 , which contains standardized 3rd to 9th grade science questions from 12 US states from the past decade.  
 Question: How was the dataset collected?
Abstractive

These actions consist of the following components:

[leftmargin=*]

Co-Reference Resolution: To support multi-turn interactions, it is sometimes necessary to use co-reference resolution techniques for effective retrieval. Query Generation: This component generates a query based on the past user-system interactions. Retrieval Model: This is the core ranking component that retrieves documents or passages from a large collection. Result Generation: The retrieved documents can be too long to be presented using some interfaces. 
 Question: What are the different modules in Macaw?

Extractive