instruction:
In this task, you will be presented with a context from an academic paper and a question based on the context. You have to classify the questions into "Extractive", "Abstractive", or "Yes-no" questions. Extractive questions can be answered by concatenating extracts taken from a context into a summary while answering abstractive questions involves paraphrasing the context using novel sentences. Yes-no question is a question whose expected answer is one of two choices, one that affirms the question and one that denies the question. Typically, the choices are either yes or no.
question:
The ratio of correct `translations' (matches) was used as an evaluation measure. 
 Question: What evaluation metric do they use?
answer:
Abstractive


question:
We introduce a surrogate training objective that avoids these problems and as a result is fully continuous.  Specifically, we form a continuous function softLB that seeks to approximate the result of running our decoder on input INLINEFORM0 and then evaluating the result against INLINEFORM1 using INLINEFORM2 . 
 Question: Do they provide a framework for building a sub-differentiable for any final loss metric?
answer:
Yes-no


question:
In order to evaluate the models trained on generated data, we manually annotated a named entities dataset comprising 53453 tokens and 2566 sentences selected from over 250 news texts from ilur.am. During annotation, we generally relied on categories and guidelines assembled by BBN Technologies for TREC 2002 question answering track. 
 Question: did they use a crowdsourcing platform for manual annotations?
answer:

Yes-no