Detailed Instructions: In this task, you will be presented with a context from an academic paper and a question based on the context. You have to classify the questions into "Extractive", "Abstractive", or "Yes-no" questions. Extractive questions can be answered by concatenating extracts taken from a context into a summary while answering abstractive questions involves paraphrasing the context using novel sentences. Yes-no question is a question whose expected answer is one of two choices, one that affirms the question and one that denies the question. Typically, the choices are either yes or no.
Problem:Furthermore, the gains in WER over the baseline are significantly larger for the Density Ratio method than for Shallow Fusion, with up to 28% relative reduction in WER (17.5% $
ightarrow $ 12.5%) compared to up to 17% relative reduction (17.5% $
ightarrow $ 14.5%) for Shallow Fusion, in the no fine-tuning scenario.
Question: What metrics are used for evaluation?
Solution:
Abstractive