In this task, you will be presented with a context from an academic paper and you have to write an answerable question based on the context. Your questions can be extractive, abstractive, or yes-no questions.

Ex Input:
We are in line with recent work BIBREF16 , proposing to shift evaluation from absolute values to more exploratory evaluations focusing on weaknesses and strengths of the embeddings and not so much in generic scores. For example, one metric could consist in checking whether for any given word, all words that are known to belong to the same class are closer than any words belonging to different classes, independently of the actual cosine.

Ex Output:
What new metrics are suggested to track progress?


Ex Input:
When evaluating classifiers it is common to use accuracy, precision and recall as well as Hamming loss.  It has been shown that when calculating precision and recall on multi-label classifiers, it can be advantageous to use micro averaged precision and recall BIBREF6 . The formulas for micro averaged precision are expressed as DISPLAYFORM0 DISPLAYFORM1

Ex Output:
what evaluation metrics are discussed?


Ex Input:
We seek a function controlled by gates that can mix states across timesteps, but which acts independently on each channel of the state vector. The simplest option, which BIBREF12 term “dynamic average pooling”, uses only a forget gate: DISPLAYFORM0

Ex Output:

What pooling function is used?