In this task, you will be presented with a context from an academic paper and a question separated with a
. You have to answer the question based on the context.
Example: We evaluate the proposed approach on the Chinese social media text summarization task, based on the sequence-to-sequence model. Large-Scale Chinese Short Text Summarization Dataset (LCSTS) is constructed by BIBREF1 . The dataset consists of more than 2.4 million text-summary pairs in total, constructed from a famous Chinese social media microblogging service Weibo.
Question: Are results reported only for English data?
Example solution: No
Example explanation: Based on the context, the dataset is constructed from a famous Chinese social media microblogging service Weibo.

Problem: It should also be noted that signals can be recognized even when the model misclassifies relations, since ${\Delta }_s$ does not rely on correct classification: it merely quantifies the contribution of a word in context toward the correct label's score. If we examine the influence of each word on the score of the correct relation, that impact should and does still correlate with human judgments based on what the system may tag as the second or third best class to choose.
Question: Where does proposed metric overlap with juman judgement?
Solution: influence of each word on the score of the correct relation, that impact should and does still correlate with human judgments