Teacher: In this task, you will be presented with a context from an academic paper and a question separated with a
. You have to answer the question based on the context.
Teacher: Now, understand the problem? If you are still confused, see the following example:
We evaluate the proposed approach on the Chinese social media text summarization task, based on the sequence-to-sequence model. Large-Scale Chinese Short Text Summarization Dataset (LCSTS) is constructed by BIBREF1 . The dataset consists of more than 2.4 million text-summary pairs in total, constructed from a famous Chinese social media microblogging service Weibo.
Question: Are results reported only for English data?
Solution: No
Reason: Based on the context, the dataset is constructed from a famous Chinese social media microblogging service Weibo.

Now, solve this instance: In other cases, the model points out plausible signals which were passed over by an annotator, and may be considered errors in the gold standard. For example, the model easily notices that question marks indicate the solutionhood relation, even where these were skipped by annotators in favor of marking WH words instead:

. [RGB]230, 230, 230Which [RGB]230, 230, 230previous [RGB]230, 230, 230Virginia [RGB]230, 230, 230Governor(s) [RGB]230, 230, 230do [RGB]230, 230, 230you [RGB]230, 230, 230most [RGB]230, 230, 230admire [RGB]230, 230, 230and [RGB]230, 230, 230why [RGB]12, 12, 12? $\xrightarrow[\text{pred:solutionhood}]{\text{gold:solutionhood}}$ [RGB]230, 230, 230Thomas [RGB]230, 230, 230Jefferson [RGB]183, 183, 183. However, it also picks up on a recurring tendency in how-to guides in which the second person pronoun referring to the reader is often the benefactee of some action, which contributes to the purpose reading and helps to disambiguate so, despite not being considered a signal by annotators.
Question: Where does proposed metric differ from juman judgement?
Student:
model points out plausible signals which were passed over by an annotator it also picks up on a recurring tendency in how-to guides in which the second person pronoun referring to the reader is often the benefactee of some action