In this task, you will be presented with a context from an academic paper and a question separated with a
. You have to answer the question based on the context.

[EX Q]: Table TABREF23 shows the Pearson, Spearman and Kendall correlation of Rouge and Sera, with pyramid scores. Interestingly, we observe that many variants of Rouge scores do not have high correlations with human pyramid scores. The lowest F-score correlations are for Rouge-1 and Rouge-L (with INLINEFORM0 =0.454). Weak correlation of Rouge-1 shows that matching unigrams between the candidate summary and gold summaries is not accurate in quantifying the quality of the summary.
Question: What different correlations result when using different variants of ROUGE scores?
[EX A]: we observe that many variants of Rouge scores do not have high correlations with human pyramid scores

[EX Q]: The third person plural pronoun 'they' has no gender in English and most other languages (unlike the third person singular pronouns 'he', 'she', and 'it'). If these sentences are translated into French, then 'they' in the first sentence should be translated 'elles', as referring to Jane and Susan, and 'they' in the second sentence should be translated 'ils', as referring to Fred and George. A couple of examples: The word 'sie' in German serves as both the formal second person prounoun (always capitalized), the third person feminine singular, and the third person plural.
Question: What language do they explore?
[EX A]: English French German

[EX Q]: Analysis of challenges from UTD
Our system relies on the pseudotext produced by ZRTools (the only freely available UTD system we are aware of), which presents several challenges for MT. Assigning wrong words to a cluster
Since UTD is unsupervised, the discovered clusters are noisy. Splitting words across different clusters
Although most UTD matches are across speakers, recall of cross-speaker matches is lower than for same-speaker matches. As a result, the same word from different speakers often appears in multiple clusters, preventing the model from learning good translations. UTD is sparse, giving low coverage We found that the patterns discovered by ZRTools match only 28% of the audio. This low coverage reduces training data size, affects alignment quality, and adversely affects translation, which is only possible when pseudoterms are present.
Question: what challenges are identified?
[EX A]:
Assigning wrong words to a cluster Splitting words across different clusters sparse, giving low coverage