In this task, you will be presented with a context from an academic paper and a question separated with a 
. You have to answer the question based on the context.

[EX Q]: Tests and Gold-Standard Data for Intrinsic Evaluation
Using the gold standard data (described below), we performed three types of tests:

Class Membership Tests: embeddings corresponding two member of the same semantic class (e.g. “Months of the Year", “Portuguese Cities", “Smileys") should be close, since they are supposed to be found in mostly the same contexts.

Class Distinction Test: this is the reciprocal of the previous Class Membership test. Embeddings of elements of different classes should be different, since words of different classes ere expected to be found in significantly different contexts.

Word Equivalence Test: embeddings corresponding to synonyms, antonyms, abbreviations (e.g. “porque" abbreviated by “pq") and partial references (e.g. “slb and benfica") should be almost equal, since both alternatives are supposed to be used be interchangeable in all contexts (either maintaining or inverting the meaning).

Therefore, in our tests, two words are considered:

distinct if the cosine of the corresponding embeddings is lower than 0.70 (or 0.80).

to belong to the same class if the cosine of their embeddings is higher than 0.70 (or 0.80).

equivalent if the cosine of the embeddings is higher that 0.85 (or 0.95). 
 Question: What intrinsic evaluation metrics are used?
[EX A]: Class Membership Tests Class Distinction Test Word Equivalence Test

[EX Q]: . While automated evaluation metrics like ROUGE measure lexical similarity between machine and human summaries, humans can better measure how coherent and readable a summary is. Our evaluation study investigates whether tuning the PG-net model increases summary coherence, by asking evaluators to select which of three summaries for the same document they like most: the PG-net model trained on CNN/DM; the model trained on student reflections; and finally the model trained on CNN/DM and tuned on student reflections. 20 evaluators were recruited from our institution and asked to each perform 20 annotations. Summaries are presented to evaluators in random order. Evaluators are then asked to select the summary they feel to be most readable and coherent. 
 Question: Who were the human evaluators used?
[EX A]: 20 evaluators were recruited from our institution and asked to each perform 20 annotations

[EX Q]: Table TABREF44 shows average results of our automatic and human evaluations. 
 Question: How big is the difference in performance between proposed model and baselines?
[EX A]:
Metric difference between Aloha and best baseline score:
Hits@1/20: +0.061 (0.3642 vs 0.3032)
MRR: +0.0572(0.5114 vs 0.4542)
F1: -0.0484 (0.3901 vs 0.4385)
BLEU: +0.0474 (0.2867 vs 0.2393)