In this task, you will be presented with a context from an academic paper and a question separated with a 
. You have to answer the question based on the context.

Q: Tests and Gold-Standard Data for Intrinsic Evaluation
Using the gold standard data (described below), we performed three types of tests:

Class Membership Tests: embeddings corresponding two member of the same semantic class (e.g. “Months of the Year", “Portuguese Cities", “Smileys") should be close, since they are supposed to be found in mostly the same contexts.

Class Distinction Test: this is the reciprocal of the previous Class Membership test. Embeddings of elements of different classes should be different, since words of different classes ere expected to be found in significantly different contexts.

Word Equivalence Test: embeddings corresponding to synonyms, antonyms, abbreviations (e.g. “porque" abbreviated by “pq") and partial references (e.g. “slb and benfica") should be almost equal, since both alternatives are supposed to be used be interchangeable in all contexts (either maintaining or inverting the meaning).

Therefore, in our tests, two words are considered:

distinct if the cosine of the corresponding embeddings is lower than 0.70 (or 0.80).

to belong to the same class if the cosine of their embeddings is higher than 0.70 (or 0.80).

equivalent if the cosine of the embeddings is higher that 0.85 (or 0.95). 
 Question: What intrinsic evaluation metrics are used?

A: Class Membership Tests Class Distinction Test Word Equivalence Test
****
Q: In our experiments we use the same data-sets from other works BIBREF32, BIBREF23, BIBREF33 as well as other datasets that we collected by us using a similar criterion (described in Section SECREF4). We use thirty different discussions that took place between March 2015 and June 2019, half of them with controversy and half without it. We considered discussions in four different languages: English, Portuguese, Spanish and French, occurring in five regions over the world: South and North America, Western Europe, Central and Southern Asia. We also studied these discussions taking first 140 characters and then 280 from each tweet to analyze the difference in performance and computing time wrt the length of the posts. 
 Question: What datasets did they use?

A: BIBREF32, BIBREF23, BIBREF33 discussions in four different languages: English, Portuguese, Spanish and French, occurring in five regions over the world: South and North America, Western Europe, Central and Southern Asia. 
****
Q: We have implemented the following interfaces for Macaw:

[leftmargin=*]

File IO: This interface is designed for experimental purposes, such as evaluating the performance of a conversational search technique on a dataset with multiple queries. This is not an interactive interface.

Standard IO: This interactive command line interface is designed for development purposes to interact with the system, see the logs, and debug or improve the system.

Telegram: This interactive interface is designed for interaction with real users (see FIGREF4). Telegram is a popular instant messaging service whose client-side code is open-source. We have implemented a Telegram bot that can be used with different devices (personal computers, tablets, and mobile phones) and different operating systems (Android, iOS, Linux, Mac OS, and Windows). This interface allows multi-modal interactions (text, speech, click, image). It can be also used for speech-only interactions. For speech recognition and generation, Macaw relies on online APIs, e.g., the services provided by Google Cloud and Microsoft Azure. In addition, there exist multiple popular groups and channels in Telegram, which allows further integration of social networks with conversational systems. For example, see the Naseri and Zamani's study on news popularity in Telegram BIBREF12. 
 Question: What interface does Macaw currently have?

A:
File IO Standard IO Telegram
****