TASK DEFINITION: In this task, you will be presented with a context from an academic paper and a question separated with a
. You have to answer the question based on the context.
PROBLEM: To study the lexical and semantic diversities of responses, we performed three analyses. First, we aggregated all worker responses to a particular question into a single list corresponding to that question. Second, we compared the diversity of individual responses between Control and AUI for each question. To measure diversity for a question, we computed the number of responses divided by the number of unique responses to that question. We call this the response density.
Question: How was lexical diversity measured?

SOLUTION: By computing number of unique responses and number of responses divided by the number of unique responses to that question for each of the questions

PROBLEM: Fenerbahce We have decided to consider tweets about popular sports clubs as our domain for stance detection. Hence we have determined our targets as Galatasaray (namely Target-1) and Fenerbahce (namely, Target-2) which are two of the most popular football clubs in Turkey. Then, we have annotated the stance information in the tweets for these targets as Favor or Against. For the purposes of the current study, we have not annotated any tweets with the Neither class.
Question: How were the tweets annotated?

SOLUTION: tweets are annotated with only Favor or Against for two targets - Galatasaray and Fenerbahce

PROBLEM: The full dataset comprises both the sample mentioned above and the 819 pairs from our preliminary work, totalling 2677 pairs. We split the vSTS dataset into training, validation and test partitions sampling at random and preserving the overall score distributions. In total, we use 1338 pairs for training, 669 for validation, and the rest of the 670 pairs were used for the final testing.
Question: How big is vSTS training data?

SOLUTION:
1338 pairs for training