In this task, you will be presented with a context from an academic paper and a question separated with a
. You have to answer the question based on the context.

Q: Compared with 2015, our QA-SRL gold yielded 1593 annotations, with 989 core and 604 adjuncts, while theirs yielded 1315 annotations, 979 core and 336 adjuncts.
Question: How much more coverage is in the new dataset?

A: 278 more annotations
****
Q: A meta-research study found only 1 in 9 qualitative papers in Human-Computer Interaction reported inter-rater reliability metrics BIBREF31. Another related area are meta-research and methods papers focused on identifying or preventing low-effort responses from crowdworkers sometimes called "spam" or "random" responses, or alternatively "fraudsters" or "cheaters." Rates of "self-agreement" are often used, determining if the same person labels the same item differently at a later stage.
Question: What are the key issues around whether the gold standard data produced in such an annotation is reliable?

A: only 1 in 9 qualitative papers in Human-Computer Interaction reported inter-rater reliability metrics low-effort responses from crowdworkers
****
Q: We introduce our proposed diversity, density, and homogeneity metrics with their detailed formulations and key intuitions.
Question: Did they propose other metrics?

A:
Yes
****