instruction:
In this task, you will be presented with a context from an academic paper and a question separated with a
. You have to answer the question based on the context.
question:
To collect article-level labels, we utilized a platform in the company that has been used by the market research team to collect surveys from the subscribers of different news publishers. The survey works as follows: The user is first presented with a set of selected pages (usually 4 pages and around 20 articles) from the print paper the day before. The user can select an article each time that he or she has read, and answer some questions about it. We added 3 questions to the existing survey that asked the level of partisanship, the polarity of partisanship, and which pro- or anti- entities the article presents. We also asked the political standpoint of the user. The complete survey can be found in Appendices.
Question: Did they crowdsource the annotations?
answer:
Yes

question:
Experiments ::: Baselines
For comparison, we select several public models as baselines including semantic parsing models: BiDAF BIBREF3, an MRC model which utilizes a bi-directional attention flow network to encode the question and passage;

QANet BIBREF12, which utilizes convolutions and self-attentions as the building blocks of encoders to represent the question and passage;

BERT BIBREF23, a pre-trained bidirectional Transformer-based language model which achieves state-of-the-art performance on lots of public MRC datasets recently;

and numerical MRC models: NAQANet BIBREF6, a numerical version of QANet model.

NAQANet+, an enhanced version of NAQANet implemented by ourselves, which further considers real number (e.g. "2.5"), richer arithmetic expression, data augmentation, etc. Syn Dep BIBREF6, the neural semantic parsing model (KDG) BIBREF22 with Stanford dependencies based sentence representations;

OpenIE BIBREF6, KDG with open information extraction based sentence representations;

SRL BIBREF6, KDG with semantic role labeling based sentence representations;

and traditional MRC models:
Question: what are the existing models they compared with?
answer:
Syn Dep OpenIE SRL BiDAF QANet BERT NAQANet NAQANet+

question:
In case of polysemous words, only the first word sense (usually the most common) is taken into account.
Question: How do they handle polysemous words in their entity library?
answer:
only the first word sense (usually the most common) is taken into account