TASK DEFINITION: In this task, you will be presented with a context from an academic paper and a question separated with a 
. You have to answer the question based on the context.
PROBLEM: As in the Sequicity framework, we report entity match rate, BLEU score and Success F1 score.  
 Question: What were the evaluation metrics used?

SOLUTION: entity match rate BLEU score Success F1 score

PROBLEM: Experiments ::: Baselines
For comparison, we select several public models as baselines including semantic parsing models: BiDAF BIBREF3, an MRC model which utilizes a bi-directional attention flow network to encode the question and passage;

QANet BIBREF12, which utilizes convolutions and self-attentions as the building blocks of encoders to represent the question and passage;

BERT BIBREF23, a pre-trained bidirectional Transformer-based language model which achieves state-of-the-art performance on lots of public MRC datasets recently;

and numerical MRC models: NAQANet BIBREF6, a numerical version of QANet model.

NAQANet+, an enhanced version of NAQANet implemented by ourselves, which further considers real number (e.g. “2.5”), richer arithmetic expression, data augmentation, etc. Syn Dep BIBREF6, the neural semantic parsing model (KDG) BIBREF22 with Stanford dependencies based sentence representations;

OpenIE BIBREF6, KDG with open information extraction based sentence representations;

SRL BIBREF6, KDG with semantic role labeling based sentence representations;

and traditional MRC models: 
 Question: what are the existing models they compared with?

SOLUTION: Syn Dep OpenIE SRL BiDAF QANet BERT NAQANet NAQANet+

PROBLEM: We define the Effective Word Score of score x as

EFWS(x) = N(+x) - N(-x),

where N(x) is the number of words in the tweet with polarity score x. 
 Question: How is effective word score calculated?

SOLUTION:
We define the Effective Word Score of score x as

EFWS(x) = N(+x) - N(-x),

where N(x) is the number of words in the tweet with polarity score x.