z-uo/qasper-squad huggingface dataset:
Paper: What we write about when we write about causality: Features of causal statements across large-scale social discourse
Data was collected from a 10% uniform sample of Twitter posts made during 2013, specifically the Gardenhose API. Twitter activity consists of short posts called tweets which are limited to 140 characters. Retweets, where users repost a tweet to spread its content, were not considered. (The spread of causal statements will be considered in future work.) We considered only English-language tweets for this study. To avoid cross-language effects, we kept only tweets with a user-reported language of `English' and, as a second constraint, individual tweets needed to match more English stopwords than any other language's set of stopwords. Stopwords considered for each language were determined using NLTK's database BIBREF29 . A tweet will be referred to as a `document' for the rest of this work.
Question: What is the source of the "control" corpus?
Answer: Randomly selected from a Twitter dump, temporally matched to causal documents
Paper: What we write about when we write about causality: Features of causal statements across large-scale social discourse
Documents were further studied by annotating their unigrams with Parts-of-Speech (POS) and Named Entities (NE) tags. POS tagging was done using NLTK v3.1 BIBREF29 which implements an averaged perceptron classifier BIBREF32 trained on the Brown Corpus BIBREF33 . (POS tagging is affected by punctuation; we show in the Appendix that our results are relatively robust to the removal of punctuation.) POS tags denote the nouns, verbs, and other grammatical constructs present in a document. Named Entity Recognition (NER) was performed using the 4-class, distributional similarity tagger provided as part of the Stanford CoreNLP v3.6.0 toolkit BIBREF34 . NER aims to identify and classify proper words in a text. The NE classifications considered were: Organization, Location, Person, and Misc. The Stanford NER tagger uses a conditional random field model BIBREF35 trained on diverse sets of manually-tagged English-language data (CoNLL-2003) BIBREF34 . Conditional random fields allow dependencies between words so that `New York' and `New York Times', for example, are classified separately as a location and organization, respectively. These taggers are commonly used and often provide reasonably accurate results, but there is always potential ambiguity in written text and improving upon these methods remains an active area of research.
Question: What are the selection criteria for "causal statements"?
Answer: Presence of only the exact unigrams 'caused', 'causing', or 'causes'
Paper: What we write about when we write about causality: Features of causal statements across large-scale social discourse
For a better understanding of the higher-order language structure present in text phrases, cause-trees were constructed. A cause-tree starts with a root cause word (either `caused', `causing' or `causes'), then the two most probable words following (preceding) the root are identified. Next, the root word plus one of the top probable words is combined into a bigram and the top two most probable words following (preceding) this bigram are found. Repeatedly applying this process builds a binary tree representing the $n$ -grams that begin with (terminate at) the root word. This process can continue until a certain $n$ -gram length is reached or until there are no more documents long enough to search.
Question: Do they use expert annotations, crowdsourcing, or only automatic methods to analyze the corpora?
Answer: Only automatic methods
Paper: What we write about when we write about causality: Features of causal statements across large-scale social discourse
Sentimental analysis was applied to estimate the emotional content of documents. Two levels of analysis were used: a method where individual unigrams were given crowdsourced numeric sentiment scores, and a second method involving a trained classifier that can incorporate document-level phrase information.
Question: how do they collect the comparable corpus?
Answer: Randomly from a Twitter dump
Paper: What we write about when we write about causality: Features of causal statements across large-scale social discourse
Lastly, we applied topic modeling to the causal corpus to determine what are the topical foci most discussed in causal statements. Topics were built from the causal corpus using Latent Dirichlet Allocation (LDA) BIBREF39 . Under LDA each document is modeled as a bag-of-words or unordered collection of unigrams. Topics are considered as mixtures of unigrams by estimating conditional distributions over unigrams: $P(w|T)$ , the probability of unigram $w$ given topic $T$ and documents are considered as mixtures of topics via $P(T|d)$ , the probability of topic $T$ given document $d$ . These distributions are then found via statistical inference given the observed distributions of unigrams across documents. The total number of topics is a parameter chosen by the practitioner. For this study we used the MALLET v2.0.8RC3 topic modeling toolkit BIBREF40 for model inference. By inspecting the most probable unigrams per topic (according to $P(w|T)$ ), we found 10 topics provided meaningful and distinct topics.
Question: How do they collect the control corpus?
Answer: Randomly from Twitter
Paper: Dataset and Neural Recurrent Sequence Labeling Model for Open-Domain Factoid Question Answering
Question answering (QA) with neural network, i.e. neural QA, is an active research direction along the road towards the long-term AI goal of building general dialogue agents BIBREF0 . Unlike conventional methods, neural QA does not rely on feature engineering and is (at least nearly) end-to-end trainable. It reduces the requirement for domain specific knowledge significantly and makes domain adaption easier. Therefore, it has attracted intensive attention in recent years.
Question: What languages do they experiment with?
Paper: Dataset and Neural Recurrent Sequence Labeling Model for Open-Domain Factoid Question Answering
In this work, we focus on open-domain factoid QA. Taking Figure FIGREF3 as an example, we formalize the problem as follows: given each question Q, we have one or more evidences E, and the task is to produce the answer A, where an evidence is a piece of text of any length that contains relevant information to answer the question. The advantage of this formalization is that evidences can be retrieved from web or unstructured knowledge base, which can improve system coverage significantly.
Question: What are the baselines?
Paper: Dataset and Neural Recurrent Sequence Labeling Model for Open-Domain Factoid Question Answering
Figure FIGREF4 shows the structure of our model. The model consists of three components: (1) question LSTM for computing question representation; (2) evidence LSTMs for evidence analysis; and (3) a CRF layer for sequence labeling. The question LSTM in a form of a single layer LSTM equipped with a single time attention takes the question as input and generates the question representation INLINEFORM0 . The three-layer evidence LSTMs takes the evidence, question representation INLINEFORM1 and optional features as input and produces “features” for the CRF layer. The CRF layer takes the “features” as input and produces the label sequence. The details will be given in the following sections.
Question: What was the inter-annotator agreement?
Paper: Dataset and Neural Recurrent Sequence Labeling Model for Open-Domain Factoid Question Answering
Following BIBREF19 , we define INLINEFORM0 as a function mapping its input INLINEFORM1 , previous state INLINEFORM2 and output INLINEFORM3 to current state INLINEFORM4 and output INLINEFORM5 : DISPLAYFORM0 
Question: Did they use a crowdsourcing platform?
Paper: Unsupervised Ranking Model for Entity Coreference Resolution
Entity coreference resolution has become a critical component for many Natural Language Processing (NLP) tasks. Systems requiring deep language understanding, such as information extraction BIBREF2 , semantic event learning BIBREF3 , BIBREF4 , and named entity linking BIBREF5 , BIBREF6 all benefit from entity coreference information.
Question: Are resolution mode variables hand crafted?
Paper: Unsupervised Ranking Model for Entity Coreference Resolution
In the following, $D = \lbrace m_0, m_1, \ldots , m_n\rbrace $ represents a generic input document which is a sequence of coreference mentions, including the artificial root mention (denoted by $m_0$ ). The method to detect and extract these mentions is discussed later in Section "Mention Detection" . Let $C = \lbrace c_1, c_2, \ldots , c_n\rbrace $ denote the coreference assignment of a given document, where each mention $m_i$ has an associated random variable $c_i$ taking values in the set $\lbrace 0, i, \ldots , i-1\rbrace $ ; this variable specifies $m_i$ 's selected antecedent ( $c_i \in \lbrace 1, 2, \ldots , i-1\rbrace $ ), or indicates that it begins a new coreference chain ( $c_i = 0$ ).
Question: What are resolution model variables?
Answer: Variables in the set {str, prec, attr} indicating in which mode the mention should be resolved.
Paper: Unsupervised Ranking Model for Entity Coreference Resolution
The following is a straightforward way to build a generative model for coreference: 
Question: Is the model presented in the paper state of the art?
Answer: No, supervised models perform better for this task.
Paper: The First Evaluation of Chinese Human-Computer Dialogue Technology
Recently, human-computer dialogue has been emerged as a hot topic, which has attracted the attention of both academia and industry. In research, the natural language understanding (NLU), dialogue management (DM) and natural language generation (NLG) have been promoted by the technologies of big data and deep learning BIBREF0 , BIBREF1 , BIBREF2 , BIBREF3 , BIBREF4 , BIBREF5 , BIBREF6 , BIBREF7 , BIBREF8 . Following the development of machine reading comprehension BIBREF9 , BIBREF10 , BIBREF11 , BIBREF12 , BIBREF13 , BIBREF14 , the NLU technology has made great progress. The development of DM technology is from rule-based approach and supervised learning based approach to reinforcement learning based approach BIBREF15 . The NLG technology is through pattern-based approach, sentence planning approach and end-to-end deep learning approach BIBREF16 , BIBREF17 , BIBREF18 . In application, there are massive products that are based on the technology of human-computer dialogue, such as Apple Siri, Amazon Echo, Microsoft Cortana, Facebook Messenger and Google Allo etc.
Question: What problems are found with the evaluation scheme?
Paper: The First Evaluation of Chinese Human-Computer Dialogue Technology
The First Evaluation of Chinese Human-Computer Dialogue Technology includes two tasks, namely user intent classification and online testing of task-oriented dialogue.
Question: How is the data annotated?
Paper: The First Evaluation of Chinese Human-Computer Dialogue Technology
In using of human-computer dialogue based applications, human may have various intent, for example, chit-chatting, asking questions, booking air tickets, inquiring weather, etc. Therefore, after receiving an input message (text or ASR result) from a user, the first step is to classify the user intent into a specific domain for further processing. Table TABREF7 shows an example of user intent with category information.
Question: What collection steps do they mention?
Paper: The First Evaluation of Chinese Human-Computer Dialogue Technology
For the task-oriented dialogue systems, the best way for evaluation is to use the online human-computer dialogue. After finishing an online human-computer dialogue with a dialogue system, the human then manually evaluate the system by using the metrics of user satisfaction degree, dialogue fluency, etc. Therefore, in the task 2, we use an online testing of task-oriented dialogue for dialogue systems. For a human tester, we will give a complete intent with an initial sentence, which is used to start the online human-computer dialogue. Table TABREF12 shows an example of the task-oriented human-computer dialogue. Here “U” and “R” denote user and robot respectively. The complete intent is as following:
Question: How many intents were classified?
Paper: The First Evaluation of Chinese Human-Computer Dialogue Technology
In the evaluation, all the data for training, developing and test is provided by the iFLYTEK Corporation.
Question: What was the result of the highest performing system?
Answer: For task 1 best F1 score was 0.9391 on closed and 0.9414 on open test.
For task2 best result had: Ratio 0.3175 , Satisfaction 64.53, Fluency 0, Turns -1 and Guide 2
Paper: The First Evaluation of Chinese Human-Computer Dialogue Technology
There are 74 participants who are signing up the evaluation. The final number of participants is 28 and the number of submitted systems is 43. Table TABREF14 and TABREF15 show the evaluation results of the closed test and open test of the task 1 respectively. Due to the space limitation, we only present the top 5 results of task 1. We will add the complete lists of the evaluation results in the version of full paper.
Question: What metrics are used in the evaluation?
Paper: Multi-style Generative Reading Comprehension
Question answering has been a long-standing research problem. Recently, reading comprehension (RC), a challenge to answer a question given textual evidence provided in a document set, has received much attention. Here, current mainstream studies have treated RC as a process of extracting an answer span from one passage BIBREF0 , BIBREF1 or multiple passages BIBREF2 , which is usually done by predicting the start and end positions of the answer BIBREF3 , BIBREF4 .
Question: How do they measure the quality of summaries?
Paper: Multi-style Generative Reading Comprehension
The task considered in this paper, is defined as:
Question: Does their model also take the expected answer style as input?
Paper: Multi-style Generative Reading Comprehension
Our proposed model, Masque, is based on multi-source abstractive summarization; the answer our model generates can be viewed as a summary from the question and multiple passages. It is also style-controllable; one model can generate the answer with the target style.
Question: What do they mean by answer styles?
Answer: well-formed sentences vs concise answers
Paper: Multi-style Generative Reading Comprehension
Given a question and passages, the question-passages reader matches them so that the interactions among the question (passage) words conditioned on the passages (question) can be captured.
Question: Is there exactly one "answer style" per dataset?
Paper: Multi-style Generative Reading Comprehension
The passage ranker maps the output of the modeling layer, $\lbrace M^{p_k}\rbrace $ , to the relevance score of each passage. To obtain a fixed-dimensional pooled representation of each passage sequence, this layer takes the output for the first passage word, $M^{p_k}_1$ , which corresponds to the beginning-of-sentence token. It calculates the relevance of each $k$ -th passage to the question as: 
Question: What are the baselines that Masque is compared against?
Answer: BiDAF, Deep Cascade QA, S-Net+CES2S, BERT+Multi-PGNet, Selector+CCG, VNET, DECAPROP, MHPGM+NOIC, ConZNet, RMR+A2D
Paper: Multi-style Generative Reading Comprehension
The answer possibility classifier maps the output of the modeling layer, $\lbrace M^{p_k}\rbrace $ , to the probability of the answer possibility. The classifier takes the output for the first word, $M^{p_k}_1$ , for all passages and concatenates them to obtain a fixed-dimensional representation. It calculates the answer possibility to the question as: 
Question: What is the performance achieved on NarrativeQA?
Answer: Bleu-1: 54.11, Bleu-4: 30.43, METEOR: 26.13, ROUGE-L: 59.87
Paper: Multi-style Generative Reading Comprehension
Given the outputs provided by the reader, the decoder generates a sequence of answer words one element at a time. It is auto-regressive BIBREF24 , consuming the previously generated words as additional input at each decoding step.
Question: What is an "answer style"?
Answer: well-formed sentences vs concise answers
Paper: A Cascade Sequence-to-Sequence Model for Chinese Mandarin Lip Reading
Lip reading, also known as visual speech recognition, aims to predict the sentence being spoken, given a silent video of a talking face. In noisy environments, where speech recognition is difficult, visual speech recognition offers an alternative way to understand speech. Besides, lip reading has practical potential in improved hearing aids, security, and silent dictation in public spaces. Lip reading is essentially a difficult problem, as most lip reading actuations, besides the lips and sometimes tongue and teeth, are latent and ambiguous. Several seemingly identical lip movements can produce different words.
Question: What was the previous state of the art model for this task?
Paper: A Cascade Sequence-to-Sequence Model for Chinese Mandarin Lip Reading
In this section, we present CSSMCM, a lip reading model for Chinese Mandarin. As mention in Section SECREF1 , pinyin and tone are both important for Chinese Mandarin lip reading. Pinyin represents how to pronounce a Chinese character and is related to mouth movement. Tone can alleviate the ambiguity of visemes (several speech sounds that look the same) to some extent and can be inferred from visible movements. Based on this, the lip reading task is defined as follow: DISPLAYFORM0 
Question: What syntactic structure is used to model tones?
Paper: A Cascade Sequence-to-Sequence Model for Chinese Mandarin Lip Reading
The pinyin prediction sub-network transforms video sequence into pinyin sequence, which corresponds to INLINEFORM0 in Equation ( EQREF6 ). This sub-network is based on the sequence-to-sequence architecture with attention mechanism BIBREF8 . We name the encoder and decoder the video encoder and pinyin decoder, for the encoder process video sequence, and the decoder predicts pinyin sequence. The input video sequence is first fed into the VGG model BIBREF9 to extract visual feature. The output of conv5 of VGG is appended with global average pooling BIBREF10 to get the 512-dim feature vector. Then the 512-dim feature vector is fed into video encoder. The video encoder can be denoted as: DISPLAYFORM0 
Question: What visual information characterizes tones?
Paper: Dissecting Content and Context in Argumentative Relation Analysis
In recent years we have witnessed a great surge in activity in the area of computational argument analysis (e.g. BIBREF0 , BIBREF1 , BIBREF2 , BIBREF3 ), and the emergence of dedicated venues such as the ACL Argument Mining workshop series starting in 2014 BIBREF4 .
Question: Do they report results only on English data?
Paper: Dissecting Content and Context in Argumentative Relation Analysis
It is well-known that the rhetorical and argumentative structure of texts bear great similarities. For example, BIBREF5 , BIBREF6 , BIBREF0 observe that elementary discourse units (EDUs) in RST BIBREF7 share great similarity with elementary argumentative units (EAUs) in argumentation analysis. BIBREF8 experiment with a modified version of the Microtext corpus BIBREF9 , which is an extensively annotated albeit small corpus. Similar to us, they separate argumentative units from discursive contextual markers. While BIBREF8 conduct a human evaluation to investigate the separation of Logos and Pathos aspects of arguments, our work investigates how (de-)contextualization of argumentative units affects automatic argumentative relation classification models.
Question: How do they demonstrate the robustness of their results?
Paper: Dissecting Content and Context in Argumentative Relation Analysis
In this section, we describe different formulations of the argumentative relation classification task and describe features used by our replicated model. In order to test our hypotheses, we propose to group all features into three distinct types.
Question: What baseline and classification systems are used in experiments?
Paper: Dissecting Content and Context in Argumentative Relation Analysis
Now, we introduce a classification of three different prediction models used in the argumentative relation prediction literature. We will inspect all of them and show that all can suffer from severe issues when focusing (too much) on the context.
Question: How are the EAU text spans annotated?
Answer: Answer with content missing: (Data and pre-processing section) The data is suited for our experiments because the annotators were explicitly asked to provide annotations on a clausal level.
Paper: Dissecting Content and Context in Argumentative Relation Analysis
Our feature implementation follows the feature descriptions for Stance recognition and link identification in BIBREF13 . These features and variations of them have been used successfully in several successive works (cf. BIBREF1 , BIBREF16 , BIBREF15 ).
Question: How are elementary argumentative units defined?
Paper: Gibberish Semantics: How Good is Russian Twitter in Word Semantic Similarity Task?
Word semantic similarity task is an important part of contemporary NLP. It can be applied in many areas, like word sense disambiguation, information retrieval, information extraction and others. It has long history of improvements, starting with simple models, like bag-of-words (often weighted by TF-IDF score), continuing with more complex ones, like LSA BIBREF0 , which attempts to find “latent” meanings of words and phrases, and even more abstract models, like NNLM BIBREF1 . Latest results are based on neural network experience, but are far more simple: various versions of Word2Vec, Skip-gram and CBOW models BIBREF2 , which currently show the State-of-the-Art results and have proven success with morphologically complex languages like Russian BIBREF3 , BIBREF4 .
Question: Which Twitter corpus was used to train the word vectors?
Answer: They collected tweets in Russian language using a heuristic query specific to Russian