niryuu/sni-each-converted huggingface dataset:
We train language models on two languages: One model is estimated on the German newscrawl distributed by WMT'18 comprising 260M sentences or 6B tokens. Another model is trained on the English newscrawl data comprising 193M sentences or 5B tokens. We learn a joint Byte-Pair-Encoding (BPE; Sennrich et al., 2016) vocabulary of 37K types on the German and English newscrawl and train the language models with this vocabulary. We consider two benchmarks: Most experiments are run on the WMT'18 English-German (en-de) news translation task and we validate our findings on the WMT'18 English-Turkish (en-tr) news task. For WMT'18 English-German, the training corpus consists of all available bitext excluding the ParaCrawl corpus and we remove sentences longer than 250 tokens as well as sentence-pairs with a source/target length ratio exceeding 1.5. This results in 5.18M sentence pairs. We tokenize all data with the Moses tokenizer BIBREF8 and apply the BPE vocabulary learned on the monolingual corpora. 
 Question: What dataset do they use?
Output: Extractive
On the other hand, Go-Explore Seq2Seq shows promising results by solving almost half of the unseen games. Figure FIGREF62 (in Appendix SECREF60) shows that most of the lost games are in the hardest set, where a very long sequence of actions is required for winning the game. These results demonstrate both the relative effectiveness of training a Seq2Seq model on Go-Explore trajectories, but they also indicate that additional effort needed for designing reinforcement learning algorithms that effectively generalize to unseen games. 
 Question: How do the authors show that their learned policy generalize better than existing solutions to unseen games?
Output: Extractive
To be able to employ the trained model, test sets are first translated to English via machine translation and then inference takes place.   For machine translation, Google translation API is used. 
 Question: how did the authors translate the reviews to other languages?
Output: Abstractive
The dataset consists of 5000 reviews in bahasa Indonesia. 
 Question: Does the dataset contain non-English reviews?
Output: Yes-no
We choose the state-of-the-art Transformer BIBREF1 model and the conventional RNN-Search model BIBREF0 as our test bed. 
 Question: Which model architectures do they test their word importance approach on?
Output: Extractive
Using DSC loss improves the F1 score by +0.58 for MRPC and +0.73 for QQP. 
 Question: What are method improvements of F1 for paraphrase identification?
Output: Extractive
We experiment with five benchmark attacking methods for texts: FGM, FGVM, DeepFool BIBREF5, HotFlip BIBREF3) and TYC BIBREF4. 
 Question: What are the benchmark attacking methods?
Output: Extractive
We use the annotations from experts for an abstract if it exists otherwise use crowd annotations.  
 Question: How do they match annotators to instances?
Output: Abstractive
We focus primarily on the word vector representations (word embeddings) created specifically using the twitter dataset. GloVe BIBREF13 is an unsupervised learning algorithm for obtaining vector representations for words. 200-dimensional GloVe embeddings trained on 2 Billion tweets are integrated. Edinburgh embeddings BIBREF14 are obtained by training skip-gram model on Edinburgh corpus BIBREF15 . Since tweets are abundant with emojis, Emoji embeddings BIBREF16 which are learned from the emoji descriptions have been used. Embeddings for each tweet are obtained by summing up individual word vectors and then dividing by the number of tokens in the tweet. 
 Question: what pretrained word embeddings were used?
Output: Abstractive
For instance, a message can be regarded as harmless on its own, but when taking previous threads into account it may be seen as abusive, and vice versa. This aspect makes detecting abusive language extremely laborious even for human annotators; therefore it is difficult to build a large and reliable dataset BIBREF10 . 
 Question: What examples of the difficulties presented by the context-dependent nature of online aggression do they authors give?
Output: Extractive
We initialized the embeddings of these words with 300 dimensional Glove embeddings BIBREF31 .  
 Question: Do they use pretrained embeddings?
Output: Yes-no
Across all languages, 145 human annotators were asked to score all 1,888 pairs (in their given language). We finally collect at least ten valid annotations for each word pair in each language. All annotators were required to abide by the following instructions:

1. Each annotator must assign an integer score between 0 and 6 (inclusive) indicating how semantically similar the two words in a given pair are. A score of 6 indicates very high similarity (i.e., perfect synonymy), while zero indicates no similarity.

2. Each annotator must score the entire set of 1,888 pairs in the dataset. The pairs must not be shared between different annotators.

3. Annotators are able to break the workload over a period of approximately 2-3 weeks, and are able to use external sources (e.g. dictionaries, thesauri, WordNet) if required.

4. Annotators are kept anonymous, and are not able to communicate with each other during the annotation process. 
 Question: How were the datasets annotated?
Output: Extractive
As before, the training and test sets contain some 30,000 and 5,000 sentence pairs, respectively 
 Question: How many samples did they generate for the artificial language?
Output: Abstractive
Next, we compared our VTN model with an RNN-based seq2seq VC model called ATTS2S BIBREF8. This model is based on the Tacotron model BIBREF32 with the help of context preservation loss and guided attention loss to stabilize training and maintain linguistic consistency after conversion. We followed the configurations in BIBREF8 but used mel spectrograms instead of WORLD features. 
 Question: What is the baseline model?
Output: Abstractive
Baselines. We use one strong non-DNN baseline, NBSVM (with unigrams or bigrams features) BIBREF23 and six DNN baselines. The first DNN baseline is CNN BIBREF25, which does not handle noisy labels. The other five were designed to handle noisy labels.

The comparison results are shown in Table TABREF12. From the results, we can make the following observations. (1) Our NetAb model achieves the best ACC and F1 on all datasets except for F1 of negative class on Laptop. 
 Question: Is the model evaluated against a CNN baseline?
Output: Yes-no
We collect data by using the Twitter API for saved data, available for public use. For our experiments we collect 3200 tweets filtered by keywords like “fire”, “earthquake”, “theft”, “robbery”, “drunk driving”, “drunk driving accident” etc. Later, we manually label tweets with and labels for classification as stage one. Our dataset contains 1313 tweet with positive label and 1887 tweets with a negative label . We create another dataset with the positively labeled tweets and provide them with category labels like “fire”, “accident”, “earthquake” etc. 
 Question: Are the tweets specific to a region?
Output: Yes-no
We apply our adaptively sparse Transformers on four machine translation tasks.  IWSLT 2017 German $\rightarrow $ English BIBREF27: 200K sentence pairs.

KFTT Japanese $\rightarrow $ English BIBREF28: 300K sentence pairs.

WMT 2016 Romanian $\rightarrow $ English BIBREF29: 600K sentence pairs.

WMT 2014 English $\rightarrow $ German BIBREF30: 4.5M sentence pairs. 
 Question: What tasks are used for evaluation?
Output: Abstractive
BIBREF6 introduce the KG-A2C, which uses a knowledge graph based state-representation to aid in the section of actions in a combinatorially-sized action-space—specifically they use the knowledge graph to constrain the kinds of entities that can be filled in the blanks in the template action-space. They test their approach on Zork1, showing the combination of the knowledge graph and template action selection resulted in improvements over existing methods. They note that their approach reaches a score of 40 which corresponds to a bottleneck in Zork1 where the player is eaten by a “grue” (resulting in negative reward) if the player has not first lit a lamp. 
 Question: What are the baselines?
Output: Extractive
In this section, we evaluate our method and compare its performance against the competitive approaches. We use INLINEFORM0 -fold evaluation protocol with INLINEFORM1 with random dataset split. We measure the performance using standard accuracy metric which we define as a ratio between correctly classified data samples from test dataset and all test samples. 
 Question: What evaluation metrics are used?
Output: Extractive
Thus we propose two kinds of relation-specific meta information: relation meta and gradient meta corresponding to afore mentioned two perspectives respectively. In our proposed framework MetaR, relation meta is the high-order representation of a relation connecting head and tail entities. Gradient meta is the loss gradient of relation meta which will be used to make a rapid update before transferring relation meta to incomplete triples during prediction. 
 Question: What meta-information is being transferred?
Output: Abstractive
A higher score indicates better step ordering (with a maximum score of 2). tab:coherencemetrics shows that our personalized models achieve average recipe-level coherence scores of 1.78-1.82, surpassing the baseline at 1.77. On average, human evaluators preferred personalized model outputs to baseline 63% of the time, confirming that personalized attention improves the semantic plausibility of generated recipes. 
 Question: What were their results on the new dataset?
Output: Extractive
$D_a$ contained all of the tweets collected on the attack day of the five attacks mentioned in section 4.2. And $D_b$ contained all of the tweets collected before the five attacks. There are 1180 tweets in $D_a$ and 7979 tweets in $D_b$. The tweets on the attack days ($D_a$) are manually annotated and only 50 percent of those tweets are actually about a DDoS attack. 
 Question: Do twitter users tend to tweet about the DOS attack when it occurs? How much data supports this assumption?
Output: Abstractive
We therefore use reduction in softmax probability of the correct relation as our signaling strength metric for the model. We call this metric ${\Delta }_s$ (for delta-softmax), which can be written as:

where $rel$ is the true relation of the EDU pair, $t_i$ represents the token at index $i$ of $N$ tokens, and $X_{mask=i}$ represents the input sequence with the masked position $i$ (for $i \in 1 \ldots N$ ignoring separators, or $\phi $, the empty set). 
 Question: How is the delta-softmax calculated?
Output: Abstractive
(2) Different from LEM and DPEMM, AEM uses a generator network to capture the event-related patterns and is able to mine events from different text sources (short and long). Moreover, unlike traditional inference procedure, such as Gibbs sampling used in LEM and DPEMM, AEM could extract the events more efficiently due to the CUDA acceleration; 
 Question: What alternative to Gibbs sampling is used?
Output: Extractive
To tackle this issue we applying a back-translation method BIBREF13, where we translate indirect and physical harassment tweets of the train set from english to german, french and greek. After that, we translate them back to english in order to achieve data augmentation. 
 Question: What language(s) is/are represented in the dataset?
Output: Extractive
Maximum matching (MM) is one of the most popular fundamental and structural segmentation algorithms for word segmentation BIBREF19 .  However, it does not solve the problem of ambiguous words and unknown words that do not exist in the dictionary. 
 Question: What are the limitations of existing Vietnamese word segmentation systems?
Output: Extractive
To further improve the performance, we adopt system combination on the decoding lattice level. By combining systems, we can take advantage of the strength of each model that is optimized for different domains.  The best result for vlsp2018 of 4.85% WER is obtained by the combination weights 0.6:0.4 where 0.6 is given to the general language model and 0.4 is given to the conversation one. On the vlsp2019 set, the ratio is change slightly by 0.7:0.3 to deliver the best result of 15.09%. 
 Question: What is the language model combination technique used in the paper?
Output: Extractive
I also performed several case studies. I obtained document embeddings, in the same latent space as the topic embeddings, by summing the posterior mean vectors INLINEFORM0 for each token, and visualized them in two dimensions using INLINEFORM1 -SNE BIBREF24 (all vectors were normalized to unit length). The state of the Union addresses (Figure FIGREF27 ) are embedded almost linearly by year, with a major jump around the New Deal (1930s), and are well separated by party at any given time period.  
 Question: What is an example of a computational social science NLP task?
Output: Abstractive
We also evaluate all five models on downstream tasks from the VecEval suite BIBREF13 , using only the tasks for which training and evaluation data is freely available: chunking, sentiment and question classification, and natural language identification (NLI). The default settings from the suite are used, but we run only the fixed settings, where the embeddings themselves are not tunable parameters of the models, forcing the system to use only the information already in the embeddings.

Finally, we use LV-N, LV-M, and FT to generate OOV word representations for the following words: 1) “hellooo”: a greeting commonly used in instant messaging which emphasizes a syllable. 2) “marvelicious”: a made-up word obtained by merging “marvelous” and “delicious”. 3) “louisana”: a misspelling of the proper name “Louisiana”. 4) “rereread”: recursive use of prefix “re”. 5) “tuzread”: made-up prefix “tuz”. 
 Question: How do they evaluate their resulting word embeddings?
Output: Extractive
In this section, we propose new simple disentanglement models that perform better than prior methods, and re-examine prior work. 
 Question: Did they experiment with the corpus?
Output: Yes-no
For comparison, we also report the performance of the error detection system by Rei2016, trained using the same FCE dataset. 
 Question: What was the baseline used?
Output: Extractive
Also, we compared our models with other existing works on this dataset including OpATT BIBREF6 and Neural Content Planning with conditional copy (NCP+CC) BIBREF4. 
 Question: What is the state-of-the-art model for the task?
Output: Extractive
We evaluate our model and baseline models on three versions of the dataset. The first one (Inc) only considers the original data, containing naturally incorrect tweets, and achieves accuracy of 80$\%$ against BERT's 72$\%$. The second version (Corr) considers the corrected tweets, and shows higher accuracy given that it is less noisy.  
 Question: Should their approach be applied only when dealing with incomplete data?
Output: Yes-no
For the two single sentence tasks—the syntax-oriented CoLA task and the SST sentiment task—we find somewhat deteriorated performance. 
 Question: Does the additional training on supervised tasks hurt performance in some tasks?
Output: Yes-no
For all tasks, we use a TensorFlow implementation. AraNet predicts age, dialect, gender, emotion, irony, and sentiment from social media posts. It delivers state-of-the-art and competitive performance on these tasks and has the advantage of using a unified, simple framework based on the recently-developed BERT model.  
 Question: Did they experiment on all the tasks?
Output: Yes-no
Our official scores (column Ens Test in Table TABREF19 ) have placed us second (EI-Reg, EI-Oc), fourth (V-Reg) and fifth (V-Oc) on the SemEval AIT-2018 leaderboard. 
 Question: What were the scores of their system?
Output: Extractive
The dataset consists of queries and the corresponding image search results. Each token in each query is given a language tag based on the user-set home language of the user making the search on Google Images. 
 Question: Do the images have multilingual annotations or monolingual ones?
Output: Abstractive
For the NER task, we consider both Chinese datasets, i.e., OntoNotes4.0 BIBREF34 and MSRA BIBREF35, and English datasets, i.e., CoNLL2003 BIBREF36 and OntoNotes5.0 BIBREF37. Table shows experimental results on NER datasets. For English datasets including CoNLL2003 and OntoNotes5.0, our proposed method outperforms BERT-MRCBIBREF38 by +0.29 and +0.96 respectively. We observe huge performance boosts on Chinese datasets, achieving F1 improvements by +0.97 and +2.36 on MSRA and OntoNotes4.0, respectively. As far as we are concerned, we are setting new SOTA performances on all of the four NER datasets. 
 Question: What are method's improvements of F1 for NER task for English and Chinese datasets?
Output: Extractive
We introduce a surrogate training objective that avoids these problems and as a result is fully continuous.  Specifically, we form a continuous function softLB that seeks to approximate the result of running our decoder on input INLINEFORM0 and then evaluating the result against INLINEFORM1 using INLINEFORM2 . 
 Question: Do they provide a framework for building a sub-differentiable for any final loss metric?
Output: Yes-no
Table TABREF19 displays the performance of the 4 baselines on the ReviewQA's test set. These results are the performance achieved by our own implementation of these 4 models. 
 Question: What tasks were evaluated?
Output: Extractive
It may be difficult to spot patterns by just looking at a collection of images. Another method is to tag all descriptions with part-of-speech information, so that it becomes possible to see e.g. which adjectives are most commonly used for particular nouns. One method readers may find particularly useful is to leverage the structure of Flickr30K Entities BIBREF8 . Following this, I applied Louvain clustering BIBREF9 to the coreference graph, resulting in clusters of expressions that refer to similar entities. 
 Question: Which methods are considered to find examples of biases and unwarranted inferences??
Output: Extractive
Results on LangID and NoLangID are compared to the system presented by deri2016grapheme, which is identified in our results as wFST. 
 Question: what was the baseline?
Output: Extractive
According to the categories mentioned in BIBREF5 , irony can be classified into 3 classes: verbal irony by means of a polarity contrast, the sentences containing expression whose polarity is inverted between the intended and the literal evaluation; other types of verbal irony, the sentences that show no polarity contrast between the literal and intended meaning but are still ironic; and situational irony, the sentences that describe situations that fail to meet some expectations. As ironies in the latter two categories are obscure and hard to understand, we decide to only focus on ironies in the first category in this work.   Because of the lack of previous work and baselines on irony generation, we implement our model based on style transfer.  
 Question: What are the difficulties in modelling the ironic pattern?
Output: Extractive
In total around 500 different workers were involved in the annotation. about 500 
 Question: How many people participated in their evaluation study of table-to-text models?
Output: Abstractive
The answer styles considered in the experiments corresponded to the two tasks. The NLG task requires a well-formed answer that is an abstractive summary of the question and ten passages, averaging 16.6 words. The Q&A task also requires an abstractive answer but prefers a more concise answer than the NLG task, averaging 13.1 words, where many of the answers do not contain the context of the question. 
 Question: Is there exactly one "answer style" per dataset?
Output: Yes-no
We work with a dataset composed of 3,206 news articles, each labeled real or fake, with a perfect 50/50 split between 1,603 real and fake articles, respectively. Fake articles were sourced from online sites that were tagged as fake news sites by the non-profit independent media fact-checking organization Verafiles and the National Union of Journalists in the Philippines (NUJP). Real articles were sourced from mainstream news websites in the Philippines, including Pilipino Star Ngayon, Abante, and Bandera. 
 Question: What is the source of the dataset?
Output: Abstractive
In the sixth international workshop on Vietnamese Language and Speech Processing (VLSP 2019), the Hate Speech Detection (HSD) task is proposed as one of the shared-tasks to handle the problem related to controlling content in SNSs. The fundamental idea of this system is how to make a system that has the diversity of viewing an input. That because of the variety of the meaning in Vietnamese language especially with the acronym, teen code type. 
 Question: Is the data all in Vietnamese?
Output: Yes-no
Finally, the morphosyntactic annotation was performed automatically, which may lead to errors, especially in the case of noisy web text. 
 Question: Did they use a crowdsourcing platform for annotations?
Output: Yes-no
First, we simply calculate the average unique predictions produced by both INLINEFORM0 and INLINEFORM1 in experiments shown in Section SECREF36 . 
 Question: What two metrics are proposed?
Output: Extractive
Each classifier is implemented with the following specifications:

Naïve Bayes (NB): Multinomial NB with additive smoothing constant 1

Logistic Regression (LR): Linear LR with L2 regularization constant 1 and limited-memory BFGS optimization

Support Vector Machine (SVM): Linear SVM with L2 regularization constant 1 and logistic loss function

Random Forests (RF): Averaging probabilistic predictions of 10 randomized decision trees

Gradient Boosted Trees (GBT): Tree boosting with learning rate 1 and logistic loss function Along with traditional machine learning approaches, we investigate neural network based models to evaluate their efficacy within a larger dataset. In particular, we explore Convolutional Neural Networks (CNN), Recurrent Neural Networks (RNN), and their variant models. 
 Question: What learning models are used on the dataset?
Output: Extractive
To address this drawback in ROUGE, we propose a new evaluation metric: Critical Information Completeness (CIC). Formally, CIC is a recall of semantic slot information between a candidate summary and a reference summary. CIC is defined as follows:

where $V$ stands for a set of delexicalized values in the reference summary, $Count_{match}(v)$ is the number of values co-occurring in the candidate summary and reference summary, and $m$ is the number of values in set $V$. In our experiments, CIC is computed as the arithmetic mean over all the dialog domains to retain the overall performance.

CIC is a suitable complementary metric to ROUGE because it accounts for the most important information within each dialog domain. CIC can be applied to any summarization task with predefined essential entities. 
 Question: How does new evaluation metric considers critical informative entities?
Output: Abstractive
However, recent work has found that many NLI datasets contain biases, or annotation artifacts, i.e., features present in hypotheses that enable models to perform surprisingly well using only the hypothesis, without learning the relationship between two texts BIBREF2 , BIBREF3 , BIBREF4 . For instance, in some datasets, negation words like “not” and “nobody” are often associated with a relationship of contradiction. As a ramification of such biases, models may not generalize well to other datasets that contain different or no such biases. 
 Question: Is such bias caused by bad annotation?
Output: Yes-no
Fenerbahçe We have decided to consider tweets about popular sports clubs as our domain for stance detection.  Hence we have determined our targets as Galatasaray (namely Target-1) and Fenerbahçe (namely, Target-2) which are two of the most popular football clubs in Turkey.  Then, we have annotated the stance information in the tweets for these targets as Favor or Against. For the purposes of the current study, we have not annotated any tweets with the Neither class. 
 Question: How were the tweets annotated?
Output: Abstractive
Datasets
For training English-Hindi NMT systems, we use the IITB English-Hindi parallel corpus BIBREF22 ( INLINEFORM0 sentences from the training set) and the ILCI English-Hindi parallel corpus ( INLINEFORM1 sentences). The ILCI (Indian Language Corpora Initiative) multilingual parallel corpus BIBREF23 spans multiple Indian languages from the health and tourism domains. We use the 520-sentence dev-set of the IITB parallel corpus for validation. For each child task, we use INLINEFORM2 sentences from ILCI corpus as the test set. 
 Question: Which dataset(s) do they experiment with?
Output: Extractive
Hypothesis 4 (H4) : The proposed template-based synthesis model outperforms a simple word replacement model. Table TABREF13 supports H4 by showing that the proposed synthesis model outperforms the WordNet baseline in training (rows 7, 8 and 19, 20) except Stat2016, and tuning (10, 11 and 22, 23) over all courses. It also shows that while adding synthetic data from the baseline is not always helpful, adding synthetic data from the template model helps to improve both the training and the tuning process. In both CS and ENGR courses, tuning with synthetic data enhances all ROUGE scores compared to tuning with only the original data. (rows 9 and 11). As for Stat2015, R-1 and R-$L$ improved, while R-2 decreased. For Stat2016, R-2 and R-$L$ improved, and R-1 decreased (rows 21 and 23). Training with both student reflection data and synthetic data compared to training with only student reflection data yields similar improvements, supporting H3 (rows 6, 8 and 18, 20). 
 Question: Is the template-based model realistic?  
Output: Yes-no
The quality class of a Wikipedia article is assigned by Wikipedia reviewers or any registered user, who can discuss through the article's talk page to reach consensus. The arXiv dataset BIBREF2 consists of three subsets of academic articles under the arXiv repository of Computer Science (cs), from the three subject areas of: Artificial Intelligence (cs.ai), Computation and Language (cs.cl), and Machine Learning (cs.lg). In line with the original dataset formulation BIBREF2 , a paper is considered to have been accepted (i.e. is positively labeled) if it matches a paper in the DBLP database or is otherwise accepted by any of the following conferences: ACL, EMNLP, NAACL, EACL, TACL, NIPS, ICML, ICLR, or AAAI. Failing this, it is considered to be rejected (noting that some of the papers may not have been submitted to one of these conferences). 
 Question: Where do they get their ground truth quality judgments?
Output: Extractive
We use a benchmark dataset created by ceccarelli2013learning from the CoNLL 2003 data.  
 Question: What is the benchmark dataset?
Output: Extractive
In addition, we evaluate the effectiveness of pre-training by comparing it with a jointly-trained model of forward translation and back-translation. Experimental results show that the encoder-decoder-reconstructor offers significant improvement in BLEU scores and alleviates the problem of repeating and missing words in the translation on English-Japanese translation task, and the encoder-decoder-reconstructor can not be trained well without pre-training, so it proves that we have to train the forward translation model in a manner similar to the conventional attention-based NMT as pre-training. 
 Question: Is pre-training effective in their evaluation?
Output: Yes-no
Maximum Entropy theory is applied to solve Vietnamese word segmentation BIBREF15 , BIBREF22 , BIBREF23 . There are several studies about the Vietnamese word segmentation task over the last decade. Dinh et al. started this task with Weighted Finite State Transducer (WFST) approach and Neural Network approach BIBREF9 . In addition, machine learning approaches are studied and widely applied to natural language processing and word segmentation as well. In fact, several studies used support vector machines (SVM) and conditional random fields (CRF) for the word segmentation task BIBREF7 , BIBREF8 . 
 Question: Which approaches have been applied to solve word segmentation in Vietnamese?
Output: Extractive
We use a new dataset of GD statements from 1970 to 2016, the UN General Debate Corpus (UNGDC), to examine the international development agenda in the UN BIBREF3 .  
 Question: Is the dataset multilingual?
Output: Yes-no
As can be seen, the proposed DSC loss outperforms the best baseline results by a large margin, i.e., outperforming BERT-tagger by +1.86 in terms of F1 score on CTB5, +1.80 on CTB6 and +2.19 on UD1.4. 
 Question: What are method's improvements of F1 w.r.t. baseline BERT tagger for Chinese POS datasets?
Output: Extractive
Considering the improvements over the majority baseline achieved by the RNN model for both non-English (on the average 22.76% relative improvement; 15.82% relative improvement on Spanish, 72.71% vs. 84.21%, 30.53% relative improvement on Turkish, 56.97% vs. 74.36%, 37.13% relative improvement on Dutch, 59.63% vs. 81.77%, and 7.55% relative improvement on Russian, 79.60% vs. 85.62%) and English test sets (27.34% relative improvement), we can draw the conclusion that our model is robust to handle multiple languages. 
 Question: which non-english language was the had the worst results?
Output: Extractive
This allows Macaw to support multi-modal interactions, such as text, speech, image, click, etc. 
 Question: What modalities are supported by Macaw?
Output: Extractive
We focus on social media posts from the website Twitter, which are an excellent testing ground for character based models due to the noisy nature of text. Heavy use of slang and abundant misspellings means that there are many orthographically and semantically similar tokens, and special characters such as emojis are also immensely popular and carry useful semantic information. In our moderately sized training dataset of 2 million tweets, there were about 0.92 million unique word types. 
 Question: Does the paper clearly establish that the challenges listed here exist in this dataset and task?
Output: Yes-no
One example illustrates that the mistakes the model makes can be associated with changes in the party policy. 
 Question: Do changes in policies of the political actors account for all of the mistakes the model made?
Output: Yes-no
Experiments ::: Baselines
To comprehensively evaluate our AGDT, we compare the AGDT with several competitive models. 
 Question: Is the model evaluated against other Aspect-Based models?
Output: Yes-no
In this step we extract the AMR graphs of the summary sentences using story sentence AMRs. We divide this task in two parts. First is finding the important sentences from the story and then extracting the key information from those sentences using their AMR graphs. 
 Question: How are sentences selected from the summary graph?
Output: Extractive
We use the publicly available dataset KVRET BIBREF5 in our experiments. There are 2,425 dialogues for training, 302 for validation and 302 for testing, as shown in the upper half of Table TABREF12. 
 Question: What is the size of the dataset?
Output: Abstractive
The summaries generated by our ILP-based approach were compared with the corresponding manual summaries using the ROUGE BIBREF22 unigram score. 
 Question: What evaluation metrics were used for the summarization task?
Output: Extractive
We use the same baseline as used by lang2011unsupervised which has been shown to be difficult to outperform. This baseline assigns a semantic role to a constituent based on its syntactic function, i.e. the dependency relation to its head. 
 Question: What baseline model is used?
Output: Extractive
We also consider the following baselines. BOW-Tags represents locations using a bag-of-words representation, using the same tag weighting as the embedding model. BOW-KL(Tags) uses the same representation but after term selection, using the same KL-based method as the embedding model. BOW-All combines the bag-of-words representation with the structured information, encoded as proposed in BIBREF7 . GloVe uses the objective from the original GloVe model for learning location vectors, i.e. this variant differs from EGEL-Tags in that instead of INLINEFORM1 we use the number of co-occurrences of tag INLINEFORM2 near location INLINEFORM3 , measured as INLINEFORM4 . 
 Question: what are the existing approaches?
Output: Extractive
We opted for an automatic approach instead, that can be scaled arbitrarily and at little cost: we generate synthetic sentence pairs $(, \tilde{})$ by randomly perturbing 1.8 million segments $$ from Wikipedia. We use three techniques: mask-filling with BERT, backtranslation, and randomly dropping out words.  
 Question: How are the synthetic examples generated?
Output: Abstractive
After excluding all the words that are not used by at least three separate users in our training set, we build our AllWords model by counting the frequencies of all the remaining words and training a multinomial Naive Bayes classifier.  
 Question: What model did they use for their system?
Output: Extractive
CIC is a suitable complementary metric to ROUGE because it accounts for the most important information within each dialog domain. CIC can be applied to any summarization task with predefined essential entities. 
 Question: Is new evaluation metric extension of ROGUE?
Output: Yes-no
To investigate the trade-off between speed and performance, we compare our technique to standard models with and without attention on a Sequence Copy Task of varying length like in BIBREF14 . For this purpose we use 4 large machine translation datasets of WMT'17 on the following language pairs: English-Czech (en-cs, 52M examples), English-German (en-de, 5.9M examples), English-Finish (en-fi, 2.6M examples), and English-Turkish (en-tr, 207,373 examples). 
 Question: Which datasets are used in experiments?
Output: Abstractive
The input of our model are the words in the input text $x[1], ... , x[n]$ and query $q[1], ... , q[n]$ . We concatenate pre-trained word embeddings from GloVe BIBREF40 and character embeddings trained by CharCNN BIBREF41 to represent input words. The $2d$ -dimension embedding vectors of input text $x_1, ... , x_n$ and query $q_1, ... , q_n$ are then fed into a Highway Layer BIBREF42 to improve the capability of word embeddings and character embeddings as

$$\begin{split} g_t &= {\rm sigmoid}(W_gx_t+b_g) \\ s_t &= {\rm relu } (W_xx_t+b_x) \\ u_t &= g_t \odot s_t + (1 - g_t) \odot x_t~. \end{split}$$ (Eq. 18) The same Highway Layer is applied to $q_t$ and produces $v_t$ . Next, $u_t$ and $v_t$ are fed into a Bi-Directional Long Short-Term Memory Network (BiLSTM) BIBREF44 respectively in order to model the temporal interactions between sequence words: Then we feed $\mathbf {U}$ and $\mathbf {V}$ into the attention flow layer BIBREF27 to model the interactions between the input text and query. Therefore, we introduce Self-Matching Layer BIBREF29 in our model as

$$\begin{split} o_t &= {\rm BiLSTM}(o_{t-1}, [h_t, c_t]) \\ s_j^t &= w^T {\rm tanh}(W_hh_j+\tilde{W_h}h_t)\\ \alpha _i^t &= {\rm exp}(s_i^t)/\Sigma _{j=1}^n{\rm exp}(s_j^t)\\ c_t &= \Sigma _{i=1}^n\alpha _i^th_i ~. \end{split}$$ (Eq. 20) Finally we feed the embeddings $\mathbf {O} = [o_1, ... , o_n]$ into a Pointer Network BIBREF39 to decode the answer sequence as

$$\begin{split} p_t &= {\rm LSTM}(p_{t-1}, c_t) \\ s_j^t &= w^T {\rm tanh}(W_oo_j+W_pp_{t-1})\\ \beta _i^t &= {\rm exp}(s_i^t)/\Sigma _{j=1}^n{\rm exp}(s_j^t)\\ c_t &= \Sigma _{i=1}^n\beta _i^to_i~. \end{split}$$ (Eq. 21) Therefore, the probability of generating the answer sequence $\textbf {a}$ is as follows

$${\rm P}(\textbf {a}|\mathbf {O}) = \prod _t {\rm P}(a^t | a^1, ... , a^{t-1}, \mathbf {O})~.$$ (Eq. 23) 
 Question: What QA models were used?
Output: Abstractive
Suppose there are INLINEFORM0 mentions in documents on average, among these global models, NCEL not surprisingly has the lowest time complexity INLINEFORM1 since it only considers adjacent mentions, where INLINEFORM2 is the number of sub-GCN layers indicating the iterations until convergence. 
 Question: Do they only use adjacent entity mentions or use more than that in some cases (next to adjacent)?
Output: Abstractive
Table TABREF14 presents the distribution of the tweets by country before and after the filtering process. A large portion of the samples is from India because the MeToo movement has peaked towards the end of 2018 in India. There are very few samples from Russia likely because of content moderation and regulations on social media usage in the country. 
 Question: Do the tweets come from a specific region?
Output: Yes-no
What similarities and/or differences do these topics have with non-violent, non-Islamic religious material addressed specifically to women? As these questions suggest, to understand what, if anything, makes extremist appeals distinctive, we need a point of comparison in terms of the outreach efforts to women from a mainstream, non-violent religious group. For this purpose, we rely on an online Catholic women's forum. Comparison between Catholic material and the content of ISIS' online magazines allows for novel insight into the distinctiveness of extremist rhetoric when targeted towards the female population. To accomplish this task, we employ topic modeling and an unsupervised emotion detection method. 
 Question: How are similarities and differences between the texts from violent and non-violent religious groups analyzed?
Output: Abstractive
The baseline approach is based on BIBREF20 . It divides the task of interpreting commands for behavioral navigation into two steps: path generation, and path verification. For path generation, this baseline uses a standard sequence-to-sequence model augmented with an attention mechanism, similar to BIBREF23 , BIBREF6 . For path verification, the baseline uses depth-first search to find a route in the graph that matches the sequence of predicted behaviors. If no route matches perfectly, the baseline changes up to three behaviors in the predicted sequence to try to turn it into a valid path. 
 Question: What baselines did they compare their model with?
Output: Abstractive
Grammatical error correction (GEC) is a challenging task due to the variability of the type of errors and the syntactic and semantic dependencies of the errors on the surrounding context. Most of the grammatical error correction systems use classification and rule-based approaches for correcting specific error types. However, these systems use several linguistic cues as features. The standard linguistic analysis tools like part-of-speech (POS) taggers and parsers are often trained on well-formed text and perform poorly on ungrammatical text. This introduces further errors and limits the performance of rule-based and classification approaches to GEC. As a consequence, the phrase-based statistical machine translation (SMT) approach to GEC has gained popularity because of its ability to learn text transformations from erroneous text to correct text from error-corrected parallel corpora without any additional linguistic information.  We model our GEC system based on the phrase-based SMT approach. However, traditional phrase-based SMT systems treat words and phrases as discrete entities. We take advantage of continuous space representation by adding two neural network components that have been shown to improve SMT systems BIBREF3 , BIBREF4 . To train NNJM, we use the publicly available implementation, Neural Probabilistic Language Model (NPLM) BIBREF14 . The latest version of Moses can incorporate NNJM trained using NPLM as a feature while decoding. Similar to NNGLM, we use the parallel text used for training the translation model in order to train NNJM. We use a source context window size of 5 and a target context window size of 4. We select a source context vocabulary of 16,000 most frequent words from the source side. The target context vocabulary and output vocabulary is set to the 32,000 most frequent words. We use a single hidden layer to speed up training and decoding with an input embedding dimension of 192 and 512 hidden layer nodes.  
 Question: Do they use pretrained word representations in their neural network models?
Output: Yes-no
We adopt the annotation machinery of BIBREF5 implemented using Amazon's Mechanical Turk, and annotate each predicate by 2 trained workers independently, while a third consolidates their annotations into a final set of roles and arguments. In this consolidation task, the worker validates questions, merges, splits or modifies answers for the same role according to guidelines, and removes redundant roles by picking the more naturally phrased questions.  
 Question: What is different in the improved annotation protocol?
Output: Abstractive
Through our experiments, we make subtle points related to: (a) the performance of our features, (b) how our approach compares against human ability to detect drunk-texting, (c) most discriminative stylistic features, and (d) an error analysis that points to future work. 
 Question: Do the authors mention any confounds to their study?
Output: Yes-no
We extract user's age by applying regular expression patterns to profile descriptions (such as "17 years old, self-harm, anxiety, depression") BIBREF41 . We compile "age prefixes" and "age suffixes", and use three age-extraction rules: 1. I am X years old 2. Born in X 3. X years old, where X is a "date" or age (e.g., 1994). We selected a subset of 1464 users INLINEFORM0 from INLINEFORM1 who disclose their gender in their profile description. 
 Question: Where does the information on individual-level demographics come from?
Output: Abstractive
In Table TABREF26 we report the results of our model on the three datasets standardly used for the evaluation of emotion classification, which we have described in Section SECREF3 . 
 Question: What was their performance on emotion detection?
Output: Abstractive
We draw on a recently released corpus of state speeches delivered during the annual UN General Debate that provides the first dataset of textual output from states that is recorded at regular time-series intervals and includes a sample of all countries that deliver speeches BIBREF11 .  
 Question: Which dataset do they use?
Output: Extractive
Based on the gained insights, we have improved the state-of-the-art in XNLI for the Translate-Test and Zero-Shot approaches by a substantial margin. 
 Question: Is the improvement over state-of-the-art statistically significant?
Output: Yes-no
One way to analyze the model is to compute model gradients with respect to input features BIBREF26, BIBREF25. Figure FIGREF37 shows that in this particular example, the most important model inputs are verbs possibly associated with the entity butter, in addition to the entity's mentions themselves. It further shows that the model learns to extract shallow clues of identifying actions exerted upon only the entity being tracked, regardless of other entities, by leveraging verb semantics. 
 Question: What evidence do they present that the model attends to shallow context clues?
Output: Abstractive
Given the fact that the research on offensive language detection has to a large extent been focused on the English language, we set out to explore the design of models that can successfully be used for both English and Danish. 
 Question: What is the challenge for other language except English
Output: Abstractive
We describe our rules for WikiSQL here. We first detect WHERE values, which exactly match to table cells. After that, if a cell appears at more than one column, we choose the column name with more overlapped words with the question, with a constraint that the number of co-occurred words is larger than 1. By default, a WHERE operator is INLINEFORM0 , except for the case that surrounding words of a value contain keywords for INLINEFORM1 and INLINEFORM2 . Then, we deal with the SELECT column, which has the largest number of co-occurred words and cannot be same with any WHERE column. By default, the SELECT AGG is NONE, except for matching to any keywords in Table TABREF8 . The coverage of our rule on training set is 78.4%, with execution accuracy of 77.9%. Our rule for KBQA is simple without using a curated mapping dictionary. The pipeline of rules in SequentialQA is similar to that of WikiSQL. Compared to the grammar of WikiSQL, the grammar of SequentialQA has additional actions including copying the previous-turn logical form, no greater than, no more than, and negation. 
 Question: How many rules had to be defined?
Output: Abstractive
Three metrics in text simplification are chosen in this paper. BLEU BIBREF5 is one traditional machine translation metric to assess the degree to which translated simplifications differed from reference simplifications. FKGL measures the readability of the output BIBREF23 . A small FKGL represents simpler output. SARI is a recent text-simplification metric by comparing the output against the source and reference simplifications BIBREF20 . 
 Question: what evaluation metrics did they use?
Output: Extractive
This framework has potential applications when comparing different gold standards, considering the design choices for a new gold standard and performing qualitative error analyses for a proposed approach. 
 Question: Have they made any attempt to correct MRC gold standards according to their findings? 
Output: Yes-no
terms extracted from Yahoo! Answers tend to be more related, in terms of the number of correlated terms, to attributes related to religion or ethnicity compared to terms from Twitter. However, for two particular attributes (i.e., Price and Buddhist), the number of correlated terms from Twitter is higher than the ones from Yahoo! Answers .  
 Question: On Twitter, do the demographic attributes and answers show more correlations than on Yahoo! Answers?
Output: Yes-no
A context is upward entailing (shown by [... $\leavevmode {\color {red!80!black}\uparrow }$ ]) that allows an inference from ( "Introduction" ) to ( "Introduction" ), where French dinner is replaced by a more general concept dinner.  On the other hand, a downward entailing context (shown by [... $\leavevmode {\color {blue!80!black}\downarrow }$ ]) allows an inference from ( "Introduction" ) to ( "Introduction" ), where workers is replaced by a more specific concept new workers. All [ workers $\leavevmode {\color {blue!80!black}\downarrow }$ ] [joined for a French dinner $\leavevmode {\color {red!80!black}\uparrow }$ ] All workers joined for a dinner All new workers joined for a French dinner Not all [new workers $\leavevmode {\color {red!80!black}\uparrow }$ ] joined for a dinner Not all workers joined for a dinner 
 Question: How do they define upward and downward reasoning?
Output: Abstractive
Stanford - Twitter Sentiment Corpus (STS Corpus): STS Corpus contains 1,600K training tweets collected by a crawler from BIBREF0 . BIBREF0 constructed a test set manually with 177 negative and 182 positive tweets. The Stanford test set is small. However, it has been widely used in different evaluation tasks BIBREF0 BIBREF5 BIBREF13 .

Sanders - Twitter Sentiment Corpus: This dataset consists of hand-classified tweets collected by using search terms: INLINEFORM0 , #google, #microsoft and #twitter. We construct the dataset as BIBREF14 for binary classification.

Health Care Reform (HCR): This dataset was constructed by crawling tweets containing the hashtag #hcr BIBREF15 . Task is to predict positive/negative tweets BIBREF14 . Table IV shows the result of our model for sentiment classification against other models. We compare our model performance with the approaches of BIBREF0 BIBREF5 on STS Corpus.  For Sanders and HCR datasets, we compare results with the model of BIBREF14 that used a ensemble of multiple base classifiers (ENS) such as NB, Random Forest (RF), SVM and Logistic Regression (LR).  
 Question: Are results reported only on English datasets?
Output: Yes-no