niryuu/sni-each-converted huggingface dataset:
Most importantly, all three languages have error-corrected corpora for testing purposes, though work on their automatic grammatical error correction is extremely limited (see Section SECREF3 ). 
 Question: Do they introduce errors in the data or does the data already contain them?
Output:  all three languages have error-corrected corpora for testing purposes
Finally, we combine our CNN and RNN models using a voting process. For each sentence in the test set, we apply several CNN and RNN models presented in Tables TABREF12 and TABREF14 and predict the class with the most votes. In case of a tie, we pick one of the most frequent classes randomly. 
 Question: How does their simple voting scheme work?
Output: we apply several CNN and RNN models presented in Tables TABREF12 and TABREF14 and predict the class with the most votes In case of a tie, we pick one of the most frequent classes randomly
Intuitively, when the Ranker makes more accurate passage selections, the Reasoner will work with less noisy data and thus is easier to succeed. Specifically, the Reasoner learns to extract the linking entity from chains selected by a well-trained Ranker, and it benefits the Ranker training by providing extra rewards. 
 Question: How do two models cooperate to select the most confident chains?
Output: Reasoner learns to extract the linking entity from chains selected by a well-trained Ranker, and it benefits the Ranker training by providing extra rewards
To test the robustness of the taggers against the OOV problem, we also conduct experiments using artificially constructed unnormalized text by corrupting words in the normal dev set. Again, the CNN tagger outperforms the two baselines by a very large margin. 
 Question: How do they confirm their model working well on out-of-vocabulary problems?
Output: conduct experiments using artificially constructed unnormalized text by corrupting words in the normal dev set
Baselines We thoroughly compare our approaches to the following baselines:

Direct source$\rightarrow $target: A standard NMT model trained on given source$\rightarrow $target parallel data.

Multilingual: A single, shared NMT model for multiple translation directions BIBREF6.

Many-to-many: Trained for all possible directions among source, target, and pivot languages.

Many-to-one: Trained for only the directions to target language, i.e., source$\rightarrow $target and pivot$\rightarrow $target, which tends to work better than many-to-many systems BIBREF27. 
 Question: What are multilingual models that were outperformed in performed experiment?
Output: Direct source$\rightarrow $target: A standard NMT model trained on given source$\rightarrow $target Multilingual: A single, shared NMT model for multiple translation directions Many-to-many: Trained for all possible directions among source, target, and pivot languages Many-to-one: Trained for only the directions to target language
Different from scaled dot-product attention, Gaussian-masked directional attention expects to pay attention to the adjacent characters of each positions and cast the localness relationship between characters as a fix Gaussian weight for attention. We assume that the Gaussian weight only relys on the distance between characters. 
 Question: How does Gaussian-masked directional multi-head attention works?
Output: pays attentions to adjacent characters and casts a localness relationship between the characters as a fixed Gaussian weight assuming the weight relies on the distance between characters
We test our proposed approach on three public short text datasets.  SearchSnippets. This dataset was selected from the results of web search transaction using predefined phrases of 8 different domains by Phan et al. BIBREF41 . StackOverflow. We use the challenge data published in Kaggle.com.  Biomedical. We use the challenge data published in BioASQ's official website.  
 Question: What datasets did they use?
Output: SearchSnippets StackOverflow Biomedical
We ran UTD over all 104 telephone calls, which pair 11 hours of audio with Spanish transcripts and their crowdsourced English translations. The transcripts contain 168,195 Spanish word tokens (10,674 types), and the translations contain 159,777 English word tokens (6,723 types). 
 Question: what is the size of the speech corpus?
Output: 104 telephone calls transcripts contain 168,195 Spanish word tokens  translations contain 159,777 English word tokens
The two popular segmentation methods are morpheme segmentation BIBREF4 and Byte Pair Encoding (BPE) BIBREF5. After word segmentation, we additionally add an specific symbol behind each separated subword unit, which aims to assist the NMT model to identify the morpheme boundaries and capture the semantic information effectively.  We utilize the Zemberek with a morphological disambiguation tool to segment the Turkish words into morpheme units, and utilize the morphology analysis tool BIBREF12 to segment the Uyghur words into morpheme units.  
 Question: How does the word segmentation method work?
Output: morpheme segmentation BIBREF4 and Byte Pair Encoding (BPE) BIBREF5 Zemberek BIBREF12
Looking into the future, an excellent extension from the works surveyed in this paper would be to give more independence to the several learning methods (e.g. less human intervention) involved in the studies as well as increasing the size of the output images. 
 Question: What challenges remain unresolved?
Output: give more independence to the several learning methods (e.g. less human intervention) involved in the studies increasing the size of the output images
We first use state-of-the-art PDTB taggers for our baseline BIBREF13 , BIBREF12 for the evaluation of the causality prediction of our models ( BIBREF12 requires sentences extracted from the text as its input, so we used our parser to extract sentences from the message). 
 Question: What baselines did they consider?
Output: state-of-the-art PDTB taggers
We report the experimental results for our two tasks in terms of the correct classification rate (CCR). For sentiment analysis, we have a three-class problem (positive, negative, and neutral), where the classes are mutually exclusive. The CCR, averaged for a set of tweets, is defined to be the number of correctly-predicted sentiments over the number of groundtruth sentiments in these tweets. For NER, we consider that each tweet may reference up to four candidates, i.e., targeted entities. The CCR, averaged for a set of tweets, is the number of correctly predicted entities (candidates) over the number of groundtruth entities (candidates) in this set. 
 Question: What measures are used for evaluation?
Output: correct classification rate (CCR)
To further improve the performance, we adopt system combination on the decoding lattice level. By combining systems, we can take advantage of the strength of each model that is optimized for different domains.  The best result for vlsp2018 of 4.85% WER is obtained by the combination weights 0.6:0.4 where 0.6 is given to the general language model and 0.4 is given to the conversation one. On the vlsp2019 set, the ratio is change slightly by 0.7:0.3 to deliver the best result of 15.09%. 
 Question: What is the language model combination technique used in the paper?
Output: system combination on the decoding lattice level combination weights
We create a new manually annotated multimodal hate speech dataset formed by $150,000$ tweets, each one of them containing text and an image.  
 Question: How large is the dataset?
Output:  $150,000$ tweets
We compare our approach to two other, the first one uses almost the same tweet data as we use for training, and the second one is the CrowdFlower dataset annotated for emotions.

In the first one Wang et al. BIBREF21 downloaded over 5M tweets which included one of 131 emotional hashtags based on Parrott's three-level categorization of emotions in seven categories: joy, sadness, anger, love, fear, thankfulness, surprise.  In the second one, the reported results are from a paper by BIBREF33 in which they used maximum entropy classifier with bag of words model to classify various emotional datasets. Here we only report part of their result for CrowdFlower dataset that can be mapped to one of our seven labels. 
 Question: What baseline is used?
Output:  Wang et al. BIBREF21 paper by BIBREF33 in which they used maximum entropy classifier with bag of words model to classify various emotional datasets
The focus of this section is on recently published datasets and LID research applicable to the South African context. An in depth survey of algorithms, features, datasets, shared tasks and evaluation methods may be found in BIBREF0.

The datasets for the DSL 2015 & DSL 2017 shared tasks BIBREF1 are often used in LID benchmarks and also available on Kaggle . The recently published JW300 parallel corpus BIBREF2 covers over 300 languages with around 100 thousand parallel sentences per language pair on average. The WiLI-2018 benchmark dataset BIBREF4 for monolingual written natural language identification includes around 1000 paragraphs of 235 languages. The NCHLT text corpora BIBREF7 is likely a good starting point for a shared LID task dataset for the South African languages BIBREF8. 
 Question: Which datasets are employed for South African languages LID?
Output: DSL 2015 DSL 2017 JW300 parallel corpus  NCHLT text corpora
MIMIC-III is a freely available, deidentified database containing electronic health records of patients admitted to an Intensive Care Unit (ICU) at Beth Israel Deaconess Medical Center between 2001 and 2012 
 Question: what datasets were used?
Output: MIMIC-III
The overall Total Accuracy score reported in table TABREF19 using the entire feature set is 549.  
 Question: Do they experiment with the dataset?
Output: Yes
Two metrics, the accuracy (ACC) and the normalized mutual information metric (NMI), are used to measure the clustering performance BIBREF38 , BIBREF48 .  
 Question: What were the evaluation metrics used?
Output: accuracy normalized mutual information
Our baseline is TransE since that the score function of our models is based on TransE. 
 Question: What baselines are used for comparison?
Output: TransE
Three clear domains can be noticed in the behavior of $\langle cc \rangle $ versus $\wp $, at $t_f$, as shown in Fig. FIGREF15 (blue squares). Phase I: $\langle cc \rangle $ increases smoothly for $\wp < 0.4$, indicating that for this domain there is a small correlation between word neighborhoods. Full vocabularies are attained also for $\wp < 0.4$; Phase II: a drastic transition appears at the critical domain $\wp ^* \in (0.4,0.6)$, in which $\langle cc \rangle $ shifts abruptly towards 1. An abrupt change in $V(t_f)$ versus $\wp $ is also found (Fig. FIGREF16) for $\wp ^*$; Phase III: single-word languages dominate for $\wp > 0.6$. The maximum value of $\langle cc \rangle $ indicate that word neighborhoods are completely correlated. 
 Question: What are three possible phases for language formation?
Output: Phase I: $\langle cc \rangle $ increases smoothly for $\wp < 0.4$, indicating that for this domain there is a small correlation between word neighborhoods. Full vocabularies are attained also for $\wp < 0.4$ Phase II: a drastic transition appears at the critical domain $\wp ^* \in (0.4,0.6)$, in which $\langle cc \rangle $ shifts abruptly towards 1. An abrupt change in $V(t_f)$ versus $\wp $ is also found (Fig. FIGREF16) for $\wp ^*$ Phase III: single-word languages dominate for $\wp > 0.6$. The maximum value of $\langle cc \rangle $ indicate that word neighborhoods are completely correlated
We mix standard linguistic features, such as Part-Of-Speech (POS) and chunk tag, together with several gazetteers specifically built for classical music, and a series of features representing tokens' left and right context. 
 Question: What kind of corpus-based features are taken into account?
Output: standard linguistic features, such as Part-Of-Speech (POS) and chunk tag series of features representing tokens' left and right context
300 test queries are randomly selected out, and annotators are asked to independently score the results of these queries with different points in terms of their quality: (1) Good (3 points): The response is grammatical, semantically relevant to the query, and more importantly informative and interesting; (2) Acceptable (2 points): The response is grammatical, semantically relevant to the query, but too trivial or generic (e.g.,“我不知道(I don't know)", “我也是(Me too)”, “我喜欢(I like it)" etc.); (3) Failed (1 point): The response has grammar mistakes or irrelevant to the query. 
 Question: How is human evaluation performed, what were the criteria?
Output: (1) Good (3 points): The response is grammatical, semantically relevant to the query, and more importantly informative and interesting (2) Acceptable (2 points): The response is grammatical, semantically relevant to the query, but too trivial or generic (3) Failed (1 point): The response has grammar mistakes or irrelevant to the query
We employ a huge dataset of 45,000 self-reported depressed users introduced in BIBREF46 where a lexicon of depression symptoms consisting of 1500 depression-indicative terms was created with the help of psychologist clinician and employed for collecting self-declared depressed individual's profiles. A subset of 8,770 users (24 million time-stamped tweets) containing 3981 depressed and 4789 control users (that do not show any depressive behavior) were verified by two human judges BIBREF46 . 
 Question: How is the data annotated?
Output: The data are self-reported by Twitter users and then verified by two human experts.
Combining the modulus part and the phase part, HAKE maps entities into the polar coordinate system, where the radial coordinate and the angular coordinates correspond to the modulus part and the phase part, respectively. 
 Question: How are entities mapped onto polar coordinate system?
Output: radial coordinate and the angular coordinates correspond to the modulus part and the phase part, respectively
In this work, we introduce a methodology that provides VQA algorithms with the ability to generate human interpretable attention maps which effectively ground the answer to the relevant image regions. We accomplish this by leveraging region descriptions and object annotations available in the Visual Genome dataset, and using these to automatically construct attention maps that can be used for attention supervision, instead of requiring human annotators to manually provide grounding labels. 
 Question: How do they obtain region descriptions and object annotations?
Output: they are available in the Visual Genome dataset
We began with evaluating standard MT paradigms, i.e., PBSMT BIBREF3 and NMT BIBREF1 . As for PBSMT, we also examined two advanced methods: pivot-based translation relying on a helping language BIBREF10 and induction of phrase tables from monolingual data BIBREF14 .

As for NMT, we compared two types of encoder-decoder architectures: attentional RNN-based model (RNMT) BIBREF2 and the Transformer model BIBREF18 . In addition to standard uni-directional modeling, to cope with the low-resource problem, we examined two multi-directional models: bi-directional model BIBREF11 and multi-to-multi (M2M) model BIBREF8 .

After identifying the best model, we also examined the usefulness of a data augmentation method based on back-translation BIBREF17 . 
 Question: what was the baseline?
Output: pivot-based translation relying on a helping language BIBREF10 nduction of phrase tables from monolingual data BIBREF14  attentional RNN-based model (RNMT) BIBREF2 Transformer model BIBREF18 bi-directional model BIBREF11 multi-to-multi (M2M) model BIBREF8 back-translation BIBREF17
We use a standard set of supervised as well as unsupervised benchmark tasks from the literature to evaluate our trained models, following BIBREF16 Sentence embeddings are evaluated for various supervised classification tasks as follows. The predefined training split is used to tune the L2 penalty parameter using cross-validation and the accuracy and F1 scores are computed on the test set.  We perform unsupervised evaluation of the learnt sentence embeddings using the sentence cosine similarity, on the STS 2014 BIBREF31 and SICK 2014 BIBREF32 datasets. These similarity scores are compared to the gold-standard human judgements using Pearson's INLINEFORM0 BIBREF33 and Spearman's INLINEFORM1 BIBREF34 correlation scores.  
 Question: What metric is used to measure performance?
Output: Accuracy and F1 score for supervised tasks, Pearson's and Spearman's correlation for unsupervised tasks
On the same architecture, our RCRN outperforms ablative baselines BiLSTM by INLINEFORM2 and 3L-BiLSTM by INLINEFORM3 on average across 16 datasets. 
 Question: By how much do they outperform BiLSTMs in Sentiment Analysis?
Output: Proposed RCRN outperforms ablative baselines BiLSTM by +2.9% and 3L-BiLSTM by +1.1% on average across 16 datasets.
Our data consists of two sets used to train and evaluate our automatic speech recognition system. Four major evaluation campaigns have enabled the creation of wide corpora of French broadcast speech: ESTER1 BIBREF13, ESTER2 BIBREF14, ETAPE BIBREF15 and REPERE BIBREF16. 
 Question: Which corpora does this paper analyse?
Output: ESTER1 ESTER2 ETAPE REPERE
We consider pre-training on the audio data (without labels) of WSJ, part of clean Librispeech (about 80h) and full Librispeech as well as a combination of all datasets (§ SECREF7 ).  Our experimental results on the WSJ benchmark demonstrate that pre-trained representations estimated on about 1,000 hours of unlabeled speech can substantially improve a character-based ASR system and outperform the best character-based result in the literature, Deep Speech 2.  
 Question: Which unlabeled data do they pretrain with?
Output: 1000 hours of WSJ audio data
To conclude this section, our model reliable corrects grammatical, spelling and word order errors on , with more mixed performance on lexical choice errors and some unnecessary paraphrasing of the input. 
 Question: What error types is their model more reliable for?
Output: grammatical, spelling and word order errors
Data-driven models for morphological analysis are constructed using training data INLINEFORM0 consisting of INLINEFORM1 training examples. The baseline model BIBREF5 we compare with regards the output space of the model as a subset INLINEFORM2 where INLINEFORM3 is the set of all tag sets seen in this training data. Specifically, they solve the task as a multi-class classification problem where the classes are individual tag sets. In low-resource scenarios, this indicates that INLINEFORM4 and even for those tag sets existing in INLINEFORM5 we may have seen very few training examples. The conditional probability of a sequence of tag sets given the sentence is formulated as a 0th order CRF. DISPLAYFORM0 
 Question: What other cross-lingual approaches is the model compared to?
Output: The baseline model BIBREF5 we compare with regards the output space of the model as a subset INLINEFORM2 where INLINEFORM3 is the set of all tag sets seen in this training data.
The dataset contains a total of 9710 passages, with an average of 6.24 sentences per passage, 16.16 words per sentence, and an average length of 86 words. 
 Question: What is the size of this dataset?
Output: 9710 passages, with an average of 6.24 sentences per passage, 16.16 words per sentence, and an average length of 86 words
We collect a novel dataset of 230K+ recipe texts and 1M+ user interactions (reviews) over 18 years (2000-2018) from Food.com. 
 Question: Where do they get the recipes from?
Output: from Food.com
We use OpenNMT BIBREF24 as the implementation of the NMT system for all experiments BIBREF5 . PBMT-R is a phrase-based method with a reranking post-processing step BIBREF18 . Hybrid performs sentence splitting and deletion operations based on discourse representation structures, and then simplifies sentences with PBMT-R BIBREF25 . SBMT-SARI BIBREF19 is syntax-based translation model using PPDB paraphrase database BIBREF26 and modifies tuning function (using SARI). Dress is an encoder-decoder model coupled with a deep reinforcement learning framework, and the parameters are chosen according to the original paper BIBREF20 . 
 Question: what state of the art methods did they compare with?
Output: OpenNMT PBMT-R Hybrid SBMT-SARI Dress
. We address various different challenges: dialogue act annotated data is not available for customer service on Twitter, the task of dialogue act annotation is subjective, existing taxonomies do not capture the fine-grained information we believe is valuable to our task, and tweets, although concise in nature, often consist of overlapping dialogue acts to characterize their full intent. 
 Question: Which dialogue acts are more suited to the twitter domain?
Output: overlapping dialogue acts
Given a multiple-choice question $qa$ with question text $q$ and answer choices A= $\lbrace a_i\rbrace $ , we select the most relevant tuples from $T$ and $S$ as follows. Selecting from Tuple KB: We use an inverted index to find the 1,000 tuples that have the most overlapping tokens with question tokens $tok(qa).$ . 
 Question: Is an entity linking process used?
Output: No
As shown in Table , the proposed PS-rnn-elmo shows a significant MAP performance improvement compared to the previous best model, CompClip-LM (0.696 to 0.734 absolute). 
 Question: How much better performance of proposed model compared to answer-selection models?
Output: significant MAP performance improvement compared to the previous best model, CompClip-LM (0.696 to 0.734 absolute)
We compared our multi-turn emotionally engaging dialog model (denoted as MEED) with two baselines—the vanilla sequence-to-sequence model (denoted as S2S) and HRAN. 
 Question: What two baseline models are used?
Output:  sequence-to-sequence model (denoted as S2S) HRAN
the TransE model Embedding models The Unstructured model BIBREF22 The TransH model BIBREF26 DISTMULT BIBREF45 ConvE BIBREF51 and ConvKB BIBREF52 The Path Ranking Algorithm (PRA) BIBREF21 
 Question: What models does this overview cover?
Output: This article presented a brief overview of embedding models of entity and relationships for KB completion.
Performing the Welch's t-test, both changes after weGAN training are statistically significant at a INLINEFORM0 significance level.  
 Question: Do they evaluate grammaticality of generated text?
Output: No
We compare the performance of the following models:

- IMG-only: This is a simple baseline where we just pass the image through a VGG19 and use the embedding of the image to predict the answer from a fixed vocabulary.

- QUES-only: This is a simple baseline where we just pass the question through a LSTM and use the embedding of the question to predict the answer from a fixed vocabulary.

- SANBIBREF2: This is a state of the art VQA model which is an encoder-decoder model with a multi-layer stacked attention BIBREF26 mechanism. It obtains a representation for the image using a deep CNN and a representation for the query using LSTM. It then uses the query representation to locate relevant regions in the image and uses this to pick an answer from a fixed vocabulary.

- SANDYBIBREF1: This is the best performing model on the DVQA dataset and is a variant of SAN. Unfortunately, the code for this model is not available and the description in the paper was not detailed enough for us to reimplement it. Hence, we report the numbers for this model only on DVQA (from the original paper).

- VOES: This is our model as described in section SECREF3 which is specifically designed for questions which do not have answers from a fixed vocabulary.

- VOES-Oracle: blackThis is our model where the first three stages of VOES are replaced by an Oracle, i.e., the QA model answers questions on a table that has been generated using the ground truth annotations of the plot. With this we can evaluate the performance of the WikiTableQA model when it is not affected by the VED model's errors.

- SAN-VOES: Given the complementary strengths of SAN-VQA and VOES, we train a hybrid model with a binary classifier which given a question decides whether to use the SAN or the VOES model. The data for training this binary classifier is generated by comparing the predictions of a trained SAN model and a trained VOES model on the training dataset. For a given question, the label is set to 1 (pick SAN) if the performance of SAN was better than that of VOES. We ignore questions where there is a tie. The classifier is a simple LSTM based model which computes a representation for the question using an LSTM and uses this representation to predict 1/0. At test time, we first pass the question through this model and depending on the output of this model use SAN or VOES. 
 Question: What models other than SAN-VOES are trained on new PlotQA dataset?
Output: IMG-only QUES-only SAN SANDY  VOES-Oracle VOES
The evaluation results are quite favorable for both targets and particularly higher for Target-1, considering the fact that they are the initial experiments on the data set. 
 Question: Which SVM approach resulted in the best performance?
Output: Target-1
We first experiment with a suite of non-neural models, including Support Vector Machines (SVMs), logistic regression, Naïve Bayes, Perceptron, and decision trees.  We finally experiment with neural models, although our dataset is relatively small. We train both a two-layer bidirectional Gated Recurrent Neural Network (GRNN) BIBREF20 and Convolutional Neural Network (CNN) (as designed in BIBREF21) with parallel filters of size 2 and 3, as these have been shown to be effective in the literature on emotion detection in text (e.g., BIBREF22, BIBREF23).  
 Question: What supervised methods are used?
Output: Support Vector Machines (SVMs), logistic regression, Naïve Bayes, Perceptron, and decision trees a two-layer bidirectional Gated Recurrent Neural Network (GRNN) BIBREF20 and Convolutional Neural Network (CNN) (as designed in BIBREF21)
Among all available parameters to tune the D2V algorithm released by Gensim, six of them were selected for optimisation BIBREF14. The window_size parameter affects the size of the sliding window used to parse texts. The alpha parameter represents the learning rate of the network. The sample setting allows the model to reduce the importance given to high-frequency words. The dm parameter defines the training used architecture (PV-DM or PV-DBOW). The hs option defines whether hierarchical softmax or negative sampling is used during the training. Finally, the vector_size parameter affects the number of dimensions composing the resulting vector. 
 Question: What six parameters were optimized with grid search?
Output: window_size alpha sample dm hs vector_size
We compare our method with the models trained using Adobe internal NLU tool, Pytext BIBREF18 and Rasa BIBREF19 NLU tools. 
 Question: What are the baselines?
Output: Adobe internal NLU tool Pytext Rasa
From the results which we get, it is evident that the transformer model achieves higher BLEU score than both Attention encoder-decoder and sequence-sequence model. 
 Question: How were their results compared to state-of-the-art?
Output: transformer model achieves higher BLEU score than both Attention encoder-decoder and sequence-sequence model
CFQ contains the most query patterns by an order of magnitude and also contains significantly more queries and questions than the other datasets.  
 Question: How authors justify that question answering dataset presented is realistic?
Output: CFQ contains the most query patterns by an order of magnitude and also contains significantly more queries and questions than the other datasets
The most intuitive way to evaluate the text answer is to directly compute the Exact Match (EM) and Macro-averaged F1 scores (F1) between the predicted text answer and the ground-truth text answer. 
 Question: What evaluation metrics were used?
Output: Exact Match (EM) Macro-averaged F1 scores (F1)
However, we found that explicitly initializing these embeddings with vectors pre-trained over a large collection of unlabeled data significantly improved performance (see Section "Effects of initialized embeddings and corrupt-sampling schemes" ). 
 Question: What is the new initialization method proposed in this paper?
Output: They initialize their word and entity embeddings with vectors pre-trained over a large corpus of unlabeled data.
Manual verification of Twitter profiles: We verified each profile discovered manually by examining the profile picture, profile background image, recent tweets, and recent pictures posted by a user. 
 Question: How is gang membership verified?
Output: Manual verification
However, adversarial misspellings constitute a longstanding real-world problem. Spammers continually bombard email servers, subtly misspelling words in efforts to evade spam detection while preserving the emails' intended meaning BIBREF1 , BIBREF2 . 
 Question: Why is the adversarial setting appropriate for misspelling recognition?
Output: Adversarial misspellings are a real-world problem
To tackle this issue, we investigate how to improve the generalization of BERT-based MCQA models with the constraint of limited training data using four representative MCQA datasets: DREAM, MCTest, TOEFL, and SemEval-2018 Task 11. 
 Question: What four representative datasets are used for bechmark?
Output: DREAM, MCTest, TOEFL, and SemEval-2018 Task 11
At the end of the annotation process, we have annotated 700 tweets, where 175 tweets are in favor of and 175 tweets are against Target-1, and similarly 175 tweets are in favor of and 175 are against Target-2.  
 Question: How many tweets did they collect?
Output: 700
Similar equations were discovered using Euclidean distance computed between the context vector representations of the equations ( INLINEFORM2 ). We give additional example results in Appendix B. 
 Question: How do they define similar equations?
Output: By using Euclidean distance computed between the context vector representations of the equations
We train three systems (S1, S2 and S3) with the corpora summarised in Table TABREF5. The first two systems are transformer models trained on different amounts of data (6M vs. 18M parallel sentences as seen in the Table). The third system includes a modification to consider the information of full coreference chains throughout a document augmenting the sentence to be translated with this information and it is trained with the same amount of sentence pairs as S1. 
 Question: Which three neural machine translation systems are analyzed?
Output: first two systems are transformer models trained on different amounts of data The third system includes a modification to consider the information of full coreference chains
Note that the question answering tasks we consider here are multimodal in that while the context is a procedural text, the question and the multiple choice answers are composed of images. 
 Question: What multimodality is available in the dataset?
Output: context is a procedural text, the question and the multiple choice answers are composed of images
We therefore use reduction in softmax probability of the correct relation as our signaling strength metric for the model. We call this metric ${\Delta }_s$ (for delta-softmax), which can be written as:

where $rel$ is the true relation of the EDU pair, $t_i$ represents the token at index $i$ of $N$ tokens, and $X_{mask=i}$ represents the input sequence with the masked position $i$ (for $i \in 1 \ldots N$ ignoring separators, or $\phi $, the empty set). 
 Question: How is the delta-softmax calculated?
Output: Answer with content missing: (Formula) Formula is the answer.
What similarities and/or differences do these topics have with non-violent, non-Islamic religious material addressed specifically to women? As these questions suggest, to understand what, if anything, makes extremist appeals distinctive, we need a point of comparison in terms of the outreach efforts to women from a mainstream, non-violent religious group. For this purpose, we rely on an online Catholic women's forum. Comparison between Catholic material and the content of ISIS' online magazines allows for novel insight into the distinctiveness of extremist rhetoric when targeted towards the female population. To accomplish this task, we employ topic modeling and an unsupervised emotion detection method. 
 Question: How are similarities and differences between the texts from violent and non-violent religious groups analyzed?
Output: By using topic modeling and unsupervised emotion detection on ISIS materials and articles from Catholic women forum
In contrast, in order to exploit section information, in this paper we propose to capture a distributed representation of both the global (the whole document) and the local context (e.g., the section/topic) when deciding if a sentence should be included in the summary 
 Question: What do they mean by global and local context?
Output: global (the whole document) local context (e.g., the section/topic)
A “coding scheme” is defined, which is a set of labels, annotations, or codes that items in the corpus may have. Schemes include formal definitions or procedures, and often include examples, particularly for borderline cases. Next, coders are trained with the coding scheme, which typically involves interactive feedback. Training sometimes results in changes to the coding scheme, in which the first round becomes a pilot test. Then, annotators independently review at least a portion of the same items throughout the entire process, with a calculation of “inter-annotator agreement” or “inter-rater reliability.” Finally, there is a process of “reconciliation” for disagreements, which is sometimes by majority vote without discussion and other times discussion-based. 
 Question: What are the core best practices of structured content analysis?
Output: “coding scheme” is defined coders are trained with the coding scheme Training sometimes results in changes to the coding scheme calculation of “inter-annotator agreement” or “inter-rater reliability.” there is a process of “reconciliation” for disagreements
Corpus utilizados ::: Corpus 5KL
Este corpus fue constituido con aproximadamente 5 000 documentos (en su mayor parte libros) en español. Los documentos originales, en formatos heterogéneos, fueron procesados para crear un único documento codificado en utf8. Las frases fueron segmentadas automáticamente, usando un programa en PERL 5.0 y expresiones regulares, para obtener una frase por línea.

Las características del corpus 5KL se encuentran en la Tabla TABREF4. Este corpus es empleado para el entrenamiento de los modelos de aprendizaje profundo (Deep Learning, Sección SECREF4).

El corpus literario 5KL posee la ventaja de ser muy extenso y adecuado para el aprendizaje automático. Tiene sin embargo, la desventaja de que no todas las frases son necesariamente “frases literarias”. Muchas de ellas son frases de lengua general: estas frases a menudo otorgan una fluidez a la lectura y proporcionan los enlaces necesarios a las ideas expresadas en las frases literarias.

Otra desventaja de este corpus es el ruido que contiene. El proceso de segmentación puede producir errores en la detección de fronteras de frases. También los números de página, capítulos, secciones o índices producen errores. No se realizó ningún proceso manual de verificación, por lo que a veces se introducen informaciones indeseables: copyrights, datos de la edición u otros. Estas son, sin embargo, las condiciones que presenta un corpus literario real.

Corpus utilizados ::: Corpus 8KF
Un corpus heterogéneo de casi 8 000 frases literarias fue constituido manualmente a partir de poemas, discursos, citas, cuentos y otras obras. Se evitaron cuidadosamente las frases de lengua general, y también aquellas demasiado cortas ($N \le 3$ palabras) o demasiado largas ($N \ge 30$ palabras). El vocabulario empleado es complejo y estético, además que el uso de ciertas figuras literarias como la rima, la anáfora, la metáfora y otras pueden ser observadas en estas frases.

Las características del corpus 8KF se muestran en la Tabla TABREF6. Este corpus fue utilizado principalmente en los dos modelos generativos: modelo basado en cadenas de Markov (Sección SECREF13) y modelo basado en la generación de Texto enlatado (Canned Text, Sección SECREF15). 
 Question: What datasets are used?
Output: Corpus 5KL Corpus 8KF
We extend the Amazon Conversational Bot Toolkit (CoBot) BIBREF6 which is a flexible event-driven framework. CoBot provides ASR results and natural language processing pipelines through the Alexa Skills Kit (ASK) BIBREF7. Gunrock corrects ASR according to the context (asr) and creates a natural language understanding (NLU) (nlu) module where multiple components analyze the user utterances. A dialog manager (DM) (dm) uses features from NLU to select topic dialog modules and defines an individual dialog flow. Each dialog module leverages several knowledge bases (knowledge). Then a natural language generation (NLG) (nlg) module generates a corresponding response. Finally, we markup the synthesized responses and return to the users through text to speech (TTS) (tts). 
 Question: What the system designs introduced?
Output: Amazon Conversational Bot Toolkit natural language understanding (NLU) (nlu) module dialog manager knowledge bases natural language generation (NLG) (nlg) module text to speech (TTS) (tts)
We implement three baseline models for comparison: Noun WordNet Semantic Text Exchange Model (NWN-STEM), General WordNet Semantic Text Exchange Model (GWN-STEM), and Word2Vec Semantic Text Exchange Model (W2V-STEM). 
 Question: What are the baseline models mentioned in the paper?
Output: Noun WordNet Semantic Text Exchange Model (NWN-STEM) General WordNet Semantic Text Exchange Model (GWN-STEM) Word2Vec Semantic Text Exchange Model (W2V-STEM)
We assess the optimal number of topics that need to be specified for the STM analysis. We follow the recommendations of the original STM paper and focus on exclusivity and semantic coherence measures. Highly frequent words in a given topic that do not appear very often in other topics are viewed as making that topic exclusive.  Following BIBREF8 we generate a set of candidate models ranging between 3 and 50 topics. We then plot the exclusivity and semantic coherence (numbers closer to 0 indicate higher coherence), with a linear regression overlaid (Figure FIGREF3 ). Models above the regression line have a “better” exclusivity-semantic coherence trade off. We select the 16-topic model, which has the largest positive residual in the regression fit, and provides higher exclusivity at the same level of semantic coherence. The topic quality is usually evaluated by highest probability words, which is presented in Figure FIGREF4 . 
 Question: How are the main international development topics that states raise identified?
Output:  They focus on exclusivity and semantic coherence measures: Highly frequent words in a given topic that do not appear very often in other topics are viewed as making that topic exclusive. They select select the 16-topic model, which has the largest positive residual in the regression fit, and provides higher exclusivity at the same level of semantic coherence.
holistic: The baseline model which maps the holistic image feature and LSTM-encoded question feature to a common space and perform element-wise multiplication between them.

TraAtt: The traditional attention model, implementation of WTL model BIBREF9 using the same $3\times 3$ regions in SalAtt model.

RegAtt: The region attention model which employs our novel attention method, same as the SalAtt model but without region pre-selection.

ConAtt: The convolutional region pre-selection attention model which replaces the BiLSTM in SalAtt model with a weight-sharing linear mapping, implemented by a convolutional layer.

Besides, we also compare our SalAtt model with the popular baseline models i.e. iBOWIMG BIBREF4 , VQA BIBREF1 , and the state-of-the-art attention-based models i.e. WTL BIBREF9 , NMN BIBREF21 , SAN BIBREF14 , AMA BIBREF33 , FDA BIBREF34 , D-NMN BIBREF35 , DMN+ BIBREF8 on two tasks of COCO-VQA. 
 Question: To which previous papers does this work compare its results?
Output: holistic TraAtt RegAtt ConAtt ConAtt iBOWIMG  VQA VQA WTL  NMN  SAN  AMA  FDA  D-NMN DMN+
Our analysis shows that attention models traditional alignment in some cases more closely while it captures information beyond alignment in others. For instance, attention agrees with traditional alignments to a high degree in the case of nouns. However, it captures other information rather than only the translational equivalent in the case of verbs. One can see that attention loss varies substantially across different POS tags. In particular, we focus on the cases of NOUN and VERB which are the most frequent POS tags in the dataset. As shown, the attention of NOUN is the closest to alignments on average. But the average attention loss for VERB is almost two times larger than the loss for NOUN. 
 Question: What useful information does attention capture?
Output: it captures other information rather than only the translational equivalent in the case of verbs
Our method is tested on Twitter datasets. This microblogging platform has been widely used to analyze discussions and polarization BIBREF12, BIBREF13, BIBREF14, BIBREF15, BIBREF2. It is a natural choice for these kind of problems, as it represents one of the main fora for public debate in online social media BIBREF15, it is a common destination for affiliative expressions BIBREF16 and is often used to report and read news about current events BIBREF17. An extra advantage of Twitter for this kind of studies is the availability of real-time data generated by millions of users. Other social media platforms offer similar data-sharing services, but few can match the amount of data and the accompanied documentation provided by Twitter. One last asset of Twitter for our work is given by retweets, whom typically indicate endorsement BIBREF18 and hence become a useful concept to model discussions as we can set “who is with who". However, our method has a general approach and it could be used a priori in any social network. In this work we report excellent result tested on Twitter but in future work we are going to test it in other social networks. 
 Question: What social media platform is observed?
Output: Twitter
Last, we evaluate our approaches in 9 commonly used text classification datasets. We evaluate our methods on several commonly used datasets whose themes range from sentiment, web-page, science to medical and healthcare. 
 Question: What NLP tasks do they consider?
Output: text classification for themes including sentiment, web-page, science, medical and healthcare
In natural language processing (NLP), taking into account the morphological complexity inherent to each language could be important for improving or adapting the existing methods, since the amount of semantic and grammatical information encoded at the word level, may vary significantly from language to language. This information could be useful for linguistic analysis and for measuring the impact of different word form normalization tools depending of the language. 
 Question: what is the practical application for this paper?
Output: Improve existing NLP methods. Improve linguistic analysis. Measure impact of word normalization tools.
In our profile verification process, we observed that most gang member profiles portray a context representative of gang culture. Some examples of these profile pictures are shown in Figure FIGREF32 , where the user holds or points weapons, is seen in a group fashion which displays a gangster culture, or is showing off graffiti, hand signs, tattoos and bulk cash. 
 Question: What are the differences in the use of images between gang member and the rest of the Twitter population?
Output: user holds or points weapons, is seen in a group fashion which displays a gangster culture, or is showing off graffiti, hand signs, tattoos and bulk cash
We leveraged an existing, annotated Twitter dataset that was constructed based on a hierarchical model of depression-related symptoms BIBREF12 , BIBREF13 . The dataset contains 9,473 annotations for 9,300 tweets. Each tweet is annotated as no evidence of depression (e.g., “Citizens fear an economic depression") or evidence of depression (e.g., “depressed over disappointment").  
 Question: Do they evaluate only on English datasets?
Output: Yes
We then train models on WMT English-German (EnDe) without BT noise, and instead explicitly tag the synthetic data with a reserved token. We repeat these experiments with WMT English-Romanian (EnRo), where NoisedBT underperforms standard BT and TaggedBT improves over both techniques. We perform our experiments on WMT18 EnDe bitext, WMT16 EnRo bitext, and WMT15 EnFr bitext respectively. We use WMT Newscrawl for monolingual data (2007-2017 for De, 2016 for Ro, 2007-2013 for En, and 2007-2014 for Fr). For bitext, we filter out empty sentences and sentences longer than 250 subwords. We remove pairs whose whitespace-tokenized length ratio is greater than 2. This results in about 5.0M pairs for EnDe, and 0.6M pairs for EnRo. We do not filter the EnFr bitext, resulting in 41M sentence pairs. 
 Question: What datasets was the method evaluated on?
Output: WMT18 EnDe bitext WMT16 EnRo bitext WMT15 EnFr bitext We perform our experiments on WMT18 EnDe bitext, WMT16 EnRo bitext, and WMT15 EnFr bitext respectively. We use WMT Newscrawl for monolingual data (2007-2017 for De, 2016 for Ro, 2007-2013 for En, and 2007-2014 for Fr). For bitext, we filter out empty sentences and sentences longer than 250 subwords. We remove pairs whose whitespace-tokenized length ratio is greater than 2. This results in about 5.0M pairs for EnDe, and 0.6M pairs for EnRo. We do not filter the EnFr bitext, resulting in 41M sentence pairs.
In phase 1 (also referred to as the “exploration” phase) the algorithm explores the state space through keeping track of previously visited states by maintaining an archive. During this phase, instead of resuming the exploration from scratch, the algorithm starts exploring from promising states in the archive to find high performing trajectories. 
 Question: How is trajectory with how rewards extracted?
Output: explores the state space through keeping track of previously visited states by maintaining an archive
We use two datasets, NELL-One and Wiki-One which are constructed by BIBREF11 . 
 Question: What datasets are used to evaluate the approach?
Output: NELL-One, Wiki-One
We notice that the concatenation consistently outperforms the gated-attention mechanism for both training and testing instructions. We suspect that the gated-attention is useful in the scenarios where objects are described in terms of multiple attributes, but it has no to harming effect when it comes to the order connectors. 
 Question: Why they conclude that the usage of Gated-Attention provides no competitive advantage against concatenation in this setting?
Output: concatenation consistently outperforms the gated-attention mechanism for both training and testing instructions
Experiments were conducted using the AMI IHM meeting corpus BIBREF18 to evaluated the speech recognition performance of various language models.  
 Question: Which dataset do they use?
Output:  AMI IHM meeting corpus
In this work, we analyze a set of tweets related to a specific classical music radio channel, BBC Radio 3, interested in detecting two types of musical named entities, Contributor and Musical Work. 
 Question: What language is the Twitter content in?
Output: English
As previous systems collect relevant data from knowledge bases after observing questions during evaluation BIBREF24 , BIBREF25 , we also explore using this option. Namely, we build a customized text corpus based on questions in commonsense reasoning tasks.  
 Question: Do they fine-tune their model on the end task?
Output: Yes
We apply the Generative Pre-trained Transformer (GPT) BIBREF2 as our pre-trained language model.  
 Question: What pretrained LM is used?
Output: Generative Pre-trained Transformer (GPT)
Compared to the in-context-parsing-based system, the combination with exact string matching yields a gain in recall of over 6%, and the combination with inflectional string matching yields an even bigger gain of almost 8%, at precision losses of 0.6% and 0.8%, respectively. 
 Question: What compleentary PIE extraction methods are used to increase reliability further?
Output: exact string matching inflectional string matching