niryuu/sni-each-converted huggingface dataset:
The Base is trained only on clean data while Base+Noise is trained on both the clean and the noisy data without noise handling. Global-CM uses a global confusion matrix for all noisy instances to model the noise as proposed by BIBREF3 and presented in Section SECREF3. The same architecture is used for Global-ID-CM, but the confusion matrix is initialized with the identity matrix (instead of Formula DISPLAY_FORM5) and only adapted during training.

The cluster-based models we propose in Section SECREF4 are Brown-CM and K-Means-CM. We experimented with numbers of clusters of 5, 10, 25 and 50. The models that select only the largest groups $G$ are marked as *-Freq and select either 30% or 50% of the clusters. The interpolation models have the postfix *-IP with $\lambda \in \lbrace 0.3, 0.5, 0.7\rbrace $ . The combination of both is named *-Freq-IP. As for all other hyperparameters, the choice was taken on the development set.

We implemented the Cleaning BIBREF15 and Dynamic-CM BIBREF14 models. Both were not developed for sequence labeling tasks and therefore needed to be adapted. For the Cleaning model, we followed the instructions by BIBREF3. The embedding and prediction components of the Dynamic-CM model were replaced according to our base model. The output of the dense layer was used as input to the dynamic matrix generation. We experimented with and without their proposed trace loss. 
 Question: What is baseline used?
Output: Base  Base+Noise Cleaning  Dynamic-CM   Global-CM  Global-ID-CM Brown-CM   K-Means-CM
Self-Attention on Question. As the question has integrated previous utterances, the model needs to directly relate previously mentioned concept with the current question. This is helpful for concept carry-over and coreference resolution. We thus employ self-attention on question. 
 Question: Does the model incorporate coreference and entailment?
Output: As the question has integrated previous utterances, the model needs to directly relate previously mentioned concept with the current question. This is helpful for concept carry-over and coreference resolution.
In order to decode spatial connections between the electrodes from the channel covariance matrix, we use a CNN BIBREF19 , in particular a four-layered 2D CNN stacking two convolutional and two fully connected hidden layers. The INLINEFORM0 feature map at a given CNN layer with input INLINEFORM1 , weight matrix INLINEFORM2 and bias INLINEFORM3 is obtained as: INLINEFORM4 . At this first level of hierarchy, the network is trained with the corresponding labels as target outputs, optimizing a cross-entropy cost function. In parallel, we apply a four-layered recurrent neural network on the channel covariance matrices to explore the hidden temporal features of the electrodes. Namely, we exploit an LSTM BIBREF20 consisting of two fully connected hidden layers, stacked with two LSTM layers and trained in a similar manner as CNN. 
 Question: How was the spatial aspect of the EEG signal computed?
Output: we use a CNN BIBREF19 , in particular a four-layered 2D CNN stacking two convolutional and two fully connected hidden layers.
For the NMT experiment, we used the Torch implementation to train a 2-layer LSTM model with 500 hidden units in both encoder and decoder, with 12 epochs. 
 Question: Which NMT models did they experiment with?
Output: 2-layer LSTM model with 500 hidden units in both encoder and decoder
In later phase, phrase-based word embedding can be incorporated for improved vocabulary mapping. To get more accurate target code for each line, Abstract Syntax Tree(AST) can be beneficial. 
 Question: What additional techniques could be incorporated to further improve accuracy?
Output: phrase-based word embedding Abstract Syntax Tree(AST)
It also needs to be noted that due to the issues in actual practice, the objective of this work is to generate candidate prescriptions to facilitate the prescribing procedure instead of substituting the human practitioners completely. 
 Question: Why did they think this was a good idea?
Output: They think it will help human TCM practitioners make prescriptions.
BioASQ organizers provide the training and testing data. The training data consists of questions, gold standard documents, snippets, concepts, and ideal answers (which we did not use in this paper, but we used last year BIBREF2). The test data is split between phases A and B. The Phase A dataset consists of the questions, unique ids, question types. The Phase B dataset consists of the questions, golden standard documents, snippets, unique ids and question types. 
 Question: What dataset did they use?
Output: BioASQ  dataset
Traditional text-to-speech (TTS) systems are composed of complex pipelines BIBREF0 , these often include accoustic frontends, duration model, acoustic prediction model and vocoder models The open source LJSpeech Dataset was used to train our TTS model. The architecture of our model utilizes RNN-based Seq2Seq model for generating mel spectrogram from text 
 Question: Which dataset(s) do they evaluate on?
Output: LJSpeech
For all experiments, the dimensions of word embeddings and recurrent hidden states are both set to 512. 
 Question: Do they use the same architecture as LSTM-s and GRUs with just replacing with the LAU unit?
Output: Yes
A single speaker narrated the 2000 sentences, which took several days.  
 Question: How many annotators participated?
Output: 1
We further use as features the embeddings of the claim, of the best-scoring snippet, and of the best-scoring sentence triplet from a Web page. We calculate these embeddings (i) as the average of the embeddings of the words in the text, and also (ii) using LSTM encodings, which we train for the task as part of a deep neural network (NN). We also use a task-specific embedding of the claim together with all the above evidence about it, which comes from the last hidden layer of the NN. 
 Question: What algorithm and embedding dimensions are used to build the task-specific embeddings?
Output:  task-specific embedding of the claim together with all the above evidence about it, which comes from the last hidden layer of the NN
We present a comparison of the proposed models to existing word embeddings approaches. These are: the Bernoulli embeddings (b-emb) BIBREF1 , continuous bag-of-words (CBOW) BIBREF5 , Distributed Memory version of Paragraph Vector (PV-DM) BIBREF11 and the Global Vectors (GloVe) BIBREF6 model. 
 Question: What word embeddings do they test?
Output: Bernoulli embeddings (b-emb) BIBREF1 , continuous bag-of-words (CBOW) BIBREF5 , Distributed Memory version of Paragraph Vector (PV-DM) BIBREF11 and the Global Vectors (GloVe) BIBREF6 model
The summaries generated by our ILP-based approach were compared with the corresponding manual summaries using the ROUGE BIBREF22 unigram score. 
 Question: What evaluation metrics were used for the summarization task?
Output: ROUGE BIBREF22 unigram score
Our first task is the recently-introduced Visual Question Answering challenge (VQA) BIBREF22 . The next set of experiments we consider focuses on GeoQA, a geographical question-answering task first introduced by Krish2013Grounded. 
 Question: What benchmark datasets they use?
Output: VQA and GeoQA
Stanford Sentiment Treebank Stanford Sentiment Treebank (SST) BIBREF14 . This concerns predicting movie review sentiment. Two datasets are derived from this corpus: (1) SST-1, containing five classes: very negative, negative, neutral, positive, and very positive. (2) SST-2, which has only two classes: negative and positive. For both, we remove phrases of length less than 4 from the training set. Subj BIBREF15 . The aim here is to classify sentences as either subjective or objective. This comprises 5000 instances of each. TREC BIBREF16 . A question classification dataset containing six classes: abbreviation, entity, description, human, location and numeric. There are 5500 training and 500 test instances. Irony BIBREF17 . This dataset contains 16,006 sentences from reddit labeled as ironic (or not). The dataset is imbalanced (relatively few sentences are ironic). Thus before training, we under-sampled negative instances to make classes sizes equal. Note that for this dataset we report the Area Under Curve (AUC), rather than accuracy, because it is imbalanced. 
 Question: What dataset/corpus is this evaluated over?
Output:  SST-1 SST-2 Subj  TREC  Irony
Different from previous work, we make our model conceptually simple and modular, so that the most important sub module, namely a five-character window context, can be pretrained using external data.  
 Question: What submodules does the model consist of?
Output: five-character window context
To simulate the scenario, we split the Microsoft COCO dataset into food and non-food domain datasets. the food dataset has 3,806 images for training and 1,775 for validation. The non-food dataset has 78,976 images for training and 38,749 for validation. Adaptation to food domain captioning 
 Question: How many examples are there in the source domain?
Output: 78,976
The focus of this publication is on representing information that is valuable for these tasks but that hitherto has largely been ignored in machine learning approaches centering around simplified language, specifically, text structure (e.g., paragraphs, lines), typography (e.g., font type, font style), and image (content, position, and dimensions) information. Information on the font type and font style (e.g., italics, bold print) of a token and its position on the physical page (for PDFs only) was specified as attributes to the token elements of the tokens layer (cf. Figure FIGREF34 for an example) 
 Question: Which information about typography is included in the corpus?
Output: font type font style Information on the font type and font style (e.g., italics, bold print) of a token and its position on the physical page
The Style baseline achieves an Accuracy of 83%, LDA features 89% and a combination of the two gets 90%. However, training on full poems reduces this to 42—52%. 
 Question: Is the outcome of the LDA analysis evaluated in any way?
Output: Yes
Cuneiform is an ancient writing system invented by the Sumerians for more than three millennia. 
 Question: What is one of the first writing systems in the world?
Output: Cuneiform
We collect a novel dataset of 230K+ recipe texts and 1M+ user interactions (reviews) over 18 years (2000-2018) from Food.com. 
 Question: How did they obtain the interactions?
Output: from Food.com
Despite this, we found that the training of our embeddings was not considerably slower than the training of order-2 equivalents such as SGNS. Explicitly, our GPU trained CBOW vectors (using the experimental settings found below) in 3568 seconds, whereas training CP-S and JCP-S took 6786 and 8686 seconds respectively. 
 Question: Do they measure computation time of their factorizations compared to other word embeddings?
Output: Yes
The comparative evaluator is trained with maximum likelihood estimation (MLE) objective, as described in eq DISPLAY_FORM6

where $\mathcal {X}$ is the set of pairwise training examples contructed as described above, $Q(x_1, x_2) \in \lbrace >,<,\approx \rbrace $ is the true label for the pair ($x_1$, $x_2$), $D_\phi ^q(x_1, x_2)$ is the probability of the comparative discriminator's prediction being $q$ ($q \in \lbrace >,<,\approx \rbrace $) for the pair ($x_1$, $x_2$). 
 Question: How they add human prefference annotation to fine-tuning process?
Output: human preference annotation is available $Q(x_1, x_2) \in \lbrace >,<,\approx \rbrace $ is the true label for the pair
We create two models both of which constitutes of three main parts: encoder, interaction and classifier and take two sequences as input. The encoder is shared among the sequences simply uses two stacked GRU layers. The interaction part consists of only attention for one model while for the another one it consists of attention and conflict combined as shown in (eqn.11) . The classifier part is simply stacked fully-connected layers.  
 Question: Which neural architecture do they use as a base for their attention conflict mechanisms?
Output: GRU-based encoder, interaction block, and classifier consisting of stacked fully-connected layers.
We use the Universal Dependencies' Hindi-English codemixed data set BIBREF9 to test the model's ability to label code-mixed data. This dataset is based on code-switching tweets of Hindi and English multilingual speakers. We use the Devanagari script provided by the data set as input tokens. 
 Question: What codemixed language pairs are evaluated?
Output: Hindi-English
We compare the performance of our Chinese word embedding vectors in the task of synonym discovery against another set of embedding vectors that was constructed with a co-occurrence model BIBREF1 . Our embeddings also proved to perform better than our benchmark dataset. 
 Question: Does this approach perform better than context-based word embeddings?
Output: Yes
We evaluate our model on two benchmark datasets BIBREF9 . The homographic dataset contains 2,250 contexts, 1,607 of which contain a pun. The heterographic dataset consists of 1,780 contexts with 1,271 containing a pun. We notice there is no standard splitting information provided for both datasets. Thus we apply 10-fold cross validation. To make direct comparisons with prior studies, following BIBREF4 , we accumulated the predictions for all ten folds and calculate the scores in the end. 
 Question: What datasets are used in evaluation?
Output: The homographic dataset contains 2,250 contexts, 1,607 of which contain a pun. The heterographic dataset consists of 1,780 contexts with 1,271 containing a pun.
We experiment with small feed-forward networks for four diverse NLP tasks: language identification, part-of-speech tagging, word segmentation, and preordering for statistical machine translation. 
 Question: What NLP tasks do the authors evaluate feed-forward networks on?
Output: language identification, part-of-speech tagging, word segmentation, and preordering for statistical machine translation
We evaluate our models on two typical tasks: text classification and text semantic matching. 
 Question: What tasks do they experiment with?
Output: text classification and text semantic matching
We model the ASP placement task as a successor of the AEP task. For all the `relevant' news entity pairs, the task is to determine the correct entity section. Each section in a Wikipedia entity page represents a different topic. For example, Barack Obama has the sections `Early Life', `Presidency', `Family and Personal Life' etc. Article-Section Ground-truth. The dataset consists of the triple INLINEFORM0 , where INLINEFORM1 , where we assume that INLINEFORM2 has already been determined as relevant. We therefore have a multi-class classification problem where we need to determine the section of INLINEFORM3 where INLINEFORM4 is cited.  
 Question: How do they determine the exact section to use the input article?
Output: They use a multi-class classifier to determine the section it should be cited
We conduct all our experiments on 635hrs of audio data for 7 Indian languages collected from $\textbf {All India Radio}$ news channel.  We collected and curated around 635Hrs of audio data for 7 Indian languages, namely Kannada, Hindi, Telugu, Malayalam, Bengali, and English. We collected the data from the All India Radio news channel where an actor will be reading news for about 5-10 mins. To cover many speakers for the dataset, we crawled data from 2010 to 2019. Since the audio is very long to train any deep neural network directly, we segment the audio clips into smaller chunks using Voice activity detector. Since the audio clips will have music embedded during the news, we use Inhouse music detection model to remove the music segments from the dataset to make the dataset clean and our dataset contains 635Hrs of clean audio which is divided into 520Hrs of training data containing 165K utterances and 115Hrs of testing data containing 35K utterances.  
 Question: How was the audio data gathered?
Output: Through the All India Radio new channel where actors read news.
We use the BLEU BIBREF30 metric on the validation set for the VQG model training. BLEU is a measure of similitude between generated and target sequences of words, widely used in natural language processing. It assumes that valid generated responses have significant word overlap with the ground truth responses. We use it because in this case we have five different references for each of the generated questions. We obtain a BLEU score of 2.07.

Our chatbot model instead, only have one reference ground truth in training when generating a sequence of words. We considered that it was not a good metric to apply as in some occasions responses have the same meaning, but do not share any words in common. Thus, we save several models with different hyperparameters and at different number of training iterations and compare them using human evaluation, to chose the model that performs better in a conversation. 
 Question: How is performance of this system measured?
Output: using the BLEU score as a quantitative metric and human evaluation for quality
(proposed) Bi-LSTM/CRF + Bi-CharLSTM with modality attention (W+C): uses the modality attention to merge word and character embeddings.

(proposed) Bi-LSTM/CRF + Bi-CharLSTM + Inception (W+C+V): takes as input visual contexts extracted from InceptionNet as well, concatenated with word and char vectors.

(proposed) Bi-LSTM/CRF + Bi-CharLSTM + Inception with modality attention (W+C+V): uses the modality attention to merge word, character, and visual embeddings as input to entity LSTM. 
 Question: Does their NER model learn NER from both text and images?
Output: Yes
FarsNet [20] [21] is the first WordNet for Persian, developed by the NLP Lab at Shahid Beheshti University and it follows the same structure as the original WordNet. 
 Question: What is the WordNet counterpart for Persian?
Output: FarsNet
Here I would use Informap algorithm BIBREF12. To make a comparison to this method, I am using CCM and SCA for distance measurement in this experiment, too. UPGMA algorithm would be used accordingly in these two cases. 
 Question: Is the proposed method compared to previous methods?
Output: Yes
Our Transformer Transducer model architecture has 18 audio and 2 label encoder layers. Every layer is identical for both audio and label encoders. The details of computations in a layer are shown in Figure FIGREF10 and Table TABREF11. All the models for experiments presented in this paper are trained on 8x8 TPU with a per-core batch size of 16 (effective batch size of 2048). 
 Question: Does model uses pretrained Transformer encoders?
Output: No
Unsupervised Evaluation
The unsupervised tasks include five tasks from SemEval Semantic Textual Similarity (STS) in 2012-2016 BIBREF30 , BIBREF31 , BIBREF32 , BIBREF33 , BIBREF34 and the SemEval2014 Semantic Relatedness task (SICK-R) BIBREF35 .

The cosine similarity between vector representations of two sentences determines the textual similarity of two sentences, and the performance is reported in Pearson's correlation score between human-annotated labels and the model predictions on each dataset.

Supervised Evaluation
It includes Semantic relatedness (SICK) BIBREF35 , SemEval (STS-B) BIBREF36 , paraphrase detection (MRPC) BIBREF37 , question-type classification (TREC) BIBREF38 , movie review sentiment (MR) BIBREF39 , Stanford Sentiment Treebank (SST) BIBREF40 , customer product reviews (CR) BIBREF41 , subjectivity/objectivity classification (SUBJ) BIBREF42 , opinion polarity (MPQA) BIBREF43 . 
 Question: How do they evaluate the sentence representations?
Output: The unsupervised tasks include five tasks from SemEval Semantic Textual Similarity (STS) in 2012-2016 BIBREF30 , BIBREF31 , BIBREF32 , BIBREF33 , BIBREF34 and the SemEval2014 Semantic Relatedness task (SICK-R) BIBREF35 .

The cosine similarity between vector representations of two sentences determines the textual similarity of two sentences, and the performance is reported in Pearson's correlation score between human-annotated labels and the model predictions on each dataset. Supervised Evaluation
It includes Semantic relatedness (SICK) BIBREF35 , SemEval (STS-B) BIBREF36 , paraphrase detection (MRPC) BIBREF37 , question-type classification (TREC) BIBREF38 , movie review sentiment (MR) BIBREF39 , Stanford Sentiment Treebank (SST) BIBREF40 , customer product reviews (CR) BIBREF41 , subjectivity/objectivity classification (SUBJ) BIBREF42 , opinion polarity (MPQA) BIBREF43 .
As we mention in the Section 2, CAEVO is the current state-of-the-art system for feature-based temporal event relation extraction BIBREF10 . It's widely used as the baseline for evaluating TB-Dense data. We adopt it as our baseline for evaluating CaTeRS and RED datasets. 
 Question: What type of baseline are established for the two datasets?
Output: CAEVO
Results on LangID and NoLangID are compared to the system presented by deri2016grapheme, which is identified in our results as wFST. 
 Question: what was the baseline?
Output: system presented by deri2016grapheme
To evaluate the responses generated by all compared methods, we compute the following automatic metrics on our test set:

BLEU: BLEU-n measures the average n-gram precision on a set of reference responses. We report BLEU-n with n=1,2,3,4.

Distinct-1 & distinct-2 BIBREF5: We count the numbers of distinct uni-grams and bi-grams in the generated responses and divide the numbers by the total number of generated uni-grams and bi-grams in the test set. These metrics can be regarded as an automatic metric to evaluate the diversity of the responses. 
 Question: What automatic metrics are used?
Output: BLEU Distinct-1 & distinct-2
We use 10-fold cross-validation, and only two types of features: n-grams and Word2Vec word embeddings.  The n-gram features include unigrams, bigrams, and trigrams, including sequences of punctuation (for example, ellipses or "!!!"), and emoticons. We use GoogleNews Word2Vec features BIBREF28 . 
 Question: What simple features are used?
Output: unigrams, bigrams, and trigrams, including sequences of punctuation Word2Vec word embeddings
The Flickr30K dataset BIBREF0 is a collection of over 30,000 images with 5 crowdsourced descriptions each. It is commonly used to train and evaluate neural network models that generate image descriptions (e.g. BIBREF2 ). This paper aims to give an overview of linguistic bias and unwarranted inferences resulting from stereotypes and prejudices. I will build on earlier work on linguistic bias in general BIBREF3 , providing examples from the Flickr30K data, and present a taxonomy of unwarranted inferences. Finally, I will discuss several methods to analyze the data in order to detect biases. 
 Question: What is the size of the dataset?
Output: 30,000
For example, the original sentence `We went shop on Saturday' and the corrected version `We went shopping on Saturday' would produce the following pattern:

(VVD shop_VV0 II, VVD shopping_VVG II)

After collecting statistics from the background corpus, errors can be inserted into error-free text. The learned patterns are now reversed, looking for the correct side of the tuple in the input sentence. We only use patterns with frequency INLINEFORM0 , which yields a total of 35,625 patterns from our training data.  
 Question: What textual patterns are extracted?
Output: (VVD shop_VV0 II, VVD shopping_VVG II)
Hence, the contributions of this work are two-fold: (1) we develop a linguistically-infused neural network model to classify reactions in social media posts, and (2) we apply our model to label 10.8M Twitter posts and 6.2M Reddit comments in order to evaluate the speed and type of user reactions to various news sources. We develop a neural network architecture that relies on content and other linguistic signals extracted from reactions and parent posts, and takes advantage of a “late fusion” approach previously used effectively in vision tasks BIBREF13 , BIBREF14 . More specifically, we combine a text sequence sub-network with a vector representation sub-network as shown in Figure FIGREF5 . The text sequence sub-network consists of an embedding layer initialized with 200-dimensional GloVe embeddings BIBREF15 followed by two 1-dimensional convolution layers, then a max-pooling layer followed by a dense layer. The vector representation sub-network consists of two dense layers. We incorporate information from both sub-networks through concatenated padded text sequences and vector representations of normalized Linguistic Inquiry and Word Count (LIWC) features BIBREF16 for the text of each post and its parent. 
 Question: What is the architecture of their model?
Output: we combine a text sequence sub-network with a vector representation sub-network as shown in Figure FIGREF5 . The text sequence sub-network consists of an embedding layer initialized with 200-dimensional GloVe embeddings BIBREF15 followed by two 1-dimensional convolution layers, then a max-pooling layer followed by a dense layer. The vector representation sub-network consists of two dense layers. We incorporate information from both sub-networks through concatenated padded text sequences and vector representations of normalized Linguistic Inquiry and Word Count (LIWC) features BIBREF16 for the text of each post and its parent.
We use the Dialogue State Tracking Competition 2 (DSTC2) dataset BIBREF27 which is the most widely used dataset for research on task-oriented chatbots.  We also used two other datasets recently open-sourced by Google Research BIBREF28 which are M2M-sim-M (dataset in movie domain) and M2M-sim-R (dataset in restaurant domain).  
 Question: What are the three datasets used?
Output: DSTC2 M2M-sim-M M2M-sim-R
In general, the model transforms the segments into object-pairs by the TC-CNN and passes sentence through bi-GRU to obtain the global representation. Then we integrate object-pairs with global representation and make a pair-wise inference to detect the relationship among the segments. Ablation studies show that the proposed SCRN at the segment level has the capacity for relational reasoning and promotes the result significantly. 
 Question: How is Relation network used to infer causality at segment level?
Output: we integrate object-pairs with global representation and make a pair-wise inference to detect the relationship among the segments
We collected all available comments in the stories from Reddit from August 2015.  
 Question: what is the source of the new dataset?
Output: Reddit
To exemplify the INLINEFORM0 score we evaluated and compared the performance of two different SBD systems over a set of YouTube videos in a multi-reference enviroment. The first system (S1) employs a Convolutional Neural Network to determine if the middle word of a sliding window corresponds to a SU boundary or not BIBREF30 . The second approach (S2) by contrast, introduces a bidirectional Recurrent Neural Network model with attention mechanism for boundary detection BIBREF31 . 
 Question: Which SBD systems did they compare?
Output: Convolutional Neural Network  bidirectional Recurrent Neural Network model with attention mechanism
Paraphrase Identification (PI) is the task of determining whether two sentences are paraphrase or not. We have implemented the cross-lingual variant of kernel functions for PI and RE tasks as described in section SECREF3 and measured the accuracy of models by testing them on the parallel data set. 
 Question: What classification task was used to evaluate the cross-lingual adaptation method described in this work?
Output: Paraphrase Identification
The system consists of: (i) Local, shallow features based mostly on orthographic, word shape and n-gram features plus their context; and (ii) three types of simple clustering features, based on unigram matching: (i) Brown BIBREF32 clusters, taking the 4th, 8th, 12th and 20th node in the path; (ii) Clark BIBREF33 clusters and, (iii) Word2vec BIBREF34 clusters, based on K-means applied over the extracted word vectors using the skip-gram algorithm. 
 Question: What shallow local features are extracted?
Output:  Local, shallow features based mostly on orthographic, word shape and n-gram features plus their context
We report segmentation performance using precision, recall, and F-measure on boundaries (BP, BR, BF), and tokens (WP, WR, WF). We also report the exact-match (X) metric which computes the proportion of correctly segmented utterances. 
 Question: How is the word segmentation task evaluated?
Output: precision, recall, and F-measure on boundaries (BP, BR, BF), and tokens (WP, WR, WF)  exact-match (X) metric
For DQA four participants answered each question, therefore we took the average INLINEFORM0 , INLINEFORM1 , and INLINEFORM2 values over the four evaluators as the result per question. 
 Question: How do they measure performance?
Output: average INLINEFORM0 , INLINEFORM1 , and INLINEFORM2 values
Slot filling models are a useful method for simple natural language understanding tasks, where information can be extracted from a sentence and used to perform some structured action As candidate tasks, we consider the actions that a user might perform via apps on their phone. Crowd-sourced data was collected simulating common use cases for four different apps: United Airlines, Airbnb, Greyhound bus service and OpenTable. The corresponding actions are booking a flight, renting a home, buying bus tickets, and making a reservation at a restaurant. 
 Question: What tasks are they experimenting with in this paper?
Output: Slot filling we consider the actions that a user might perform via apps on their phone The corresponding actions are booking a flight, renting a home, buying bus tickets, and making a reservation at a restaurant
Results form Table TABREF31 may give an idea that INLINEFORM0 is just an scaled INLINEFORM1 . While it is true that they show a linear correlation, INLINEFORM2 may produce a different system ranking than INLINEFORM3 given the integral multi-reference principle it follows. However, what we consider the most profitable about INLINEFORM4 is the twofold inclusion of all available references it performs. First, the construction of INLINEFORM5 to provide a more inclusive reference against to whom be evaluated and then, the computation of INLINEFORM6 , which scales the result depending of the agreement between references. We showed how INLINEFORM0 is an inclusive metric which not only evaluates the performance of a system against all references, but also takes into account the agreement between them.  
 Question: What makes it a more reliable metric?
Output: It takes into account the agreement between different systems
Given that there is no public dataset available with financial intents in Portuguese, we have employed the incremental approach to create our own training set for the Intent Classifier. We have created domain-specific word vectors by considering a set 246,945 documents, corresponding to of 184,001 Twitter posts and and 62,949 news articles, all related to finance . 
 Question: What datasets are used?
Output: Custom dataset with user questions; set of documents, twitter posts and news articles, all related to finance.
Results on all sets show an important difference between precision and recall, precision being significantly higher than recall. There is also a significant difference between the results obtained on development and test set (F1 = 89.60, F1 = 87.82) and the results on the supplemental test set (F1 = 71.49). The time difference between the supplemental test set and the development and test set (the headlines from the the supplemental test set being from a different time period to the training set) can probably explain these differences. 
 Question: What is the performance of the CRF model on the task described?
Output: the results obtained on development and test set (F1 = 89.60, F1 = 87.82) and the results on the supplemental test set (F1 = 71.49)
Although our approach strongly outperforms random baselines, the relatively low F1 scores indicate that predicting which word is echoed in explanations is a very challenging task. It follows that we are only able to derive a limited understanding of how people choose to echo words in explanations. The extent to which explanation construction is fundamentally random BIBREF47, or whether there exist other unidentified patterns, is of course an open question. We hope that our study and the resources that we release encourage further work in understanding the pragmatics of explanations. 
 Question: Do authors provide any explanation for intriguing patterns of word being echoed?
Output: No
Multi-task Pairwise Neural Ranking
We propose a multi-task pairwise neural ranking approach to better incorporate and distinguish the relative order between the candidate segmentations of a given hashtag. Our model adapts to address single- and multi-token hashtags differently via a multi-task learning strategy without requiring additional annotations. In this section, we describe the task setup and three variants of pairwise neural ranking models (Figure FIGREF11 ). Pairwise Neural Ranking Model Margin Ranking (MR) Loss Adaptive Multi-task Learning 
 Question: What set of approaches to hashtag segmentation are proposed?
Output: Adaptive Multi-task Learning
 Margin Ranking (MR) Loss
 Pairwise Neural Ranking Model
In ConvE, only $v_h$ and $v_r$ are reshaped and then concatenated into an input matrix which is fed to the convolution layer 
 Question: Did the authors try stacking multiple convolutional layers?
Output: No
To identify the most suitable classifier for classifying the scalars associated with each text, we perform evaluations using the stochastic gradient descent, naive bayes, decision tree, and random forest classifiers. 
 Question: What was the baseline?
Output: stochastic gradient descent, naive bayes, decision tree
In order to obtain a large number of Brazilian music lyrics, we created a crawler to navigate into the Vagalume website, extracting, for each musical genre, all the songs by all the listed authors. 
 Question: what is the source of the song lyrics?
Output: Vagalume website
Our data are collected from the yelp academic dataset, provided by Yelp.com, a popular restaurant review website. The data set contains three types of objects: business, user, and review, where business objects contain basic information about local businesses (i.e. restaurants), review objects contain review texts and star rating, and user objects contain aggregate information about a single user across all of Yelp. Table TABREF31 illustrates the general statistics of the dataset. 
 Question: Does they focus on any specific product/service domain?
Output: local businesses (i.e. restaurants)
Following titovcrosslingual, we run our experiments on the English (EN) and German (DE) sections of the CoNLL 2009 corpus BIBREF13 , and EN-DE section of the Europarl corpus BIBREF14 . 
 Question: Which parallel corpora are used?
Output: English (EN) and German (DE) sections of the CoNLL 2009 corpus BIBREF13 EN-DE section of the Europarl corpus BIBREF14
KGR10, also known as plWordNet Corpus 10.0 (PLWNC 10.0), is the result of the work on the toolchain to automatic acquisition and extraction of the website content, called CorpoGrabber BIBREF19 . It is a pipeline of tools to get the most relevant content of the website, including all subsites (up to the user-defined depth). The proposed toolchain can be used to build a big Web corpus of text documents. It requires the list of the root websites as the input. Tools composing CorpoGrabber are adapted to Polish, but most subtasks are language independent. The whole process can be run in parallel on a single machine and includes the following tasks: download of the HTML subpages of each input page URL with HTTrack, extraction of plain text from each subpage by removing boilerplate content (such as navigation links, headers, footers, advertisements from HTML pages) BIBREF20 , deduplication of plain text BIBREF20 , bad quality documents removal utilising Morphological Analysis Converter and Aggregator (MACA) BIBREF21 , documents tagging using Wrocław CRF Tagger (WCRFT) BIBREF22 . Last two steps are available only for Polish. 
 Question: How was the KGR10 corpus created?
Output: most relevant content of the website, including all subsites
The Y-axis in the figure is the success rate of the agent (measured in terms of number of dialogs that resulted in launching a skill divided by total number of dialogs), and the X-axis is the number of learning steps.  
 Question: How did they measure effectiveness?
Output: number of dialogs that resulted in launching a skill divided by total number of dialogs
We used dataset, HEOT obtained from one of the past studies done by Mathur et al. where they annotated a set of cleaned tweets obtained from twitter for the conversations happening in Indian subcontinent. A labelled dataset for a corresponding english tweets were also obtained from a study conducted by Davidson et al. 
 Question: What dataset is used?
Output: HEOT  A labelled dataset for a corresponding english tweets
We extracted 200 sentence pairs from BIBREF3 's dataset and provided each pair with a document context consisting of a preceding and a following sentence, as in the following example. 
 Question: What document context was added?
Output: Preceding and following sentence of each metaphor and paraphrase are added as document context
In this section, we analyze the Hamming distance between the projections of the sentences from the enwik9 dataset and the corresponding projections of the same sentences after applying character level perturbations. We experiment with three types of character level perturbation BIBREF11 and two types of word level perturbation operations.

Perturbation Study ::: Character Level Perturbation Operations
insert(word, n) : We randomly choose n characters from the character vocabulary and insert them at random locations into the input word. We however retain the first and last characters of the word as is. Ex. transformation: $sample \rightarrow samnple$.

swap(word, n): We randomly swap the location of two characters in the word n times. As with the insert operation, we retain the first and last characters of the word as is and only apply the swap operation to the remaining characters. Ex. transformation: $sample \rightarrow sapmle$.

duplicate(word, n): We randomly duplicate a character in the word by n times. Ex. transformation: $sample \rightarrow saample$.

Perturbation Study ::: Character Level Perturbation Operations ::: Word Level Perturbation Operations
drop(sentence, n): We randomly drop n words from the sentence. Ex. transformation: This is a big cat. $\rightarrow $ This is a cat.

duplicate(sentence, n): Similar to duplicate(word, n) above, we randomly duplicate a word in the sentence n times. Ex. transformation: This is a big cat. $\rightarrow $ This is a big big cat.

swap(sentence, n): Similar to swap(word, n), we randomly swap the location of two words in the sentence n times. Ex. transformation: This is a big cat. $\rightarrow $ This cat is big. 
 Question: How does their perturbation algorihm work?
Output: same sentences after applying character level perturbations
Methods ::: Length Encoding Method
Inspired by BIBREF11, we use length encoding to provide the network with information about the remaining sentence length during decoding. We propose two types of length encoding: absolute and relative. Let pos and len be, respectively, a token position and the end of the sequence, both expressed in terms of number characters. Then, the absolute approach encodes the remaining length:

where $i=1,\ldots ,d/2$.

Similarly, the relative difference encodes the relative position to the end. This representation is made consistent with the absolute encoding by quantizing the space of the relative positions into a finite set of $N$ integers:

where $q_N: [0, 1] \rightarrow \lbrace 0, 1, .., N\rbrace $ is simply defined as $q_N(x) = \lfloor {x \times N}\rfloor $. As we are interested in the character length of the target sequence, len and pos are given in terms of characters, but we represent the sequence as a sequence of BPE-segmented subwords BIBREF17. To solve the ambiguity, len is the character length of the sequence, while pos is the character count of all the preceding tokens. 
 Question: How do they enrich the positional embedding with length information
Output: They introduce new trigonometric encoding which besides information about position uses additional length information (abs or relative).
Advanced neural architectures based on character input (CNNs, BPE, etc) are supposed to be able to learn how to handle spelling and morphology variations themselves, even for languages with rich morphology: `just add more layers!'. Contextualised embedding models follow this tradition: as a rule, they are trained on raw text collections, with minimal linguistic pre-processing. Below, we show that this is not entirely true. 
 Question: Why is lemmatization not necessary in English?
Output: Advanced neural architectures and contextualized embedding models learn how to handle spelling and morphology variations.
We opted for an automatic approach instead, that can be scaled arbitrarily and at little cost: we generate synthetic sentence pairs $(, \tilde{})$ by randomly perturbing 1.8 million segments $$ from Wikipedia. We use three techniques: mask-filling with BERT, backtranslation, and randomly dropping out words.  
 Question: How are the synthetic examples generated?
Output: Random perturbation of Wikipedia sentences using mask-filling with BERT, backtranslation and randomly drop out
For both datasets, we follow the evaluation metrics used in the original evaluation tasks BIBREF13 . For DBQA, P@1 (Precision@1), MAP (Mean Average Precision) and MRR (Mean Reciprocal Rank) are adopted. For KBRE, since only one golden candidate is labeled for each question, only P@1 and MRR are used. 
 Question: Which metrics do they use to evaluate matching?
Output: Precision@1 Mean Average Precision Mean Reciprocal Rank
Note that GANE-AP delivers better results compared with GANE-OT, suggesting the attention parsing mechanism can further improve the low-level mutual attention matrix. 
 Question: Which of their proposed attention methods works better overall?
Output: attention parsing
We use TB-Dense and MATRES in our experiments and briefly summarize the data statistics in Table TABREF33. 
 Question: What datasets were used for this work?
Output: TB-Dense  MATRES
WikiSmall consists of ordinary and simplified sentences from the ordinary and simple English Wikipedias, which has been used as benchmark for evaluating text simplification BIBREF17 , BIBREF18 , BIBREF8 . The training set has 89,042 sentence pairs, and the test set has 100 pairs. WikiLarge is also from Wikipedia corpus whose training set contains 296,402 sentence pairs BIBREF19 , BIBREF20 . WikiLarge includes 8 (reference) simplifications for 2,359 sentences split into 2,000 for development and 359 for testing. 
 Question: what are the sizes of both datasets?
Output: training set has 89,042 sentence pairs, and the test set has 100 pairs training set contains 296,402 2,000 for development and 359 for testing
Another problem we face is that the BERT model released by Google is trained only on Arabic Wikipedia, which is almost exclusively Modern Standard Arabic (MSA). The shared task dataset contains 5,030 tweets related to different political issues and events in the Middle East taking place between 2011 and 2018. Tweets involve both MSA as well as dialects at various degrees of granularity such as Egyptian, Gulf, and Levantine. 
 Question: What dialect is used in the Google BERT model and what is used in the task data?
Output: Modern Standard Arabic (MSA) MSA as well as dialects at various degrees of granularity such as Egyptian, Gulf, and Levantine
Using a set of semantically oriented tasks that require explicit semantic cross-lingual representations, we showed that mBERT contextual embeddings do not represent similar semantic phenomena similarly and therefore they are not directly usable for zero-shot cross-lingual tasks. 
 Question: What challenges this work presents that must be solved to build better language-neutral representations?
Output: contextual embeddings do not represent similar semantic phenomena similarly and therefore they are not directly usable for zero-shot cross-lingual tasks
The final annotation categories for the dataset are: Joy, Sadness, Anger, Fear, Anticipation, Surprise, Love, Disgust, Neutral 
 Question: How many emotions do they look at?
Output: 9
As a first baseline we use Bag-of-Words, a well-known and robust text representations used in various domains BIBREF21 , combined with a standard shallow classifier, namely, a Support Vector Machine with linear kernel. We used LIBSVM implementation of SVM. 
 Question: Which shallow approaches did they experiment with?
Output: SVM
Among all single models, LFT performs the best, followed by MinAvgOut. RL is also comparable with previous state-of-the-art models VHRED (attn) and Reranking-RL. We think that this is because LFT exerts no force in pulling the model predictions away from the ground-truth tokens, but rather just makes itself aware of how dull each response is. Consequently, its responses appear more relevant than the other two approaches. Moreover, the hybrid model (last row) outperforms all other models by a large margin.  
 Question: Which one of the four proposed models performed best?
Output: the hybrid model MinAvgOut + RL
The task is set up to mimic (albeit, in an oversimplified manner) the input-output symbol alignments and local syntactic properties that models must learn in many natural language tasks, such as translation, tagging and summarization. 
 Question: Why does the proposed task a good proxy for the general-purpose sequence to sequence tasks?
Output: The task is set up to mimic (albeit, in an oversimplified manner) the input-output symbol alignments and local syntactic properties that models must learn in many natural language tasks, such as translation, tagging and summarization.
We first investigated overall detection of any NMT-Fake reviews (1,006 fake reviews and 994 real reviews). 
 Question: How many reviews in total (both generated and true) do they evaluate on Amazon Mechanical Turk?
Output: 1,006 fake reviews and 994 real reviews
We evaluate our model on two benchmark treebanks, English Penn Treebank (PTB) and Chinese Penn Treebank (CTB5.1) following standard data splitting BIBREF30, BIBREF31. POS tags are predicted by the Stanford Tagger BIBREF32. For constituent parsing, we use the standard evalb tool to evaluate the F1 score. For dependency parsing, we apply Stanford basic dependencies (SD) representation BIBREF4 converted by the Stanford parser. Following previous work BIBREF27, BIBREF33, we report the results without punctuations for both treebanks. Tables TABREF17, TABREF18 and TABREF19 compare our model to existing state-of-the-art, in which indicator Separate with our model shows the results of our model learning constituent or dependency parsing separately, (Sum) and (Concat) respectively represent the results with the indicated input token representation setting. On PTB, our model achieves 93.90 F1 score of constituent parsing and 95.91 UAS and 93.86 LAS of dependency parsing. On CTB, our model achieves a new state-of-the-art result on both constituent and dependency parsing. At last, our model is evaluated on two benchmark treebanks for both constituent and dependency parsing. The empirical results show that our parser reaches new state-of-the-art for all parsing tasks. 
 Question: How is dependency parsing empirically verified?
Output:  At last, our model is evaluated on two benchmark treebanks for both constituent and dependency parsing. The empirical results show that our parser reaches new state-of-the-art for all parsing tasks.
Knowledge Base Question Answering (KBQA) systems answer questions by obtaining information from KB tuples BIBREF0 , BIBREF1 , BIBREF2 , BIBREF3 , BIBREF4 , BIBREF5 
 Question: What does KBQA abbreviate for
Output: Knowledge Base Question Answering
After using Kytea, Japanese texts are applied LSW algorithm to replace OOV words by their synonyms.  
 Question: Are synonymous relation taken into account in the Japanese-Vietnamese task?
Output: Yes