niryuu/sni-each-converted huggingface dataset:
Since labels are unknown on INLINEFORM1 , EGL computes the expectation of the gradient norm over all possible labelings. BIBREF3 interprets EGL as “expected model change”. In the following section, we formalize the intuition for EGL and show that it follows naturally from reducing the variance of an estimator. Eq. ( EQREF7 ) indicates that to reduce INLINEFORM0 on test data, we need to minimize the expected variance INLINEFORM1 over the test set. A practical issue is that we do not know INLINEFORM0 in advance. We could instead substitute an estimate INLINEFORM1 from a pre-trained model, where it is reasonable to assume the INLINEFORM2 to be close to the true INLINEFORM3 . The batch selection then works by taking the samples that have largest gradient norms, DISPLAYFORM0

For RNNs, the gradients for each potential label can be obtained by back-propagation. Another practical issue is that EGL marginalizes over all possible labelings, but in speech recognition, the number of labelings scales exponentially in the number of timesteps. Therefore, we only marginalize over the INLINEFORM0 most probable labelings. They are obtained by beam search decoding, as in BIBREF7 . The EGL method in BIBREF3 is almost the same as Eq. ( EQREF8 ), except the gradient's norm is not squared in BIBREF3 . 
 Question: How do they calculate variance from the model outputs?
Output: reducing the variance of an estimator  EGL method in BIBREF3 is almost the same as Eq. ( EQREF8 ), except the gradient's norm is not squared in BIBREF3
Our AV in-cabin data-set includes 30 hours of multimodal data collected from 30 passengers (15 female, 15 male) in 20 rides/sessions. 10 types of passenger intents are identified and annotated as: Set/Change Destination, Set/Change Route (including turn-by-turn instructions), Go Faster, Go Slower, Stop, Park, Pull Over, Drop Off, Open Door, and Other (turn music/radio on/off, open/close window/trunk, change AC/temp, show map, etc.). Relevant slots are identified and annotated as: Location, Position/Direction, Object, Time-Guidance, Person, Gesture/Gaze (this, that, over there, etc.), and None. In addition to utterance-level intent types and their slots, word-level intent keywords are annotated as Intent as well.  
 Question: What intents does the paper explore?
Output: Set/Change Destination Set/Change Route Go Faster Go Slower Stop Park Pull Over Drop Off Open Door Other
Throughout this article, we use two different embedding spaces. The first is the widely used representation built on GoogleNews BIBREF8 . The second is taken from BIBREF2 , and was trained on a Reddit dataset BIBREF9 . 
 Question: Which embeddings do they detect biases in?
Output: Word embeddings trained on GoogleNews and Word embeddings trained on Reddit dataset
Our analysis found that gang members have a penchant for using just a small set of emoji symbols that convey their anger and violent behavior through their tweets. Figure FIGREF24 illustrates the emoji distribution for the top 20 most frequent emojis used by gang member profiles in our dataset. The fuel pump emoji was the most frequently used emoji by the gang members, which is often used in the context of selling or consuming marijuana. The pistol emoji is the second most frequent in our dataset, which is often used with the guardsman emoji or the police cop emoji in an `emoji chain'. Figure FIGREF28 presents some prototypical `chaining' of emojis used by gang members. The chains may reflect their anger at law enforcement officers, as a cop emoji is often followed up with the emoji of a weapon, bomb, or explosion. We found that 32.25% of gang members in our dataset have chained together the police and the pistol emoji, compared to just 1.14% of non-gang members. Moreover, only 1.71% of non-gang members have used the hundred points emoji and pistol emoji together in tweets while 53% of gang members have used them. A variety of the angry face emoji such as devil face emoji and imp emoji were also common in gang member tweets. 
 Question: What are the differences in the use of emojis between gang member and the rest of the Twitter population?
Output: 32.25% of gang members in our dataset have chained together the police and the pistol emoji, compared to just 1.14% of non-gang members only 1.71% of non-gang members have used the hundred points emoji and pistol emoji together in tweets while 53% of gang members have used them gang members have a penchant for using just a small set of emoji symbols that convey their anger and violent behavior
The analysis of results from Tables TABREF17 , TABREF18 and TABREF19 show that 12 of 15 best results were obtained using new word embeddings. The evaluation results presented in Table TABREF20 (the chosen best embeddings models from Table TABREF19 ) prove that the best group of word embeddings is EC. The highest type F1-score was obtained for EC1 model, built using binary FastText Skip-gram method utilising subword information, with vector dimension equal to 300 and negative sampling equal to 10. The ability of the model to provide vector representation for the unknown words seems to be the most important. 
 Question: What conclusions are drawn from these experiments?
Output: best results were obtained using new word embeddings best group of word embeddings is EC The highest type F1-score was obtained for EC1 model, built using binary FastText Skip-gram method utilising subword information ability of the model to provide vector representation for the unknown words seems to be the most important
For evaluating the proposed model on behavior related data, we employ the Couples Therapy Corpus (CoupTher) BIBREF21 and Cancer Couples Interaction Dataset (Cancer) BIBREF22. These are the targeted conditions under which a behavior-gated language model can offer improved performance. We utilize the Couple's Therapy Corpus as an in-domain experimental corpus since our behavior classification model is also trained on the same.  
 Question: On which dataset is model trained?
Output: Couples Therapy Corpus (CoupTher) BIBREF21
We perform experiments (§ SECREF5 ) on two English-Japanese translation corpora to evaluate the method's utility in improving translation accuracy and reducing the time required for training. 
 Question: What language pairs did they experiment with?
Output: English-Japanese
This study focuses on Switchboard-300, a standard 300-hour English conversational speech recognition task. 
 Question: How big is Switchboard-300 database?
Output: 300-hour English conversational speech
We use the same baseline as used by lang2011unsupervised which has been shown to be difficult to outperform. This baseline assigns a semantic role to a constituent based on its syntactic function, i.e. the dependency relation to its head. 
 Question: What baseline model is used?
Output: same baseline as used by lang2011unsupervised
In recent years, there has been a rapid growth in the usage of social media. People post their day-to-day happenings on regular basis. BIBREF0 propose four tasks for detecting drug names, classifying medication intake, classifying adverse drug reaction and detecting vaccination behavior from tweets. We participated in the Task2 and Task4. 
 Question: Was the system only evaluated over the second shared task?
Output: No
We use the following subset of GLUE tasks BIBREF4 for fine-tuning:

MRPC: the Microsoft Research Paraphrase Corpus BIBREF13

STS-B: the Semantic Textual Similarity Benchmark BIBREF14

SST-2: the Stanford Sentiment Treebank, two-way classification BIBREF15

QQP: the Quora Question Pairs dataset

RTE: the Recognizing Textual Entailment datasets

QNLI: Question-answering NLI based on the Stanford Question Answering Dataset BIBREF3

MNLI: the Multi-Genre Natural Language Inference Corpus, matched section BIBREF16 
 Question: What subset of GLUE tasks is used?
Output: MRPC STS-B SST-2 QQP RTE QNLI MNLI
We define the Effective Word Score of score x as

EFWS(x) = N(+x) - N(-x),

where N(x) is the number of words in the tweet with polarity score x. 
 Question: How is effective word score calculated?
Output: We define the Effective Word Score of score x as

EFWS(x) = N(+x) - N(-x),

where N(x) is the number of words in the tweet with polarity score x.
The SemEval-2010 benchmark dataset BIBREF0 is composed of 244 scientific articles collected from the ACM Digital Library (conference and workshop papers). 
 Question: how many articles are in the dataset?
Output: 244
Each testing instance was paired with 99 randomly sampled negative instances. Each recommendation model ranks the 100 instances according to its predicted results. The ranked list is judged by Hit Ratio (HR) BIBREF49 and Normalized Discount Cumulative Gain (NDCG) BIBREF50 at the position 10. HR@10 is a recall-based metric, measuring the percentage of the testing item being correctly recommended in the top-10 position. NDCG@10 is a ranked evaluation metric which considers the position of the correct hit in the ranked result. Overall, our AMRAN outperforms all baselines, achieving 0.657 HR@10 and 0.410 NDCG@10. It improves HR@10 by 5.3% and NDCG@10 by 3% over the best baseline (i.e., $NAIS_{concat}$). 
 Question: What is the model accuracy?
Output: Overall, our AMRAN outperforms all baselines, achieving 0.657 HR@10 and 0.410 NDCG@10.
For reliable multi-hop reasoning using OpenIE tuples, we can add inter-tuple connections to the support graph search, controlled by a small number of rules over the OpenIE predicates. Learning such rules for the Science domain is an open problem and potential avenue of future work. 
 Question: Is their method capable of multi-hop reasoning?
Output: Yes
After adversarial modifications, the performance of the original target models (those without the “-adv” suffix) drops dramatically (e.g. the overall accuracy of BERT on Quora drops from 94.6% to 24.1%), revealing that the target models are vulnerable to our adversarial examples. 
 Question: How much dramatically results drop for models on generated adversarial examples?
Output: BERT on Quora drops from 94.6% to 24.1%
Both beat the non-attention baseline by a significant margin. For the attention baseline, we use the standard parametrized attention BIBREF2 . 
 Question: Which baseline methods are used?
Output: standard parametrized attention and a non-attention baseline
Therefore, we just implement a simple one-layer RNN with LSTM cell to classify pre-processed sentences into ironic sentences and non-ironic sentences because LSTM networks are widely used in irony detection. 
 Question: How did the authors find ironic data on twitter?
Output: They developed a classifier to find ironic sentences in twitter data
We address this tension by training vector space models to represent the data, in which each unique word in a large corpus is represented by a vector (embedding) in high-dimensional space. The geometry of the resulting vector space captures many semantic relations between words.  We address this tension by training vector space models to represent the data, in which each unique word in a large corpus is represented by a vector (embedding) in high-dimensional space. The geometry of the resulting vector space captures many semantic relations between words.  
 Question: Do they model semantics 
Output: Yes
Table TABREF34 shows the inference speed of our method when implementing the sequnece modeling layer with the LSTM-based, CNN-based, and Transformer-based architecture, respectively.  From the table, we can see that our method has a much faster inference speed than Lattice-LSTM when using the LSTM-based sequence modeling layer, and it was also much faster than LR-CNN, which used an CNN architecture to implement the sequence modeling layer. And as expected, our method with the CNN-based sequence modeling layer showed some advantage in inference speed than those with the LSTM-based and Transformer-based sequence model layer. 
 Question:  What percentage of improvement in inference speed is obtained by the proposed method over the newest state-of-the-art methods?
Output: Across 4 datasets, the best performing proposed model (CNN) achieved an average of 363% improvement over the state of the art method (LR-CNN)
This baseline has an F1 of 41.59% and accuracy of 71.22%. 
 Question: what are their evaluation metrics?
Output: F1 accuracy
The vector gate is inspired by BIBREF11 and BIBREF12 , but is different to the former in that the gating mechanism acts upon each dimension of the word and character-level vectors, and different to the latter in that it does not rely on external sources of information for calculating the gating mechanism. 
 Question: Where do they employ feature-wise sigmoid gating?
Output: gating mechanism acts upon each dimension of the word and character-level vectors
The Software Ontology (SWO) BIBREF5 is included because its set of CQs is of substantial size and it was part of Ren et al.'s set of analysed CQs. The CQ sets of Dem@Care BIBREF8 and OntoDT BIBREF9 were included because they were available. CQs for the Stuff BIBREF6 and African Wildlife (AWO) BIBREF7 ontologies were added to the set, because the ontologies were developed by one of the authors (therewith facilitating in-depth domain analysis, if needed), they cover other topics, and are of a different `type' (a tutorial ontology (AWO) and a core ontology (Stuff)), thus contributing to maximising diversity in source selection. 
 Question: How many domains of ontologies do they gather data from?
Output: 5 domains: software, stuff, african wildlife, healthcare, datatypes
The competition is divided into five subtasks which involve standard classification, ordinal classification and distributional estimation. For a more detailed description see BIBREF0 . 
 Question: What were the five English subtasks?
Output:  five subtasks which involve standard classification, ordinal classification and distributional estimation. For a more detailed description see BIBREF0
When we include the LM embeddings in our system overall performance increases from 90.87% to 91.93% INLINEFORM0 for the CoNLL 2003 NER task, a more then 1% absolute F1 increase, and a substantial improvement over the previous state of the art. We also establish a new state of the art result (96.37% INLINEFORM1 ) for the CoNLL 2000 Chunking task. 
 Question: what results do they achieve?
Output: 91.93% F1 score on CoNLL 2003 NER task and 96.37% F1 score on CoNLL 2000 Chunking task
We tried many configurations of our network models, but report results with only three configurations of Transformers.

Transformer Type 1: This network is a small to medium-sized network consisting of 4 Transformer layers. Each layer utilizes 8 attention heads with a depth of 512 and a feed-forward depth of 1024.

Transformer Type 2: The second model is small in size, using 2 Transformer layers. The layers utilize 8 attention heads with a depth of 256 and a feed-forward depth of 1024.

Transformer Type 3: The third type of model is minimal, using only 1 Transformer layer. This network utilizes 8 attention heads with a depth of 256 and a feed-forward depth of 512. 
 Question: What neural configurations are explored?
Output: tried many configurations of our network models, but report results with only three configurations Transformer Type 1 Transformer Type 2 Transformer Type 3
For answer retrieval, a dataset is created by INLINEFORM4 , which gives INLINEFORM5 accuracy and INLINEFORM6 coverage, respectively. 
 Question: Do they employ their indexing-based method to create a sample of a QA Wikipedia dataset?
Output: Yes
In a second step, we will train a neural monolingual translation system, that translates from the output of the PBMT system INLINEFORM0 to a better target sentence INLINEFORM1 . 
 Question: Do they train the NMT model on PBMT outputs?
Output: Yes
We extract unique users and tweets from the combined result set to generate a dataset of about 60,000 unique tweets, pertaining to 51,104 unique users. 
 Question: How many followers did they analyze?
Output: 51,104
We need good amount of data to try deep learning state-of-the-art algorithms. There are lot of open datasets available for names, locations, organisations, but not for topics as defined in Abstract above. Also defining and inferring topics is an individual preference and there are no fix set of rules for its definition. But according to our definition, we can use wikipedia titles as our target topics. English wikipedia dataset has more than 18 million titles if we consider all versions of them till now. We had to clean up the titles to remove junk titles as wikipedia title almost contains all the words we use daily.  After doing some more cleaning we were left with 10 million titles. We have a dump of 15 million English news articles published in past 4 years. Further, we reduced number of articles by removing duplicate and near similar articles. We used our pre-trained doc2vec models and cosine similarity to detect almost similar news articles. 
 Question: How large is the dataset they used?
Output: English wikipedia dataset has more than 18 million a dump of 15 million English news articles
We have training and testing sets in three different languages: English, Chinese and Korean. When fine-tuning, we simply adopted the official training script of BERT, with default hyperparameters, to fine-tune each model until training loss converged. 
 Question: what does the model learn in zero-shot setting?
Output: we simply adopted the official training script of BERT, with default hyperparameters, to fine-tune each model until training loss converged
. While automated evaluation metrics like ROUGE measure lexical similarity between machine and human summaries, humans can better measure how coherent and readable a summary is. Our evaluation study investigates whether tuning the PG-net model increases summary coherence, by asking evaluators to select which of three summaries for the same document they like most: the PG-net model trained on CNN/DM; the model trained on student reflections; and finally the model trained on CNN/DM and tuned on student reflections. 20 evaluators were recruited from our institution and asked to each perform 20 annotations. Summaries are presented to evaluators in random order. Evaluators are then asked to select the summary they feel to be most readable and coherent. 
 Question: Who were the human evaluators used?
Output: 20 evaluators were recruited from our institution and asked to each perform 20 annotations
As shown in Equation ( EQREF6 ), tone prediction sub-network ( INLINEFORM0 ) takes video and pinyin sequence as inputs and predict corresponding tone sequence. Video context vectors INLINEFORM0 and pinyin context vectors INLINEFORM1 are fused when predicting a tone character at each decoder step. The video encoder is the same as in Section SECREF7 and the pinyin encoder is: DISPLAYFORM0 The input video sequence is first fed into the VGG model BIBREF9 to extract visual feature. The output of conv5 of VGG is appended with global average pooling BIBREF10 to get the 512-dim feature vector. Then the 512-dim feature vector is fed into video encoder. 
 Question: What visual information characterizes tones?
Output: video sequence is first fed into the VGG model BIBREF9 to extract visual feature
To test this hypothesis we first trained UG-WGAN in English, Chinese and German following the procedure described in Section "UG-WGAN" . For this experiment we trained UG-WGAN on the English and Russian language following the procedure described in Section "UG-WGAN" .  To do so we trained 6 UG-WGAN models on the following languages: English, Russian, Arabic, Chinese, German, Spanish, French. 
 Question: What are the languages they consider in this paper?
Output: The languages considered were English, Chinese, German, Russian, Arabic, Spanish, French
We only used models of dimension 300 and, for Word2Vec, Wang2Vec, and FastText, skip-gram architectured models. 
 Question: what word embedding techniques did they experiment with?
Output: Word2Vec, Wang2Vec, and FastText
The baselines are: (a) trained on S-SQuAD, (b) trained on T-SQuAD and then fine-tuned on S-SQuAD, and (c) previous best model trained on S-SQuAD BIBREF5 by using Dr.QA BIBREF20 . The proposed approach (row (f)) outperforms previous best model (row (c)) by 2% EM score and over 1.5% F1 score. 
 Question: What was the previous best model?
Output: (c) previous best model trained on S-SQuAD BIBREF5 by using Dr.QA BIBREF20
We downloaded 76 videos from a tutorial website about an image editing program .  
 Question: What is the source of the triples?
Output: a tutorial website about an image editing program
Excluding the embedding weights, our model requires 100k parameters , while BIBREF8 requires 250k parameters. 
 Question: how much was the parameter difference between their model and previous methods?
Output: our model requires 100k parameters , while BIBREF8 requires 250k parameters
Input of the model is the concatenation of word embedding and another embedding indicating whether this word is predicate: $ \mathbf {x}_t = [\mathbf {W}_{\text{emb}}(w_t), \mathbf {W}_{\text{mask}}(w_t = v)]. $ 
 Question: What's the input representation of OpenIE tuples into the model?
Output: word embeddings
We use an off-the-shelf parser, in this case Stanford CoreNLP BIBREF11 , to create binary constituency parses. 
 Question: How do they obtain parsed source sentences?
Output: Stanford CoreNLP BIBREF11
Most of the above-discussed systems either shows high performance on (a) Twitter dataset or (b) Facebook dataset (given in the TRAC-2018), but not on both English code-mixed datasets. This may be due to the text style or level of complexities of both datasets. 
 Question: How have the differences in communication styles between Twitter and Facebook increase the complexity of the problem?
Output: Systems do not perform well both in Facebook and Twitter texts
In this section, we describe our approach to classify user reactions into one of eight types of discourse: agreement, answer, appreciation, disagreement, elaboration, humor, negative reaction, or question, or as none of the given labels, which we call “other”, using linguistically-infused neural network models. 
 Question: What are the nine types?
Output: agreement answer appreciation disagreement elaboration humor negative reaction question other
We evaluate the generations with the ROUGE BIBREF29 and METEOR BIBREF30 metrics using the true sentences as targets. 
 Question: What metrics are used for evaluation?
Output: ROUGE BIBREF29 and METEOR BIBREF30
More specifically, in our model we use the context of the word to predict its label and by doing so our model learns label-aware context for each word in the sentence. In order to improve the interactivity between the word representation and its context, we increase the mutual information between the word representations and its context. 
 Question: How does their model utilize contextual information for each work in the given sentence in a multi-task setting? setting?
Output: we use the context of the word to predict its label and by doing so our model learns label-aware context for each word in the sentence
The straight line in figures FIGREF39, FIGREF43 and FIGREF51 is the result of a supervised LDA algorithm which is used as a baseline.  
 Question: Was performance of the weakly-supervised model compared to the performance of a supervised model?
Output: Yes
The corpus includes 2000 sentences.  
 Question: How long is the dataset?
Output: 2000
A selection of results from this study are that:

Non-English code is a large-scale phenomena.

Transliteration is common in identifiers for all languages.

Languages clusters into three distinct groups based on how speakers use identifiers/comments/transliteration.

Non-latin script users write comments in their L1 script but write identifiers in English.

Right-to-left (RTL) language scripts, such as Arabic, have no observed prevalence on GitHub identifiers, implying that existing coders who speak RTL languages have substantial barriers in using their native script in code. 
 Question: What are results of public code repository study?
Output: Non-English code is a large-scale phenomena. Transliteration is common in identifiers for all languages. Languages clusters into three distinct groups based on how speakers use identifiers/comments/transliteration. Non-latin script users write comments in their L1 script but write identifiers in English. Right-to-left (RTL) language scripts, such as Arabic, have no observed prevalence on GitHub identifiers
Word embeddings have risen in popularity for NLP applications due to the success of models designed specifically for the big data setting.  Note that “big” datasets are not always available, particularly in computational social science NLP applications, where the data of interest are often not obtained from large scale sources such as the internet and social media, but from sources such as press releases BIBREF11 , academic journals BIBREF10 , books BIBREF12 , and transcripts of recorded speech BIBREF13 , BIBREF14 , BIBREF15  have proposed a model-based method for training interpretable corpus-specific word embeddings for computational social science, using mixed membership representations, Metropolis-Hastings-Walker sampling, and NCE. Experimental results for prediction, supervised learning, and case studies on state of the Union addresses and NIPS articles, indicate that high-quality embeddings and topics can be obtained using the method. The results highlight the fact that big data is not always best, as domain-specific data can be very valuable, even when it is small. 
 Question: Why is big data not appropriate for this task?
Output: Training embeddings from small-corpora can increase the performance of some tasks
Our dataset is annotated based on Chinese pathology reports provided by the Department of Gastrointestinal Surgery, Ruijin Hospital. It contains 17,833 sentences, 826,987 characters and 2,714 question-answer pairs. 
 Question: How big is dataset of pathology reports collected from Ruijing Hospital?
Output: 17,833 sentences, 826,987 characters and 2,714 question-answer pairs
We chose three different datasets for the experiments: SNLI, MultiNLI and SICK. The Stanford Natural Language Inference (SNLI) corpus BIBREF4 is a dataset of 570k human-written sentence pairs manually labeled with the labels entailment, contradiction, and neutral.  The Multi-Genre Natural Language Inference (MultiNLI) corpus BIBREF5 consisting of 433k human-written sentence pairs labeled with entailment, contradiction and neutral. SICK BIBREF6 is a dataset that was originally constructed to test compositional distributional semantics (DS) models.  
 Question: Which datasets were used?
Output: SNLI, MultiNLI and SICK
Although MCDN doesn't obtain the highest precision, it increases F1-score by 10.2% and 3% compared with the existing best systems $LS \cup KLD \cup CONN$ and $KLD \cup LS \cup LS_{inter}$. 
 Question: What performance did proposed method achieve, how much better is than previous state-of-the-art?
Output: increases F1-score by 10.2% and 3% compared with the existing best systems $LS \cup KLD \cup CONN$ and $KLD \cup LS \cup LS_{inter}$
In particular, for the NLG task, our single model outperformed competing models in terms of both Rouge-L and Bleu-1. 
 Question: How do they measure the quality of summaries?
Output: Rouge-L Bleu-1
The stories from each scenario were distributed among four different annotators.  
 Question: How many subjects have been used to create the annotations?
Output:  four different annotators
Recent attempts can be divided into two categories: (i) those which tries to incorporate additional information to further improve the performance of knowledge graph embedding, e.g., entity types or concepts BIBREF13, relations paths BIBREF17, textual descriptions BIBREF11, BIBREF12 and logical rules BIBREF23; (ii) those which tries to design more complicated strategies, e.g., deep neural network models BIBREF24. 
 Question: What are recent works on knowedge graph embeddings authors mention?
Output: entity types or concepts BIBREF13 relations paths BIBREF17  textual descriptions BIBREF11, BIBREF12 logical rules BIBREF23 deep neural network models BIBREF24
First, we consider WeedsPrec BIBREF8 which captures the features of INLINEFORM0 which are included in the set of a broader term's features, INLINEFORM1 : DISPLAYFORM0

Second, we consider invCL BIBREF11 which introduces a notion of distributional exclusion by also measuring the degree to which the broader term contains contexts not used by the narrower term. Although most unsupervised distributional approaches are based on the DIH, we also consider the distributional SLQS model based on on an alternative informativeness hypothesis BIBREF10 , BIBREF4 . For completeness, we also include cosine similarity as a baseline in our evaluation. 
 Question: Which distributional methods did they consider?
Output: WeedsPrec BIBREF8 invCL BIBREF11 SLQS model cosine similarity
We start the training process by applying the rule INLINEFORM4 to a set of natural language questions INLINEFORM5 . The resulting dataset is considered as the training data to initialize both the semantic parser and the question generator. Afterwards, both models are improved following the back-translation protocol that target sequences should follow the real data distribution, yet source sequences can be generated with noises. 
 Question: How is the back-translation model trained?
Output:  applying the rule INLINEFORM4 to a set of natural language questions INLINEFORM5 both models are improved following the back-translation protocol that target sequences should follow the real data distribution
Same as BIBREF5 , in the first sub-network, pinyin sequence is predicted from the video. Different from BIBREF5 , which predicts pinyin characters from video, pinyin is taken as a whole in CSSMCM, also known as syllables. As we know, Mandarin Chinese is a syllable-based language and syllables are their logical unit of pronunciation. 
 Question: What syntactic structure is used to model tones?
Output: syllables
More than 2,100 texts were paired with 15 questions each, resulting in a total number of approx. 32,000 annotated questions. For 13% of the questions, the workers did not agree on one of the 4 categories with a 3 out of 5 majority, so we did not include these questions in our dataset.

The distribution of category labels on the remaining 87% is shown in Table TABREF10 . 14,074 (52%) questions could be answered. Out of the answerable questions, 10,160 could be answered from the text directly (text-based) and 3,914 questions required the use of commonsense knowledge (script-based). After removing 135 questions during the validation, the final dataset comprises 13,939 questions, 3,827 of which require commonsense knowledge (i.e. 27.4%). This ratio was manually verified based on a random sample of questions. 
 Question: what dataset statistics are provided?
Output: More than 2,100 texts were paired with 15 questions each, resulting in a total number of approx. 32,000 annotated questions. 13% of the questions are not answerable.  Out of the answerable questions, 10,160 could be answered from the text directly (text-based) and 3,914 questions required the use of commonsense knowledge (script-based).  The final dataset comprises 13,939 questions, 3,827 of which require commonsense knowledge (i.e. 27.4%).
It was collected using crowdsourcing BIBREF4. The data collection was done using an Android application. 
 Question: what is the source of the data?
Output: Android application
To help the human representative quickly determine the cause of the escalation, we generate a visualization of the user's turns using the attention weights to highlight the turns influential in the escalation decision. 
 Question: Do they explain model predictions solely on attention weights?
Output: Yes
We mainly concern with the two following structures of the embedding space.

Semantic similarity structure: Semantically similar entities are close to each other in the embedding space, and vice versa. Semantic direction structure: There exist semantic directions in the embedding space, by which only one semantic aspect changes while all other aspects stay the same. It can be identified by a vector difference, such as the subtraction between two embedding vectors. 
 Question: What are the uncanny semantic structures of the embedding space?
Output: Semantic similarity structure Semantic direction structure
The type and correctness of all the question answer pairs are verified by at least two annotators. 
 Question: What was the inter-annotator agreement?
Output: correctness of all the question answer pairs are verified by at least two annotators
Figure 1 shows a simple linear model with rank constraint. 
 Question: What are their baseline methods?
Output: simple linear model with rank constraint Hierarchical softmax N-gram features
We then evaluate the ranks of the target entities $t$ with the commonly used mean rank (MR), mean reciprocal rank (MRR), as well as Hits@1, Hits@3, and Hits@10. 
 Question: what was the evaluation metrics studied in this work?
Output: mean rank (MR), mean reciprocal rank (MRR), as well as Hits@1, Hits@3, and Hits@10
In our experiments, the RNN takes in spectrograms of utterances, passing them through two 2D-convolutional layers, followed by seven bi-directional recurrent layers and a fully-connected layer with softmax activation. All recurrent layers are batch normalized. At each timestep, the softmax activations give a probability distribution over the characters. CTC loss BIBREF8 is then computed from the timestep-wise probabilities. 
 Question: Which model do they use for end-to-end speech recognition?
Output: RNN
We obtained 9,892 stories of sexual harassment incidents that was reported on Safecity. Those stories include a text description, along with tags of the forms of harassment, e.g. commenting, ogling and groping. A dataset of these stories was published by Karlekar and Bansal karlekar2018safecity. In addition to the forms of harassment, we manually annotated each story with the key elements (i.e. “harasser", “time", “location", “trigger"), because they are essential to uncover the harassment patterns. An example is shown in Figure FIGREF3. Furthermore, we also assigned each story classification labels in five dimensions (Table TABREF4). The detailed definitions of classifications in all dimensions are explained below. 
 Question: What is the size of the dataset?
Output:  9,892 stories of sexual harassment incidents
The reviews we used have been extracted from TripAdvisor and originally proposed in BIBREF10 , BIBREF11 .  
 Question: Where are the hotel reviews from?
Output: TripAdvisor
SIGHAN Bakeoff defines two types of evaluation settings, closed test limits all the data for learning should not be beyond the given training set, while open test does not take this limitation BIBREF21. 
 Question: What is meant by closed test setting?
Output: closed test limits all the data for learning should not be beyond the given training set, while open test does not take this limitation
To quantitatively evaluate the performance of our bidirectional recurrent based approach, we adopt METEOR metric because of its robust performance. 
 Question: what metrics were used for evaluation?
Output: METEOR
To test whether topic models can be used for dating poetry or attributing authorship, we perform supervised classification experiments with Random Forest Ensemble classifiers.  
 Question: What is the algorithm used for the classification tasks?
Output: Random Forest Ensemble classifiers
Experimental Protocol ::: Datasets ::: Training Dataset
(i) TTS System dataset: We trained our TTS system with a mixture of neutral and newscaster style speech. For a total of 24 hours of training data, split in 20 hours of neutral (22000 utterances) and 4 hours of newscaster styled speech (3000 utterances).

(ii) Embedding selection dataset: As the evaluation was carried out only on the newscaster speaking style, we restrict our linguistic search space to the utterances associated to the newscaster style: 3000 sentences.

Experimental Protocol ::: Datasets ::: Evaluation Dataset
The systems were evaluated on two datasets:

(i) Common Prosody Errors (CPE): The dataset on which the baseline Prostron model fails to generate appropriate prosody. This dataset consists of complex utterances like compound nouns (22%), “or" questions (9%), “wh" questions (18%). This set is further enhanced by sourcing complex utterances (51%) from BIBREF24.

(ii) LFR: As demonstrated in BIBREF25, evaluating sentences in isolation does not suffice if we want to evaluate the quality of long-form speech. Thus, for evaluations on LFR we curated a dataset of news samples. The news style sentences were concatenated into full news stories, to capture the overall experience of our intended use case. 
 Question: What dataset is used for train/test of this method?
Output: Training datasets: TTS System dataset and embedding selection dataset. Evaluation datasets: Common Prosody Errors dataset and LFR dataset.
We now seek to know if a pre-trained multi-BERT has ability to solve RC tasks in the zero-shot setting. 
 Question: What model is used as a baseline?  
Output: pre-trained multi-BERT
Wizard interface: the interface shown to participants with the Wizard role provides possible actions on the right-hand side of the browser window. These actions could be verbal, such as sending a message, or non-verbal, such as switching on/off a button to activate a robot.  Dialogue structure: we introduced structured dialogues through a Finite State Machine (FSM) that controls the current dialogue state and offers multiple suitable and relevant state transitions (actions) to the Wizard depending on the point in the interaction, the state of the world and the history. A graph of dialogue states, transitions and utterances is loaded when the system is initialised, and each chat room has its own dialogue state, which changes through actions. System-changing actions: actions trigger transitions between the states in the FSM. We differentiate two types of actions:

Verbal actions, such as the dialogue options available at that moment. The Wizard can select one of several predefined messages to send, or type their own message if needed. Free text messages do not change the dialogue state in the FSM, so it is important to minimise their use by providing enough dialogue options to the Wizard. Predefined messages can also trigger other associated events such as pop-ups or follow-up non-verbal actions.

Non-verbal actions, such as commands to trigger events. These can take any form, but we used buttons to control robots in our data collection.

Submitting an action would change the dialogue state in the FSM, altering the set of actions available in the subsequent turn visible to the Wizard. Some dialogue options are only possible at certain states, in a similar way as to how non-verbal actions are enabled or disabled depending on the state.  
 Question: How does framework made sure that dialogue will not breach procedures?
Output: The Wizard can select one of several predefined messages to send, or type their own message if needed. Free text messages do not change the dialogue state in the FSM, so it is important to minimise their use by providing enough dialogue options to the Wizard. Predefined messages can also trigger other associated events such as pop-ups or follow-up non-verbal actions.
In particular, the best-performing systems at SemEval-2015 and SemEval-2016 used deep convolutional networks BIBREF53 , BIBREF54  
 Question: What is the current SOTA for sentiment analysis on Twitter at the time of writing?
Output: deep convolutional networks BIBREF53 , BIBREF54
We use a graph to model the social media context, relating tweets to one another, authors to tweets and other authors. Figure FIGREF7 shows the graph, composed of three types of nodes: tweets (T), users (U) and the “world” (W). Edges are created between nodes and weighted as follows: T-T the unigram cosine similarity between tweets, T-U weighted 100 between a tweet and its author, U-U weighted 1 between two users in a “follows” relationship and U-W weighted 0.001 to ensure a connected graph for the mad algorithm. 
 Question: What information is contained in the social graph of tweet authors?
Output:  the graph, composed of three types of nodes: tweets (T), users (U) and the “world” (W). Edges are created between nodes and weighted as follows: T-T the unigram cosine similarity between tweets, T-U weighted 100 between a tweet and its author, U-U weighted 1 between two users in a “follows” relationship and U-W weighted 0.001 to ensure a connected graph for the mad algorithm.
By choosing English (En) as the pivot language, we perform pivot alignments for identical English segments on Europarl Fr-En and En-De parallel corpora BIBREF18 , constructing a multi-parallel corpus of Fr-En-De. Then each of the Fr*-De and Fr-De* pseudo parallel corpora is established from the multi-parallel data by applying the pivot language-based translation described in the previous section. For automatic translation, we utilize a pre-trained and publicly released NMT model for En $\rightarrow $ De and train another NMT model for En $\rightarrow $ Fr using the WMT'15 En-Fr parallel corpus BIBREF19 . 
 Question: Where do they collect the synthetic data?
Output: Yes
We achieve significant gains over our strong baseline by leveraging decompositions from our best decomposition model, trained with USeq2Seq on FastText pseudo-decompositions; we find a 3.1 F1 gain on the original dev set, 11 F1 gain on the multi-hop dev set, and 10 F1 gain on the out-of-domain dev set.  
 Question: How large is the improvement over the baseline?
Output: 3.1 F1 gain on the original dev set 11 F1 gain on the multi-hop dev set 10 F1 gain on the out-of-domain dev set.
Contextualized word embeddings, sentence embeddings, such as deep contextualized word representations BIBREF20 , BERT BIBREF22 , encode the complex characteristics and meanings of words in various context by jointly training a bidirectional language model.  The second method is to use sentence embeddings, BERT. It is used to a generate single 786-dimensional sentence embedding from 10k-dimensional one-hot vector or distribution over previous words and then merge into a single context vector with two different merging methods. Using this approach, we can obtain a more dense, informative, fixed-length vectors to encode conversational-context information, $e^k_{context}$ to be used in next $k$ -th utterance prediction. We use contextual gating mechanism in our decoder network to combine the conversational-context embeddings with speech and word embeddings effectively. Our gating is contextual in the sense that multiple embeddings compute a gate value that is dependent on the context of multiple utterances that occur in a conversation.  Let $e_w = e_w(y_{u-1})$ be our previous word embedding for a word $y_{u-1}$ , and let $e_s = e_s(x^k_{1:T})$ be a speech embedding for the acoustic features of current $k$ -th utterance $x^k_{1:T}$ and $e_c = e_c(s_{k-1-n:k-1})$ be our conversational-context embedding for $n$ -number of preceding utterances ${s_{k-1-n:k-1}}$ . Then using a gating mechanism:

$$g = \sigma (e_c, e_w, e_s)$$ (Eq. 15)

where $\sigma $ is a 1 hidden layer DNN with $\texttt {sigmoid}$ activation, the gated embedding $e$ is calcuated as

$$e = g \odot (e_c, e_w, e_s) \\ h = \text{LSTM}(e)$$ (Eq. 16)

and fed into the LSTM decoder hidden layer.  The output of the decoder $h$ is then combined with conversational-context embedding $e_c$ again with a gating mechanism,

$$g = \sigma (e_C, h) \\ \hat{h} = g \odot (e_c, h)$$ (Eq. 17)

Then the next hidden layer takes these gated activations, $\hat{h}$ , and so on. 
 Question: How are sentence embeddings incorporated into the speech recognition system?
Output: BERT generates sentence embeddings that represent words in context. These sentence embeddings are merged into a single conversational-context vector that is used to calculate a gated embedding and is later combined with the output of the decoder h to provide the gated activations for the next hidden layer.
Motivated by phrase based SMT, we retrieve neighbors which have high local, sub-sentence level overlap with the source sentence. We adapt our approach to retrieve n-grams instead of sentences. We represent every sentence by their reduced n-gram set. For every n-gram in INLINEFORM0 , we find the closest n-gram in the training set using the IDF similarity defined above. 
 Question: Where do they retrieve neighbor n-grams from in their approach?
Output: represent every sentence by their reduced n-gram set
We used a dataset provided by Shortir, an Indonesian news aggregator and summarizer company. The dataset contains roughly 20K news articles. Each article has the title, category, source (e.g., CNN Indonesia, Kumparan), URL to the original article, and an abstractive summary which was created manually by a total of 2 native speakers of Indonesian 
 Question: Did they use a crowdsourcing platform for the summaries?
Output: No
Using our methodology, we tagged 10,000 Arabic tweet dataset for offensiveness, where offensive tweets account for roughly 19% of the tweets. Further, we labeled tweets as vulgar or hate speech.  
 Question: How many tweets are in the dataset?
Output: 10,000 Arabic tweet dataset
. Grammatically correct text is needed as the starting point for inserting artificial errors, and we used two different sources: 1) the corrected version of the same FCE training set on which the system is trained (450K tokens), and 2) example sentences extracted from the English Vocabulary Profile (270K tokens). 
 Question: Which languages are explored in this paper?
Output: English
In this paper, we first highlight the importance of TV and radio broadcast as a source of data for ASR, and the potential impact it can have. We then perform a statistical analysis of gender representation in a data set composed of four state-of-the-art corpora of French broadcast, widely used within the speech community. Finally we question the impact of such a representation on the systems developed on this data, through the perspective of an ASR system. 
 Question: What tasks did they use to evaluate performance for male and female speakers?
Output: ASR
Our experiments are performed on an actual endangered language, Mboshi (Bantu C25), a language spoken in Congo-Brazzaville, using the bilingual French-Mboshi 5K corpus of BIBREF17. On the Mboshi side, we consider alphabetic representation with no tonal information. On the French side,we simply consider the default segmentation into words. 
 Question: What is the dataset used in the paper?
Output: French-Mboshi 5K corpus
We identified some limitations during the process, which we describe in this section.

When deciding publisher partisanship, the number of people from whom we computed the score was small. For example, de Stentor is estimated to reach 275K readers each day on its official website. Deciding the audience leaning from 55 samples was subject to sampling bias. Besides, the scores differ very little between publishers. None of the publishers had an absolute score higher than 1, meaning that even the most partisan publisher was only slightly partisan. Deciding which publishers we consider as partisan and which not is thus not very reliable.

The article-level annotation task was not as well-defined as on a crowdsourcing platform. We included the questions as part of an existing survey and didn't want to create much burden to the annotators. Therefore, we did not provide long descriptive text that explained how a person should annotate an article. We thus run under the risk of annotator bias. This is one of the reasons for a low inter-rater agreement. 
 Question: What limitations are mentioned?
Output: deciding publisher partisanship, risk annotator bias because of short description text provided to annotators
We used the Universal Dependencies Treebank UD v2.1 BIBREF0 for our experiments. We picked four low-resource/high-resource language pairs, each from a different family: Danish/Swedish (da/sv), Russian/Bulgarian (ru/bg), Finnish/Hungarian (fi/hu), Spanish/Portuguese (es/pt). Picking languages from different families would ensure that we obtain results that are on average consistent across languages. 
 Question: What languages are explored?
Output: Danish/Swedish (da/sv), Russian/Bulgarian (ru/bg), Finnish/Hungarian (fi/hu), Spanish/Portuguese (es/pt)