niryuu/sni-each-converted huggingface dataset:
We focus on the data shared by BIBREF3, and empirically test to what extent changes in the evaluation design affect the outcome of the human evaluation. In this study, we address three aspects that we consider to be particularly relevant for human evaluation of MT, with a special focus on testing human–machine parity: the choice of raters, the use of linguistic context, and the construction of reference translations. We empirically test and discuss the impact of these factors on human evaluation of MT in Sections SECREF3–SECREF5.  
 Question: What empricial investigations do they reference?
Output: empirically test to what extent changes in the evaluation design affect the outcome of the human evaluation
The goal here being to assess if D2V could effectively replace the related-document function on PubMed, five different document similarity evaluations were designed as seen on figure FIGREF9.  Methods ::: Evaluation ::: String length
To assess whether a similar length could lead to convergence of two documents, the size of the query document $D_{x}$ has been compared with the top-close document $C_{x}$ for 10,000 document randomly selected from the TeS after some pre-processing steps (stopwords and spaces were removed from both documents). Methods ::: Evaluation ::: Words co-occurrences
A matrix of words co-occurrence was constructed on the total corpus from PubMed. Briefly, each document was lowered and tokenized. A matrix was filled with the number of times that two words co-occur in a single document. Then, for 5,000 documents $D_{x}$ from the TeS, all models were queried for the top-close document $C_{x}$. All possible combinations between all words $WD_{x} \in D_{x}$ and all words $WC_{x} \in C_{x}$ (excluding stopwords) were extracted, 500 couples were randomly selected and the number of times each of them was co-occurring was extracted from the matrix. The average value of this list was calculated, reflecting the proximity between D and C regarding their words content. This score was also calculated between each $D_{x}$ and the top-close document $C_{x}$ returned by the pmra algorithm. Methods ::: Evaluation ::: Stems co-occurrences
The evaluation task explained above was also applied on 10,000 stemmed texts (using the Gensim’s PorterStemmer to only keep word’s roots). The influence of the conjugation form or other suffixes can be assessed. Methods ::: Evaluation ::: MeSH similarity
It is possible to compare the ability of both pmra and D2V to bring closer articles which were indexed with common labels. To do so, 5,000 documents $D_{x}$ randomly selected from the TeS were sent to both pmra and D2V architectures, and the top-five closer articles $C_{x}$ were extracted. The following rules were then applied to each MeSH found associated with $D_{x}$ for each document $C_{x_i}$ : add 1 to the score if this MeSH term is found in both $D_{x}$ and $C_{x_i}$, add 3 if this MeSH is defined as major topic and add 1 for each qualifier in common between $D_{x}$ and Cxi regarding this particular MeSH term. Then, the mean of these five scores was calculated for both pmra and D2V. 
 Question: What four evaluation tasks are defined to determine what influences proximity?
Output: String length Words co-occurrences Stems co-occurrences MeSH similarity
We map each relation type $R(x,y)$ to at least one parametrized natural-language question $q_x$ whose answer is $y$ . 
 Question: How is the input triple translated to a slot-filling task?
Output: The relation R(x,y) is mapped onto a question q whose answer is y
Task: given a caption and a paired image (if used), the goal is to label every token in a caption in BIO scheme (B: beginning, I: inside, O: outside) BIBREF27 .  
 Question: Can named entities in SnapCaptions be discontigious?
Output: No
System description ::: Architecture The components involved in the process are: QnAMaker Portal: This is the Graphical User Interface (GUI) for using QnAMaker.  QnaMaker Management APIs: This is used for the extraction of Question-Answer (QA) pairs from semi-structured content.  Azure Search Index: Stores the KB with questions and answers as indexable columns, thus acting as a retrieval layer. QnaMaker WebApp: Acts as a layer between the Bot, Management APIs, and Azure Search Index.  Bot: Calls the WebApp with the User's query to get results. 
 Question: What components is the QnAMaker composed of?
Output: QnAMaker Portal QnaMaker Management APIs Azure Search Index QnaMaker WebApp Bot
We adopt a DNN-based acoustic model BIBREF0 with 11 hidden layers and the alignment used to train the model is derived from a HMM-GMM model trained with SAT criterion. 
 Question: What are the deep learning architectures used in the task?
Output: DNN-based acoustic model BIBREF0
Training Dataset UM Inventory BIBREF5 is a public dataset created by researchers from the University of Minnesota, containing about 37,500 training samples with 75 abbreviation terms. Existing work reports abbreviation disambiguation results on 50 abbreviation terms BIBREF6, BIBREF5, BIBREF17. However, after carefully reviewing this dataset, we found that it contains many samples where medical professionals disagree: wrong samples and uncategorized samples. Due to these mistakes and flaws of this dataset, we removed the erroneous samples and eventually selected 30 abbreviation terms as our training dataset that can be made public. 
 Question: What existing dataset is re-examined and corrected for training?
Output:  UM Inventory
Although cursing is frequent in tweets, they represent just 1.15% of all words used BIBREF21 . In contrast, we found 5.72% of all words posted by gang member accounts to be classified as a curse word, which is nearly five times more than the average curse word usage on Twitter. The clouds also reflect the fact that gang members often talk about drugs and money with terms such as smoke, high, hit, and money, while ordinary users hardly speak about finances and drugs. We also noticed that gang members talk about material things with terms such as got, money, make, real, need whereas ordinary users tend to vocalize their feelings with terms such as new, like, love, know, want, look, make, us. 
 Question: What are the differences in language use between gang member and the rest of the Twitter population?
Output: Although cursing is frequent in tweets, they represent just 1.15% of all words used BIBREF21 . In contrast, we found 5.72% of all words posted by gang member accounts to be classified as a curse word gang members talk about material things with terms such as got, money, make, real, need whereas ordinary users tend to vocalize their feelings with terms such as new, like, love, know, want, look, make, us
The dataset consists of more than $0.3$ million records and has been made available for future research. 
 Question: What is the size of the dataset?
Output: $0.3$ million records
We have shown that transformer-based encoders together with GRU models provide the best performance for question representation. Notably, we demonstrated that using pre-trained text representations provide consistent performance improvements across several hyper-parameter configurations. We have also shown that using an object detector fine-tuned with external data provides large improvements in accuracy. Moreover, we have shown that attention mechanisms are paramount for learning top performing networks, once they allow producing question-aware image representations that are capable of encoding spatial relations. It became clear that Top-Down is the preferred attention method, given its results with ReLU activation. 
 Question: What components are identified as core components for training VQA models?
Output: pre-trained text representations transformer-based encoders together with GRU models attention mechanisms are paramount for learning top performing networks Top-Down is the preferred attention method
Trinomials are likely to appear in exactly one order, and when they appear in more than one order the last word is almost always the same across all instances.  
 Question: Are there any new finding in analasys of trinomials that was not present binomials?
Output: Trinomials are likely to appear in exactly one order
We evaluate the QRNN architecture on a popular document-level sentiment classification benchmark, the IMDb movie review dataset BIBREF17 . The dataset consists of a balanced sample of 25,000 positive and 25,000 negative reviews, divided into equal-size train and test sets, with an average document length of 231 words BIBREF18 . We compare only to other results that do not make use of additional unlabeled data (thus excluding e.g., BIBREF19 ). 
 Question: What sentiment classification dataset is used?
Output: the IMDb movie review dataset BIBREF17
The learning model for retrieval is trained by an oracle constructed using distant supervision. Using the answer labels in the training set, we can find appropriate articles that include the information requested in the question. First, the question and article are embedded into vector sequences, using the same method as the comprehension model. We do not use anonymization here, to retain simplicity. Otherwise, the anonymization procedure would have to be repeated several times for a potentially large collection of documents. These vector sequences are next fed to a Bi-GRU, to produce the outputs $v$ (for the question) and $H_c$ (for the document) similar to the previous section. 
 Question: How can a neural model be used for a retrieval if the input is the entire Wikipedia?
Output: Using the answer labels in the training set, we can find appropriate articles that include the information requested in the question.
There are almost 1.1 million sentences in the collection. There are 119 different relation types (unique predicates), having from just a few relations to a few million relations. 
 Question: How big is their dataset?
Output: 1.1 million sentences 119 different relation types (unique predicates)
This work also contributes a new dataset of INLINEFORM0 pairs of free-form natural language instructions and high-level navigation plans. This dataset was collected through Mechanical Turk using 100 simulated environments with a corresponding topological map and, to the best of our knowledge, it is the first of its kind for behavioral navigation. The dataset opens up opportunities to explore data-driven methods for grounding navigation commands into high-level motion plans. While the dataset was collected with simulated environments, no structure was imposed on the navigation instructions while crowd-sourcing data. Thus, many instructions in our dataset are ambiguous. Moreover, the order of the behaviors in the instructions is not always the same. For instance, a person said “turn right and advance” to describe part of a route, while another person said “go straight after turning right” in a similar situation. The high variability present in the natural language descriptions of our dataset makes the problem of decoding instructions into behaviors not trivial. See Appendix A of the supplementary material for additional details on our data collection effort. 
 Question: Did the collection process use a WoZ method?
Output: No
For the CNN-Dailymail dataset, the Lead-3 model is considered a strong baseline; both the abstractive BIBREF16 and extractive BIBREF14 state-of-the art methods on this dataset beat this baseline only marginally. For the proxy report section of the AMR bank, we consider the Lead-1-AMR model as the baseline. 
 Question: Which other methods do they compare with?
Output: Lead-3 Lead-1-AMR
In tab:blueperpl we report BLEU scores for the reconstruction of test set sentences from their content and feature representations, as well as the model perplexities of the reconstruction. For both models, we use beam decoding with a beam size of eight. Note that for both models, the all and top classification accuracy tends to be quite similar, though for the Baseline they are often almost exactly the same when the Baseline has little to no diversity in the outputs. 
 Question: What metrics are used for automatic evaluation?
Output: classification accuracy BLEU scores model perplexities of the reconstruction
In this chapter, we apply an ensemble of deep learning and linguistics to tackle both the problem of aspect extraction and subjectivity detection. 
 Question: How are aspects identified in aspect extraction?
Output: apply an ensemble of deep learning and linguistics t
Lastly, the attention model, MDREA, also outperforms the best existing research results (WAP 0.690 to 0.688) BIBREF19 . 
 Question: By how much does their model outperform the state of the art results?
Output: the attention model, MDREA, also outperforms the best existing research results (WAP 0.690 to 0.688)
Four factual attributes pertaining to the 200 colleges are extracted from the U.S. News Statistics, which consists of Undergraduate Enrollment, Male/Female Ratio, Private/Public, and Region (Northeast, South, West, and Midwest).  
 Question: Which major geographical regions are studied?
Output: Northeast U.S, South U.S., West U.S. and Midwest U.S.
Recommendation diversity. As defined in BIBREF18 , we calculate recommendation diversity as the average dissimilarity of all pairs of tags in the list of recommended tags. 
 Question: how is diversity measured?
Output: average dissimilarity of all pairs of tags in the list of recommended tags
We tested the natural language representation against the visual-based and feature representations on several tasks, with varying difficulty. The tasks included a basic scenario, a health gathering scenario, a scenario in which the agent must take cover from fireballs, a scenario in which the agent must defend itself from charging enemies, and a super scenario, where a mixture of the above scenarios was designed to challenge the agent. 
 Question: What experiments authors perform?
Output: a basic scenario, a health gathering scenario, a scenario in which the agent must take cover from fireballs, a scenario in which the agent must defend itself from charging enemies, and a super scenario, where a mixture of the above scenarios
In this scenario we have a paired dataset that contains article-comment parallel contents INLINEFORM0 , and an unpaired dataset that contains the documents (articles or comments) INLINEFORM1 . The supervised model is trained on INLINEFORM2 so that we can learn the matching or mapping between articles and comments. By sharing the encoder of the supervised model and the unsupervised model, we can jointly train both the models with a joint objective function: DISPLAYFORM0 
 Question: Which paired corpora did they use in the other experiment?
Output: dataset that contains article-comment parallel contents INLINEFORM0 , and an unpaired dataset that contains the documents (articles or comments) INLINEFORM1
In the experiments, we consider four data sets, two of them newly created and the remaining two already public: CNN, TIME, 20 Newsgroups, and Reuters-21578. The code and the two new data sets are available at github.com/baiyangwang/emgan. For the pre-processing of all the documents, we transformed all characters to lower case, stemmed the documents, and ran the word2vec model on each corpora to obtain word embeddings with a size of 300. In all subsequent models, we only consider the most frequent INLINEFORM0 words across all corpora in a data set. 
 Question: Which corpora do they use?
Output: CNN, TIME, 20 Newsgroups, and Reuters-21578
For the NER task, we consider both Chinese datasets, i.e., OntoNotes4.0 BIBREF34 and MSRA BIBREF35, and English datasets, i.e., CoNLL2003 BIBREF36 and OntoNotes5.0 BIBREF37. Table shows experimental results on NER datasets. For English datasets including CoNLL2003 and OntoNotes5.0, our proposed method outperforms BERT-MRCBIBREF38 by +0.29 and +0.96 respectively. We observe huge performance boosts on Chinese datasets, achieving F1 improvements by +0.97 and +2.36 on MSRA and OntoNotes4.0, respectively. As far as we are concerned, we are setting new SOTA performances on all of the four NER datasets. 
 Question: What are method's improvements of F1 for NER task for English and Chinese datasets?
Output: English datasets including CoNLL2003 and OntoNotes5.0, our proposed method outperforms BERT-MRCBIBREF38 by +0.29 and +0.96 respectively Chinese datasets, achieving F1 improvements by +0.97 and +2.36 on MSRA and OntoNotes4.0, respectively
In this section, we evaluate our method and compare its performance against the competitive approaches. We use INLINEFORM0 -fold evaluation protocol with INLINEFORM1 with random dataset split. We measure the performance using standard accuracy metric which we define as a ratio between correctly classified data samples from test dataset and all test samples. 
 Question: What evaluation metrics are used?
Output: standard accuracy metric
We recruited 100 crowdworkers on Amazon Mechanical Turk (AMT) and measured completion times and accuracies for typing randomly sampled sentences from the Yelp corpus.  
 Question: How many participants were trying this communication game?
Output: 100
Our features consisted of direct similarity over ConceptNet Numberbatch embeddings, the relationships inferred over ConceptNet by SME, features that compose ConceptNet with other resources (WordNet and Wikipedia), and a purely corpus-based feature that looks up two-word phrases in the Google Books dataset. 
 Question: What features did they train on?
Output: direct similarity over ConceptNet Numberbatch embeddings, the relationships inferred over ConceptNet by SME, features that compose ConceptNet with other resources (WordNet and Wikipedia), and a purely corpus-based feature that looks up two-word phrases in the Google Books dataset
The final result submitted in public leaderboard is 0.73019 and in private leaderboard is 0.58455. It is quite different in bad way. That maybe is the result of the model too overfit on train set tuning on public test set. 
 Question: What is private dashboard?
Output: Private dashboard is leaderboard where competitors can see results after competition is finished - on hidden part of test set (private test set).
The key idea behind our proposed interactive MRC (iMRC) is to restrict the document context that a model observes at one time. Concretely, we split a supporting document into its component sentences and withhold these sentences from the model. Given a question, the model must issue commands to observe sentences in the withheld set; we equip models with actions such as Ctrl+F (search for token) and stop for searching through partially observed documents. A model searches iteratively, conditioning each command on the input question and the sentences it has observed previously. Thus, our task requires models to `feed themselves' rather than spoon-feeding them with information. This casts MRC as a sequential decision-making problem amenable to reinforcement learning (RL). 
 Question: How do they train models in this setup?
Output: Thus, our task requires models to `feed themselves' rather than spoon-feeding them with information. This casts MRC as a sequential decision-making problem amenable to reinforcement learning (RL).
We propose a natural language generation model based on BERT, making good use of the pre-trained language model in the encoder and decoder process, and the model can be trained end-to-end without handcrafted features. During model training, the objective of our model is sum of the two processes, jointly trained using "teacher-forcing" algorithm. During training we feed the ground-truth summary to each decoder and minimize the objective.

$$L_{model} = \hat{L}_{dec} + \hat{L}_{refine}$$ (Eq. 23)

At test time, each time step we choose the predicted word by $\hat{y} = argmax_{y^{\prime }} P(y^{\prime }|x)$ , use beam search to generate the draft summaries, and use greedy search to generate the refined summaries. 
 Question: How are the different components of the model trained? Is it trained end-to-end?
Output: the objective of our model is sum of the two processes, jointly trained using "teacher-forcing" algorithm we feed the ground-truth summary to each decoder and minimize the objective At test time, each time step we choose the predicted word by $\hat{y} = argmax_{y^{\prime }} P(y^{\prime }|x)$ , use beam search to generate the draft summaries, and use greedy search to generate the refined summaries. the model can be trained end-to-end
With regards to complexity and intangibility of ethics and morals, we restrict ourselves to a rather basic implementation of this construct, following the theories of deontological ethics. These ask which choices are morally required, forbidden, or permitted instead of asking which kind of a person we should be or which consequences of our actions are to be preferred. Thus, norms are understood as universal rules of what to do and what not to do. Therefore, we focus on the valuation of social acceptance in single verbs and single verbs with surrounding context information —e.g. trust my friend or trust a machine— to figure out which of them represent a Do and which tend to be a Don't.  
 Question: How do the authors define deontological ethical reasoning?
Output: These ask which choices are morally required, forbidden, or permitted norms are understood as universal rules of what to do and what not to do
We introduce a surrogate training objective that avoids these problems and as a result is fully continuous.  Specifically, we form a continuous function softLB that seeks to approximate the result of running our decoder on input INLINEFORM0 and then evaluating the result against INLINEFORM1 using INLINEFORM2 . 
 Question: Do they provide a framework for building a sub-differentiable for any final loss metric?
Output: Yes
We then propose a group of new interesting tasks, such as analogy query and analogy browsing, and discuss how they can be used in modern digital libraries. 
 Question: What new interesting tasks can be solved based on the uncanny semantic structures of the embedding space?
Output:  analogy query analogy browsing
For every gendered character in the dataset, we ask annotators to create a new character with a persona of the opposite gender that is otherwise identical except for referring nouns or pronouns. 
 Question: In the targeted data collection approach, what type of data is targetted?
Output: Gendered characters in the dataset
Attention-base translation model We use the system of BIBREF6 , a convolutional sequence to sequence model. 
 Question: Which translation system do they use to translate to English?
Output: Attention-based translation model with convolution sequence to sequence model
In Spanish, serigos2017using extracted anglicisms from a corpus of Argentinian newspapers by combining dictionary lookup (aided by TreeTagger and the NLTK lemmatizer) with automatic filtering of capitalized words and manual inspection. In serigos2017applying, a character n-gram module was added to estimate the probabilities of a word being English or Spanish. moreno2018configuracion used different pattern-matching filters and lexicon lookup to extract anglicism cadidates from a corpus of tweets in US Spanish. 
 Question: Does the paper mention other works proposing methods to detect anglicisms in Spanish?
Output: Yes
To remain consistent with experiments performed with LSTM's we use the morfessor for the subword tokenization in the Finnish Language. 
 Question: Is the LSTM baseline a sub-word model?
Output: Yes
In this study we use transcripts and results of Oxford-style debates from the public debate series “Intelligence Squared Debates” (IQ2 for short). 
 Question: what debates dataset was used?
Output: Intelligence Squared Debates
We introduce our proposed diversity, density, and homogeneity metrics with their detailed formulations and key intuitions. 
 Question: Did they propose other metrics?
Output: Yes
In this experiment, we compare our Enitity-GCN against recent prior work on the same task. We present test and development results (when present) for both versions of the dataset in Table 2 . From BIBREF0 , we list an oracle based on human performance as well as two standard reading comprehension models, namely BiDAF BIBREF3 and FastQA BIBREF6 . We also compare against Coref-GRU BIBREF12 , MHPGM BIBREF11 , and Weaver BIBREF10 . Additionally, we include results of MHQA-GRN BIBREF23 , from a recent arXiv preprint describing concurrent work. They jointly train graph neural networks and recurrent encoders. We report single runs of our two best single models and an ensemble one on the unmasked test set (recall that the test set is not publicly available and the task organizers only report unmasked results) as well as both versions of the validation set. 
 Question: What baseline did they compare Entity-GCN to?
Output: Human, FastQA, BiDAF, Coref-GRU, MHPGM, Weaver / Jenga, MHQA-GRN
The baseline model for our experiments is explained in the paper by Alec Go [1]. The model uses the Naive Bayes, SVM, and the Maximum Entropy classifiers for their experiment. 
 Question: What previously proposed methods is this method compared against?
Output: Naive Bayes SVM Maximum Entropy classifiers
In a cooperation with researchers from the German Institute for International Educational Research we identified the following current controversial topics in education in English-speaking countries: (1) homeschooling, (2) public versus private schools, (3) redshirting — intentionally delaying the entry of an age-eligible child into kindergarten, allowing their child more time to mature emotionally and physically BIBREF51 , (4) prayer in schools — whether prayer in schools should be allowed and taken as a part of education or banned completely, (5) single-sex education — single-sex classes (males and females separate) versus mixed-sex classes (“co-ed”), and (6) mainstreaming — including children with special needs into regular classes. 
 Question: Do they report results only on English data?
Output: Yes
Secondly, a well-known problem of uneven yet unknown distribution of word senses is alleviated by modifying a naïve Bayesian classifier. Thanks to this correction, the classifier is no longer biased towards senses that have more training data. Finally, the aggregated feature values corresponding to target word senses are used to build a naïve Bayesian classifier adjusted to a situation of unknown a priori probabilities. 
 Question: How do they deal with unknown distribution senses?
Output: The Näive-Bayes classifier is corrected so it is not biased to most frequent classes
To derive contextualized representations from textual features, the book title and blurb are concatenated and then fed through BERT The non-text features are generated in a separate preprocessing step. The metadata features are represented as a ten-dimensional vector (two dimensions for gender, see Section SECREF10). Author embedding vectors have a length of 200 (see Section SECREF22). In the next step, all three representations are concatenated and passed into a MLP with two layers, 1024 units each and ReLu activation function. 
 Question: How do they combine text representations with the knowledge graph embeddings?
Output: all three representations are concatenated and passed into a MLP
We test the character and word-level variants by predicting hashtags for a held-out test set of posts. 
 Question: What other tasks do they test their method on?
Output: None
These recordings contain background noise (as is expected in a mall) and have different languages (Indians speak a mixture of English and Hindi). 
 Question: What languages are considered?
Output: English Hindi
We compare the performance of a standard Transformer Base model and our semi-parametric NMT approach on an English-French translation task. 
 Question: To which systems do they compare their results against?
Output: standard Transformer Base model
In particular, we use Semeval 2014 BIBREF34 Twitter Sentiment Analysis Dataset for the training Sarcasm Datasets Used in the Experiment
This dataset was created by BIBREF8 . The tweets were downloaded from Twitter using #sarcasm as a marker for sarcastic tweets. It is a monolingual English dataset which consists of a balanced distribution of 50,000 sarcastic tweets and 50,000 non-sarcastic tweets.

Since sarcastic tweets are less frequently used BIBREF8 , we also need to investigate the robustness of the selected features and the model trained on these features on an imbalanced dataset. To this end, we used another English dataset from BIBREF8 . It consists of 25,000 sarcastic tweets and 75,000 non-sarcastic tweets.

We have obtained this dataset from The Sarcasm Detector. It contains 120,000 tweets, out of which 20,000 are sarcastic and 100,000 are non-sarcastic. We randomly sampled 10,000 sarcastic and 20,000 non-sarcastic tweets from the dataset. Visualization of both the original and subset data show similar characteristics. 
 Question: Which benchmark datasets are used?
Output: Semeval 2014 BIBREF34 Twitter Sentiment Analysis Dataset   dataset was created by BIBREF8  English dataset from BIBREF8  dataset from The Sarcasm Detector
The final collection of Facebook pages for the experiments described in this paper is as follows: FoxNews, CNN, ESPN, New York Times, Time magazine, Huffington Post Weird News, The Guardian, Cartoon Network, Cooking Light, Home Cooking Adventure, Justin Bieber, Nickelodeon, Spongebob, Disney. 
 Question: Which Facebook pages did they look at?
Output: FoxNews, CNN, ESPN, New York Times, Time magazine, Huffington Post Weird News, The Guardian, Cartoon Network, Cooking Light, Home Cooking Adventure, Justin Bieber, Nickelodeon, Spongebob, Disney
Experiment 1: scene variation in modified referring expressions We recruited 58 pairs of participants (116 participants total) over Amazon's Mechanical Turk who were each paid $1.75 for their participation. 
 Question: Does the paper describe experiments with real humans?
Output: Yes
To simplify the task, we set up a binary classification: candidates who have been liked or shortlisted are considered part of the hirable class and others part of the not hirable class. 
 Question: How is "hirability" defined?
Output: candidates who have been liked or shortlisted are considered part of the hirable class
The dataset consists of queries and the corresponding image search results. Each token in each query is given a language tag based on the user-set home language of the user making the search on Google Images. 
 Question: Do the images have multilingual annotations or monolingual ones?
Output: monolingual
irremediable annotation discrepancies Some shortcomings of recall come from irremediable annotation discrepancies. Largely, we are hamstrung by differences in choice of attributes to annotate. When one resource marks gender and the other marks case, we can't infer the gender of the word purely from its surface form. The resources themselves would need updating to encode the relevant morphosyntactic information. Some languages had a very low number of overlapping forms, and no tag matches or near-matches between them: Arabic, Hindi, Lithuanian, Persian, and Russian. A full list of observed, irremediable discrepancies is presented alongside the codebase. We were able to find specific instances of incorrectly applied UniMorph annotation, as well as specific instances of cross-lingual inconsistency in both resources.  
 Question: What are the main sources of recall errors in the mapping?
Output: irremediable annotation discrepancies differences in choice of attributes to annotate The resources themselves would need updating to encode the relevant morphosyntactic information. Some languages had a very low number of overlapping forms, and no tag matches or near-matches between them the two annotations encode distinct information incorrectly applied UniMorph annotation cross-lingual inconsistency in both resources
Fortunately, 180221 of 231162 author names could be matched successfully. There are many reasons for the remaining uncovered cases. 9073 Latin names could not be found in the name dictionary ENAMDICT and 14827 name matchings between the names' Latin and kanji representations did not succeed. These names might be missing at all in the dictionary, delivered in a very unusual format that the tool does not cover, or might not be Japanese or human names at all. Of course, Japanese computer scientists sometimes also cooperate with foreign colleagues but our tool expects Japanese names and is optimized for them. 
 Question: How successful are they at matching names of authors in Japanese and English?
Output: 180221 of 231162 author names could be matched successfully
Baselines. We use one strong non-DNN baseline, NBSVM (with unigrams or bigrams features) BIBREF23 and six DNN baselines. The first DNN baseline is CNN BIBREF25, which does not handle noisy labels. The other five were designed to handle noisy labels.

The comparison results are shown in Table TABREF12. From the results, we can make the following observations. (1) Our NetAb model achieves the best ACC and F1 on all datasets except for F1 of negative class on Laptop. 
 Question: Is the model evaluated against a CNN baseline?
Output: Yes
The dataset was split timewise into training/validation/test sets, and hyperparameters were optimized to maximize the AUC-ROC on the validation set.  
 Question: What evaluation metrics were used?
Output: AUC-ROC
Especially for Davidson-dataset, some tweets with specific language (written within the African American Vernacular English) and geographic restriction (United States of America) are oversampled such as tweets containing disparage words “nigga", “faggot", “coon", or “queer", result in high rates of misclassification. However, these misclassifications do not confirm the low performance of our classifier because annotators tended to annotate many samples containing disrespectful words as hate or offensive without any presumption about the social context of tweeters such as the speaker’s identity or dialect, whereas they were just offensive or even neither tweets.  
 Question: What biases does their model capture?
Output: Data annotation biases where tweet containing disrespectful words are annotated as hate or offensive without any presumption about the social context of tweeters
As a result, the neutral class in SNLI-VE has substantial labelling errors. Vu BIBREF3 estimated ${\sim }31\%$ errors in this class, and ${\sim }1\%$ for the contradiction and entailment classes. 
 Question: What is the class with highest error rate in SNLI-VE?
Output: neutral class
Our full dataset consists of all subreddits on Reddit from January 2013 to December 2014, for which there are at least 500 words in the vocabulary used to estimate our measures, in at least 4 months of the subreddit's history. We compute our measures over the comments written by users in a community in time windows of months, for each sufficiently active month, and manually remove communities where the bulk of the contributions are in a foreign language.  
 Question: How did the select the 300 Reddit communities for comparison?
Output: They selected all the subreddits from January 2013 to December 2014 with at least 500 words in the vocabulary and at least 4 months of the subreddit's history. They also removed communities with the bulk of the contributions are in foreign language.
The answer styles considered in the experiments corresponded to the two tasks. The NLG task requires a well-formed answer that is an abstractive summary of the question and ten passages, averaging 16.6 words. The Q&A task also requires an abstractive answer but prefers a more concise answer than the NLG task, averaging 13.1 words, where many of the answers do not contain the context of the question. 
 Question: What is an "answer style"?
Output: well-formed sentences vs concise answers
In our experiment, we use a conversational Mandarin-English code-switching speech corpus called SEAME Phase II (South East Asia Mandarin-English).  
 Question: What languages are explored in this paper?
Output: Mandarin English
Thus, SIM has many fewer parameters than existing dialogue state tracking models. To compensate for the exclusion of slot-specific parameters, we incorporate better feature representation of user utterance and dialogue states using syntactic information and convolutional neural networks (CNN).  
 Question: How do they prevent the model complexity increasing with the increased number of slots?
Output: They exclude slot-specific parameters and incorporate better feature representation of user utterance and dialogue states using syntactic information and convolutional neural networks (CNN).
We created this CORD-19-NER dataset with comprehensive named entity annotation on the CORD-19 corpus (2020-03-13).  The corpus is generated from the 29,500 documents in the CORD-19 corpus (2020-03-13).  
 Question: What is the size of this dataset?
Output: 29,500 documents
To collect article-level labels, we utilized a platform in the company that has been used by the market research team to collect surveys from the subscribers of different news publishers. The survey works as follows: The user is first presented with a set of selected pages (usually 4 pages and around 20 articles) from the print paper the day before. The user can select an article each time that he or she has read, and answer some questions about it. We added 3 questions to the existing survey that asked the level of partisanship, the polarity of partisanship, and which pro- or anti- entities the article presents. We also asked the political standpoint of the user. The complete survey can be found in Appendices. 
 Question: Did they crowdsource the annotations?
Output: Yes
But we also showed that in some cases the saliency maps seem to not capture the important input features.  The second observation we can make is that the saliency map doesn't seem to highlight the right things in the input for the summary it generates 
 Question: Is the explanation from saliency map correct?
Output: No
In this work, we explore the applicability of our method to three popular architectures of this layer: the LSTM-based, the CNN-based, and the transformer-based. Table TABREF46 shows performance of our method with different sequence modeling architectures. From the table, we can first see that the LSTM-based architecture performed better than the CNN- and transformer- based architectures.  
 Question: Which are the sequence model architectures this method can be transferred across?
Output: The sequence model architectures which this method is transferred to are: LSTM and Transformer-based models
We perform experiments on document-level variants of the SQuAD dataset BIBREF1 . 
 Question: What datasets have this method been evaluated on?
Output: document-level variants of the SQuAD dataset
We evaluate our method on IWSLT16 German-English (DE-EN) translation in both directions, WMT15 English-German (EN-DE) translation in both directions, and NIST Chinese-to-English (ZH$\rightarrow $EN) translation.  
 Question: What corpora is used?
Output: IWSLT16 WMT15 NIST
We call this lifelong interactive learning and inference (LiLi). Lifelong learning is reflected by the facts that the newly acquired facts are retained in the KB and used in inference for future queries, and that the accumulated knowledge in addition to the updated KB including past inference performances are leveraged to guide future interaction and learning. 
 Question: In what way does LiLi imitate how humans acquire knowledge and perform inference during an interactive conversation?
Output: newly acquired facts are retained in the KB and used in inference for future queries, and that the accumulated knowledge in addition to the updated KB including past inference performances are leveraged to guide future interaction and learning
For training and testing we use the train-validation-test split of WikiTableQuestions BIBREF0 , a dataset containing 22,033 pairs of questions and answers based on 2,108 Wikipedia tables. This dataset is also used by our baselines, BIBREF0 , BIBREF3 . 
 Question: Does the dataset they use differ from the one used by Pasupat and Liang, 2015?
Output: No
We compute our measures over the comments written by users in a community in time windows of months, for each sufficiently active month, and manually remove communities where the bulk of the contributions are in a foreign language.  
 Question: Do they report results only on English data?
Output: No
We adopt the following two datasets:

Quora BIBREF1: The Quora Question Pairs dataset contains question pairs annotated with labels indicating whether the two questions are paraphrases. We use the same dataset partition as BIBREF5, with 384,348/10,000/10,000 pairs in the training/development/test set respectively.

MRPC BIBREF34: The Microsoft Research Paraphrase Corpus consists of sentence pairs collected from online news. Each pair is annotated with a label indicating whether the two sentences are semantically equivalent. There are 4,076/1,725 pairs in the training/test set respectively. 
 Question: What are benhmark datasets for paraphrase identification?
Output: Quora MRPC
To enlarge our experiment base we added new debates, more detailed information about each one is shown in Table TABREF24 in UNKREF6. To select new discussions and to determine if they are controversial or not we looked for topics widely covered by mainstream media, and that have generated ample discussion, both online and offline. For non-controversy discussions we focused on “soft news" and entertainment, but also to events that, while being impactful and/or dramatic, did not generate large controversies. To validate that intuition, we manually checked a sample of tweets, being unable to identify any clear instance of controversy On the other side, for controversial debates we focused on political events such as elections, corruption cases or justice decisions. 
 Question: What controversial topics are experimented with?
Output: political events such as elections, corruption cases or justice decisions
We collected a corpus of poems and a corpus of vernacular literature from online resources. The poem corpus contains 163K quatrain poems from Tang Poems and Song Poems, the vernacular literature corpus contains 337K short paragraphs from 281 famous books, the corpus covers various literary forms including prose, fiction and essay. Note that our poem corpus and a vernacular corpus are not aligned. We further split the two corpora into a training set and a validation set. 
 Question: What dataset is used for training?
Output: We collected a corpus of poems and a corpus of vernacular literature from online resources
We explored three different aspects of inconsistency and designed metrics for their measurements.  A word level feature using tf-idf BIBREF22 is added for robustness. 
 Question: What features do they extract?
Output: Inconsistency in Noun Phrase Structures  Inconsistency Between Clauses Inconsistency Between Named Entities and Noun Phrases Word Level Feature Using TF-IDF
In our experiments, we used TF.IDF-based features over the title and over the content of the article we wanted to classify. We had these features twice – once for the title and once for the the content of the article, as we wanted to have two different representations of the same article. Thus, we used a total of 1,100 TF.IDF-weighted features (800 content + 300 title), limiting the vocabulary to the top 800 and 300 words, respectively (which occurred in more than five articles). 
 Question: what lexical features did they experiment with?
Output: TF.IDF-based features
We explore an architecture based on a stack of dilated convolution layers, effectively operating on a broader scale than with standard convolutions while limiting model size. Standard convolutional networks cannot capture long temporal patterns with reasonably small models due to the increase in computational cost yielded by larger receptive fields. Dilated convolutions skip some input values so that the convolution kernel is applied over a larger area than its own. The network therefore operates on a larger scale, without the downside of increasing the number of parameters. 
 Question: What are dilated convolutions?
Output: Similar to standard convolutional networks but instead they skip some input values effectively operating on a broader scale.
This allows Macaw to support multi-modal interactions, such as text, speech, image, click, etc. 
 Question: What modalities are supported by Macaw?
Output: text, speech, image, click, etc
A simple, yet recurrent text model is the well-known word co-occurrence network. After optional textual pre-processing steps, in a co-occurrence network each different word becomes a node and edges are established via co-occurrence in a desired window. A common strategy connects only adjacent words in the so called word adjacency networks. 
 Question: On what model architectures are previous co-occurence networks based?
Output: in a co-occurrence network each different word becomes a node and edges are established via co-occurrence in a desired window connects only adjacent words in the so called word adjacency networks
The extensive use of emojis has drawn a growing attention from researchers BIBREF4 , BIBREF5 because the emojis convey fruitful semantical and sentimental information to visually complement the textual information which is significantly useful in understanding the embedded emotional signals in texts BIBREF6  However, the previous literatures lack in considerations of the linguistic complexities and diversity of emoji. Therefore, previous emoji embedding methods fail to handle the situation when the semantics or sentiments of the learned emoji embeddings contradict the information from the corresponding contexts BIBREF5 , or when the emojis convey multiple senses of semantics and sentiments such as ( and ). In practice, emojis can either summarize and emphasis the original tune of their contexts, or express more complex semantics such as irony and sarcasm by being combined with contexts of contradictory semantics or sentiments. Conventional emoji analysis can only extract single embedding of each emoji, and such embeddings will confuse the following sentiment analysis model by inconsistent sentiment signals from the input texts and emojis 
 Question: What is the motivation for training bi-sense embeddings?
Output:  previous emoji embedding methods fail to handle the situation when the semantics or sentiments of the learned emoji embeddings contradict the information from the corresponding contexts BIBREF5 , or when the emojis convey multiple senses of semantics and sentiments
Car-speak is abstract language that pertains to a car's physical attribute(s). In this instance the physical attributes that the term “fast” pertains to could be the horsepower, or it could be the car's form factor (how the car looks). However, we do not know exactly which attributes the term “fast” refers to. We train a series of classifiers in order to classify car-speak. We train three classifiers on the review vectors that we prepared in Section SECREF8. The classifiers we use are K Nearest Neighbors (KNN), Random Forest (RF), Support Vector Machine (SVM), and Multi-layer Perceptron (MLP) BIBREF13. 
 Question: Is car-speak language collection of abstract features that classifier is later trained on?
Output: No