niryuu/sni-each-converted huggingface dataset:
Thus, for this work, we build upon Hybrid Code Networks (HCN) BIBREF4 since HCNs achieve state-of-the-art performance in a data-efficient way for task-oriented dialogs, and propose AE-HCNs which extend HCNs with an autoencoder (Figure FIGREF8 ).  AE-HCN(-CNN) outperforms HCN on Test-OOD by a large margin about 17(20) points on average while keeping the minimum performance trade-off compared to Test. 
Output: By how much does their method outperform state-of-the-art OOD detection?
We manually investigate the errors made by artificial neural networks for morphological inflection in a target language after pretraining on different source languages. For our qualitative analysis, we make use of the validation set. Therefore, we show validation set accuracies in Table TABREF19 for comparison. We manually annotate the outputs for the first 75 development examples for each source–target language combination. All found errors are categorized as belonging to one of the following categories.
Output: How is the performance on the task evaluated?
Wav2vec BIBREF22 proposed a multi-layer convolutional neural network optimized via a noise contrastive binary classification and was applied to WSJ ASR tasks. Our experiments consisted of three different setups: 1) a fully-supervised system using all labeled data; 2) an SSL system using wav2vec features; 3) an SSL system using our proposed DeCoAR features. All models used were based on deep BLSTMs with the CTC loss criterion.
Output: What are baseline models on WSJ eval92 and LibriSpeech test-clean?
We compare our model with the following state-of-the-art joint entity and relation extraction models:

(1) SPTree BIBREF4: This is an end-to-end neural entity and relation extraction model using sequence LSTM and Tree LSTM. (2) Tagging BIBREF5: This is a neural sequence tagging model which jointly extracts the entities and relations using an LSTM encoder and an LSTM decoder. (3) CopyR BIBREF6: This model uses an encoder-decoder approach for joint extraction of entities and relations. (4) HRL BIBREF11: This model uses a reinforcement learning (RL) algorithm with two levels of hierarchy for tuple extraction. (5) GraphR BIBREF14: This model considers each token in a sentence as a node in a graph, and edges connecting the nodes as relations between them. (6) N-gram Attention BIBREF9: This model uses an encoder-decoder approach with N-gram attention mechanism for knowledge-base completion using distantly supervised data.
Output: What is previous work authors reffer to?
In particular, we investigated attention to nouns, verbs, pronouns, subjects, objects, and negation words, and special BERT tokens across the tasks.
Output: What handcrafter features-of-interest are used?
Even if the optional lemmatization step is applied, one can still aim at further reducing the graph complexity by merging similar vertices. This step is called meta vertex construction. The meta-vertex construction step works as follows. Let INLINEFORM0 represent the set of vertices, as defined above. A meta vertex INLINEFORM1 is comprised of a set of vertices that are elements of INLINEFORM2 , i.e. INLINEFORM3 . Let INLINEFORM4 denote the INLINEFORM5 -th meta vertex. We construct a given INLINEFORM6 so that for each INLINEFORM7 , INLINEFORM8 's initial edges (prior to merging it into a meta vertex) are rewired to the newly added INLINEFORM9 . Note that such edges connect to vertices which are not a part of INLINEFORM10 . Thus, both the number of vertices, as well as edges get reduced substantially. This feature is implemented via the following procedure: Meta vertex candidate identification. Edit distance and word lengths distance are used to determine whether two words should be merged into a meta vertex (only if length distance threshold is met, the more expensive edit distance is computed). The meta vertex creation. As common identifiers, we use the stemmed version of the original vertices and if there is more than one resulting stem, we select the vertex from the identified candidates that has the highest centrality value in the graph and its stemmed version is introduced as a novel vertex (meta vertex).
Output: How are meta vertices computed?
The second data batch consists of event-related tweets for five natural disasters occurring in the U.S. in 2018. These are: the East Coast Bomb Cyclone (Jan. 2 - 6); the Mendocino, California wildfires (Jul. 27 - Sept. 18); Hurricane Florence (Aug. 31 - Sept. 19); Hurricane Michael (Oct. 7 - 16); and the California Camp Fires (Nov. 8 - 25). 
Output: Which five natural disasters were examined?
The comparative evaluator is trained with maximum likelihood estimation (MLE) objective, as described in eq DISPLAY_FORM6

where $\mathcal {X}$ is the set of pairwise training examples contructed as described above, $Q(x_1, x_2) \in \lbrace >,<,\approx \rbrace $ is the true label for the pair ($x_1$, $x_2$), $D_\phi ^q(x_1, x_2)$ is the probability of the comparative discriminator's prediction being $q$ ($q \in \lbrace >,<,\approx \rbrace $) for the pair ($x_1$, $x_2$).
Output: How they add human prefference annotation to fine-tuning process?
Maximum Entropy theory is applied to solve Vietnamese word segmentation BIBREF15 , BIBREF22 , BIBREF23 . There are several studies about the Vietnamese word segmentation task over the last decade. Dinh et al. started this task with Weighted Finite State Transducer (WFST) approach and Neural Network approach BIBREF9 . In addition, machine learning approaches are studied and widely applied to natural language processing and word segmentation as well. In fact, several studies used support vector machines (SVM) and conditional random fields (CRF) for the word segmentation task BIBREF7 , BIBREF8 .
Output: Which approaches have been applied to solve word segmentation in Vietnamese?
In the future, we aim to use the corpus for annotation projects such as parts-of-speech tagging, named entity recognition. Furthermore, the generated word embeddings will be utilized for the automatic construction of Sindhi WordNet.
Output: Are trained word embeddings used for any other NLP task?
For the benchmarks, we selected five systems. We picked first the langid.py library which is frequently used to compare systems in the literature. Since our work is in neural-network LID, we selected two neural network systems from the literature, specifically the encoder-decoder EquiLID system of BIBREF6 and the GRU neural network LanideNN system of BIBREF10. Finally, we included CLD2 and CLD3, two implementations of the Naïve Bayes LID software used by Google in their Chrome web browser BIBREF4, BIBREF0, BIBREF8 and sometimes used as a comparison system in the LID literature BIBREF7, BIBREF6, BIBREF8, BIBREF2, BIBREF10.
Output: Which existing language ID systems are tested?
As it is reported that conservatives and liberals exhibit different behaviors on online social platforms BIBREF19BIBREF20BIBREF21, we further assigned a political bias label to different US outlets (and therefore news articles) following the procedure described in BIBREF2. In order to assess the robustness of our method, we performed classification experiments by training only on left-biased (or right-biased) outlets of both disinformation and mainstream domains and testing on the entire set of sources, as well as excluding particular sources that outweigh the others in terms of samples to avoid over-fitting.
Output: How is the political bias of different sources included in the model?
We collect HLA data from TV Tropes BIBREF3, a knowledge-based website dedicated to pop culture, containing information on a plethora of characters from a variety of sources. Similar to Wikipedia, its content is provided and edited collaboratively by a massive user-base. These attributes are determined by human viewers and their impressions of the characters, and are correlated with human-like characteristics. We believe that TV Tropes is better for our purpose of fictional character modeling than data sources used in works such as BIBREF25 shuster2019engaging because TV Tropes' content providers are rewarded for correctly providing content through community acknowledgement.
Output: How does dataset model character's profiles?
We conducted the human evaluation using Amazon Mechanical Turk to assess subjective quality. We recruit master level workers (who have good prior approval rates) to perform a human comparison between generated responses from two systems (which are randomly sampled from comparison systems). The workers are required to judge each utterance from 1 (bad) to 3 (good) in terms of informativeness and naturalness. Informativeness indicates the extent to which generated utterance contains all the information specified in the dialog act. Naturalness denotes whether the utterance is as natural as a human does. To reduce judgement bias, we distribute each question to three different workers. Finally, we collected in total of 5800 judges.
Output: What was the criteria for human evaluation?
Finally, when investigating the relatedness between European vs. non European languages (cf. (En/Fr)$\rightarrow $Ar), we obtain similar results than those obtained in the monolingual experiment (macro F-score 62.4 vs. 68.0) and best results are achieved by Ar $\rightarrow $(En/Fr). This shows that there are pragmatic devices in common between both sides and, in a similar way, similar text-based patterns in the narrative way of the ironic tweets.
Output: Do the authors identify any cultural differences in irony use?
After adversarial modifications, the performance of the original target models (those without the “-adv” suffix) drops dramatically (e.g. the overall accuracy of BERT on Quora drops from 94.6% to 24.1%), revealing that the target models are vulnerable to our adversarial examples.
Output: How much dramatically results drop for models on generated adversarial examples?
Given the fact that the research on offensive language detection has to a large extent been focused on the English language, we set out to explore the design of models that can successfully be used for both English and Danish.
Output: What is the challenge for other language except English
We began with evaluating standard MT paradigms, i.e., PBSMT BIBREF3 and NMT BIBREF1 . As for PBSMT, we also examined two advanced methods: pivot-based translation relying on a helping language BIBREF10 and induction of phrase tables from monolingual data BIBREF14 .

As for NMT, we compared two types of encoder-decoder architectures: attentional RNN-based model (RNMT) BIBREF2 and the Transformer model BIBREF18 . In addition to standard uni-directional modeling, to cope with the low-resource problem, we examined two multi-directional models: bi-directional model BIBREF11 and multi-to-multi (M2M) model BIBREF8 .

After identifying the best model, we also examined the usefulness of a data augmentation method based on back-translation BIBREF17 .
Output: what was the baseline?
In the second round, we collected 293 annotations from 12 annotators. After Korektor, there are 4262 unique sentences (including 150 seed sentences) that form the COSTRA 1.0 dataset.
Output: How are possible sentence transformations represented in dataset, as new sentences?
We conduct our experiments using the evaluation methodology of SemEval 2010 Task 14: Word Sense Induction & Disambiguation BIBREF5 . 
Output: What evaluation is conducted?
For the 2019 Workshop on Neural Generation of Text (WNGT) Efficiency shared task BIBREF0, the Notre Dame Natural Language Processing (NDNLP) group looked at a method of inducing sparsity in parameters called auto-sizing in order to reduce the number of parameters in the Transformer at the cost of a relatively minimal drop in performance.
Output: What is WNGT 2019 shared task?
Our dataset is annotated based on Chinese pathology reports provided by the Department of Gastrointestinal Surgery, Ruijin Hospital. It contains 17,833 sentences, 826,987 characters and 2,714 question-answer pairs. 
Output: How big is QA-CTS task dataset?
In this section, we describe our data collection for these 3 components. We first describe a series of pilot studies that we conducted in order to collect commonsense inference questions (Section SECREF4 ). In Section SECREF5 , we discuss the resulting data collection of questions, texts and answers via crowdsourcing on Amazon Mechanical Turk (henceforth MTurk). Section SECREF17 gives information about some necessary postprocessing steps and the dataset validation. 
Output: how was the data collected?
We define hateful speech to be the language which explicitly or implicitly threatens or demeans a person or a group based upon a facet of their identity such as gender, ethnicity, or sexual orientation.
Output: What is their definition of hate speech?
We collected the dataset with 1900 dialogs and 8533 turns. 
Output: What is the average length of dialog?
Crowd workers were asked to mark whether a triple was correct, namely, did the triple reflect the consequence of the sentence.
Output: What is the role of crowd-sourcing?
Table TABREF18 shows the Spearman correlation values of GM$\_$KL model evaluated on the benchmark word similarity datasets: SL BIBREF20, WS, WS-R, WS-S BIBREF21, MEN BIBREF22, MC BIBREF23, RG BIBREF24, YP BIBREF25, MTurk-287 and MTurk-771 BIBREF26, BIBREF27, and RW BIBREF28.  Table TABREF19 shows the evaluation results of GM$\_$KL model on the entailment datasets such as entailment pairs dataset BIBREF29 created from WordNet with both positive and negative labels, a crowdsourced dataset BIBREF30 of 79 semantic relations labelled as entailed or not and annotated distributionally similar nouns dataset BIBREF31
Output: What are the qualitative experiments performed on benchmark datasets?
Among all available parameters to tune the D2V algorithm released by Gensim, six of them were selected for optimisation BIBREF14. The window_size parameter affects the size of the sliding window used to parse texts. The alpha parameter represents the learning rate of the network. The sample setting allows the model to reduce the importance given to high-frequency words. The dm parameter defines the training used architecture (PV-DM or PV-DBOW). The hs option defines whether hierarchical softmax or negative sampling is used during the training. Finally, the vector_size parameter affects the number of dimensions composing the resulting vector.
Output: What six parameters were optimized with grid search?
Baselines We thoroughly compare our approaches to the following baselines:

Direct source$\rightarrow $target: A standard NMT model trained on given source$\rightarrow $target parallel data.

Multilingual: A single, shared NMT model for multiple translation directions BIBREF6.

Many-to-many: Trained for all possible directions among source, target, and pivot languages.

Many-to-one: Trained for only the directions to target language, i.e., source$\rightarrow $target and pivot$\rightarrow $target, which tends to work better than many-to-many systems BIBREF27.
Output: What are multilingual models that were outperformed in performed experiment?
The Wikipedia revision dumps that were previously introduced by Leskovec et al. leskovec2010governance contain eight GB (compressed size) revision edits with meta data.
Output: How large is the Wikipedia revision dump dataset?
The final annotation categories for the dataset are: Joy, Sadness, Anger, Fear, Anticipation, Surprise, Love, Disgust, Neutral
Output: How many emotions do they look at?
Conditional Random Fields
Conditional Random Fields (CRF) BIBREF10 are a standard approach when dealing with sequential data in the context of sequence labeling. BiLSTM-CRF
Prior to the emergence of deep neural language models, BiLSTM-CRF models BIBREF11 had achieved state-of-the-art results for the task of sequence labeling. Multi-Task Learning
Multi-Task Learning (MTL) BIBREF15 has become popular with the progress in deep learning. BioBERT
Deep neural language models have recently evolved to a successful method for representing text. In particular, Bidirectional Encoder Representations from Transformers (BERT) outperformed previous state-of-the-art methods by a large margin on various NLP tasks BIBREF17.
Output: What baselines do they introduce?
To generate it, we explore two implicit user-feedback labelling strategies: five-minute reuse and one-day return. 
Output: What feedback labels are used?
Specifically, we design three tasks for seq2seq model pre-training, namely Sentence Reordering (SR), Next Sentence Generation (NSG), and Masked Document Generation (MDG). 
Output: What is masked document generation?
The corpus consists of 53 documents, which contain an average number of 156.1 sentences per document, each with 19.55 tokens on average. The corpus comprises 8,275 sentences and 167,739 words in total.
Output: How large is the corpus?
The reports are published by Météo France and the Met Office, its British counterpart. They are publicly available on the respective websites of the organizations. Both corpora span on the same period as the corresponding time series and given their daily nature, it yields a total of 4,261 and 4,748 documents respectively.
Output: How big is dataset used for training/testing?
As detailed before, we adapted the cluster-linking dataset from rupnik2016news to evaluate our online crosslingual clustering approach.
Output: What are the sources of the datasets?
In this chapter, we apply an ensemble of deep learning and linguistics to tackle both the problem of aspect extraction and subjectivity detection.
Output: How are aspects identified in aspect extraction?
Although humor is a universal construct, there is a wide variety between what each individual may find humorous. We attempt to focus on a subset of the population where we can quantitatively measure reactions: the popular Reddit r/Jokes thread. This forum is highly popular - with tens of thousands of jokes being posted monthly and over 16 million members. Although larger joke datasets exist, the r/Jokes thread is unparalleled in the amount of rated jokes it contains. To the best of our knowledge there is no comparable source of rated jokes in any other language. These Reddit posts consist of the body of the joke, the punchline, and the number of reactions or upvotes. Although this type of humor may only be most enjoyable to a subset of the population, it is an effective way to measure responses to jokes in a large group setting.
Output: What kind of humor they have evaluated?
Multiple papers have shown that 'shallow' naive Bayes classifiers BIBREF12, BIBREF8, BIBREF13, BIBREF14, SVMs BIBREF15 and similar models work very well for doing LID. The DSL 2017 paper BIBREF1, for example, gives an overview of the solutions of all of the teams that competed on the shared task and the winning approach BIBREF16 used an SVM with character n-gram, parts of speech tag features and some other engineered features. The winning approach for DSL 2015 used an ensemble naive Bayes classifier. The fasttext classifier BIBREF17 is perhaps one of the best known efficient 'shallow' text classifiers that have been used for LID . Multiple papers have proposed hierarchical stacked classifiers (including lexicons) that would for example first classify a piece of text by language group and then by exact language BIBREF18, BIBREF19, BIBREF8, BIBREF0. Researchers have investigated deeper LID models li
Output: What is the approach of previous work?
Next, for emergent language, we show that the MASC architecture can achieve very high localization accuracy, significantly outperforming the baseline that does not include this mechanism.
Output: What evaluation metrics did the authors look at?
Baselines. As none of the existing KBC methods can solve the OKBC problem, we choose various versions of LiLi as baselines.

Single: Version of LiLi where we train a single prediction model INLINEFORM0 for all test relations.

Sep: We do not transfer (past learned) weights for initializing INLINEFORM0 , i.e., we disable LL.

F-th): Here, we use a fixed prediction threshold 0.5 instead of relation-specific threshold INLINEFORM0 .

BG: The missing or connecting links (when the user does not respond) are filled with “@-RelatedTo-@" blindly, no guessing mechanism.

w/o PTS: LiLi does not ask for additional clues via past task selection for skillset improvement.
Output: What baseline is used in the experiments?
As a warm-up procedure, the first 100 posts were annotated by two annotators (the author and the supervisor) and the results compared. The remainder of the annotation task was performed by the author, resulting in 3600 annotated samples.
Output: Who were the annotators?
In order to verify the reliability of our technique in coverage expansion for infrequent words we did a set of experiments on the Rare Word similarity dataset BIBREF6 . The dataset comprises 2034 pairs of rare words, such as ulcerate-change and nurturance-care, judged by 10 raters on a [0,10] scale. Table TABREF15 shows the results on the dataset for three pre-trained word embeddings (cf. § SECREF2 ), in their initial form as well as when enriched with additional words from WordNet.
Output: How are rare words defined?
A frequency filter ratio INLINEFORM0 is set to filter out the low-frequency words (rare words) from the lookup table
Output: how are rare words defined?
Paraphrases can be obtained by translating an English string into a foreign language and then back-translating it into English. 
Output: It looks like learning to paraphrase questions, a neural scoring model and a answer selection model cannot be trained end-to-end. How are they trained?
It includes a total 708 hours of French (Fr), German (De), Dutch (Nl), Russian (Ru), Spanish (Es), Italian (It), Turkish (Tr), Persian (Fa), Swedish (Sv), Mongolian (Mn) and Chinese (Zh) speeches, with French and German ones having the largest durations among existing public corpora.
Output: Which languages are part of the corpus?
The observed performance of both the BasicDCGAN and MultitaskDCGAN using 3-classes is comparable to the state-of-the-art, with 49.80% compared to 49.99% reported in BIBREF16 
Output: What model achieves state of the art performance on this task?
We consider two types of architectures: a bi-directional language model to augment the sequence to sequence encoder and a uni-directional model to augment the decoder. Both use self-attention BIBREF16 and the uni-directional model contains INLINEFORM0 transformer blocks, followed by a word classifier to predict the next word on the right. bi-directional language model to augment the sequence to sequence encoder
Output: What language model architectures are used?
We provide three pairs of short/long datasets from different domains (movies and restaurants) and from different languages (English and Korean) suitable for the task: Mov_en, Res_en, and Mov_ko. Most of the datasets are from previous literature and are gathered differently The Mov_en datasets are gathered from different websites; the short dataset consists of hand-picked sentences by BIBREF19 from document-level reviews from the Rotten Tomatoes website, while the long dataset consists of reviews from the IMDB website obtained by BIBREF20.
Output: What dierse domains and languages are present in new datasets?
We compare our sentence classification with the LSTM baseline and evaluate the biggest set of PICO sentence data available at this point BIBREF13. SCIBERT was chosen as an additional baseline model for fine-tuning because it provided the best representation of embedded PICO sentences. 
Output: What baselines did they consider?
First, we simply calculate the average unique predictions produced by both INLINEFORM0 and INLINEFORM1 in experiments shown in Section SECREF36 .
Output: What two metrics are proposed?
In a second step, we will train a neural monolingual translation system, that translates from the output of the PBMT system INLINEFORM0 to a better target sentence INLINEFORM1 .
Output: Do they train the NMT model on PBMT outputs?
To identify new keywords in the selected microposts, we again leverage crowdsourcing, as humans are typically better than machines at providing specific explanations BIBREF18, BIBREF19. In the crowdsourcing task, workers are first asked to find those microposts where the model predictions are deemed correct. Then, from those microposts, workers are asked to find the keyword that best indicates the class of the microposts as predicted by the model.
Output: How is the keyword specific expectation elicited from the crowd?
Finally, the morphosyntactic annotation was performed automatically, which may lead to errors, especially in the case of noisy web text.
Output: Did they use a crowdsourcing platform for annotations?
To fine-tune BERT, we used a proprietary corpus that consists of hundreds of thousands of legal agreements.
Output: How big is dataset used for fine-tuning BERT?
In this section, we present the details of the analysis performed on the data obtained pertaining to Twitter messages from January 2020 upto now, that is the time since the news of the Coronavirus outbreak in China was spread across nations. Also, the data prominently captures the tweets in English, Spanish, and French languages.
Output: Do they collect only English data?
In natural language processing (NLP), taking into account the morphological complexity inherent to each language could be important for improving or adapting the existing methods, since the amount of semantic and grammatical information encoded at the word level, may vary significantly from language to language. This information could be useful for linguistic analysis and for measuring the impact of different word form normalization tools depending of the language.
Output: what is the practical application for this paper?
We designed 3 evaluation sets: (1) Base Set (1,264 samples) held out from the simulated data. (2) Augmented Set (1,280 samples) built by adding two out-of-distribution symptoms, with corresponding dialogue contents and queries, to the Base Set (“bleeding” and “cold”, which never appeared in training data). (3) Real-World Set (944 samples) manually delineated from the the symptom checking portions (approximately 4 hours) of real-world dialogues, and annotated as evaluation samples.
Output: How do they select instances to their hold-out test set?
Where possible, we tried to restrict the content of each text to the abstract and conclusion of the original work. However, since in many cases these sections were too short, we also considered other parts of the original works such as introduction or discussion sections. To ensure that the extracted text portions are appropriate for the AV task, each original work was preprocessed manually. More precisely, we removed tables, formulas, citations, quotes and sentences that include non-language content such as mathematical constructs or specific names of researchers, systems or algorithms.
Output: What is a self-compiled corpus?
We also run additional CTC experiments with 36 layers Transformer (total parameters $\pm $120 millions). The baseline with 36 layers has the same performance with 24 layers, but by adding the proposed methods, the 36 layer performance improved to give the best results. 
Output: How many layers do they use in their best performing network?
The results in Table TABREF7 show that SRCC performs much better in knowledge extraction. The two documents' contents contain the same idea expressed by terms in a different order that John had asked Mary to marry him before she left. It is obvious that cosine similarity cannot recognize this association, but SRCC has successfully recognized it and produced a similarity value of -0.285714.
Output: How do they evaluate knowledge extraction performance?
We use the Fisher Spanish speech corpus BIBREF11, which consists of 819 phone calls, with an average duration of 12 minutes, amounting to a total of 160 hours of data.
Output: What language do they look at?
As shown in Table , the proposed PS-rnn-elmo shows a significant MAP performance improvement compared to the previous best model, CompClip-LM (0.696 to 0.734 absolute).
Output: How much better performance of proposed model compared to answer-selection models?
The Business Language Testing Service (BULATS) test of Cambridge Assessment English BIBREF27 is a multi-level computer-based English test. It consists of read speech and free-speaking components, with the candidate responding to prompts. The BULATS spoken test has five sections, all with materials appropriate to business scenarios.  In this work, non-native speech from the BULATS test is used as both training and test data for the speaker verification systems. To investigate how the systems generalise, data for testing is also taken from the Cambridge Assessment English Linguaskill online test. Like BULATS, this is also a multi-level test and has a similar format composed of the same five sections as described before but assesses general English ability.
Output: What standard large speaker verification corpora is used for evaluation?
Dialogue Collection: before the formal data collection starts, we required the workers to make a small number of dialogues and gave them feedback about the dialogue quality. Then, well-trained workers were paired to converse according to the given goals. The workers were also asked to annotate both user states and system states.

Dialogue Annotation: we used some rules to automatically annotate dialogue acts according to user states, system states, and dialogue histories. To evaluate the quality of the annotation of dialogue acts and states, three experts were employed to manually annotate dialogue acts and states for 50 dialogues. The results show that our annotations are of high quality. Finally, each dialogue contains a structured goal, a task description, user states, system states, dialogue acts, and utterances.
Output: How was the corpus annotated?
We train our models on Sentiment140 and Amazon product reviews. Both of these datasets concentrates on sentiment represented by a short text.  For training the softmax model, we divide the text sentiment to two kinds of emotion, positive and negative. And for training the tanh model, we convert the positive and negative emotion to [-1.0, 1.0] continuous sentiment score, while 1.0 means positive and vice versa. 
Output: Was the introduced LSTM+CNN model trained on annotated data in a supervised fashion?
Results show that the CJFA encoder obtains significantly better phone classification accuracy than the VAE baseline and also than the CJFS encoder. These results are replicated for speaker recognition tasks.
Output: Which approach out of two proposed in the paper performed better in experiments?
We use two popular knowledge bases: WordNet BIBREF0 and Freebase BIBREF1 in this paper. Specifically, we use WN18 (a subset of WordNet) BIBREF24 and FB15K (a subset of Freebase) BIBREF2 since their text descriptions are easily publicly available. Table 1 lists statistics of the two datasets.
Output: What datasets are used to evaluate this paper?
Approach ::: Method: Training and Testing ::: Experiment 1: Representation
Some of the problems encountered by prior approaches seem to be attributable to the use of infix notation. In this experiment, we compare translation BLEU-2 scores to spot the differences in representation interpretability. Approach ::: Method: Training and Testing ::: Experiment 2: State-of-the-art
This experiment compares our networks to recent previous work. We count a given test score by a simple “correct versus incorrect" method. The answer to an expression directly ties to all of the translation terms being correct, which is why we do not consider partial precision. We compare average accuracies over 3 test trials on different randomly sampled test sets from each MWP dataset.
Output: How is this problem evaluated?
We developed an AdaBoost-based classifier to detect our new fake reviews, consisting of 200 shallow decision trees (depth 2).
Output: What kind of model do they use for detection?
This dataset is aimed to contribute to developing a partisan news detector. There are several ways that the dataset can be used to devise the system. For example, it is possible to train the detector using publisher-level labels and test with article-level labels. It is also possible to use semi-supervised learning and treat the publisher-level part as unsupervised, or use only the article-level part. We also released the raw survey data so that new mechanisms to decide the article-level labels can be devised.
Output: What examples of applications are mentioned?
The training data released for this task contains 22 drug labels, referred to as Training-22, with gold standard annotations.  As Training-22 is a relatively small dataset, we additionally utilize an external dataset with 180 annotated drug labels dubbed NLM-180 BIBREF5 (more later). 
Output: What training data did they use?
We need good amount of data to try deep learning state-of-the-art algorithms. There are lot of open datasets available for names, locations, organisations, but not for topics as defined in Abstract above. Also defining and inferring topics is an individual preference and there are no fix set of rules for its definition. But according to our definition, we can use wikipedia titles as our target topics. English wikipedia dataset has more than 18 million titles if we consider all versions of them till now. We had to clean up the titles to remove junk titles as wikipedia title almost contains all the words we use daily.  After doing some more cleaning we were left with 10 million titles. We have a dump of 15 million English news articles published in past 4 years. Further, we reduced number of articles by removing duplicate and near similar articles. We used our pre-trained doc2vec models and cosine similarity to detect almost similar news articles.
Output: How large is the dataset they used?
Our work is further connected to the topic of interpretability and transparency of machine learning models BIBREF11, BIBREF35, BIBREF12, for which humans are increasingly involved, for instance for post-hoc evaluations of the model's interpretability. 
Output: How are the interpretability merits of the approach demonstrated?
We achieved first place for 8 directions: German$\leftrightarrow $English, German$\leftrightarrow $French, Chinese$\leftrightarrow $English, English$\rightarrow $Lithuanian, English$\rightarrow $Finnish, and Russian$\rightarrow $English, and three other directions were placed second (ranked by teams), which included Lithuanian$\rightarrow $English, Finnish$\rightarrow $English, and English$\rightarrow $Kazakh.
Output: Which language directions are machine translation systems of WMT evaluated on?
The final results for our F1-score from this classification technique are 0.52 and 0.31 for the single and multi-turn data respectively.

We compare our results for both the single-turn and multi-turn experiments to the accuracies on the test data based off the BLEU score.
Output: which existing metrics do they compare with?
GPT-2 achieves the highest score and the $n$-gram the lowest.
Output: Which of the model yields the best performance?
Interesting prior work on quantifying social norm violation has taken a heavily data-driven focus BIBREF8 , BIBREF9 . 
Output: Does this paper propose a new task that others can try to improve performance on?
We evaluate our approaches on two public Chinese span-extraction machine reading comprehension datasets: CMRC 2018 (simplified Chinese) BIBREF8 and DRCD (traditional Chinese) BIBREF9. The statistics of the two datasets are listed in Table TABREF29.
Output: Is this a span-based (extractive) QA task?
The second, Continuous Bag-of-Words (CBOW), the model is given the sequence of words without the middle one and attempts to predict this omitted word. 
Output: Which word embeddings are analysed?
In addition, in order to estimate annotation reliability and provide for better evaluation, every question in the test set is answered by at least two additional experts.
Output: Are the answers double (and not triple) annotated?
Motivated by phrase based SMT, we retrieve neighbors which have high local, sub-sentence level overlap with the source sentence. We adapt our approach to retrieve n-grams instead of sentences. We represent every sentence by their reduced n-gram set. For every n-gram in INLINEFORM0 , we find the closest n-gram in the training set using the IDF similarity defined above.
Output: Where do they retrieve neighbor n-grams from in their approach?
For POS tagging, we observe error reductions of respectively 0.71% for GSD, 0.81% for Sequoia, 0.7% for Spoken and 0.28% for ParTUT. For parsing, we observe error reductions in LAS of 2.96% for GSD, 3.33% for Sequoia, 1.70% for Spoken and 1.65% for ParTUT. On the XNLI benchmark, CamemBERT obtains improved performance over multilingual language models on the TRANSLATE-TRAIN setting (81.2 vs. 80.2 for XLM) while using less than half the parameters (110M vs. 250M). However, its performance still lags behind models trained on the original English training set in the TRANSLATE-TEST setting, 81.2 vs. 82.91 for RoBERTa. Both improvements result in a 2.36 point increase in the F1 score with respect to the best SEM architecture (BiLSTM-CRF), giving CamemBERT the state of the art for NER on the FTB.
Output: How much better was results of CamemBERT than previous results on these tasks?
We make copies of the monolingual model for each language and add additional crosslingual latent variables (CLVs) to couple the monolingual models, capturing crosslingual semantic role patterns. Concretely, when training on parallel sentences, whenever the head words of the arguments are aligned, we add a CLV as a parent of the two corresponding role variables. 
Output: Which additional latent variables are used in the model?
Violations The most challenging features are all related to Violations. Low performance on Infl/Agr Violations, which marks morphological violations (He washed yourself, This is happy), is especially striking because a relatively high proportion (29%) of these sentences are Simple. These models are likely to be deficient in encoding morphological features is that they are word level models, and do not have direct access sub-word information like inflectional endings, which indicates that these features are difficult to learn effectively purely from lexical distributions.
Output: Do the authors have a hypothesis as to why morphological agreement is hardly learned by any model?
In spite of the windowing approach, the class distribution is still skewed, and an accuracy metric would reflect the particular class distribution in our data set. Therefore, we adopt the unweighted average recall (UAR) metric commonly used in emotion classification research. UAR is a reweighted accuracy where the samples of both classes are weighted equally in aggregate. UAR thus simulates a uniform class distribution. To match the objective, our classifiers are trained on appropriately weighted training data. Note that chance performance for UAR is by definition 50%, making results more comparable across different data sets.
Output: What they use as a metric of finding hot spots in meeting?
Contexts are either ground-truth contexts from that dataset, or they are Wikipedia passages retrieved using TF-IDF BIBREF24 based on a HotpotQA question. A new non-overlapping set of contexts was again constructed from Wikipedia via HotpotQA using the same method as Round 1. In addition to contexts from Wikipedia for Round 3, we also included contexts from the following domains: News (extracted from Common Crawl), fiction (extracted from BIBREF27, and BIBREF28), formal spoken text (excerpted from court and presidential debate transcripts in the Manually Annotated Sub-Corpus (MASC) of the Open American National Corpus), and causal or procedural text, which describes sequences of events or actions, extracted from WikiHow. Finally, we also collected annotations using the longer contexts present in the GLUE RTE training data, which came from the RTE5 dataset BIBREF29.
Output: What data sources do they use for creating their dataset?
Our user study compares the correctness of three scenarios:

Parser correctness - our baseline is the percentage of examples where the top query returned by the semantic parser was correct.

User correctness - the percentage of examples where the user selected a correct query from the top-7 generated by the parser.

Hybrid correctness - correctness of queries returned by a combination of the previous two scenarios. The system returns the query marked by the user as correct; if the user marks all queries as incorrect it will return the parser's top candidate.
Output: Do they conduct a user study where they show an NL interface with and without their explanation?
We train language models on two languages: One model is estimated on the German newscrawl distributed by WMT'18 comprising 260M sentences or 6B tokens. Another model is trained on the English newscrawl data comprising 193M sentences or 5B tokens. We learn a joint Byte-Pair-Encoding (BPE; Sennrich et al., 2016) vocabulary of 37K types on the German and English newscrawl and train the language models with this vocabulary. We consider two benchmarks: Most experiments are run on the WMT'18 English-German (en-de) news translation task and we validate our findings on the WMT'18 English-Turkish (en-tr) news task. For WMT'18 English-German, the training corpus consists of all available bitext excluding the ParaCrawl corpus and we remove sentences longer than 250 tokens as well as sentence-pairs with a source/target length ratio exceeding 1.5. This results in 5.18M sentence pairs. We tokenize all data with the Moses tokenizer BIBREF8 and apply the BPE vocabulary learned on the monolingual corpora.
Output: What dataset do they use?
We implement three baseline models for comparison: Noun WordNet Semantic Text Exchange Model (NWN-STEM), General WordNet Semantic Text Exchange Model (GWN-STEM), and Word2Vec Semantic Text Exchange Model (W2V-STEM).
Output: What are the baseline models mentioned in the paper?
The ensembles were formed by simply averaging the predictions from the constituent single models.
Output: How does their ensemble method work?
The corpus of supervisor assessment has 26972 sentences. The summary statistics about the number of words in a sentence is: min:4 max:217 average:15.5 STDEV:9.2 Q1:9 Q2:14 Q3:19.
Output: What is the average length of the sentences?
The Hausa data used in this paper is part of the LORELEI language pack. It consists of Broad Operational Language Translation (BOLT) data gathered from news sites, forums, weblogs, Wikipedia articles and twitter messages. We use a split of 10k training and 1k test instances. The Yorùbá NER data used in this work is the annotated corpus of Global Voices news articles recently released by BIBREF22. The dataset consists of 1,101 sentences (26k tokens) divided into 709 training sentences, 113 validation sentences and 279 test sentences based on 65%/10%/25% split ratio.
Output: How much labeled data is available for these two languages?
Slot Filling is an information extraction task which has become popular in the last years BIBREF3 . The task aims at extracting information about persons, organizations or geo-political entities from a large collection of news, web and discussion forum documents.
Output: What is the task of slot filling?
In order to compare with the previous methods, we evaluate our model on AIDA-B and other datasets.  AIDA-CoNLL BIBREF14 is annotated on Reuters news articles. It contains training (AIDA-Train), validation (AIDA-A) and test (AIDA-B) sets.

ACE2004 BIBREF15 is a subset of the ACE2004 Coreference documents.

MSNBC BIBREF16 contains top two stories in the ten news categories(Politics, Business, Sports etc.)

AQUAINT BIBREF17 is a news corpus from the Xinhua News Service, the New York Times, and the Associated Press.

WNED-CWEB BIBREF18 is randomly picked from the FACC1 annotated ClueWeb 2012 dataset.

WNED-WIKI BIBREF18 is crawled from Wikipedia pages with its original hyperlink annotation.
Output: What datasets used for evaluation?
our method still can improve the state-of-the-art accuracy BIBREF7 from 60.32% to 60.34% Although we only have 57% of testing questions can benefit from the basic questions, our method still can improve the state-of-the-art accuracy BIBREF7 from 60.32% to 60.34%, 
Output: What accuracy do they approach with their proposed method?
Logician is trained under the attention-based sequence-to-sequence paradigm, with three mechanisms: restricted copy mechanism to ensure literally honestness, coverage mechanism to alleviate the under extraction and over extraction problem, and gated dependency attention mechanism to incorporate dependency information. 
Output: How is Logician different from traditional seq2seq models?
To test whether topic models can be used for dating poetry or attributing authorship, we perform supervised classification experiments with Random Forest Ensemble classifiers. 
Output: What is the algorithm used for the classification tasks?
Embedding: We developed different variations of our models with a simple lookup table embeddings learned from scratch and using high-performance contextual embeddings, which are ELMo BIBREF11, BERT BIBREF16 and ClinicalBERT BIBREF13 (trained and provided by the authors).
Output: What embeddings are used?
For each candidate mention span, we retrieve the top 10 entities according to the Freebase API. We then create a lattice in which the nodes correspond to mention-entity pairs, scored by their Freebase API scores, and the edges encode the fact that no joint assignment of entities to mentions can contain overlapping spans. We take the top 10 paths through the lattice as possible entity disambiguations. For each possibility, we generate $n$ -best paraphrases that contains the entity mention spans. In the end, this process creates a total of $10n$ paraphrases. 
Output: How many paraphrases are generated per question?
The split methods (beyond random split) are the following:

Output length: Variation of the setup described by BIBREF2 where the train set consists of examples with output (sparql query or action sequence) length $\le \hspace{-2.5pt} N$, while the test set consists of examples with output length $> \hspace{-2.5pt} N$. For CFQ, we use $N = 7$ constraints. For scan, we use $N = 22$ actions.

Input length: Variation of the above setup, in which the train set consists of examples with input (question or command) length $\le N$, while test set consists of examples with input length $> N$. For CFQ, we use $N=19$ grammar leaves. For SCAN, we use $N=8$ tokens.

Output pattern: Variation of setup described by BIBREF8, in which the split is based on randomly assigning clusters of examples sharing the same output (query or action sequence) pattern. Query patterns are determined by anonymizing entities and properties; action sequence patterns collapse primitive actions and directions.

Input pattern: Variation of the previous setup in which the split is based on randomly assigning clusters of examples sharing the same input (question or command) pattern. Question patterns are determined by anonymizing entity and property names ; command patterns collapse verbs and the interchangeable pairs left/right, around/opposite, twice/thrice.
Output: What are other approaches into creating compositional generalization benchmarks?
We conduct all our experiments on 635hrs of audio data for 7 Indian languages collected from $\textbf {All India Radio}$ news channel.  We collected and curated around 635Hrs of audio data for 7 Indian languages, namely Kannada, Hindi, Telugu, Malayalam, Bengali, and English. We collected the data from the All India Radio news channel where an actor will be reading news for about 5-10 mins. To cover many speakers for the dataset, we crawled data from 2010 to 2019. Since the audio is very long to train any deep neural network directly, we segment the audio clips into smaller chunks using Voice activity detector. Since the audio clips will have music embedded during the news, we use Inhouse music detection model to remove the music segments from the dataset to make the dataset clean and our dataset contains 635Hrs of clean audio which is divided into 520Hrs of training data containing 165K utterances and 115Hrs of testing data containing 35K utterances. 
Output: How was the audio data gathered?