niryuu/sni-each-converted huggingface dataset:
The third person plural pronoun `they' has no gender in English and most other languages (unlike the third person singular pronouns `he', `she', and `it').  If these sentences are translated into French, then `they' in the first sentence should be translated `elles', as referring to Jane and Susan, and `they' in the second sentence should be translated `ils', as referring to Fred and George. A couple of examples: The word `sie' in German serves as both the formal second person prounoun (always capitalized), the third person feminine singular, and the third person plural. 
Output: What language do they explore?
This section introduces our probabilistic model that infers keyword expectation and trains the target model simultaneously.
Output: What type of classifiers are used?
A single speaker narrated the 2000 sentences, which took several days. 
Output: How many annotators participated?
Our first baseline is ROUGE-L BIBREF1 , since it is the most commonly used metric for compression tasks. We compare to the best n-gram-overlap metrics from toutanova2016dataset; We further compare to the negative LM cross-entropy, i.e., the log-probability which is only normalized by sentence length. Our next baseline is perplexity, which corresponds to the exponentiated cross-entropy: Due to its popularity, we also performed initial experiments with BLEU BIBREF17 . 
Output: Is ROUGE their only baseline?
We test our proposed approach for binary classification on either sarcasm or irony, on seven benchmark datasets retrieved from different media sources. Below we describe each dataset, please see Table TABREF1 below for a summary. Twitter: We use the Twitter dataset provided for the SemEval 2018 Task 3, Irony Detection in English Tweets BIBREF18  Finally, we use the dataset by BIBREF20 , who collected a user self-annotated corpus of tweets with the #sarcasm hashtag. Reddit: BIBREF21 collected SARC, a corpus comprising of 600.000 sarcastic comments on Reddit. We use main subset, SARC 2.0, and the political subset, SARC 2.0 pol. Online Dialogues: We utilize the Sarcasm Corpus V1 (SC-V1) and the Sarcasm Corpus V2 (SC-V2), which are subsets of the Internet Argument Corpus (IAC). Compared to other datasets in our selection, these differ mainly in text length and structure complexity BIBREF22 .  To augment each dataset with our external data, we first filter out tweets that are not in English using language guessing systems.
Output: Do they evaluate only on English?
We use TB-Dense and MATRES in our experiments and briefly summarize the data statistics in Table TABREF33.
Output: What datasets were used for this work?
By choosing English (En) as the pivot language, we perform pivot alignments for identical English segments on Europarl Fr-En and En-De parallel corpora BIBREF18 , constructing a multi-parallel corpus of Fr-En-De. Then each of the Fr*-De and Fr-De* pseudo parallel corpora is established from the multi-parallel data by applying the pivot language-based translation described in the previous section. For automatic translation, we utilize a pre-trained and publicly released NMT model for En $\rightarrow $ De and train another NMT model for En $\rightarrow $ Fr using the WMT'15 En-Fr parallel corpus BIBREF19 .
Output: Where do they collect the synthetic data?
In order to achieve these configuration the TIMIT data was split. Fig. FIGREF12 illustrates the split of the data into 8 subsets (A–H). The TIMIT dataset contains speech from 462 speakers in training and 168 speakers in the test set, with 8 utterances for each speaker. The TIMIT training and test set are split into 8 blocks, where each block contains 2 utterances per speaker, randomly chosen. Thus each block A,B,C,D contains data from 462 speakers with 924 utterances taken from the training sets, and each block E,F,G,H contains speech from 168 test set speakers with 336 utterances.

For Task a training of embeddings and the classifier is identical, namely consisting of data from blocks (A+B+C+E+F+G). The test data is the remainder, namely blocks (D+H). For Task b the training of embeddings and classifiers uses (A+B+E+F) and (C+G) respectively, while again using (D+H) for test. Task c keeps both separate: embeddings are trained on (A+B+C+D), classifiers on (E+G) and tests are conducted on (F+H). Note that H is part of all tasks, and that Task c is considerably easier as the number of speakers to separate is only 168, although training conditions are more difficult.
Output: What TIMIT datasets are used for testing?
We formalize this intuition in a model called NextSum, which selects the next summary sentence based not only on properties of the source text, but also on the previously selected sentences in the summary.
Output: How does nextsum work?
We have observed that the number of epochs is an important parameter and its increase leads to results that rank our two worst models almost equal, or even better than others.
Output: How does different parameter settings impact the performance and semantic capacity of resulting model?
There are 8757 news records in our preprocessed data set. 
Output: How large is the dataset?
To achieve this, a method is proposed that transforms the goal-labels of the used dataset (DSTC2) into labels whose behaviour can be replicated during deployment.
Output: what corpus is used to learn behavior?
In order to find the object type, SimplerVoice, first, builds an ontology-based knowledge tree. We propose to use 2 methods to generate the suitable verbs for the target object: heuristics-based, and n-grams model.
Output: Which model do they use to generate key messages?
In order to train and evaluate open-domain factoid QA system for real-world questions, we build a new Chinese QA dataset named as WebQA.
Output: What languages do they experiment with?
We propose a novel problem of relationship recommendation (RSR). Different from the reciprocal recommendation problem on DSNs, our RSR task operates on regular social networks (RSN), estimating long-term and serious relationship compatibility based on social posts such as tweets.
Output: Is this a task other people have worked on?
UTD aims to automatically identify and cluster repeated terms (e.g. words or phrases) from speech. To circumvent the exhaustive DTW-based search limited by INLINEFORM0 time BIBREF6 , we exploit the scalable UTD framework in the Zero Resource Toolkit (ZRTools) BIBREF7 , which permits search in INLINEFORM1 time.
Output: How is the vocabulary of word-like or phoneme-like units automatically discovered?
The systems were trained on all parallel data available for the WMT 2016. The news commentary corpus, the European parliament proceedings and the common crawl corpus sum up to 3.7M sentences and around 90M words.
Output: Which dataset do they use?
One-Vs-The-Rest strategy was adopted for the task of multi-class classification and I reported F-score, micro-F, macro-F and weighted-F scores using 10-fold cross-validation. 
Output: What metrics are considered?
Argus Dataset AI2-8grade/CK12 Dataset We consider this dataset as preliminary since it was not reviewed by a human and many hypotheses are apparently unprovable by the evidence we have gathered (i.e. the theoretical top accuracy is much lower than 1.0).  MCTest Dataset We use an official extension of the dataset for RTE evaluation that again textually merges questions and answers.
Output: what datasets did they use?
We evaluate our cross-lingual pre-training based transfer approach against several strong baselines on two public datatsets, Europarl BIBREF31 and MultiUN BIBREF32, which contain multi-parallel evaluation data to assess the zero-shot performance.
Output: which datasets did they experiment with?
We used the Levenshtein distance metric BIBREF8 implemented in Apache Lucene library BIBREF9 . Another simple approach is the aforementioned diacritical swapping, which is a term that we introduce here for referring to a solution inspired by the work of BIBREF4 . A promising method, adapted from work on correcting texts by English language learners BIBREF11 , expands on the concept of selecting a correction nearest to the spelling error according to some notion of distance. Here, the Levenshtein distance is used in a weighted sum to cosine distance between word vectors. Our ELMo-augmented LSTM is bidirectional.
Output: What methods are tested in PIEWi?
For experiments that use parallel data to initialize foreign specific parameters, we use the same datasets in the work of BIBREF6. Specifically, we use United Nations Parallel Corpus BIBREF18 for en-ru, en-ar, en-zh, and en-fr. We collect en-hi parallel data from IIT Bombay corpus BIBREF19 and en-vi data from OpenSubtitles 2018.
Output: What datasets are used for evaluation?
We built a dataset for the task of hate speech detection in memes with 5,020 images that were weakly labeled into hate or non-hate memes, depending on their source.
Output: How is each instance of the dataset annotated?
For each test set, we use the corresponding training questions $Q_\mathit {tr}$ to retrieve domain-relevant sentences from S. Specifically, for each multiple-choice question $(q,A) \in Q_\mathit {tr}$ and each choice $a \in A$ , we use all non-stopword tokens in $q$ and $a$ as an ElasticSearch query against S. We take the top 200 hits, run Open IE v4, and aggregate the resulting tuples over all $a \in A$ and over all questions in $Q_\mathit {tr}$ to create the tuple KB (T).
Output: What method was used to generate the OpenIE extractions?
We evaluate with all commonly-used metrics in question generation BIBREF13: BLEU-1 (B1), BLEU-2 (B2), BLEU-3 (B3), BLEU-4 (B4) BIBREF17, METEOR (MET) BIBREF18 and ROUGE-L (R-L) BIBREF19. We use the evaluation script released by Chen2015MicrosoftCC.
Output: What metrics do they use?
Observe that in only about 12% (7 of the top 59) of the most cited areas of research, women received higher average citations than men. These include: sentiment analysis, information extraction, document summarization, spoken dialogue, cross lingual (research), dialogue, systems, language generation. (Of course, note that some of the 59 areas, as estimated using title term bigrams, are overlapping. Also, we did not include large scale in the list above because the difference in averages is very small and it is not really an area of research.) Thus, the citation gap is common across a majority of the high-citations areas within NLP.
Output: Which NLP area have the highest average citation for woman author?
We only used models of dimension 300 and, for Word2Vec, Wang2Vec, and FastText, skip-gram architectured models.
Output: what word embedding techniques did they experiment with?
We present a comparison of the proposed models to existing word embeddings approaches. These are: the Bernoulli embeddings (b-emb) BIBREF1 , continuous bag-of-words (CBOW) BIBREF5 , Distributed Memory version of Paragraph Vector (PV-DM) BIBREF11 and the Global Vectors (GloVe) BIBREF6 model.
Output: What word embeddings do they test?
These features are typically obtained by training a deep neural network jointly on several languages for which labelled data is available. The bottom layers of the network are normally shared across all training languages. The network then splits into separate parts for each of the languages, or has a single shared output. The final output layer has phone labels or HMM states as targets. The final shared layer often has a lower dimensionality than the input layer, and is therefore referred to as a `bottleneck'.
Output: What are bottleneck features?
The corpus consists of 53 documents, which contain an average number of 156.1 sentences per document, each with 19.55 tokens on average.
Output: How many documents are in the new corpus?
This section reports the results of the experiments conducted on two data sets for evaluating the performances of wDTW and wTED against other baseline methods. We denote the following distance/similarity measures. WMD: The Word Mover's Distance introduced in Section SECREF1 . VSM: The similarity measure introduced in Section UID12 . PV-DTW: PV-DTW is the same as Algorithm SECREF21 except that the distance between two paragraphs is not based on Algorithm SECREF20 but rather computed as INLINEFORM0 where INLINEFORM1 is the PV embedding of paragraph INLINEFORM2 . PV-TED: PV-TED is the same as Algorithm SECREF23 except that the distance between two paragraphs is not based on Algorithm SECREF20 but rather computed as INLINEFORM0 .
Output: Which are the state-of-the-art models?
We have shown that transformer-based encoders together with GRU models provide the best performance for question representation. Notably, we demonstrated that using pre-trained text representations provide consistent performance improvements across several hyper-parameter configurations. We have also shown that using an object detector fine-tuned with external data provides large improvements in accuracy. Moreover, we have shown that attention mechanisms are paramount for learning top performing networks, once they allow producing question-aware image representations that are capable of encoding spatial relations. It became clear that Top-Down is the preferred attention method, given its results with ReLU activation.
Output: What components are identified as core components for training VQA models?
They used 20-million-words data randomly sampled from the raw text released by the CoNLL 2017 Shared Task - Automatically Annotated Raw Texts and Word Embeddings BIBREF8, which is a combination of Wikipedia dump and common crawl.  For example, we compared the Latvian model by ELMoForManyLangs with a model we trained on a complete (wikidump + common crawl) Latvian corpus, which has about 280 million tokens.
Output: How larger are the training sets of these versions of ELMo compared to the previous ones?
Using our methodology, we tagged 10,000 Arabic tweet dataset for offensiveness, where offensive tweets account for roughly 19% of the tweets. Further, we labeled tweets as vulgar or hate speech. 
Output: How many tweets are in the dataset?
We find that all the factors we tested can qualitatively affect how a model generalizes on the question formation task. These factors are the type of recurrent unit, the type of attention, and the choice of sequential vs. tree-based model structure.
Output: What architectural factors were investigated?
We compare our proposed discrete CVAE (DCVAE) with the two-stage sampling approach to three categories of response generation models:

Baselines: Seq2seq, the basic encoder-decoder model with soft attention mechanism BIBREF30 used in decoding and beam search used in testing; MMI-bidi BIBREF5, which uses the MMI to re-rank results from beam search.

CVAE BIBREF14: We adjust the original work which is for multi-round conversation for our single-round setting. For a fair comparison, we utilize the same keywords used in our network pre-training as the knowledge-guided features in this model.

Other enhanced encoder-decoder models: Hierarchical Gated Fusion Unit (HGFU) BIBREF12, which incorporates a cue word extracted using pointwise mutual information (PMI) into the decoder to generate meaningful responses; Mechanism-Aware Neural Machine (MANM) BIBREF13, which introduces latent embeddings to allow for multiple diverse response generation.
Output: What other kinds of generation models are used in experiments?
(proposed) Bi-LSTM/CRF + Bi-CharLSTM with modality attention (W+C): uses the modality attention to merge word and character embeddings.

(proposed) Bi-LSTM/CRF + Bi-CharLSTM + Inception (W+C+V): takes as input visual contexts extracted from InceptionNet as well, concatenated with word and char vectors.

(proposed) Bi-LSTM/CRF + Bi-CharLSTM + Inception with modality attention (W+C+V): uses the modality attention to merge word, character, and visual embeddings as input to entity LSTM.
Output: Does their NER model learn NER from both text and images?
Because our conversion rules are interpretable, we identify shortcomings in both resources, using each as validation for the other. We were able to find specific instances of incorrectly applied UniMorph annotation, as well as specific instances of cross-lingual inconsistency in both resources.
Output: Do they look for inconsistencies between different languages' annotations in UniMorph?
In this study, we experiment four different classification models including logistic regression (LR), recurrent neural network (RNN) BIBREF35, convolutional neural network (CNN) BIBREF36 and Google BERT BIBREF37. 
Output: Which document classifiers do they experiment with?
We use 16 different datasets from several popular review corpora used in BIBREF20 . These datasets consist of 14 product review datasets and two movie review datasets. We use CoNLL 2000 BIBREF22 sequence labeling dataset for both POS Tagging and Chunking tasks.
Output: What dataset did they use?
. While automated evaluation metrics like ROUGE measure lexical similarity between machine and human summaries, humans can better measure how coherent and readable a summary is. Our evaluation study investigates whether tuning the PG-net model increases summary coherence, by asking evaluators to select which of three summaries for the same document they like most: the PG-net model trained on CNN/DM; the model trained on student reflections; and finally the model trained on CNN/DM and tuned on student reflections. 20 evaluators were recruited from our institution and asked to each perform 20 annotations. Summaries are presented to evaluators in random order. Evaluators are then asked to select the summary they feel to be most readable and coherent.
Output: Who were the human evaluators used?
Table 1 shows an example of difficult and easy examples according to our definition of difficulty. The underlined text demarcates the (consensus) reference label provided by domain experts. In the difficult examples, crowd workers marked text distinct from these reference annotations; whereas in the easy cases they reproduced them with reasonable fidelity. The difficult sentences usually exhibit complicated structure and feature jargon.
Output: Is an instance a sentence or an IE tuple?
Annotator votes for one of the seven emotions, namely Ekman’s six basic emotions BIBREF1, plus the neutral.
Output: What labels does the dataset have?
We defined the reward as being 1 for successfully completing the task, and 0 otherwise. A discount of $0.95$ was used to incentivize the system to complete dialogs faster rather than slower, yielding return 0 for failed dialogs, and $G = 0.95^{T-1}$ for successful dialogs, where $T$ is the number of system turns in the dialog.   0.95^{T-1} reward  0.95^{T-1} 
Output: What is the reward model for the reinforcement learning appraoch?
The experiment dataset comes from Microsoft Research (MSR) . It contains three domains: movie, taxi, and restaurant. The total count of dialogues per domain and train/valid/test split is reported in Table TABREF11. At every turn both user and agent acts are annotated, we use only the agent side as targets in our experiment. The acts are ordered in the dataset (each output sentence aligns with one act).
Output: What datasets are used for training/testing models?
KryptoOracle has been built in the Apache ecosystem and uses Apache Spark. Data structures in Spark are based on resilient distributed datasets (RDD), a read only multi-set of data which can be distributed over a cluster of machines and is fault tolerant.  Spark RDD has the innate capability to recover itself because it stores all execution steps in a lineage graph. In case of any faults in the system, Spark redoes all the previous executions from the built DAG and recovers itself to the previous steady state from any fault such as memory overload. Spark RDDs lie in the core of KryptoOracle and therefore make it easier for it to recover from faults. Moreover, faults like memory overload or system crashes may require for the whole system to hard reboot. However, due to the duplicate copies of the RDDs in Apache Hive and the stored previous state of the machine learning model, KryptoOracle can easily recover to the previous steady state.


Output: How is the architecture fault-tolerant?
The unlabeled set of development data was used in the training of both the UBM and the i-vector extractor.
Output: Do they single out a validation set from the fixed SRE training set?
The data used for this task is drawn from the Switchboard conversational English corpus BIBREF31 .
Output: Which dataset do they use?
In this paper, we present an extraction-then-synthesis framework for machine reading comprehension shown in Figure 1 , in which the answer is synthesized from the extraction results. We build an evidence extraction model to predict the most important sub-spans from the passages as evidence, and then develop an answer synthesis model which takes the evidence as additional features along with the question and passage to further elaborate the final answers.
Output: Which framework they propose in this paper?
In this paper, we present an extraction-then-synthesis framework for machine reading comprehension shown in Figure 1 , in which the answer is synthesized from the extraction results. We build an evidence extraction model to predict the most important sub-spans from the passages as evidence, and then develop an answer synthesis model which takes the evidence as additional features along with the question and passage to further elaborate the final answers.
Output: What two components are included in their proposed framework?
For each object, $O_i$ in the input Open IE tuple $(S; P; O_1; O_2 \ldots )$ , we add a triple $(S; P; O_i)$ to this table.
Output: Are the OpenIE extractions all triples?
We test the character and word-level variants by predicting hashtags for a held-out test set of posts.
Output: What other tasks do they test their method on?
Modules: Modules are the basic building blocks of different models. In LeafNATS, we provide ready-to-use modules for constructing recurrent neural network (RNN)-based sequence-to-sequence (Seq2Seq) models for NATS, e.g., pointer-generator network BIBREF1 . These modules include embedder, RNN encoder, attention BIBREF24 , temporal attention BIBREF6 , attention on decoder BIBREF2 and others. We also use these basic modules to assemble a pointer-generator decoder module and the corresponding beam search algorithms. The embedder can also be used to realize the embedding-weights sharing mechanism BIBREF2 .
Output: What models are included in the toolkit?
Our system learns Perceptron models BIBREF37 using the Machine Learning machinery provided by the Apache OpenNLP project with our own customized (local and clustering) features.  The local features constitute our baseline system on top of which the clustering features are added.
Output: what are the baselines?
The final collection of Facebook pages for the experiments described in this paper is as follows: FoxNews, CNN, ESPN, New York Times, Time magazine, Huffington Post Weird News, The Guardian, Cartoon Network, Cooking Light, Home Cooking Adventure, Justin Bieber, Nickelodeon, Spongebob, Disney.
Output: Which Facebook pages did they look at?
We use three creative English datasets with distinct linguistic characteristics: (1) A corpus of 740 classical and contemporary English poems, (2) a corpus of 14950 metaphor sentences retrieved from a metaphor database website and (3) a corpus of 1500 song lyrics ranging across genres.  We first pre-train our generator on the Gutenberg dataset BIBREF24 for 20 epochs and then fine-tune BIBREF19 them to our target datasets with a language modeling objective.
Output: Which datasets are used?
We proposed joint learning NLP models that use convolutional neural network (CNN) BIBREF8 and bi-directional long short-term memory (BiLSTM) BIBREF9, BIBREF10 as basic units. Our models can automatically extract the key elements from the sexual harassment stories and at the same time categorize the stories in different dimensions. The proposed models outperformed the single task models, and achieved higher than previously reported accuracy in classifications of harassment forms BIBREF6.
Output: What model did they use?
In total around 500 different workers were involved in the annotation. about 500
Output: How many people participated in their evaluation study of table-to-text models?
Table TABREF31 shows the F-score, the Area Under the ROC Curve (AUC) and the mean accuracy (ACC) of the proposed models when different inputs are available. Fig. FIGREF32 shows the Precision vs Recall plot and the ROC curve (which plots the True Positive Rate vs the False Positive Rate) of the different models.
Output: What metrics are used to benchmark the results?
Tourist: I can't go straight any further.

Guide: ok. turn so that the theater is on your right.

Guide: then go straight

Tourist: That would be going back the way I came

Guide: yeah. I was looking at the wrong bank

Tourist: I'll notify when I am back at the brooks brothers, and the bank.

Tourist: ACTION:TURNRIGHT

Guide: make a right when the bank is on your left

Tourist: ACTION:FORWARD ACTION:FORWARD ACTION:TURNRIGHT

Tourist: Making the right at the bank.

Tourist: ACTION:FORWARD ACTION:FORWARD

Tourist: I can't go that way.

Tourist: ACTION:TURNLEFT

Tourist: Bank is ahead of me on the right

Tourist: ACTION:FORWARD ACTION:FORWARD ACTION:TURNLEFT

Guide: turn around on that intersection

Tourist: I can only go to the left or back the way I just came.

Tourist: ACTION:TURNLEFT

Guide: you're in the right place. do you see shops on the corners?

Guide: If you're on the corner with the bank, cross the street

Tourist: I'm back where I started by the shop and the bank.

Tourist: ACTION:TURNRIGHT
Output: What language do the agents talk in?
Benchmark Evaluation ::: Classifier Models
SVM: A linear support vector machine with bag-of-words sentence representations.

MLP: A multi-layer perceptron with USE embeddings BIBREF4 as input.

FastText: A shallow neural network that averages embeddings of n-grams BIBREF5.

CNN: A convolutional neural network with non-static word embeddings initialized with GloVe BIBREF6.

BERT: A neural network that is trained to predict elided words in text and then fine-tuned on our data BIBREF1.

Platforms: Several platforms exist for the development of task-oriented agents. We consider Google's DialogFlow and Rasa NLU with spacy-sklearn.
Output: Which classifiers are evaluated?
Hypothesis 4 (H4) : The proposed template-based synthesis model outperforms a simple word replacement model. Table TABREF13 supports H4 by showing that the proposed synthesis model outperforms the WordNet baseline in training (rows 7, 8 and 19, 20) except Stat2016, and tuning (10, 11 and 22, 23) over all courses. It also shows that while adding synthetic data from the baseline is not always helpful, adding synthetic data from the template model helps to improve both the training and the tuning process. In both CS and ENGR courses, tuning with synthetic data enhances all ROUGE scores compared to tuning with only the original data. (rows 9 and 11). As for Stat2015, R-1 and R-$L$ improved, while R-2 decreased. For Stat2016, R-2 and R-$L$ improved, and R-1 decreased (rows 21 and 23). Training with both student reflection data and synthetic data compared to training with only student reflection data yields similar improvements, supporting H3 (rows 6, 8 and 18, 20).
Output: Is the template-based model realistic?
Similar to BIBREF9, we experiment with SVM with RBF kernel, with features that represent (1) the simple characteristics of the argument tree and (2) the linguistic characteristics of the claim.
Output: What models that rely only on claim-specific linguistic features are used as baselines?
Looking at the current WinoGrande leaderboard, it appears that the previous state of the art is based on RoBERTa BIBREF2, which can be characterized as an encoder-only transformer architecture.
Output: What is the previous state of the art?
To learn better graph representation of multi-party dialogues, we adopt the dialogues with 8-15 utterances and 3-7 speakers. To simplify the task, we filter the dialogues with long sentences (more than 20 words). Finally, we obtain 52,053 dialogues and 460,358 utterances.
Output: How large is the proposed dataset?
In our experiments, we compared about 60 elementary metrics, which can be organised as follows:

MT metrics

BLEU, ROUGE, METEOR, TERp

Variants of BLEU: BLEU_1gram, BLEU_2gram, BLEU_3gram, BLEU_4gram and seven smoothing methods from NLTK BIBREF32 .

Intermediate components of TERp inspired by BIBREF18 : e.g. number of insertions, deletions, shifts...

Readability metrics and other sentence-level features: FKGL and FRE, numbers of words, characters, syllables...

Metrics based on the baseline QuEst features (17 features) BIBREF28 , such as statistics on the number of words, word lengths, language model probability and INLINEFORM0 -gram frequency.

Metrics based on other features: frequency table position, concreteness as extracted from BIBREF33 's BIBREF33 list, language model probability of words using a convolutional sequence to sequence model from BIBREF34 , comparison methods using pre-trained fastText word embeddings BIBREF35 or Skip-thought sentence embeddings BIBREF36 .
Output: what approaches are compared?
We ensemble BERT and E-BERT by (a) mean-pooling their outputs (AVG) or (b) concatenating the entity and its name with a slash symbol (CONCAT), e.g.: Jean_Marais / Jean Mara ##is.
Output: What are the two ways of ensembling BERT and E-BERT?
We also applied our model to MCTest dataset which requires machines to answer multiple-choice reading comprehension questions about fictional stories. 
Output: Do they experiment with their proposed model on any other dataset other than MovieQA?
In this work, we explore the applicability of our method to three popular architectures of this layer: the LSTM-based, the CNN-based, and the transformer-based. Table TABREF46 shows performance of our method with different sequence modeling architectures. From the table, we can first see that the LSTM-based architecture performed better than the CNN- and transformer- based architectures. 
Output: Which are the sequence model architectures this method can be transferred across?
The annotation scheme was developed to capture three aspects of classroom talk that are theorized in the literature as important to discussion quality and learning opportunities: argumentation (the process of systematically reasoning in support of an idea), specificity (the quality of belonging or relating uniquely to a particular subject), and knowledge domain (area of expertise represented in the content of the talk).
Output: how do they measure discussion quality?
Table TABREF32 shows the perplexity of the models on each of the test sets. As expected, each model performs better on the test set they have been trained on. When applied to a different test set, both see an increase in perplexity. However, the Leipzig model seems to have more trouble generalizing: its perplexity nearly doubles on the SwissCrawl test set and raises by twenty on the combined test set.
Output: How is language modelling evaluated?
Besides, we also use two of its adversarial sets, namely AddSent and AddOneSent BIBREF6 , to evaluate the robustness to noise of MRC models. The passages in the adversarial sets contain misleading sentences, which are aimed at distracting MRC models.  Specifically, we not only evaluate the performance of KAR on the development set and the test set, but also do this on the adversarial sets. 
Output: How do the authors examine whether a model is robust to noise or not?
From the description of music genres provided above emerges that there is a limited number of super-genres and derivation lines BIBREF19, BIBREF20, as shown in figure FIGREF1.

From a computational perspective, genres are classes and, although can be treated by machine learning algorithms, they do not include information about the relations between them. In order to formalize the relations between genres for computing purposes, we define a continuous genre scale from the most experimental and introverted super-genre to the most euphoric and inclusive one. We selected from Wikipedia the 77 genres that we mentioned in bold in the previous paragraph and asked to two independent raters to read the Wikipedia pages of the genres, listen to samples or artists of the genres (if they did not know already) and then annotate the following dimensions:
Output: How many genres did they collect from?
The dataset contains a total of 9710 passages, with an average of 6.24 sentences per passage, 16.16 words per sentence, and an average length of 86 words.
Output: What is the size of this dataset?
In order to represent individual sentences, we insert external [cls] tokens at the start of each sentence, and each [cls] symbol collects features for the sentence preceding it. We also use interval segment embeddings to distinguish multiple sentences within a document. This way, document representations are learned hierarchically where lower Transformer layers represent adjacent sentences, while higher layers, in combination with self-attention, represent multi-sentence discourse. Position embeddings in the original Bert model have a maximum length of 512; we overcome this limitation by adding more position embeddings that are initialized randomly and fine-tuned with other parameters in the encoder.
Output: What is novel about their document-level encoder?
Contributors record voice clips by reading from a bank of donated sentences.
Output: What domains are covered in the corpus?
We calculate the probability of a sentence with a long-short term memory (LSTM, hochreiter1997long) LM, i.e., a special type of RNN LM, which has been trained on a large corpus. We train our LSTM LMs on the English Gigaword corpus BIBREF15 , which consists of news data.
Output: what language models do they use?
The library design of Torch-Struct follows the distributions API used by both TensorFlow and PyTorch BIBREF29.
Output: Does API provide ability to connect to models written in some other deep learning framework?
For our classification experiments, we use the MH17 Twitter dataset introduced by BIBREF4, a dataset collected in order to study the flow of (dis)information about the MH17 plane crash on Twitter.
Output: What dataset is used for this study?
We collect $37,263$ concept-sets as the inputs, each of which contains three to five common concepts. These concept-sets are sampled from several large corpora of image/video captions, such that the concepts inside them are more likely to co-occur in natural scenes. The expected concept-sets in our task are supposed to be likely co-occur in natural, daily-life scenes . The concepts in images/videos captions, which usually describe scenes in our daily life, thus possess the desired property. We therefore collect a large amount of caption sentences from a variety of datasets, including VATEX BIBREF4, LSMDC BIBREF12, ActivityNet BIBREF13, and SNLI BIBREF15, forming 1,040,330 sentences in total.
Output: Where do the concept sets come from?
We recruited 102 pairs of participants from Amazon Mechanical Turk and randomly assigned speaker and listener roles
Output: Was this experiment done in a lab?
For each model, we examined word-level perplexity, R@3 in next-word prediction, latency (ms/q), and energy usage (mJ/q). To explore the perplexity–recall relationship, we collected individual perplexity and recall statistics for each sentence in the test set.
Output: What aspects have been compared between various language models?
The seed lexicon consists of positive and negative predicates. If the predicate of an extracted event is in the seed lexicon and does not involve complex phenomena like negation, we assign the corresponding polarity score ($+1$ for positive events and $-1$ for negative events) to the event. It is a 
Output: What is the seed lexicon?
Grammatical error correction (GEC) is a challenging task due to the variability of the type of errors and the syntactic and semantic dependencies of the errors on the surrounding context. Most of the grammatical error correction systems use classification and rule-based approaches for correcting specific error types. However, these systems use several linguistic cues as features. The standard linguistic analysis tools like part-of-speech (POS) taggers and parsers are often trained on well-formed text and perform poorly on ungrammatical text. This introduces further errors and limits the performance of rule-based and classification approaches to GEC. As a consequence, the phrase-based statistical machine translation (SMT) approach to GEC has gained popularity because of its ability to learn text transformations from erroneous text to correct text from error-corrected parallel corpora without any additional linguistic information.  We model our GEC system based on the phrase-based SMT approach. However, traditional phrase-based SMT systems treat words and phrases as discrete entities. We take advantage of continuous space representation by adding two neural network components that have been shown to improve SMT systems BIBREF3 , BIBREF4 . To train NNJM, we use the publicly available implementation, Neural Probabilistic Language Model (NPLM) BIBREF14 . The latest version of Moses can incorporate NNJM trained using NPLM as a feature while decoding. Similar to NNGLM, we use the parallel text used for training the translation model in order to train NNJM. We use a source context window size of 5 and a target context window size of 4. We select a source context vocabulary of 16,000 most frequent words from the source side. The target context vocabulary and output vocabulary is set to the 32,000 most frequent words. We use a single hidden layer to speed up training and decoding with an input embedding dimension of 192 and 512 hidden layer nodes. 
Output: Do they use pretrained word representations in their neural network models?
We perform experiments on document-level variants of the SQuAD dataset BIBREF1 .
Output: What datasets have this method been evaluated on?
The corpus of supervisor assessment has 26972 sentences. 
Output: What is the size of the real-life dataset?
In phase 1 (also referred to as the “exploration” phase) the algorithm explores the state space through keeping track of previously visited states by maintaining an archive. During this phase, instead of resuming the exploration from scratch, the algorithm starts exploring from promising states in the archive to find high performing trajectories.
Output: How is trajectory with how rewards extracted?
We use the publicly available dataset KVRET BIBREF5 in our experiments.  This dataset includes dialogues in 3 domains: calendar, weather, navigation (POI) which is suitable for our mix-domain dialogue experiments. 
Output: Which domains did they explored?
In this section, we consider two kinds of relational classification tasks: (1) relation prediction and (2) relation extraction.  We hope to design a simple and clear experiment setup to conduct error analysis for relational prediction. Therefore, we consider a typical method TransE BIBREF3 as the subject as well as FB15K BIBREF3 as the dataset. For relation extraction, we consider the supervised relation extraction setting and TACRED dataset BIBREF10 . As for the subject model, we use the best model on TACRED dataset — position-aware neural sequence model. 
Output: Which competitive relational classification models do they test?
Each tweet was tokenized using NLTK TweetTokenizer and classified as one of 10 potential accounts from which it may have originated. The accounts were chosen based on the distinct topics each is known to typically tweet about.
Output: What text classification task is considered?
Our experiments are performed on an actual endangered language, Mboshi (Bantu C25), a language spoken in Congo-Brazzaville, using the bilingual French-Mboshi 5K corpus of BIBREF17. On the Mboshi side, we consider alphabetic representation with no tonal information. On the French side,we simply consider the default segmentation into words.
Output: What is the dataset used in the paper?
This model uses a particular RNN cell in order to store just relevant information about the given question.  Dynamic Memory stores information of entities present in $T$ . The addition of the $s_t^T q$ term in the gating function is our main contribution. 
Output: How does the model recognize entities and their relation to answers at inference time when answers are not accessible?
In test batch 4, our system (called FACTOIDS) achieved highest recall score of ‘0.7033’ but low precision of 0.1119, leaving open the question of how could we have better balanced the two measures.
Output: What was their highest recall score?
We use the additional INLINEFORM0 training articles labeled by publisher as an unsupervised data set to further train the BERT model. We first investigate the impact of pre-training on BERT-BASE's performance.  On the same computer, fine tuning the model on the small training set took only about 35 minutes for sequence length 100. 
Output: How are the two different models trained?
To evaluate our sampling strategy, we compare it with the pretrained model and fine-tuned model on 16 different corpora. The results show that our approach outperforms those baselines on all corpora: it achieves 1.6% lower phone error rate on average.
Output: By how much do they, on average, outperform the baseline multilingual model on 16 low-resource tasks?
We compare our approach with several methods BIBREF1 , BIBREF31 , BIBREF11 , BIBREF8 , BIBREF10 , BIBREF39 in two cross-domain settings. Using string kernels, Giménez-Pérez et al. BIBREF10 reported better performance than SST BIBREF31 and KE-Meta BIBREF11 in the multi-source domain setting. In addition, we compare our approach with SFA BIBREF1 , CORAL BIBREF8 and TR-TrAdaBoost BIBREF39 in the single-source setting. Transductive string kernels. We present a simple and straightforward approach to produce a transductive similarity measure suitable for strings. Our transductive kernel classifier (TKC) approach is composed of two learning iterations. 
Output: What machine learning algorithms are used?
The first, referred to here as seq2seq, is the standard RNN-based neural machine translation system with attention BIBREF0 . This baseline does not use the parsed data. The multi-source systems improve strongly over both baselines, with improvements of up to 1.5 BLEU over the seq2seq baseline and up to 1.1 BLEU over the parse2seq baseline.
Output: Whas is the performance drop of their model when there is no parsed input?
We first evaluate our converted embedding models on multi-language lexical similarity and relatedness tasks, as a sanity check, to make sure the word sense induction process did not hurt the general performance of the embeddings. Then, we test the sense embeddings on WSD task.
Output: Was any extrinsic evaluation carried out?
One of the several formats into which FHIR can be serialized is RDF. However, because RDF was designed as an abstract information model and FHIR was designed for operational use in a healthcare setting, there is the potential for a slight mismatch between the models. This comes up in two ways: One, RDF makes statements of fact, whereas FHIR makes records of events. Two, RDF is intended to have the property of monotonicity, meaning that previous facts cannot be invalidated by new facts.
Output: How are FHIR and RDF combined?
The Base is trained only on clean data while Base+Noise is trained on both the clean and the noisy data without noise handling. Global-CM uses a global confusion matrix for all noisy instances to model the noise as proposed by BIBREF3 and presented in Section SECREF3. The same architecture is used for Global-ID-CM, but the confusion matrix is initialized with the identity matrix (instead of Formula DISPLAY_FORM5) and only adapted during training.

The cluster-based models we propose in Section SECREF4 are Brown-CM and K-Means-CM. We experimented with numbers of clusters of 5, 10, 25 and 50. The models that select only the largest groups $G$ are marked as *-Freq and select either 30% or 50% of the clusters. The interpolation models have the postfix *-IP with $\lambda \in \lbrace 0.3, 0.5, 0.7\rbrace $ . The combination of both is named *-Freq-IP. As for all other hyperparameters, the choice was taken on the development set.

We implemented the Cleaning BIBREF15 and Dynamic-CM BIBREF14 models. Both were not developed for sequence labeling tasks and therefore needed to be adapted. For the Cleaning model, we followed the instructions by BIBREF3. The embedding and prediction components of the Dynamic-CM model were replaced according to our base model. The output of the dense layer was used as input to the dynamic matrix generation. We experimented with and without their proposed trace loss.
Output: What is baseline used?
Recurrent Neural Network (RNN) Language Models
are trained to output the probability distribution of the upcoming word given a context, without explicitly representing the structure of the context BIBREF9, BIBREF10. ActionLSTM
models the linearized bracketed tree structure of a sentence by learning to predict the next action required to construct a phrase-structure parse BIBREF16. Generative Recurrent Neural Network Grammars (RNNG)
jointly model the word sequence as well as the underlying syntactic structure BIBREF18.
Output: What are the baseline models?
We evaluate our method on IWSLT16 German-English (DE-EN) translation in both directions, WMT15 English-German (EN-DE) translation in both directions, and NIST Chinese-to-English (ZH$\rightarrow $EN) translation. 
Output: What corpora is used?
Without ELMo (the same setting as 4-th row in Table 3 ), our data settings is the same as qin-EtAl:2017:Long whose performance was state-of-the-art and will be compared directly. We see that even without the pre-trained ELMo encoder, our performance is better, which is mostly attributed to our better sentence pair representations.
Output: Why does their model do better than prior models?
Automation of cQA forums can be divided into three tasks: question-comment relevance (Task A), question-question relevance (Task B), and question-external comment relevance (Task C). In our cQA tasks, the pair of objects are (question, question) or (question, comment), and the relationship is relevant/irrelevant. For task C, in addition to an original question (oriQ) and an external comment (relC), the question which relC commented on is also given (relQ). To incorporate this extra information, we consider a multitask learning framework which jointly learns to predict the relationships of the three pairs (oriQ/relQ, oriQ/relC, relQ/relC).
Output: What supplemental tasks are used for multitask learning?
In order to validate the proposed classification model in real crowdsourcing settings, Amazon Mechanical Turk (AMT) was used to obtain labels from multiple annotators for two popular datasets: Reuters-21578 BIBREF30 and LabelMe BIBREF31 . In order to first validate the proposed model for classification problems in a slightly more controlled environment, the well-known 20-Newsgroups benchmark corpus BIBREF29 was used by simulating multiple annotators with different levels of expertise. 
Output: what datasets were used?
These models define parametrized similarity scoring functions $: Q\times T\rightarrow \mathbb {R}$ , where $Q$ is the set of natural language questions and $T$ is the set of paraphrases of logical forms.
Output: Does a neural scoring function take both the question and the logical form as inputs?
First, many news articles begin with reporter names, media agencies, dates or other contents irrelevant to the content, e.g. “New York (CNN) –”, “Jones Smith, May 10th, 2018:”. We therefore apply simple regular expressions to remove these prefixes.

Second, to ensure that the summary is concise and the article contains enough salient information, we only keep articles with 10-150 words in the top three sentences and 150-1200 words in the rest, and that contain at least 6 sentences in total. In this way, we filter out i) articles with excessively long content to reduce memory consumption; ii) very short leading sentences with little information which are unlikely to be a good summary. To encourage the model to generate abstrative summaries, we also remove articles where any of the top three sentences is exactly repeated in the rest of the article.

Third, we try to remove articles whose top three sentences may not form a relevant summary.
Output: What does the data cleaning and filtering process consist of?