niryuu/sni-each-converted huggingface dataset:
The datasets used for training, validation and testing contain sentences extracted from the Europarl corpus BIBREF1 and SoNaR corpus BIBREF2. The Europarl corpus is an open-source parallel corpus containing proceedings of the European Parliament. The Dutch section consists of 2,333,816 sentences and 53,487,257 words. The SoNaR corpus comprises two corpora: SONAR500 and SONAR1. The SONAR500 corpus consists of more than 500 million words obtained from different domains. 
 Question: What are the sizes of both datasets?
Output: The Dutch section consists of 2,333,816 sentences and 53,487,257 words. The SONAR500 corpus consists of more than 500 million words obtained from different domains.
This decoding approach closely follows Algorithm SECREF7 , but along with soft back pointers, we also compute hard back pointers at each time step. After computing all the relevant quantities like model score, loss etc., we follow the hard backpointers to obtain the best sequence INLINEFORM0 . 
 Question: Do they compare partially complete sequences (created during steps of beam search) to gold/target sequences?
Output: Yes
We used fastText and SVM BIBREF16 for preliminary experiments.  
 Question: What classification models were used?
Output: fastText and SVM BIBREF16
For English all-words WSD, we train our WSD model on SemCor BIBREF24, and test it on Senseval-2 (SE2), Senseval-3 (SE3), SemEval 2013 task 12 (SE13), and SemEval 2015 task 13 (SE15). We use OntoNotes Release 5.0, which contains a number of annotations including word senses for Chinese. 
 Question: What datasets are used for testing?
Output: Senseval-2 (SE2), Senseval-3 (SE3), SemEval 2013 task 12 (SE13), and SemEval 2015 task 13 (SE15) OntoNotes Release 5.0
We used Amazon Mechanical Turk (AMT) for our annotations, restricting the task to workers in five English-speaking countries (USA, UK, Canada, New Zealand, and Australia), more than 1000 finished HITs and at least a 95% acceptance rate. 
 Question: What crowdsourcing platform did they use?
Output: Amazon Mechanical Turk (AMT)
This work analyzes the understanding of pretrained language models of factual and commonsense knowledge stored in negated statements. To this end, we introduce the negated LAMA dataset. We construct it by simply inserting negation elements (e.g., “not”) in LAMA cloze statement (e.g., “The theory of relativity was not developed by [MASK].”). 
 Question: How did they extend LAMA evaluation framework to focus on negation?
Output: To this end, we introduce the negated LAMA dataset. We construct it by simply inserting negation elements (e.g., “not”) in LAMA cloze statement
The following data sources were used to train the RNN-T and associated RNN-LMs in this study.

Source-domain baseline RNN-T: approximately 120M segmented utterances (190,000 hours of audio) from YouTube videos, with associated transcripts obtained from semi-supervised caption filtering BIBREF28.

Source-domain normalizing RNN-LM: transcripts from the same 120M utterance YouTube training set. This corresponds to about 3B tokens of the sub-word units used (see below, Section SECREF30).

Target-domain RNN-LM: 21M text-only utterance-level transcripts from anonymized, manually transcribed audio data, representative of data from a Voice Search service. This corresponds to about 275M sub-word tokens.

Target-domain RNN-T fine-tuning data: 10K, 100K, 1M and 21M utterance-level {audio, transcript} pairs taken from anonymized, transcribed Voice Search data. These fine-tuning sets roughly correspond to 10 hours, 100 hours, 1000 hours and 21,000 hours of audio, respectively. 
 Question: How is the training data collected?
Output: from YouTube videos, with associated transcripts obtained from semi-supervised caption filtering from a Voice Search service
The problem with morpheme segmentation is that the vocabulary of stem units is still very large, which leads to many rare and unknown words at the training time.  Therefore, we learn a BPE model on the stem units in the training corpus rather than the words, and then apply it on the stem unit of each word after morpheme segmentation. 
 Question: How is morphology knowledge implemented in the method?
Output: A BPE model is applied to the stem after morpheme segmentation.
As shown in Figure FIGREF7 , KAR is an end-to-end MRC model consisting of five layers:

Lexicon Embedding Layer. This layer maps the words to the lexicon embeddings. Context Embedding Layer. This layer maps the lexicon embeddings to the context embeddings. Coarse Memory Layer. This layer maps the context embeddings to the coarse memories. Refined Memory Layer. This layer maps the coarse memories to the refined memories. Answer Span Prediction Layer. This layer predicts the answer start position and the answer end position based on the above layers. 
 Question: What type of model is KAR?
Output: Lexicon Embedding Layer Context Embedding Layer Coarse Memory Layer Refined Memory Layer Answer Span Prediction Layer
We employ the standard ROUGE-1, ROUGE-2 and ROUGE-L metrics BIBREF29 to evaluate all summarization models. Following BIBREF22, BIBREF23, we use F-measure ROUGE on XSUM and CNN/DailyMail, and use limited-length recall-measure ROUGE on NYT and DUC. 
 Question: What metric was used in the evaluation step?
Output: ROUGE-1, ROUGE-2 and ROUGE-L F-measure ROUGE on XSUM and CNN/DailyMail, and use limited-length recall-measure ROUGE on NYT and DUC
Our baseline is a three-layer LSTM-LM with 1,150 hidden units at internal layers trained with the standard cross-entropy loss.  
 Question: What neural language models are explored?
Output: LSTM-LM
$\mathit {\texttt {+}PPMI}$ only falters on the RW and analogy tasks, and we hypothesize this is where $\mathit {\texttt {-}PMI}$ is useful: in the absence of positive information, negative information can be used to improve rare word representations and word analogies. 
 Question: What are the disadvantages to clipping negative PMI?
Output: It may lead to poor rare word representations and word analogies.
We work with four individual datasets. The datasets contain addition, subtraction, multiplication, and division word problems.

AI2 BIBREF2. AI2 is a collection of 395 addition and subtraction problems, containing numeric values, where some may not be relevant to the question.

CC BIBREF19. The Common Core dataset contains 600 2-step questions. The Cognitive Computation Group at the University of Pennsylvania gathered these questions.

IL BIBREF4. The Illinois dataset contains 562 1-step algebra word questions. The Cognitive Computation Group compiled these questions also.

MAWPS BIBREF20. MAWPS is a relatively large collection, primarily from other MWP datasets. We use 2,373 of 3,915 MWPs from this set. 
 Question: What datasets do they use?
Output: AI2 BIBREF2 CC BIBREF19 IL BIBREF4 MAWPS BIBREF20
The only remaining question is what makes two environments similar enough to infer the existence of a common category. There is, again, a large literature on this question (including the aforementioned language modeling, unsupervised parsing, and alignment work), but in the current work we will make use of a very simple criterion: fragments are interchangeable if they occur in at least one lexical environment that is exactly the same. 
 Question: How do they determine similar environments for fragments in their data augmentation scheme?
Output: fragments are interchangeable if they occur in at least one lexical environment that is exactly the same
The third person plural pronoun `they' has no gender in English and most other languages (unlike the third person singular pronouns `he', `she', and `it').  If these sentences are translated into French, then `they' in the first sentence should be translated `elles', as referring to Jane and Susan, and `they' in the second sentence should be translated `ils', as referring to Fred and George. A couple of examples: The word `sie' in German serves as both the formal second person prounoun (always capitalized), the third person feminine singular, and the third person plural.  
 Question: What language do they explore?
Output: English French German
We use two popular knowledge bases: WordNet BIBREF0 and Freebase BIBREF1 in this paper. Specifically, we use WN18 (a subset of WordNet) BIBREF24 and FB15K (a subset of Freebase) BIBREF2 since their text descriptions are easily publicly available. Table 1 lists statistics of the two datasets. 
 Question: What datasets are used to evaluate this paper?
Output: WordNet BIBREF0 Freebase BIBREF1 WN18 (a subset of WordNet) BIBREF24  FB15K (a subset of Freebase) BIBREF2
We found that properties of the emerging lexicon related to the combinatoriality, namely the words length distribution, the frequency of use of the different forms and a measure for the combinatoriality itself, reflect both qualitatively and quantitatively the corresponding properties as measured in human languages, provided that the memory parameter $\tau $ is sufficiently high, that is that a sufficiently high effort is required in order to understand and learn brand new forms. 
 Question: What empirical data are the Blending Game predictions compared to?
Output: words length distribution, the frequency of use of the different forms and a measure for the combinatoriality
A baseline model for automatic extraction of anglicisms was created using the annotated corpus we just presented as training material. As mentioned in Section 3, the task of detecting anglicisms can be approached as a sequence labeling problem where only certain spans of texts will be labeled as anglicism (in a similar way to an NER task). The chosen model was conditional random field model (CRF), which was also the most popular model in both Shared Tasks on Language Identification for Code-Switched Data BIBREF23, BIBREF24. 
 Question: Does the paper motivate the use of CRF as the baseline model?
Output: the task of detecting anglicisms can be approached as a sequence labeling problem where only certain spans of texts will be labeled as anglicism (in a similar way to an NER task). The chosen model was conditional random field model (CRF), which was also the most popular model in both Shared Tasks on Language Identification for Code-Switched Data
Corpus Embedding
Suppose that INLINEFORM0 is the target low-resource corpus, we are interested in optimizing the acoustic model with a much larger training corpora set INLINEFORM1 where INLINEFORM2 is the number of corpora and INLINEFORM3 . Each corpus INLINEFORM4 is a collection of INLINEFORM5 pairs where INLINEFORM6 is the input features and INLINEFORM7 is its target.

Our purpose here is to compute the embedding INLINEFORM0 for each corpus INLINEFORM1 where INLINEFORM2 is expected to encode information about its corpus INLINEFORM3 . Those embeddings can be jointly trained with the standard multilingual model BIBREF4 . First, the embedding matrix INLINEFORM4 for all corpora is initialized, the INLINEFORM5 -th row of INLINEFORM6 is corresponding to the embedding INLINEFORM7 of the corpus INLINEFORM8 . Next, during the training phase, INLINEFORM9 can be used to bias the input feature INLINEFORM10 as follows. DISPLAYFORM0

where INLINEFORM0 is an utterance sampled randomly from INLINEFORM1 , INLINEFORM2 is its hidden features, INLINEFORM3 is the parameter of the acoustic model and Encoder is the stacked bidirectional LSTM as shown in Figure. FIGREF5 . Next, we apply the language specific softmax to compute logits INLINEFORM4 and optimize them with the CTC objective BIBREF29 . The embedding matrix INLINEFORM5 can be optimized together with the model during the training process. 
 Question: How do they compute corpus-level embeddings?
Output: First, the embedding matrix INLINEFORM4 for all corpora is initialized during the training phase, INLINEFORM9 can be used to bias the input feature Next, we apply the language specific softmax to compute logits INLINEFORM4 and optimize them with the CTC objective
As a first corpus, we compiled INLINEFORM0 that represents a collection of 80 excerpts from scientific works including papers, dissertations, book chapters and technical reports, which we have chosen from the well-known Digital Bibliography & Library Project (DBLP) platform. As a second corpus, we compiled INLINEFORM0 , which represents a collection of 1,645 chat conversations of 550 sex offenders crawled from the Perverted-Justice portal. As a third corpus, we compiled INLINEFORM0 , which is a collection of 200 aggregated postings crawled from the Reddit platform. 
 Question: What size are the corpora?
Output: 80 excerpts from scientific works collection of 1,645 chat conversations collection of 200 aggregated postings
1. BERT based fine-tuning: In the first approach, which is shown in Figure FIGREF8, very few changes are applied to the BERTbase. In this architecture, only the [CLS] token output provided by BERT is used. The [CLS] output, which is equivalent to the [CLS] token output of the 12th transformer encoder, a vector of size 768, is given as input to a fully connected network without hidden layer. The softmax activation function is applied to the hidden layer to classify.

2. Insert nonlinear layers: Here, the first architecture is upgraded and an architecture with a more robust classifier is provided in which instead of using a fully connected network without hidden layer, a fully connected network with two hidden layers in size 768 is used. The first two layers use the Leaky Relu activation function with negative slope = 0.01, but the final layer, as the first architecture, uses softmax activation function as shown in Figure FIGREF8.

3. Insert Bi-LSTM layer: Unlike previous architectures that only use [CLS] as the input for the classifier, in this architecture all outputs of the latest transformer encoder are used in such a way that they are given as inputs to a bidirectional recurrent neural network (Bi-LSTM) as shown in Figure FIGREF8. After processing the input, the network sends the final hidden state to a fully connected network that performs classification using the softmax activation function.

4. Insert CNN layer: In this architecture shown in Figure FIGREF8, the outputs of all transformer encoders are used instead of using the output of the latest transformer encoder. So that the output vectors of each transformer encoder are concatenated, and a matrix is produced. The convolutional operation is performed with a window of size (3, hidden size of BERT which is 768 in BERTbase model) and the maximum value is generated for each transformer encoder by applying max pooling on the convolution output. By concatenating these values, a vector is generated which is given as input to a fully connected network. By applying softmax on the input, the classification operation is performed. 
 Question: What new fine-tuning methods are presented?
Output: BERT based fine-tuning Insert nonlinear layers Insert Bi-LSTM layer Insert CNN layer
1) Typical Methods. Three typical knowledge graph embedding methods includes TransE, TransR and TransH are selected as baselines. 2) Path-based Methods. We compare our method with two typical path-based model include PTransE, and ALL-PATHS BIBREF18. 3) Attribute-incorporated Methods. Several state-of-art attribute-incorporated methods including R-GCN BIBREF24 and KR-EAR BIBREF26 are used to compare with our methods on three real datasets. 
 Question: What seven state-of-the-art methods are used for comparison?
Output: TransE, TransR and TransH PTransE, and ALL-PATHS R-GCN BIBREF24 and KR-EAR BIBREF26
Currently, the following WSD models induced from a text corpus are available: Word senses based on cluster word features. This model uses the cluster words from the induced word sense inventory as sparse features that represent the sense.

Word senses based on context word features. This representation is based on a sum of word vectors of all cluster words in the induced sense inventory weighted by distributional similarity scores.

Super senses based on cluster word features. To build this model, induced word senses are first globally clustered using the Chinese Whispers graph clustering algorithm BIBREF9 . The edges in this sense graph are established by disambiguation of the related words BIBREF11 , BIBREF12 . The resulting clusters represent semantic classes grouping words sharing a common hypernym, e.g. “animal”. This set of semantic classes is used as an automatically learned inventory of super senses: There is only one global sense inventory shared among all words in contrast to the two previous traditional “per word” models. Each semantic class is labeled with hypernyms. This model uses words belonging to the semantic class as features.

Super senses based on context word features. This model relies on the same semantic classes as the previous one but, instead, sense representations are obtained by averaging vectors of words sharing the same class. 
 Question: Do they use a neural model for their task?
Output: No
CoVoST is currently built on that release and covers the following 11 languages: French, German, Dutch, Russian, Spanish, Italian, Turkish, Persian, Swedish, Mongolian and Chinese. 
 Question: Is Arabic one of the 11 languages in CoVost?
Output: No
We also measure the usage of words related to people's core values as reported by Boyd et al. boyd2015. The sets of words, or themes, were excavated using the Meaning Extraction Method (MEM) BIBREF10 . 
 Question: How do they obtain psychological dimensions of people?
Output: using the Meaning Extraction Method
Table TABREF21 shows results for Korean INLINEFORM0 English, using the same configurations (1, 2 and 8) as for German–English. Our results confirm that the techniques we apply are successful across datasets, and result in stronger systems than previously reported on this dataset, achieving 10.37 BLEU as compared to 5.97 BLEU reported by gu-EtAl:2018:EMNLP1. 
 Question: what are the methods they compare with in the korean-english dataset?
Output: gu-EtAl:2018:EMNLP1
Slot Filling is an information extraction task which has become popular in the last years BIBREF3 . The task aims at extracting information about persons, organizations or geo-political entities from a large collection of news, web and discussion forum documents. 
 Question: What is the task of slot filling?
Output: The task aims at extracting information about persons, organizations or geo-political entities from a large collection of news, web and discussion forum documents.
The release of the FEVER fact extraction and verification dataset BIBREF0 provides a large-scale challenge that tests a combination of retrieval and textual entailment capabilities. To verify a claim in the dataset as supported, refuted, or undecided, a system must retrieve relevant articles and sentences from Wikipedia. Then it must decide whether each of those sentences, or some combination of them, entails or refutes the claim, which is an entailment problem. Systems are evaluated on the accuracy of the claim predictions, with credit only given when correct evidence is submitted. As entailment data, premises in FEVER data differ substantially from those in the image caption data used as the basis for the Stanford Natural Language Inference (SNLI) BIBREF1 dataset. Sentences are longer (31 compared to 14 words on average), vocabulary is more abstract, and the prevalence of named entities and out-of-vocabulary terms is higher. The retrieval aspect of FEVER is not straightforward either. A claim may have small word overlap with the relevant evidence, especially if the claim is refuted by the evidence. 
 Question: What is the FEVER task?
Output: tests a combination of retrieval and textual entailment capabilities. To verify a claim in the dataset as supported, refuted, or undecided, a system must retrieve relevant articles and sentences from Wikipedia. Then it must decide whether each of those sentences, or some combination of them, entails or refutes the claim, which is an entailment problem
Manual inspection of others examples also supports our claim. 
 Question: Do they perform manual evaluation?
Output: Yes
The first metric we report is the reaction type. The second metric we report is reaction speed. A study by Jin et al. jin2013epidemiological found that trusted news stories spread faster than misinformation or rumor; Zeng et al. zeng2016rumors found that tweets which deny rumors had shorter delays than tweets of support. Our second goal is to determine if these trends are maintained for various types of news sources on Twitter and Reddit. To examine whether users react to content from trusted sources differently than from deceptive sources, we measure the reaction delay, which we define as the time elapsed between the moment the link or content was posted/tweeted and the moment that the reaction comment or tweet occurred. We report the cumulative distribution functions (CDFs) for each source type and use Mann Whitney U (MWU) tests to compare whether users respond with a given reaction type with significantly different delays to news sources of different levels of credibility. 
 Question: How is speed measured?
Output: time elapsed between the moment the link or content was posted/tweeted and the moment that the reaction comment or tweet occurred
We also perform a fine-grained syntactic analysis that shows how our method enables to control the morphological realization of first and second-person pronouns, together with verbs and adjectives related to them. 
 Question: What conclusions are drawn from the syntactic analysis?
Output:  our method enables to control the morphological realization of first and second-person pronouns, together with verbs and adjectives related to them
We use 16 different datasets from several popular review corpora used in BIBREF20 . These datasets consist of 14 product review datasets and two movie review datasets. We use CoNLL 2000 BIBREF22 sequence labeling dataset for both POS Tagging and Chunking tasks. 
 Question: What dataset did they use?
Output: 16 different datasets from several popular review corpora used in BIBREF20 CoNLL 2000 BIBREF22
We use a default train-validation-test split of 70-15-15 for each dataset, and use all 4 datasets (BF, BA, SFU and Sherlock). 
 Question: Which multiple datasets did they train on during joint training?
Output: BF, BA, SFU and Sherlock
Methodology ::: Corpus-based Approach
Contextual information is informative in the sense that, in general, similar words tend to appear in the same contexts.  In the corpus-based approach, we capture both of these characteristics and generate word embeddings specific to a domain. Methodology ::: Dictionary-based Approach
In Turkish, there do not exist well-established sentiment lexicons as in English. In this approach, we made use of the TDK (Türk Dil Kurumu - “Turkish Language Institution”) dictionary to obtain word polarities. 
 Question: What word-based and dictionary-based feature are used?
Output: generate word embeddings specific to a domain TDK (Türk Dil Kurumu - “Turkish Language Institution”) dictionary to obtain word polarities
We investigate the following six idiom resources:

Wiktionary;

the Oxford Dictionary of English Idioms (ODEI, BIBREF31);

UsingEnglish.com (UE);

the Sporleder corpus BIBREF10;

the VNC dataset BIBREF9;

and the SemEval-2013 Task 5 dataset BIBREF15. 
 Question: What dictionaries are used for automatic extraction of PIEs?
Output: Wiktionary Oxford Dictionary of English Idioms UsingEnglish.com (UE) Sporleder corpus VNC dataset SemEval-2013 Task 5 dataset
The improved performance of our attention models that actively select their optimal context, over a model with the complete thread as context, hLSTM, shows that the context inference improves intervention prediction over using the default full context. 
 Question: What aspects of discussion are relevant to instructor intervention, according to the attention mechanism?
Output: context inference
In this paper, we work on recognizing the polarity of an affective event that is represented by a score ranging from $-1$ (negative) to 1 (positive). 
 Question: What are labels available in dataset for supervision?
Output: negative positive
Dataset: Based on some earlier work, only available labelled dataset had 3189 rows of text messages of average length of 116 words and with a range of 1, 1295. 
 Question: How big is the dataset?
Output: 3189 rows of text messages
The multi-task model outperforms the single-task model at all data sizes, and the relative performance increases as the size of the training data decreases. When only 200 sentences of training data are used, the performance of the multi-task model is about 60% better than the single-task model for both the Airbnb and Greyhound apps. The relative gain for the OpenTable app is 26%. The results differ between the tasks, but none have an overall benefit from the open vocabulary system. 
 Question: Does the performance increase using their method?
Output: The multi-task model outperforms the single-task model at all data sizes but none have an overall benefit from the open vocabulary system
Conclusion 
 Question: What is triangulation?
Output: Answer with content missing: (Chapter 3) The concept can be easily explained with an example, visualized in Figure 1. Consider the Portuguese (Pt) word trabalho which, according to the MUSE Pt–En dictionary, has the words job and work as possible En translations. In turn, these two En words can be translated to 4 and 5 Czech (Cs) words respectively. By utilizing the transitive property (which translation should exhibit) we can identify the set of 7 possible Cs translations for the Pt word trabalho.
All baselines are Transformer models in their base configuration BIBREF11, using 6 encoder and decoder layers, with model and hidden dimensions of 512 and 2048 respectively, and 8 heads for all attention layers. For initial multi-task experiments, all model parameters were shared BIBREF12, but performance was down by multiple BLEU points compared to the baselines. 
 Question: What baselines non-adaptive baselines are used?
Output: Transformer models in their base configuration BIBREF11, using 6 encoder and decoder layers, with model and hidden dimensions of 512 and 2048 respectively, and 8 heads for all attention layers
replacing this single GRU with two different components first component is a sentence reader second component is the input fusion layer 
 Question: What changes they did on input module?
Output: For the DMN+, we propose replacing this single GRU with two different components. The first component is a sentence reader The second component is the input fusion layer
In this section we discuss the state of the art on conversational systems in three perspectives: types of interactions, types of architecture, and types of context reasoning. ELIZA BIBREF11 was one of the first softwares created to understand natural language processing.  Right after ELIZA came PARRY, developed by Kenneth Colby, who is psychiatrist at Stanford University in the early 1970s. A.L.I.C.E. (Artificial Linguistic Internet Computer Entity) BIBREF12 appeared in 1995 but current version utilizes AIML, an XML language designed for creating stimulus-response chat robots BIBREF13 . Cleverbot (1997-2014) is a chatbot developed by the British AI scientist Rollo Carpenter.  
 Question: What is the state of the art described in the paper?
Output: ELIZA  PARRY A.L.I.C.E. Cleverbot
The results in Table TABREF38 confirm the results of BIBREF13 and suggest that we successfully replicated a large proportion of their features. The results for all three prediction settings (one outgoing edge: INLINEFORM0 , support/attack: INLINEFORM1 and support/attack/neither: INLINEFORM2 ) across all type variables ( INLINEFORM3 , INLINEFORM4 and INLINEFORM5 ) are displayed in Table TABREF39 . All models significantly outperform the majority baseline with respect to macro F1. 
 Question: What baseline and classification systems are used in experiments?
Output: BIBREF13 majority baseline
Following the previous works, we perform 10-fold cross validation and report the average results.  
 Question: How is the robustness of the model evaluated?
Output: 10-fold cross validation
All documents are segmented into paragraphs and processed at the paragraph level (both training and inference); this is acceptable because we observe that most paragraphs are less than 200 characters. The input sequences are segmented by the BERT tokenizer, with the special [CLS] token inserted at the beginning and the special [SEP] token added at the end. 
 Question: At what text unit/level were documents processed?
Output: documents are segmented into paragraphs and processed at the paragraph level
On high level, we indirectly evaluate the content relevance between the candidate summary and the human summary using information retrieval. To accomplish this, we use the summaries as search queries and compare the overlaps of the retrieved results.  
 Question: In the proposed metric, how is content relevance measured?
Output: The content relevance between the candidate summary and the human summary is evaluated using information retrieval - using the summaries as search queries and compare the overlaps of the retrieved results.
We investigate both methods, either in isolation or combined, on two translation directions (En-It and En-De) for which the length of the target is on average longer than the length of the source. En-It, En-De in both directions 
 Question: Which languages do they focus on?
Output: two translation directions (En-It and En-De)
We further combine the supervised IR with our unsupervised model, which is trained with full unpaired data (4.8M) and different number of paired data (from 50K to 4.8M). 
 Question: How many comments were used?
Output: from 50K to 4.8M
Various automated evaluation approaches are proposed to facilitate the development and evaluation of NLG models. We summarize these evaluation approaches below. Text Overlap Metrics, including BLEU BIBREF5, METEOR BIBREF6 and ROUGE BIBREF7, are the most popular metrics employed in the evaluation of NLG models. Perplexity is commonly used to evaluate the quality of a language model. Parameterized Metrics learn a parameterized model to evaluate generated text. 
 Question: What previous automated evalution approaches authors mention?
Output: Text Overlap Metrics, including BLEU Perplexity Parameterized Metrics
In Table TABREF15 we measure BLEU BIBREF19, NIST BIBREF20, METEOR BIBREF21, ROUGE-L BIBREF22 and CIDEr BIBREF23 metrics on the 2018 E2E NLG Challenge test data using the evaluation script provided by the organizers.  We use the gold standard selection during training and validation of the text generation model, as well as the automatic evaluation. As we deploy our text generation model for manual evaluation, we use a Conditional Random Field (CRF) model to predict which events to mention. The second human evaluation aimed at judging the acceptability of the output for production use in a news agency.  In the minimum edit evaluation, carried out by the annotator who created the news corpus, only factual mistakes and grammatical errors are corrected, resulting in text which may remain awkward or unfluent. The word error rate (WER) of the generated text compared to its corrected variant as a reference is 5.6% (6.2% disregarding punctuation).  The factual errors and their types are summarized in Table TABREF23. From the total of 510 game events generated by the system, 78 of these contained a factual error, i.e. 84.7% were generated without factual errors. Most fluency issues relate to the overall flow and structure of the report. Addressing these issues would require the model to take into account multiple events in a game, and combine the information more flexibly to avoid repetition.  The second human evaluation aimed at judging the acceptability of the output for production use in a news agency. The output is evaluated in terms of its usability for a news channel labelled as being machine-generated, i.e. not aiming at the level of a human journalist equipped with substantial background information. The evaluation was carried out by two journalists from the STT agency, who split the 59 games among themselves approximately evenly. 
 Question: What evaluation criteria and metrics were used to evaluate the generated text?
Output: BLEU  NIST  METEOR  ROUGE-L CIDEr  evaluation script automatic evaluation human evaluation minimum edit evaluation word error rate (WER) factual errors and their types fluency issues acceptability of the output for production use in a news agency
We use MS COCO, Bing and Flickr datasets from BIBREF26 to train the model that generates questions. These datasets contain natural questions about images with the purpose of knowing more about the picture. As can be seen in the Figure FIGREF8, questions cannot be answered by only looking at the image. Each source contains 5,000 images with 5 questions per image, adding a total of 15,000 images with 75,000 questions. We use two datasets to train our chatbot model. The first one is the Persona-chat BIBREF15 which contains dialogues between two people with different profiles that are trying to know each other. It is complemented by the Cornell-movie dialogues dataset BIBREF27, which contains a collection of fictional conversations extracted from raw movie scripts. Persona-chat's sentences have a maximum of 15 words, making it easier to learn for machines and a total of 162,064 utterances over 10,907 dialogues. While Cornell-movie dataset contains 304,713 utterances over 220,579 conversational exchanges between 10,292 pairs of movie characters. 
 Question: How big dataset is used for training this system?
Output: For the question generation model 15,000 images with 75,000 questions. For the chatbot model, around 460k utterances over 230k dialogues.
Our experiments showed that using the objects color or shape to uniquely identify an object allows the robot successfully complete the binning task in 97.6% and 96.0% of the cases. However, using the shape alone as a unique identifier, the task could only be completed in 79.0% of the cases. 
 Question: What is task success rate achieved? 
Output: 96-97.6% using the objects color or shape and 79% using shape alone
As well as refining the annotation guidelines, the development process trained annotators who were not security experts. The development and test sets for Darkode and Hack Forums were annotated by additional team members (five for Darkode, one for Hack Forums), and then every disagreement was discussed and resolved to produce a final annotation. The authors, who are researchers in either NLP or computer security, did all of the annotation. 
 Question: Who annotated the data?
Output: annotators who were not security experts researchers in either NLP or computer security
In HotpotQA, on average we can find 6 candidate chains (2-hop) in a instance, and the human labeled true reasoning chain is unique. A predicted chain is correct if the chain only contains all supporting passages (exact match of passages).

In MedHop, on average we can find 30 candidate chains (3-hop). For each candidate chain our human annotators labeled whether it is correct or not, and the correct reasoning chain is not unique. A predicted chain is correct if it is one of the chains that human labeled as correct.

The accuracy is defined as the ratio: The accuracy is defined as the ratio: 
 Question: What benchmarks are created?
Output: Answer with content missing: (formula) The accuracy is defined as the ratio # of correct chains predicted to # of evaluation samples
In target data, the code is written in Python programming language. 
 Question: What programming language is target language?
Output: Python
Before designing this NLP toolkit, we conducted a survey among engineers and identified a spectrum of three typical personas. 
 Question: How do the authors evidence the claim that many engineers find it a big overhead to choose from multiple frameworks, models and optimization techniques?
Output: By conducting a survey among engineers
As discussed in Section SECREF2 , a variety of features that measure salience of an entity in text are available from the NLP community. We reimplemented the ones in Dunietz and Gillick BIBREF11 . This includes a variety of features, e.g. positional features, occurrence frequency and the internal POS structure of the entity and the sentence it occurs in.  
 Question: What features are used to represent the salience and relative authority of entities?
Output: Salience features positional features, occurrence frequency and the internal POS structure of the entity and the sentence it occurs in.
The relative authority of entity features:   comparative relevance of the news article to the different entities occurring in it.
Last but not least, ethics and fairness are important considerations, that deserve to be studied. In that sense, detection of individual and global bias should be prioritized in order to give useful feedbacks to practitioners. Furthermore we are considering using adversarial learning as in BIBREF33 in order to ensure fairness during the training process. 
 Question: Is there any ethical consideration in the research?
Output: No
For reliable multi-hop reasoning using OpenIE tuples, we can add inter-tuple connections to the support graph search, controlled by a small number of rules over the OpenIE predicates. Learning such rules for the Science domain is an open problem and potential avenue of future work. 
 Question: Can the method answer multi-hop questions?
Output: Yes
We hypothesized that question answering can benefit from synergistic interaction between the two tasks through parameter sharing and joint training under this multitask setting. 
 Question: Which components of QA and QG models are shared during training?
Output: parameter sharing
Querying posts on Twitter with extracted lexicons led to a collection of $19,300$ tweets. In order to have lexical diversity, we added 2500 randomly sampled tweets to our dataset. 
 Question: How many tweets were collected?
Output: $19,300$ added 2500 randomly sampled tweets
Three annotators, A1-A3, mark each tweet in the Dataset H as drunk or sober. 
 Question: Is the data acquired under distant supervision verified by humans at any stage?
Output: Yes
ch As continuous word embedding spaces exhibit similar structures across (even distant) languages BIBREF35, we use a multilingual word representation which aims to learn a linear mapping from a source to a target embedding space. 
 Question: What multilingual word representations are used?
Output:  a multilingual word representation which aims to learn a linear mapping from a source to a target embedding space
In this paper, we use three data sets from the literature to train and evaluate our own classifier. Data collected by BIBREF3 , which we term the Sexist/Racist (SR) data set, was collected using an initial Twitter search followed by analysis and filtering by the authors and their team who identified 17 common phrases, hashtags, and users that were indicative of abusive speech. BIBREF4 collected the HATE dataset by searching for tweets using a lexicon provided by Hatebase.org. The final data set we used, which we call HAR, was collected by BIBREF9 ; we removed all retweets reducing the dataset to 20,000 tweets. Tweets were labeled as “Harrassing” or “Non-Harrassing”; hate speech was not explicitly labeled, but treated as an unlabeled subset of the broader “Harrassing” category BIBREF9 . 
 Question: Which publicly available datasets are used?
Output: BIBREF3 BIBREF4 BIBREF9
However, no prior work has explored the target of the offensive language, which is important in many scenarios, e.g., when studying hate speech with respect to a specific target. 
 Question: What are the differences between this dataset and pre-existing ones?
Output: no prior work has explored the target of the offensive language
We compared several summarization methods which can be categorized into three groups: unsupervised, non-neural supervised, and neural supervised methods. For the unsupervised methods, we tested: SumBasic, which uses word frequency to rank sentences and selects top sentences as the summary BIBREF13 , BIBREF14 . Lsa, which uses latent semantic analysis (LSA) to decompose the term-by-sentence matrix of a document and extracts sentences based on the result. We experimented with the two approaches proposed in BIBREF15 and BIBREF16 respectively. LexRank, which constructs a graph representation of a document, where nodes are sentences and edges represent similarity between two sentences, and runs PageRank algorithm on that graph and extracts sentences based on the resulting PageRank values BIBREF17 . TextRank, which is very similar to LexRank but computes sentence similarity based on the number of common tokens BIBREF19 . Bayes, which represents each sentence as a feature vector and uses naive Bayes to classify them BIBREF5 . The original paper computes TF-IDF score on multi-word tokens that are identified automatically using mutual information. We did not do this identification, so our TF-IDF computation operates on word tokens. Hmm, which uses hidden Markov model where states correspond to whether the sentence should be extracted BIBREF20 . The original work uses QR decomposition for sentence selection but our implementation does not. We simply ranked the sentences by their scores and picked the top 3 as the summary. MaxEnt, which represents each sentence as a feature vector and leverages maximum entropy model to compute the probability of a sentence should be extracted BIBREF21 . The original approach puts a prior distribution over the labels but we put the prior on the weights instead. Our implementation still agrees with the original because we employed a bias feature which should be able to learn the prior label distribution. As for the neural supervised method, we evaluated NeuralSum BIBREF11 using the original implementation by the authors. We modified their implementation slightly to allow for evaluating the model with ROUGE. Note that all the methods are extractive. Our implementation code for all the methods above is available online. As a baseline, we used Lead-N which selects INLINEFORM0 leading sentences as the summary. For all methods, we extracted 3 sentences as the summary since it is the median number of sentences in the gold summaries that we found in our exploratory analysis. 
 Question: Which approaches did they use?
Output: SumBasic Lsa LexRank TextRank Bayes Hmm MaxEnt NeuralSum Lead-N
We use three encoder-decoder neural architectures as baselines: (1) LSTM+attention as an LSTM BIBREF19 with attention mechanism BIBREF20; (2) Transformer BIBREF21 and (3) Universal Transformer BIBREF22. 
 Question: What three machine architectures are analyzed?
Output: LSTM+attention Transformer  Universal Transformer
To capture this interesting property, we propose a new tagging scheme consisting of three tags, namely { INLINEFORM0 }.

INLINEFORM0 tag indicates that the current word appears before the pun in the given context.

INLINEFORM0 tag highlights the current word is a pun.

INLINEFORM0 tag indicates that the current word appears after the pun. 
 Question: What is the tagging scheme employed?
Output: A new tagging scheme that tags the words before and after the pun as well as the pun words.
We defined a sequence labeling task to extract custom entities from user input. We assumed seven (7) possible entities (see Table TABREF43) to be recognized by the model: topic, subtopic, examination mode and level, question number, intent, as well as the entity other for remaining words in the utterance.   We defined a classification problem to predict the system's next action according to the given user input. We assumed 13 custom actions (see Table TABREF42) that we considered being our labels. In the conversational dataset, each input was automatically labeled by the rule-based system with the corresponding next action and the dialogue-id. Thus, no additional post-labeling was required.  
 Question: How does the IPA label data after interacting with users?
Output: It defined a sequence labeling task to extract custom entities from user input and label the next action (out of 13  custom actions defined).
We train a single layer LSTM with a 150-dimensional hidden state for hate / not hate classification. The input dimensionality is set to 100 and GloVe BIBREF26 embeddings are used as word input representations. 
 Question: What unimodal detection models were used?
Output:  single layer LSTM with a 150-dimensional hidden state for hate / not hate classification
GE-FL reduces the heavy load of instance annotation and performs well when we provide prior knowledge with no bias. In our experiments, we observe that comparable numbers of labeled features for each class have to be supplied. We randomly select $t \in [1, 20]$ features from the feature pool for one class, and only one feature for the other. Our methods are also evaluated on datasets with different unbalanced class distributions. We manually construct several movie datasets with class distributions of 1:2, 1:3, 1:4 by randomly removing 50%, 67%, 75% positive documents. Incorporating KL divergence is robust enough to control unbalance both in the dataset and in labeled features while the other three methods are not so competitive. 
 Question: How do they define robustness of a model?
Output: ability to accurately classify texts even when the amount of prior knowledge for different classes is unbalanced, and when the class distribution of the dataset is unbalanced
Given that non-content words are distinctive enough for a classifier to determine style, we propose a suite of low-level linguistic feature counts (henceforth, controls) as our formal, content-blind definition of style. The style of a sentence is represented as a vector of counts of closed word classes (like personal pronouns) as well as counts of syntactic features like the number of SBAR non-terminals in its constituency parse, since clause structure has been shown to be indicative of style BIBREF20. Controls are extracted heuristically, and almost all rely on counts of pre-defined word lists. For constituency parses we use the Stanford Parser BIBREF21. table:controlexamples lists all the controls along with examples. 
 Question: How they model style as a suite of low-level linguistic controls, such as frequency of pronouns, prepositions, and subordinate clause constructions?
Output: style of a sentence is represented as a vector of counts of closed word classes (like personal pronouns) as well as counts of syntactic features like the number of SBAR non-terminals in its constituency parse, since clause structure has been shown to be indicative of style
The task of Word Sense Induction (WSI) can be seen as an unsupervised version of WSD. WSI aims at clustering word senses and does not require to map each cluster to a predefined sense. Instead of that, word sense inventories are induced automatically from the clusters, treating each cluster as a single sense of a word. We suggest a more advanced procedure of graph construction that uses the interpretability of vector addition and subtraction operations in word embedding space BIBREF6 while the previous algorithm only relies on the list of nearest neighbours in word embedding space. The key innovation of our algorithm is the use of vector subtraction to find pairs of most dissimilar graph nodes and construct the graph only from the nodes included in such “anti-edges”. 
 Question: Is the method described in this work a clustering-based method?
Output: Yes
The dataset from Twitter that we are using in our work, consists of a train set, a validation set and a test set. It was published for the "First workshop on categorizing different types of online harassment languages in social media". The whole dataset is divided into two categories, which are harassment and non-harassment tweets. Moreover, considering the type of the harassment, the tweets are divided into three sub-categories which are indirect harassment, sexual and physical harassment. 
 Question: What types of online harassment are studied?
Output: indirect harassment, sexual and physical harassment