niryuu/sni-each-converted huggingface dataset:
We thus also conducted human studies on Amazon MTurk to evaluate the generated responses with pairwise comparison for dialogue quality. We compare our models with an advanced decoding algorithm MMI BIBREF2 and two models, namely LSTM BIBREF0 and VHRED BIBREF7, both with additive attention. To our best knowledge, LSTM and VHRED were the primary models with which F1's were reported on the Ubuntu dataset. Following BIBREF5 (BIBREF5), we employ two criteria: Plausibility and Content Richness. The first criterion measures whether the response is plausible given the context, while the second gauges whether the response is diverse and informative.  
 Question: How is human evaluation performed, what was the criteria?
Output: Abstractive
Especially for Davidson-dataset, some tweets with specific language (written within the African American Vernacular English) and geographic restriction (United States of America) are oversampled such as tweets containing disparage words “nigga", “faggot", “coon", or “queer", result in high rates of misclassification. However, these misclassifications do not confirm the low performance of our classifier because annotators tended to annotate many samples containing disrespectful words as hate or offensive without any presumption about the social context of tweeters such as the speaker’s identity or dialect, whereas they were just offensive or even neither tweets.  
 Question: What biases does their model capture?
Output: Abstractive
We focus here on the Europarl domain, for which we have ample data in several languages, and use as in-domain training data the Europarl corpus BIBREF5 for two translation directions: English INLINEFORM0 German and English INLINEFORM1 French. 
 Question: what language pairs are explored?
Output: Abstractive
Our idea is the BQ generation for MQ and, at the same time, we only want the minimum number of BQ to represent the MQ, so modeling our problem as $LASSO$ optimization problem is an appropriate way 
 Question: What they formulate the question generation as?
Output: Abstractive
Considering the improvements over the majority baseline achieved by the RNN model for both non-English (on the average 22.76% relative improvement; 15.82% relative improvement on Spanish, 72.71% vs. 84.21%, 30.53% relative improvement on Turkish, 56.97% vs. 74.36%, 37.13% relative improvement on Dutch, 59.63% vs. 81.77%, and 7.55% relative improvement on Russian, 79.60% vs. 85.62%) and English test sets (27.34% relative improvement), we can draw the conclusion that our model is robust to handle multiple languages. 
 Question: which non-english language had the best performance?
Output: Extractive
The PolyResponse restaurant search is currently available in 8 languages and for 8 cities around the world: English (Edinburgh), German (Berlin), Spanish (Madrid), Mandarin (Taipei), Polish (Warsaw), Russian (Moscow), Korean (Seoul), and Serbian (Belgrade). 
 Question: In what 8 languages is PolyResponse engine used for restourant search and booking system?
Output: Abstractive
In this work, we introduce a methodology that provides VQA algorithms with the ability to generate human interpretable attention maps which effectively ground the answer to the relevant image regions. We accomplish this by leveraging region descriptions and object annotations available in the Visual Genome dataset, and using these to automatically construct attention maps that can be used for attention supervision, instead of requiring human annotators to manually provide grounding labels. 
 Question: How do they obtain region descriptions and object annotations?
Output: Abstractive
The experiment dataset comes from Microsoft Research (MSR) . It contains three domains: movie, taxi, and restaurant. The total count of dialogues per domain and train/valid/test split is reported in Table TABREF11. At every turn both user and agent acts are annotated, we use only the agent side as targets in our experiment. The acts are ordered in the dataset (each output sentence aligns with one act). 
 Question: What datasets are used for training/testing models? 
Output: Abstractive
In this paper, we use three encoders (NBOW, LSTM and attentive LSTM) to model the text descriptions. 
 Question: What neural models are used to encode the text?
Output: Abstractive
We compare our best F1 score with other state-of-the-art approaches in table TABREF39 , which shows our model has competitive advantage in dealing with drug-drug interaction extraction. 
 Question: What is the performance of their model?
Output: Abstractive
To further study the information encoded in the discourse embeddings, we perform t-SNE clustering BIBREF20 on them, using the best performing model CNN2-DE (global). 
 Question: How are discourse embeddings analyzed?
Output: Abstractive
Table TABREF14 show the results obtained for proposed s2sL approach in comparison to that of MLP for the tasks of Speech/Music and Neutral/Sad classification, by considering different proportions of training data. 
 Question: Which models/frameworks do they compare to?
Output: Extractive
In this paper, we present an extraction-then-synthesis framework for machine reading comprehension shown in Figure 1 , in which the answer is synthesized from the extraction results. We build an evidence extraction model to predict the most important sub-spans from the passages as evidence, and then develop an answer synthesis model which takes the evidence as additional features along with the question and passage to further elaborate the final answers. 
 Question: What two components are included in their proposed framework?
Output: Abstractive
Task: given a caption and a paired image (if used), the goal is to label every token in a caption in BIO scheme (B: beginning, I: inside, O: outside) BIBREF27 .  
 Question: Can named entities in SnapCaptions be discontigious?
Output: Yes-no
Noticeably, despite being both based on and integrated into a BERT$_\text{base}$ model, our architecture even outperforms a standalone BERT$_\text{large}$ model by a large margin. 
 Question: What models other than standalone BERT is new model compared to?
Output: Abstractive
Figure FIGREF10 illustrates the UTCNN model. As more than one user may interact with a given post, we first add a maximum pooling layer after the user matrix embedding layer and user vector embedding layer to form a moderator matrix embedding INLINEFORM0 and a moderator vector embedding INLINEFORM1 for moderator INLINEFORM2 respectively, where INLINEFORM3 is used for the semantic transformation in the document composition process, as mentioned in the previous section. The term moderator here is to denote the pseudo user who provides the overall semantic/sentiment of all the engaged users for one document. The embedding INLINEFORM4 models the moderator stance preference, that is, the pattern of the revealed user stance: whether a user is willing to show his preference, whether a user likes to show impartiality with neutral statements and reasonable arguments, or just wants to show strong support for one stance. Ideally, the latent user stance is modeled by INLINEFORM5 for each user. Likewise, for topic information, a maximum pooling layer is added after the topic matrix embedding layer and topic vector embedding layer to form a joint topic matrix embedding INLINEFORM6 and a joint topic vector embedding INLINEFORM7 for topic INLINEFORM8 respectively, where INLINEFORM9 models the semantic transformation of topic INLINEFORM10 as in users and INLINEFORM11 models the topic stance tendency. The latent topic stance is also modeled by INLINEFORM12 for each topic.

As for comments, we view them as short documents with authors only but without likers nor their own comments. Therefore we apply document composition on comments although here users are commenters (users who comment). It is noticed that the word embeddings INLINEFORM0 for the same word in the posts and comments are the same, but after being transformed to INLINEFORM1 in the document composition process shown in Figure FIGREF4 , they might become different because of their different engaged users. The output comment representation together with the commenter vector embedding INLINEFORM2 and topic vector embedding INLINEFORM3 are concatenated and a maximum pooling layer is added to select the most important feature for comments. Instead of requiring that the comment stance agree with the post, UTCNN simply extracts the most important features of the comment contents; they could be helpful, whether they show obvious agreement or disagreement. Therefore when combining comment information here, the maximum pooling layer is more appropriate than other pooling or merging layers. Indeed, we believe this is one reason for UTCNN's performance gains.

Finally, the pooled comment representation, together with user vector embedding INLINEFORM0 , topic vector embedding INLINEFORM1 , and document representation are fed to a fully connected network, and softmax is applied to yield the final stance label prediction for the post. 
 Question: How many layers does the UTCNN model have?
Output: Abstractive
Finally, we combine our CNN and RNN models using a voting process. For each sentence in the test set, we apply several CNN and RNN models presented in Tables TABREF12 and TABREF14 and predict the class with the most votes. In case of a tie, we pick one of the most frequent classes randomly. 
 Question: How does their simple voting scheme work?
Output: Extractive
We use the BLEU BIBREF30 metric on the validation set for the VQG model training. BLEU is a measure of similitude between generated and target sequences of words, widely used in natural language processing. It assumes that valid generated responses have significant word overlap with the ground truth responses. We use it because in this case we have five different references for each of the generated questions. We obtain a BLEU score of 2.07.

Our chatbot model instead, only have one reference ground truth in training when generating a sequence of words. We considered that it was not a good metric to apply as in some occasions responses have the same meaning, but do not share any words in common. Thus, we save several models with different hyperparameters and at different number of training iterations and compare them using human evaluation, to chose the model that performs better in a conversation. 
 Question: How is performance of this system measured?
Output: Abstractive
Since our methodology does not use a seed list of offensive words, it is not biased by topic, target, or dialect. Using our methodology, we tagged 10,000 Arabic tweet dataset for offensiveness, where offensive tweets account for roughly 19% of the tweets. Further, we labeled tweets as vulgar or hate speech. To date, this is the largest available dataset, which we plan to make publicly available along with annotation guidelines. We use this dataset to characterize Arabic offensive language to ascertain the topics, dialects, and users' gender that are most associated with the use of offensive language.  
 Question: In what way is the offensive dataset not biased by topic, dialect or target?
Output: Abstractive
Metrics. We use tolerance accuracy BIBREF16, which measures how far away the predicted span is from the gold standard span, as a metric. The rationale behind the metric is that, in practice, it suffices to recommend a rough span which contains the answer - a difference of a few seconds would not matter much to the user. Metrics. We used accuracy and MRR (Mean Reciprocal Ranking) as metrics.  Metrics. To evaluate our pipeline approach we use overall accuracy after filtering and accuracy given that the segment is in the top 10 videos.  
 Question: What evaluation metrics were used in the experiment?
Output: Abstractive
The accuracy of our model is 7.8% higher than the best result achieved by LSVM. The results show that this model can perform better than state-of-the-art baselines including hybrid CNN BIBREF15 and LSTM with attention BIBREF16 by 3.1% on the validation set and 1% on the test set. 
 Question: What are state of the art methods authors compare their work with? 
Output: Abstractive
We compare the performance of translation approaches based on four metrics:

[align=left,leftmargin=0em,labelsep=0.4em,font=]

As in BIBREF20 , EM is 1 if a predicted plan matches exactly the ground truth; otherwise it is 0.

The harmonic average of the precision and recall over all the test set BIBREF26 .

The minimum number of insertions, deletions or swap operations required to transform a predicted sequence of behaviors into the ground truth sequence BIBREF27 .

GM is 1 if a predicted plan reaches the ground truth destination (even if the full sequence of behaviors does not match exactly the ground truth). Otherwise, GM is 0. 
 Question: What evaluation metrics are used?
Output: Abstractive
the basic model yields good performance for recognizing explicit discourse relations as well, which is comparable with previous best result (92.05% macro F1-score and 93.09% accuracy as reported in BIBREF11 ). After untying parameters in the softmax prediction layer, implicit discourse relation classification performance was improved across all four relations, meanwhile, the explicit discourse relation classification performance was also improved. Then we also created ensemble models by applying majority voting to combine results of ten runs. From table 5 , each ensemble model obtains performance improvements compared with single model. The full model achieves performance boosting of (51.84 - 48.82 = 3.02) and (94.17 - 93.21 = 0.96) in macro F1-scores for predicting implicit and explicit discourse relations respectively.  
 Question: What discourse relations does it work best/worst for?
Output: Extractive
On high level, we indirectly evaluate the content relevance between the candidate summary and the human summary using information retrieval. To accomplish this, we use the summaries as search queries and compare the overlaps of the retrieved results.  
 Question: In the proposed metric, how is content relevance measured?
Output: Abstractive
The first challenge is to acquire very large Vietnamese corpus and to use them in building a classifier, which could further improve accuracy.  The second challenge is design and development of big data warehouse and analytic framework for Vietnamese documents, which corresponds to the rapid and continuous growth of gigantic volume of articles and/or documents from Web 2.0 applications, such as, Facebook, Twitter, and so on. The final challenge relates to building a system, which is able to incrementally learn new corpora and interactively process feedback. 
 Question: Why challenges does word segmentation in Vietnamese pose?
Output: Abstractive
The seq2seq model with global attention gives the best results with an average target BLEU score of 29.65 on the style transfer dataset, compared with an average target BLEU score of 26.97 using the seq2seq model with pointer networks. 
 Question: What is best BLEU score of language style transfer authors got?
Output: Extractive
In this section, we introduce an invertibility condition for our neural projector to tackle the optimization challenge. Specifically, we constrain our neural projector with two requirements: (1) INLINEFORM0 and (2) INLINEFORM1 exists.  
 Question: What is the invertibility condition?
Output: Abstractive
Therefore, we just implement a simple one-layer RNN with LSTM cell to classify pre-processed sentences into ironic sentences and non-ironic sentences because LSTM networks are widely used in irony detection. 
 Question: How did the authors find ironic data on twitter?
Output: Abstractive
We use three creative English datasets with distinct linguistic characteristics: (1) A corpus of 740 classical and contemporary English poems, (2) a corpus of 14950 metaphor sentences retrieved from a metaphor database website and (3) a corpus of 1500 song lyrics ranging across genres.  We first pre-train our generator on the Gutenberg dataset BIBREF24 for 20 epochs and then fine-tune BIBREF19 them to our target datasets with a language modeling objective. 
 Question: Which datasets are used?
Output: Extractive
Cloze Track User Query Track 
 Question: What two types the Chinese reading comprehension dataset consists of?
Output: Abstractive
The baseline system for the SLC task is a very simple logistic regression classifier with default parameters, where we represent the input instances with a single feature: the length of the sentence.  The baseline for the FLC task generates spans and selects one of the 18 techniques randomly.  
 Question: What was the baseline for this task?
Output: Abstractive
The task presented here was easy and simple to analyze, however, future work may be done on natural language tasks. If these properties hold it might indicate that a new evaluation paradigm for NLP should be pushed; one that emphasizes performance on uncharacteristic (but structurally sound) inputs in addition to the data typically seen in training. 
 Question: Can the findings of this paper be generalized to a general-purpose task?
Output: Yes-no
We create three additional training datasets by adding sentences involving object RCs to the original Wikipedia corpus (Section lm). To this end, we randomly pick up 30 million sentences from Wikipedia (not overlapped to any sentences in the original corpus), parse by the same parser, and filter sentences containing an object RC, amounting to 680,000 sentences.  
 Question: How do they perform data augmentation?
Output: Abstractive
Input of the model is the concatenation of word embedding and another embedding indicating whether this word is predicate: $ \mathbf {x}_t = [\mathbf {W}_{\text{emb}}(w_t), \mathbf {W}_{\text{mask}}(w_t = v)]. $ 
 Question: What's the input representation of OpenIE tuples into the model?
Output: Abstractive
n the OLID dataset, we use a hierarchical annotation model split into three levels to distinguish between whether language is offensive or not (A), and type (B) and target (C) of the offensive language. Each level is described in more detail in the following subsections and examples are shown in Table TABREF10 . Level A: Offensive language Detection
Level A discriminates between offensive (OFF) and non-offensive (NOT) tweets. Level B: Categorization of Offensive Language
Level B categorizes the type of offense and two labels are used: targeted (TIN) and untargeted (INT) insults and threats. Level C: Offensive Language Target Identification
Level C categorizes the targets of insults and threats as individual (IND), group (GRP), and other (OTH). 
 Question: What are the three layers of the annotation scheme?
Output: Extractive
Car-speak is abstract language that pertains to a car's physical attribute(s). In this instance the physical attributes that the term “fast” pertains to could be the horsepower, or it could be the car's form factor (how the car looks). However, we do not know exactly which attributes the term “fast” refers to. 
 Question: How does car speak pertains to a car's physical attributes?
Output: Extractive
We will compare to the more recent cross-lingual language model XLM BIBREF12, as well as the state-of-the-art CoNLL 2018 shared task results with predicted tokenisation and segmentation in an updated version of the paper. In French, no extensive work has been done due to the limited availability of NER corpora. We compare our model with the strong baselines settled by BIBREF49, who trained both CRF and BiLSTM-CRF architectures on the FTB and enhanced them using heuristics and pretrained word embeddings. In the TRANSLATE-TRAIN setting, we report the best scores from previous literature along with ours. BiLSTM-max is the best model in the original XNLI paper, mBERT which has been reported in French in BIBREF52 and XLM (MLM+TLM) is the best-presented model from BIBREF50. 
 Question: What is the state of the art?
Output: Abstractive
We defined the reward as being 1 for successfully completing the task, and 0 otherwise. A discount of $0.95$ was used to incentivize the system to complete dialogs faster rather than slower, yielding return 0 for failed dialogs, and $G = 0.95^{T-1}$ for successful dialogs, where $T$ is the number of system turns in the dialog.   0.95^{T-1} reward  0.95^{T-1}  
 Question: What is the reward model for the reinforcement learning appraoch?
Output: Abstractive
Our model introduces a multi-turns inference mechanism to process multi-perspective matching features.  
 Question: Which matching features do they employ?
Output: Abstractive
We compare FSDM with four baseline methods and two ablations.

NDM BIBREF7 proposes a modular end-to-end trainable network. It applies de-lexicalization on user utterances and responses.

LIDM BIBREF9 improves over NDM by employing a discrete latent variable to learn underlying dialogue acts. This allows the system to be refined by reinforcement learning.

KVRN BIBREF13 adopts a copy-augmented Seq2Seq model for agent response generation and uses an attention mechanism on the KB. It does not perform belief state tracking.

TSCP/RL BIBREF10 is a two-stage CopyNet which consists of one encoder and two copy-mechanism-augmented decoders for belief state and response generation. 
 Question: What baselines have been used in this work?
Output: Abstractive
As illustrated in Figure FIGREF1, our key idea is that we can exploit discourse relations BIBREF4 to efficiently propagate polarity from seed predicates that directly report one's emotions (e.g., “to be glad” is positive). Suppose that events $x_1$ are $x_2$ are in the discourse relation of Cause (i.e., $x_1$ causes $x_2$). If the seed lexicon suggests $x_2$ is positive, $x_1$ is also likely to be positive because it triggers the positive emotion. The fact that $x_2$ is known to be negative indicates the negative polarity of $x_1$. Similarly, if $x_1$ and $x_2$ are in the discourse relation of Concession (i.e., $x_2$ in spite of $x_1$), the reverse of $x_2$'s polarity can be propagated to $x_1$. Even if $x_2$'s polarity is not known in advance, we can exploit the tendency of $x_1$ and $x_2$ to be of the same polarity (for Cause) or of the reverse polarity (for Concession) although the heuristic is not exempt from counterexamples. We transform this idea into objective functions and train neural network models that predict the polarity of a given event. 
 Question: How are relations used to propagate polarity?
Output: Abstractive
Questions: We make use of the 7,787 science exam questions of the Aristo Reasoning Challenge (ARC) corpus BIBREF31 , which contains standardized 3rd to 9th grade science questions from 12 US states from the past decade.  
 Question: How was the dataset collected?
Output: Abstractive
Also, note that the performance for this task is not expected to achieve a perfect accuracy, as there may be situations where more than one action is reasonable, and also because writers tell a story playing with elements such as surprise or uncertainty. 
 Question: Why do they think this task is hard?  What is the baseline performance?
Output: Abstractive
Crowdsourced annotators assigned similarity to word pairs during the word similarity task.  
 Question: did they use a crowdsourcing platform for annotations?
Output: Yes-no
Results reveal that, contrary to the common belief, correlations between Rouge and the Pyramid scores are weak, which challenges its effectiveness for scientific summarization. Furthermore, we show a large variance of correlations between different Rouge variants and the manual evaluations which further makes the reliability of Rouge for evaluating scientific summaries less clear. 
 Question: What is the common belief that this paper refutes? (c.f. 'contrary to the common belief, ROUGE is not much [sic] reliable'
Output: Extractive
DrQA is a CRC baseline coming with the CoQA dataset. Note that this implementation of DrQA is different from DrQA for SQuAD BIBREF8 in that it is modified to support answering no answer questions by having a special token unknown at the end of the document.  DrQA+CoQA is the above baseline pre-tuned on CoQA dataset and then fine-tuned on $(\text{RC})_2$ .  BERT is the vanilla BERT model directly fine-tuned on $(\text{RC})_2$ . We use this baseline for ablation study on the effectiveness of pre-tuning. BERT+review first tunes BERT on domain reviews using the same objectives as BERT pre-training and then fine-tunes on $(\text{RC})_2$ . We use this baseline to show that a simple domain-adaptation of BERT is not good. BERT+CoQA first fine-tunes BERT on the supervised CoQA data and then fine-tunes on $(\text{RC})_2$ . We use this baseline to show that pre-tuning is very competitive even compared with models trained from large-scale supervised data. 
 Question: What is the baseline model used?
Output: Abstractive
The corpus of supervisor assessment has 26972 sentences.  
 Question: What is the size of the real-life dataset?
Output: Extractive
(proposed) Bi-LSTM/CRF + Bi-CharLSTM with modality attention (W+C): uses the modality attention to merge word and character embeddings.

(proposed) Bi-LSTM/CRF + Bi-CharLSTM + Inception (W+C+V): takes as input visual contexts extracted from InceptionNet as well, concatenated with word and char vectors.

(proposed) Bi-LSTM/CRF + Bi-CharLSTM + Inception with modality attention (W+C+V): uses the modality attention to merge word, character, and visual embeddings as input to entity LSTM. 
 Question: Does their NER model learn NER from both text and images?
Output: Yes-no
Traditional text-to-speech (TTS) systems are composed of complex pipelines BIBREF0 , these often include accoustic frontends, duration model, acoustic prediction model and vocoder models. The architecture of our model utilizes RNN-based Seq2Seq model for generating mel spectrogram from text. The architecture is similar to that of Tacotron 2 BIBREF4  Direct comparison of model parameters between ours and the open-source tacotron 2, our model contains 4.5 million parameters, whereas the Tacotron 2 contains around 13 million parameters with default setting. By helping our model learn attention alignment faster, we can afford to use a smaller overall model to achieve similar quality speech quality. 
 Question: Do they reduce the number of parameters in their architecture compared to other direct text-to-speech models?
Output: Yes-no
We build the iSQuAD and iNewsQA datasets based on SQuAD v1.1 BIBREF0 and NewsQA BIBREF1. iMRC: Making MRC Interactive ::: Evaluation Metric
Since iMRC involves both MRC and RL, we adopt evaluation metrics from both settings. First, as a question answering task, we use $\text{F}_1$ score to compare predicted answers against ground-truth, as in previous works. When there exist multiple ground-truth answers, we report the max $\text{F}_1$ score. Second, mastering multiple games remains quite challenging for RL agents. Therefore, we evaluate an agent's performance during both its training and testing phases. During training, we report training curves averaged over 3 random seeds. During test, we follow common practice in supervised learning tasks where we report the agent's test performance corresponding to its best validation performance . 
 Question: What are the models evaluated on?
Output: Abstractive
The annotation projection method, which we follow in this work, is one way to address this problem. It was introduced for POS tagging, base noun phrase bracketing, NER tagging, and inflectional morphological analysis BIBREF29 but it has also been used for dependency parsing BIBREF30 , role labeling BIBREF31 , BIBREF32 and semantic parsing BIBREF26 . 
 Question: Do the authors test their annotation projection techniques on tasks other than AMR?
Output: Yes-no
The encoder is a bidirectional Long-Short Term Memory(LSTM) cell BIBREF14 and the decoder a single LSTM cell with attention mechanism.  
 Question: How many attention layers are there in their model?
Output: Abstractive
The data for this project are two parts, the first part is the historical S&P 500 component stocks, which are downloaded from the Yahoo Finance. We use the data over the period of from 12/07/2017 to 06/01/2018. The second part is the news article from financial domain are collected with the same time period as stock data. Hence, only news article from financial domain are collected. The data is mainly taken from Webhose archived data, which consists of 306242 news articles present in JSON format, dating from December 2017 up to end of June 2018. 
 Question: What is the dataset used in the paper?
Output: Abstractive
Our study focused primarily on English tweets, since this was the language of our diagnostic training sample. 
 Question: Do the authors report results only on English datasets?
Output: Yes-no
We evaluate with all commonly-used metrics in question generation BIBREF13: BLEU-1 (B1), BLEU-2 (B2), BLEU-3 (B3), BLEU-4 (B4) BIBREF17, METEOR (MET) BIBREF18 and ROUGE-L (R-L) BIBREF19. We use the evaluation script released by Chen2015MicrosoftCC. 
 Question: What metrics do they use?
Output: Extractive
For example, the original sentence `We went shop on Saturday' and the corrected version `We went shopping on Saturday' would produce the following pattern:

(VVD shop_VV0 II, VVD shopping_VVG II)

After collecting statistics from the background corpus, errors can be inserted into error-free text. The learned patterns are now reversed, looking for the correct side of the tuple in the input sentence. We only use patterns with frequency INLINEFORM0 , which yields a total of 35,625 patterns from our training data.  
 Question: What textual patterns are extracted?
Output: Extractive
For each candidate mention span, we retrieve the top 10 entities according to the Freebase API. We then create a lattice in which the nodes correspond to mention-entity pairs, scored by their Freebase API scores, and the edges encode the fact that no joint assignment of entities to mentions can contain overlapping spans. We take the top 10 paths through the lattice as possible entity disambiguations. For each possibility, we generate $n$ -best paraphrases that contains the entity mention spans. In the end, this process creates a total of $10n$ paraphrases.  
 Question: How many paraphrases are generated per question?
Output: Abstractive
CoVoST is currently built on that release and covers the following 11 languages: French, German, Dutch, Russian, Spanish, Italian, Turkish, Persian, Swedish, Mongolian and Chinese. 
 Question: Is Arabic one of the 11 languages in CoVost?
Output: Yes-no
We first introduce a method that detects bottlenecks in text-games using the overall reward gained and the knowledge graph state. This method freezes the policy used to reach the bottleneck and restarts the training from there on out, additionally conducting a backtracking search to ensure that a sub-optimal policy has not been frozen. The second contribution explore how to leverage knowledge graphs to improve existing exploration algorithms for dealing with combinatorial action-spaces such as Go-Explore BIBREF9.  
 Question: What are the two new strategies?
Output: Extractive
In Table TABREF1, we summarize the quantitative results of the above previous studies. In Table TABREF1, we summarize the quantitative results of the above previous studies. 
 Question: What is the accuracy reported by state-of-the-art methods?
Output: Abstractive
We evaluated the phrase compositionality models on the adjective-noun and noun-noun phrase similarity tasks compiled by Mitchell2010, using the same evaluation scheme as in the original work. Spearman's INLINEFORM0 between phrasal similarities derived from our compositional functions and the human annotators (computed individually per annotator and then averaged across all annotators) was the evaluation measure. 
 Question: How do they score phrasal compositionality?
Output: Extractive
We use Waseem and Hovy BIBREF5, Davidson et al. BIBREF9, and Waseem et al. BIBREF10 as baselines and compare the results with our different fine-tuning strategies using pre-trained BERTbase model.  
 Question: What existing approaches do they compare to?
Output: Extractive
We then proceed to connect these mentions i) if they co-occur within the same document (we will refer to this as DOC-BASED edges), ii) if the pair of named entity mentions is identical (MATCH edges—these may connect nodes across and within documents), or iii) if they are in the same coreference chain, as predicted by the external coreference system (COREF edges).  
 Question: How did they get relations between mentions?
Output: Abstractive
Three incremental levels of document preprocessing are experimented with: raw text, text cleaning through document logical structure detection, and removal of keyphrase sparse sections of the document. In doing so, we present the first consistent comparison of different keyphrase extraction models and study their robustness over noisy text. 
 Question: what levels of document preprocessing are looked at?
Output: Extractive
Our model contains five independent decoders, one for each image in the sequence. This allows each decoder to learn a specific language model for each position of the sequence. 
 Question: Do the decoder LSTMs all have the same weights?
Output: Yes-no
We employ a huge dataset of 45,000 self-reported depressed users introduced in BIBREF46 where a lexicon of depression symptoms consisting of 1500 depression-indicative terms was created with the help of psychologist clinician and employed for collecting self-declared depressed individual's profiles. A subset of 8,770 users (24 million time-stamped tweets) containing 3981 depressed and 4789 control users (that do not show any depressive behavior) were verified by two human judges BIBREF46 . 
 Question: How is the data annotated?
Output: Abstractive
We performed experiments on synthetically generated dataset since it gives us a better control over the distribution of the data. Specifically we compared the gains obtained using our approach versus the variance of the distribution. We created dataset from the following generative process. [H] Generative Process [1] Generate data

Pick k points INLINEFORM0 as domain -1 means and a corresponding set of k points INLINEFORM1 as domain-2 means, and covariance matrices INLINEFORM2

iter INLINEFORM0 upto num INLINEFORM1 samples Sample class INLINEFORM2 Sample INLINEFORM3 Sample INLINEFORM4 Add q and a so sampled to the list of q,a pairs We generated the dataset from the above sampling process with means selected on a 2 dimensional grid of size INLINEFORM5 with variance set as INLINEFORM6 in each dimension.10000 sample points were generated. The parameter INLINEFORM7 of the above algorithm was set to 0.5 and k was set to 9 (since the points could be generated from one of the 9 gaussians with centroids on a INLINEFORM8 grid). 
 Question: How do they generate the synthetic dataset?
Output: Abstractive
The input for both tasks consists of news articles in free-text format, collected from 36 propagandist and 12 non-propagandist news outlets and then annotated by professional annotators.  
 Question: What dataset was used?
Output: Extractive
Later, BIBREF8 introduced an RNN with an external stack memory to learn simple context-free languages, such as $a^n b^m$ , $a^nb^ncb^ma^m$ , and $a^{n+m} b^n c^m$ . Similar studies BIBREF15 , BIBREF16 , BIBREF17 , BIBREF10 , BIBREF11 have explored the existence of stable counting mechanisms in simple RNNs, which would enable them to learn various context-free and context-sensitive languages BIBREF9 , on the other hand, proposed a variant of Long Short-Term Memory (LSTM) networks to learn two context-free languages, $a^n b^n$ , $a^n b^m B^m A^n$ , and one strictly context-sensitive language, $a^n b^n c^n$ . 
 Question: How do they get the formal languages?
Output: Abstractive
To investigate whether our BERT-based model can transfer knowledge beyond language, we consider image features as simple visual tokens that can be presented to the model analogously to textual token embeddings. In order to make the $o_j$ vectors (of dimension $2048+4=2052$) comparable to BERT embeddings (of dimension 768), we use a simple linear cross-modal projection layer $W$ of dimensions $2052\hspace{-1.00006pt}\times \hspace{-1.00006pt}768$. The $N$ object regions detected in an image, are thus represented as $X_{img} = (W.o_1,...,W.o_N)$. Once mapped into the BERT embedding space with $W$, the image is seen by the rest of the model as a sequence of units with no explicit indication if it is a text or an image embedding. 
 Question: How are multimodal representations combined?
Output: Abstractive
Our ASR and ST models follow the architecture in berard2018end, but have 3 decoder layers like that in pino2019harnessing. 
 Question: What is the architecture of their model?
Output: Extractive
Validated transcripts were sent to professional translators. In order to control the quality of the professional translations, we applied various sanity checks to the translations BIBREF11. We also sanity check the overlaps of train, development and test sets in terms of transcripts and voice clips (via MD5 file hashing), and confirm they are totally disjoint. 
 Question: How is the quality of the data empirically evaluated? 
Output: Extractive
The final annotation categories for the dataset are: Joy, Sadness, Anger, Fear, Anticipation, Surprise, Love, Disgust, Neutral 
 Question: How many emotions do they look at?
Output: Abstractive
Moreover, QA and QG have probabilistic correlation as both tasks relate to the joint probability between $q$ and $a$ . Given a question-answer pair $\langle q, a \rangle $ , the joint probability $P(q, a)$ can be computed in two equivalent ways.

$$P(q, a) = P(a) P(q|a) = P(q)P(a|q)$$ (Eq. 1)

The conditional distribution $P(q|a)$ is exactly the QG model, and the conditional distribution $P(a|q)$ is closely related to the QA model. Existing studies typically learn the QA model and the QG model separately by minimizing their own loss functions, while ignoring the probabilistic correlation between them.

Based on these considerations, we introduce a training framework that exploits the duality of QA and QG to improve both tasks. There might be different ways of exploiting the duality of QA and QG. In this work, we leverage the probabilistic correlation between QA and QG as the regularization term to influence the training process of both tasks. Specifically, the training objective of our framework is to jointly learn the QA model parameterized by $\theta _{qa}$ and the QG model parameterized by $\theta _{qg}$ by minimizing their loss functions subject to the following constraint.

$$P_a(a) P(q|a;\theta _{qg}) = P_q(q)P(a|q;\theta _{qa})$$ (Eq. 3)

$P_a(a)$ and $P_q(q)$ are the language models for answer sentences and question sentences, respectively. Overall, the framework includes three components, namely a QA model, a QG model and a regularization term that reflects the duality of QA and QG. The QA specific objective aims to minimize the loss function $l_{qa}(f_{qa}(a,q;\theta _{qa}), label)$ , where $label$ is 0 or 1 that indicates whether $a$ is the correct answer of $q$ or not. For each correct question-answer pair, the QG specific objective is to minimize the following loss function,

$$l_{qg}(q, a) = -log P_{qg}(q|a;\theta _{qg})$$ (Eq. 6)

where $a$ is the correct answer of $q$ . The third objective is the regularization term which satisfies the probabilistic duality constrains as given in Equation 3 . Specifically, given a correct $\langle q, a \rangle $ pair, we would like to minimize the following loss function,

$$ \nonumber l_{dual}(a,q;\theta _{qa}, \theta _{qg}) &= [logP_a(a) + log P(q|a;\theta _{qg}) \\ & - logP_q(q) - logP(a|q;\theta _{qa})]^2$$ (Eq. 9)

where $P_a(a)$ and $P_q(q)$ are marginal distributions, which could be easily obtained through language model. 
 Question: What does "explicitly leverages their probabilistic correlation to guide the training process of both models" mean?
Output: Abstractive
Second, we compute three metrics on the extracted information:

$\bullet $ Relation Generation (RG) estimates how well the system is able to generate text containing factual (i.e., correct) records. We measure the precision and absolute number (denoted respectively RG-P% and RG-#) of unique relations $r$ extracted from $\hat{y}_{1:T}$ that also appear in $s$.

$\bullet $ Content Selection (CS) measures how well the generated document matches the gold document in terms of mentioned records. We measure the precision and recall (denoted respectively CS-P% and CS-R%) of unique relations $r$ extracted from $\hat{y}_{1:T}$ that are also extracted from $y_{1:T}$.

$\bullet $ Content Ordering (CO) analyzes how well the system orders the records discussed in the description. We measure the normalized Damerau-Levenshtein distance BIBREF36 between the sequences of records extracted from $\hat{y}_{1:T}$ that are also extracted from $y_{1:T}$. 
 Question: Which qualitative metric are used for evaluation?
Output: Extractive
A single speaker narrated the 2000 sentences, which took several days.  
 Question: How many annotators participated?
Output: Abstractive
To evaluate real-world performance of our selected classifier (i.e., performance in the absence of model and parameter bias), we perform classification of the holdout set. On this set, our classifier had an accuracy and F1-score of 89.6% and 89.2%, respectively.  
 Question: What was their system's performance?
Output: Extractive
The study of the mathematical structure of grammar has indicated that the fundamental things making up sentences are not the words, but some atomic grammatical types, such as the noun-type and the sentence-type BIBREF23 , BIBREF24 , BIBREF25 . The transitive verb-type is not an atomic grammatical type, but a composite made up of two noun-types and one sentence-type. Hence, particularly interesting here is that atomic doesn't really mean smallest... 
 Question: Do they argue that all words can be derived from other (elementary) words?
Output: Yes-no
Following the formula, we can calculate the contribution of every input word makes to every output word, forming a contribution matrix of size $M \times N$, where $N$ is the output sentence length. Given the contribution matrix, we can obtain the word importance of each input word to the entire output sentence. To this end, for each input word, we first aggregate its contribution values to all output words by the sum operation, and then normalize all sums through the Softmax function. 
 Question: How do their models decide how much improtance to give to the output words?
Output: Extractive
To better demonstrate the effectiveness of the proposed model, we compare with baselines and show the results in Table TABREF12 . 
 Question: What was the score of the proposed model?
Output: Abstractive
The problem with morpheme segmentation is that the vocabulary of stem units is still very large, which leads to many rare and unknown words at the training time.  Therefore, we learn a BPE model on the stem units in the training corpus rather than the words, and then apply it on the stem unit of each word after morpheme segmentation. 
 Question: How is morphology knowledge implemented in the method?
Output: Abstractive
For those willing to have a more tightly-controlled installation of Seshat on their system, we also fully specify the manual installation steps in our online documentation). 
 Question: Is this software available to the public?
Output: Yes-no
KryptoOracle has been built in the Apache ecosystem and uses Apache Spark. Data structures in Spark are based on resilient distributed datasets (RDD), a read only multi-set of data which can be distributed over a cluster of machines and is fault tolerant.  Spark RDD has the innate capability to recover itself because it stores all execution steps in a lineage graph. In case of any faults in the system, Spark redoes all the previous executions from the built DAG and recovers itself to the previous steady state from any fault such as memory overload. Spark RDDs lie in the core of KryptoOracle and therefore make it easier for it to recover from faults. Moreover, faults like memory overload or system crashes may require for the whole system to hard reboot. However, due to the duplicate copies of the RDDs in Apache Hive and the stored previous state of the machine learning model, KryptoOracle can easily recover to the previous steady state.

 
 Question: How is the architecture fault-tolerant?
Output: Abstractive
The answer styles considered in the experiments corresponded to the two tasks. The NLG task requires a well-formed answer that is an abstractive summary of the question and ten passages, averaging 16.6 words. The Q&A task also requires an abstractive answer but prefers a more concise answer than the NLG task, averaging 13.1 words, where many of the answers do not contain the context of the question. 
 Question: What do they mean by answer styles?
Output: Abstractive
We build and test our MMT models on the Multi30K dataset BIBREF21 . Each image in Multi30K contains one English (EN) description taken from Flickr30K BIBREF22 and human translations into German (DE), French (FR) and Czech BIBREF23 , BIBREF24 , BIBREF25 . The dataset contains 29,000 instances for training, 1,014 for development, and 1,000 for test. We only experiment with German and French, which are languages for which we have in-house expertise for the type of analysis we present. In addition to the official Multi30K test set (test 2016), we also use the test set from the latest WMT evaluation competition, test 2018 BIBREF25 . 
 Question: Do they report results only on English dataset?
Output: Yes-no
One of the most important operations on a word is to obtain the set of words whose meaning is similar to the word, or whose usage in text is similar to the word. We call this set the neighbor of the word. We have observed that the neighbor of a polysemic word consists of words that resemble the primary sense of the polysemic word.  Even though a word may be a polysemic, it usually corresponds to a single vector in distributed representation. This vector is primarily determined by the major sense, which is most frequently used. The information about a word's minor sense is subtle, and the effect of a minor sense is difficult to distinguish from statistical fluctuation. To measure the effect of a minor sense, this paper proposes to use the concept of surrounding uniformity. The surrounding uniformity roughly corresponds to statistical fluctuation in the vectors that correspond to the words in the neighbor Surrounding Uniformity (SU) can be expressed as follows: $SU(\vec{w}) = \frac{|\vec{s}(\vec{w})|}{|\vec{w}| + \sum _{i}^{N}|\vec{a_i}(\vec{w})|}$

where $\vec{s}(\vec{w}) = \vec{w} + \sum _{i}^{N} \vec{a_i}(\vec{w}).$ 
 Question: How is the fluctuation in the sense of the word and its neighbors measured?
Output: Abstractive
However, no prior work has explored the target of the offensive language, which is important in many scenarios, e.g., when studying hate speech with respect to a specific target. 
 Question: What are the differences between this dataset and pre-existing ones?
Output: Extractive
The reports are published by Météo France and the Met Office, its British counterpart. They are publicly available on the respective websites of the organizations. Both corpora span on the same period as the corresponding time series and given their daily nature, it yields a total of 4,261 and 4,748 documents respectively. 
 Question: How big is dataset used for training/testing?
Output: Abstractive
A group of 50 native people who were well-versed in both English and Tamil languages acted as annotators for the evaluation. A collection of samples of about 100 sentences were taken from the test set results for comparison. This set included a randomized selection of the translation results to ensure the objectivity of evaluation. Fluency and adequacy results for the RNNMorph results are tabulated. Adequacy rating was calculated on a 5-point scale of how much of the meaning is conveyed by the translation (All, Most, Much, Little, None). The fluency rating was calculated based on grammatical correctness on a 5-point scale of (Flawless, Good, Non-native, Disfluent, Incomprehensive). For the comparison process, the RNNMorph and the RNNSearch + Word2Vec models’ sentence level translations were individually ranked between each other, permitting the two translations to have ties in the ranking. 
 Question: How were the human judgements assembled?
Output: Abstractive