niryuu/sni-each-converted huggingface dataset:
Evaluation Metrics. To evaluate the strategy formulation ability, we introduce a measure called Coverage( INLINEFORM0 ), defined as the fraction of total query data instances, for which LiLi has successfully formulated strategies that lead to winning. If LiLi wins on all episodes for a given dataset, INLINEFORM1 is 1.0. To evaluate the predictive performance, we use Avg. MCC and avg. +ve F1 score. 
 Question: What metrics are used to establish that this makes chatbots more knowledgeable and better at learning and conversation? 
Output: Extractive
The final section of the paper, then, discusses how insights gained from technologically observing opinion dynamics can inform conceptual modelling efforts and approaches to on-line opinion facilitation. As such, the paper brings into view and critically evaluates the fundamental conceptual leap from machine-guided observation to debate facilitation and intervention. 
 Question: Does the paper report the results of previous models applied to the same tasks?
Output: Yes-no
For experiments that use parallel data to initialize foreign specific parameters, we use the same datasets in the work of BIBREF6. Specifically, we use United Nations Parallel Corpus BIBREF18 for en-ru, en-ar, en-zh, and en-fr. We collect en-hi parallel data from IIT Bombay corpus BIBREF19 and en-vi data from OpenSubtitles 2018. 
 Question: What datasets are used for evaluation?
Output: Extractive
For generating a poem from images we use an existing actor-critic architecture BIBREF1. For Shakespearizing modern English texts, we experimented with various types of sequence to sequence models. We use a sequence-to-sequence model which consists of a single layer unidrectional LSTM encoder and a single layer LSTM decoder and pre-trained retrofitted word embeddings shared between source and target sentences. Since a pair of corresponding Shakespeare and modern English sentences have significant vocabulary overlap we extend the sequence-to-sequence model mentioned above using pointer networks BIBREF11 that provide location based attention and have been used to enable copying of tokens directly from the input. 
 Question: What models are used for painting embedding and what for language style transfer?
Output: Extractive
We compare eight different models in our experiments. Four of them have a Projected Layer (see Fig. FIGREF2), while the others do not have, and this is the only difference between these two groups of our models. So, we actually include four models in our experiments (having a projected layer or not). Firstly, LastStateRNN is the classic RNN model, where the last state passes through an MLP and then the LR Layer estimates the corresponding probability. In contrast, in the AvgRNN model we consider the average vector of all states that come out of the cells. The AttentionRNN model is the one that it has been presented in BIBREF9. Moreover, we introduce the MultiAttentionRNN model for the harassment language detection, which instead of one attention, it includes four attentions, one for each category. 
 Question: What are the different variations of the attention-based approach which are examined?
Output: Abstractive
The dataset from Twitter that we are using in our work, consists of a train set, a validation set and a test set. It was published for the "First workshop on categorizing different types of online harassment languages in social media". The whole dataset is divided into two categories, which are harassment and non-harassment tweets. Moreover, considering the type of the harassment, the tweets are divided into three sub-categories which are indirect harassment, sexual and physical harassment. 
 Question: What types of online harassment are studied?
Output: Extractive
Each tweet is annotated as no evidence of depression (e.g., “Citizens fear an economic depression") or evidence of depression (e.g., “depressed over disappointment"). If a tweet is annotated evidence of depression, then it is further annotated with one or more depressive symptoms, for example, depressed mood (e.g., “feeling down in the dumps"), disturbed sleep (e.g., “another restless night"), or fatigue or loss of energy (e.g., “the fatigue is unbearable") BIBREF10 . 
 Question: How is the dataset annotated?
Output: Extractive
For the human evaluation, we follow the standard approach in evaluating machine translation systems BIBREF23 , as used for question generation by BIBREF9 . We asked three workers to rate 300 generated questions between 1 (poor) and 5 (good) on two separate criteria: the fluency of the language used, and the relevance of the question to the context document and answer. 
 Question: What human evaluation metrics were used in the paper?
Output: Abstractive
First, it provides an annotation interface that allows users to define content elements, upload documents, and annotate documents Our platform is able to ingest documents in a variety of formats, including PDFs and Microsoft Word, and converts these formats into plain text before presenting them to the annotators. 
 Question: What type of documents are supported by the annotation platform?
Output: Abstractive
We use a default train-validation-test split of 70-15-15 for each dataset, and use all 4 datasets (BF, BA, SFU and Sherlock). 
 Question: Which multiple datasets did they train on during joint training?
Output: Extractive
Table TABREF23 shows the Pearson, Spearman and Kendall correlation of Rouge and Sera, with pyramid scores. Interestingly, we observe that many variants of Rouge scores do not have high correlations with human pyramid scores. The lowest F-score correlations are for Rouge-1 and Rouge-L (with INLINEFORM0 =0.454). Weak correlation of Rouge-1 shows that matching unigrams between the candidate summary and gold summaries is not accurate in quantifying the quality of the summary. 
 Question: What different correlations result when using different variants of ROUGE scores?
Output: Extractive
This decoding approach closely follows Algorithm SECREF7 , but along with soft back pointers, we also compute hard back pointers at each time step. After computing all the relevant quantities like model score, loss etc., we follow the hard backpointers to obtain the best sequence INLINEFORM0 . 
 Question: Do they compare partially complete sequences (created during steps of beam search) to gold/target sequences?
Output: Yes-no
We recruited 102 pairs of participants from Amazon Mechanical Turk and randomly assigned speaker and listener roles 
 Question: Was this experiment done in a lab?
Output: Yes-no
There are several caveats in our work: first, tweet sentiment is rarely binary (this work could be extended to a multinomial or continuous model). Second, our results are constrained to Twitter users, who are known to be more negative than the general U.S. population BIBREF9 . Third, we do not take into account the aggregate effects of continued natural disasters over time. Going forward, there is clear demand in discovering whether social networks can indicate environmental metrics in a “nowcasting" fashion. As climate change becomes more extreme, it remains to be seen what degree of predictive power exists in our current model regarding climate change sentiments with regards to natural disasters. 
 Question: Do the authors mention any confounds to their study?
Output: Yes-no
Eventually, we have a balanced testing dataset, where each term-sense pair has around 15 samples for testing (on average, each pair has 14.56 samples and the median sample number is 15), and a comparison with training dataset is shown in Figure FIGREF11. Due to the difficulty for collecting the testing dataset, we decided to only collect for a random selection of 30 terms.  
 Question: How big is dataset for testing?
Output: Abstractive
Clustering was performed separately for each specialty of doctors. We also examined the distribution of doctors' IDs in the obtained clusters. It turned out that some clusters covered almost exactly descriptions written by one doctor. This situation took place in the specialties where clusters are separated with large margins (e.g. psychiatry, pediatrics, cardiology). 
 Question: Do they explore similarity of texts across different doctors?
Output: Yes-no
The task of humor identification in social media texts is analyzed as a classification problem and several machine learning classification models are used. 
 Question: What experiments were carried out on the corpus?
Output: Extractive
In this study, we concentrate our effort on re-assessing keyphrase extraction performance on three increasingly sophisticated levels of document preprocessing described below. 
 Question: what keyphrase extraction models were reassessed?
Output: Abstractive
For the embeddings, we relied on $AraVec$ BIBREF30 for Arabic, FastText BIBREF31 for French, and Word2vec Google News BIBREF32 for English .  
 Question: What monolingual word representations are used?
Output: Abstractive
An extrinsic evaluation was carried out on the task of Open IE BIBREF7. 
 Question: Is the semantic hierarchy representation used for any task?
Output: Abstractive
To analyse the results we chose to use the test provided by BIBREF10, which consists of $19\,791$ analogies divided into 19 different categories: 6 related to the “semantic" macro-area (8915 analogies) and 13 to the “syntactic" one (10876 analogies). All the analogies are composed by two pairs of words that share a relation, schematized with the equation: $a:a^{*}=b:b^{*}$ (e.g. “man : woman = king : queen"); where $b^{*}$ is the word to be guessed (“queen"), $b$ is the word coupled to it (“king"), $a$ is the word for the components to be eliminated (“man"), and $a^{*}$ is the word for the components to be added (“woman"). 
 Question: Are the word embeddings tested on a NLP task?
Output: Yes-no
For answer retrieval, a dataset is created by INLINEFORM4 , which gives INLINEFORM5 accuracy and INLINEFORM6 coverage, respectively. 
 Question: Do they employ their indexing-based method to create a sample of a QA Wikipedia dataset?
Output: Yes-no
We quantify the use of these strategies by comparing the airtime debaters devote to talking points . For a side INLINEFORM0 , let the self-coverage INLINEFORM1 be the fraction of content words uttered by INLINEFORM2 in round INLINEFORM3 that are among their own talking points INLINEFORM4 ; and the opponent-coverage INLINEFORM5 be the fraction of its content words covering opposing talking points INLINEFORM6 .  We use all conversational features discussed above. For each side INLINEFORM0 we include INLINEFORM1 , INLINEFORM2 , and their sum. We also use the drop in self-coverage given by subtracting corresponding values for INLINEFORM3 , and the number of discussion points adopted by each side. 
 Question: what aspects of conversation flow do they look at?
Output: Abstractive
Because of this, we do not claim that this dataset can be considered a ground truth. 
 Question: How is the ground truth for fake news established?
Output: Abstractive
In this section, we consider two kinds of relational classification tasks: (1) relation prediction and (2) relation extraction.  We hope to design a simple and clear experiment setup to conduct error analysis for relational prediction. Therefore, we consider a typical method TransE BIBREF3 as the subject as well as FB15K BIBREF3 as the dataset. For relation extraction, we consider the supervised relation extraction setting and TACRED dataset BIBREF10 . As for the subject model, we use the best model on TACRED dataset — position-aware neural sequence model.  
 Question: Which competitive relational classification models do they test?
Output: Abstractive
CNN can also be employed on the sarcasm datasets in order to identify sarcastic and non-sarcastic tweets. We term the features extracted from this network baseline features, the method as baseline method and the CNN architecture used in this baseline method as baseline CNN. Since the fully-connected layer has 100 neurons, we have 100 baseline features in our experiment.  
 Question: What are the network's baseline features?
Output: Abstractive
Experimental results show that our joint model outperforms the visual-only model in all cases, and the text-only model on Wikipedia and two subsets of arXiv. 
 Question: Do the methods that work best on academic papers also work best on Wikipedia?
Output: Yes-no
Human Judgments
Following BIBREF11 , BIBREF12 and the vast amount of previous work on semantic similarity, we ask nine undergraduate subjects to assess the similarity of 360 pairs of relations from a subset of Wikidata BIBREF8 that are chosen to cover from high to low levels of similarity. In our experiment, subjects were asked to rate an integer similarity score from 0 (no similarity) to 4 (perfectly the same) for each pair.  
 Question: How do they gather human judgements for similarity between relations?
Output: Abstractive
Baselines. We compare our approach (FacTweet) to the following set of baselines: LR + Bag-of-words: We aggregate the tweets of a feed and we use a bag-of-words representation with a logistic regression (LR) classifier.

Tweet2vec: We use the Bidirectional Gated recurrent neural network model proposed in BIBREF20. We keep the default parameters that were provided with the implementation. To represent the tweets, we use the decoded embedding produced by the model. With this baseline we aim at assessing if the tweets' hashtags may help detecting the non-factual accounts.

LR + All Features (tweet-level): We extract all our features from each tweet and feed them into a LR classifier. Here, we do not aggregate over tweets and thus view each tweet independently.

LR + All Features (chunk-level): We concatenate the features' vectors of the tweets in a chunk and feed them into a LR classifier.

FacTweet (tweet-level): Similar to the FacTweet approach, but at tweet-level; the sequential flow of the tweets is not utilized. We aim at investigating the importance of the sequential flow of tweets.

Top-$k$ replies, likes, or re-tweets: Some approaches in rumors detection use the number of replies, likes, and re-tweets to detect rumors BIBREF21. Thus, we extract top $k$ replied, liked or re-tweeted tweets from each account to assess the accounts factuality. We tested different $k$ values between 10 tweets to the max number of tweets from each account. Figure FIGREF24 shows the macro-F1 values for different $k$ values. It seems that $k=500$ for the top replied tweets achieves the highest result. Therefore, we consider this as a baseline. 
 Question: What baselines were used in this work?
Output: Extractive
In terms of this, rather than starting from Twitter API Search, we look into the archived snapshots of two major news websites (CNN, NBC), and then extract the tweet blocks that are embedded in the news articles. 
 Question: How do they determine if tweets have been used by journalists?
Output: Extractive
Figure FIGREF5 shows that E-BERT performs comparable to BERT and ERNIE on unfiltered LAMA. 
 Question: Which of the two ensembles yields the best performance?
Output: Abstractive
The resulting sequence of vectors is encoded using an LSTM encoder.  
 Question: What architecture does the encoder have?
Output: Extractive
We began with evaluating standard MT paradigms, i.e., PBSMT BIBREF3 and NMT BIBREF1 . As for PBSMT, we also examined two advanced methods: pivot-based translation relying on a helping language BIBREF10 and induction of phrase tables from monolingual data BIBREF14 .

As for NMT, we compared two types of encoder-decoder architectures: attentional RNN-based model (RNMT) BIBREF2 and the Transformer model BIBREF18 . In addition to standard uni-directional modeling, to cope with the low-resource problem, we examined two multi-directional models: bi-directional model BIBREF11 and multi-to-multi (M2M) model BIBREF8 .

After identifying the best model, we also examined the usefulness of a data augmentation method based on back-translation BIBREF17 . 
 Question: what was the baseline?
Output: Extractive
Following BIBREF11 , we experiment on the non-anonymized version of . When generating summaries, we follow standard practice of tuning the maximum output length and disallow repeating the same trigram BIBREF27 , BIBREF14 . For this task we train language model representations on the combination of newscrawl and the training data. Table TABREF16 shows that pre-trained embeddings can significantly improve on top of a strong baseline transformer. We also compare to BIBREF26 who use a task-specific architecture compared to our generic sequence to sequence baseline. Pre-trained representations are complementary to their method. 
 Question: What other models do they compare to?
Output: Extractive
When we include the LM embeddings in our system overall performance increases from 90.87% to 91.93% INLINEFORM0 for the CoNLL 2003 NER task, a more then 1% absolute F1 increase, and a substantial improvement over the previous state of the art. We also establish a new state of the art result (96.37% INLINEFORM1 ) for the CoNLL 2000 Chunking task. 
 Question: what results do they achieve?
Output: Abstractive
Finally, the decoder task, which predicts the target sequence probability at time INLINEFORM3 based on previous output and context information INLINEFORM4 , can be formulated as: DISPLAYFORM0 
 Question: How does their decoder generate text?
Output: Extractive
Additionally, we also test their performance only on the items from the Zurich and Visp dialect, because most of the samples are from this two dialects. In Table TABREF15 we show the result of the comparison of the two models. 
 Question: Is the model evaluated on the graphemes-to-phonemes task?
Output: Yes-no
In contrast, our models are trained only on the 76K questions in the training set. 
 Question: Do the authors also try the model on other datasets?
Output: Yes-no
The final code-mixed tweets were forwarded to a group of three annotators who were university students and fluent in both English and Hindi. 
 Question: How many annotators tagged each text?
Output: Extractive
Features. We argue that different kinds of features like the sentiment of the text, morality, and other text-based features are critical to detect the nonfactual Twitter accounts by utilizing their occurrence during reporting the news in an account's timeline. We employ a rich set of features borrowed from previous works in fake news, bias, and rumors detection BIBREF0, BIBREF1, BIBREF8, BIBREF9.

[leftmargin=4mm]

Emotion: We build an emotions vector using word occurrences of 15 emotion types from two available emotional lexicons. We use the NRC lexicon BIBREF10, which contains $\sim $14K words labeled using the eight Plutchik's emotions BIBREF11. The other lexicon is SentiSense BIBREF12 which is a concept-based affective lexicon that attaches emotional meanings to concepts from the WordNet lexical database. It has $\sim $5.5 words labeled with emotions from a set of 14 emotional categories We use the categories that do not exist in the NRC lexicon.

Sentiment: We extract the sentiment of the tweets by employing EffectWordNet BIBREF13, SenticNet BIBREF14, NRC BIBREF10, and subj_lexicon BIBREF15, where each has the two sentiment classes, positive and negative.

Morality: Features based on morality foundation theory BIBREF16 where words are labeled in one of the following 10 categories (care, harm, fairness, cheating, loyalty, betrayal, authority, subversion, sanctity, and degradation).

Style: We use canonical stylistic features, such as the count of question marks, exclamation marks, consecutive characters and letters, links, hashtags, users' mentions. In addition, we extract the uppercase ratio and the tweet length.

Words embeddings: We extract words embeddings of the words of the tweet using $Glove\-840B-300d$ BIBREF17 pretrained model. The tweet final representation is obtained by averaging its words embeddings. 
 Question: What features are extracted?
Output: Extractive
All documents are segmented into paragraphs and processed at the paragraph level (both training and inference); this is acceptable because we observe that most paragraphs are less than 200 characters. The input sequences are segmented by the BERT tokenizer, with the special [CLS] token inserted at the beginning and the special [SEP] token added at the end. 
 Question: Was the structure of regulatory filings exploited when training the model? 
Output: Yes-no
Knowledge Base Question Answering (KBQA) systems answer questions by obtaining information from KB tuples BIBREF0 , BIBREF1 , BIBREF2 , BIBREF3 , BIBREF4 , BIBREF5 
 Question: What does KBQA abbreviate for
Output: Extractive
We explore an architecture based on a stack of dilated convolution layers, effectively operating on a broader scale than with standard convolutions while limiting model size. Standard convolutional networks cannot capture long temporal patterns with reasonably small models due to the increase in computational cost yielded by larger receptive fields. Dilated convolutions skip some input values so that the convolution kernel is applied over a larger area than its own. The network therefore operates on a larger scale, without the downside of increasing the number of parameters. 
 Question: What are dilated convolutions?
Output: Abstractive
PERPLEXITY 
 Question: Does the paper consider the use of perplexity in order to identify text anomalies?
Output: Yes-no
For every gendered character in the dataset, we ask annotators to create a new character with a persona of the opposite gender that is otherwise identical except for referring nouns or pronouns. 
 Question: In the targeted data collection approach, what type of data is targetted?
Output: Abstractive
Finally, when investigating the relatedness between European vs. non European languages (cf. (En/Fr)$\rightarrow $Ar), we obtain similar results than those obtained in the monolingual experiment (macro F-score 62.4 vs. 68.0) and best results are achieved by Ar $\rightarrow $(En/Fr). This shows that there are pragmatic devices in common between both sides and, in a similar way, similar text-based patterns in the narrative way of the ironic tweets. 
 Question: Do the authors identify any cultural differences in irony use?
Output: Yes-no
Although our approach strongly outperforms random baselines, the relatively low F1 scores indicate that predicting which word is echoed in explanations is a very challenging task. It follows that we are only able to derive a limited understanding of how people choose to echo words in explanations. The extent to which explanation construction is fundamentally random BIBREF47, or whether there exist other unidentified patterns, is of course an open question. We hope that our study and the resources that we release encourage further work in understanding the pragmatics of explanations. 
 Question: Do authors provide any explanation for intriguing patterns of word being echoed?
Output: Yes-no
An important feature when suggesting an article INLINEFORM0 to an entity INLINEFORM1 is the novelty of INLINEFORM2 w.r.t the already existing entity profile INLINEFORM3 Given an entity INLINEFORM0 and the already added news references INLINEFORM1 up to year INLINEFORM2 , the novelty of INLINEFORM3 at year INLINEFORM4 is measured by the KL divergence between the language model of INLINEFORM5 and articles in INLINEFORM6 . We combine this measure with the entity overlap of INLINEFORM7 and INLINEFORM8 . The novelty value of INLINEFORM9 is given by the minimal divergence value. Low scores indicate low novelty for the entity profile INLINEFORM10 .

 
 Question: What features are used to represent the novelty of news articles to entity pages?
Output: Abstractive
Motivated by this, we introduce resolution mode variables $\Pi = \lbrace \pi _1, \ldots , \pi _n\rbrace $ , where for each mention $j$ the variable $\pi _j \in \lbrace str, prec, attr\rbrace $ indicates in which mode the mention should be resolved. In our model, we define three resolution modes — string-matching (str), precise-construct (prec), and attribute-matching (attr) — and $\Pi $ is deterministic when $D$ is given (i.e. $P(\Pi |D)$ is a point distribution). We determine $\pi _j$ for each mention $m_j$ in the following way:

$\pi _j = str$ , if there exists a mention $m_i, i < j$ such that the two mentions satisfy the String Match sieve, the Relaxed String Match sieve, or the Strict Head Match A sieve in the Stanford multi-sieve system BIBREF1 .

$\pi _j = prec$ , if there exists a mention $m_i, i < j$ such that the two mentions satisfy the Speaker Identification sieve, or the Precise Constructs sieve.

$\pi _j = attr$ , if there is no mention $m_i, i < j$ satisfies the above two conditions. 
 Question: Are resolution mode variables hand crafted?
Output: Yes-no
We crowdsourced multiple-choice questions in two parts, encouraging workers to be imaginative and varied in their use of language. First, workers were given a seed qualitative relation q+/-( INLINEFORM0 ) in the domain, expressed in English (e.g., “If a surface has more friction, then an object will travel slower”), and asked to enter two objects, people, or situations to compare. 
 Question: Do all questions in the dataset allow the answers to pick from 2 options?
Output: Yes-no
We have implemented the following interfaces for Macaw:

[leftmargin=*]

File IO: This interface is designed for experimental purposes, such as evaluating the performance of a conversational search technique on a dataset with multiple queries. This is not an interactive interface.

Standard IO: This interactive command line interface is designed for development purposes to interact with the system, see the logs, and debug or improve the system.

Telegram: This interactive interface is designed for interaction with real users (see FIGREF4). Telegram is a popular instant messaging service whose client-side code is open-source. We have implemented a Telegram bot that can be used with different devices (personal computers, tablets, and mobile phones) and different operating systems (Android, iOS, Linux, Mac OS, and Windows). This interface allows multi-modal interactions (text, speech, click, image). It can be also used for speech-only interactions. For speech recognition and generation, Macaw relies on online APIs, e.g., the services provided by Google Cloud and Microsoft Azure. In addition, there exist multiple popular groups and channels in Telegram, which allows further integration of social networks with conversational systems. For example, see the Naseri and Zamani's study on news popularity in Telegram BIBREF12. 
 Question: What interface does Macaw currently have?
Output: Extractive
However, probably the most surprising result in the field is Zipf's law, which states that if one ranks words by their frequency in a large text, the resulting rank frequency distribution is approximately a power law, for all languages BIBREF0, BIBREF11. Another interesting characterization of texts is the Heaps-Herdan law, which describes how the vocabulary -that is, the set of different words- grows with the size of a text, the number of which, empirically, has been found to grow as a power of the text size BIBREF18, BIBREF19. It is worth noting that it has been argued that this law is a consequence Zipf's law. BIBREF20, BIBREF21 
 Question: How do Zipf and Herdan-Heap's laws differ?
Output: Abstractive
This step consists of generating a query out of the claim and querying a search engine (here, we experiment with Google and Bing) in order to retrieve supporting documents. Rather than querying the search engine with the full claim (as on average, a claim is two sentences long), we generate a shorter query following the lessons highlighted in BIBREF0 . We rank the words by means of tf-idf. We compute the idf values on a 2015 Wikipedia dump and the English Gigaword. BIBREF0 suggested that a good way to perform high-quality search is to only consider the verbs, the nouns and the adjectives in the claim; thus, we exclude all words in the claim that belong to other parts of speech. Moreover, claims often contain named entities (e.g., names of persons, locations, and organizations); hence, we augment the initial query with all the named entities from the claim's text. We use IBM's AlchemyAPI to identify named entities. Ultimately, we generate queries of 5-10 tokens, which we execute against a search engine. We then collect the snippets and the URLs in the results, skipping any result that points to a domain that is considered unreliable. 
 Question: How are the potentially relevant text fragments identified?
Output: Abstractive
In this paper, we use three data sets from the literature to train and evaluate our own classifier. Data collected by BIBREF3 , which we term the Sexist/Racist (SR) data set, was collected using an initial Twitter search followed by analysis and filtering by the authors and their team who identified 17 common phrases, hashtags, and users that were indicative of abusive speech. BIBREF4 collected the HATE dataset by searching for tweets using a lexicon provided by Hatebase.org. The final data set we used, which we call HAR, was collected by BIBREF9 ; we removed all retweets reducing the dataset to 20,000 tweets. Tweets were labeled as “Harrassing” or “Non-Harrassing”; hate speech was not explicitly labeled, but treated as an unlabeled subset of the broader “Harrassing” category BIBREF9 . Many of the false negatives we see are specific references to characters in the TV show “My Kitchen Rules”, rather than something about women in general.  While this may be a limitation of considering only the content of the tweet, it could also be a mislabel.

Debra are now my most hated team on #mkr after least night's ep. Snakes in the grass those two.

Along these lines, we also see correct predictions of innocuous speech, but find data mislabeled as hate speech:

@LoveAndLonging ...how is that example "sexism"?

@amberhasalamb ...in what way? 
 Question: Do they report results only on English data?
Output: Yes-no
For the MT task, we use the WMT 2014 En $\leftrightarrow $ Fr parallel corpus. The dataset contains 36 million En $\rightarrow $ Fr sentence pairs. We swapped the source and target sentences to obtain parallel data for the Fr $\rightarrow $ En translation task. We use these two datasets (72 million sentence pairs) to train a single multilingual NMT model to learn both these translation directions simultaneously.  
 Question: What data were they used to train the multilingual encoder?
Output: Abstractive
For experiments, we train and evaluate our models in the civil law system of mainland China. We collect and construct a large-scale real-world data set of INLINEFORM0 case documents that the Supreme People's Court of People's Republic of China has made publicly available 
 Question: what is the size of the real-world civil case dataset?
Output: Abstractive
Despite the greater similarity between our task and the 2013 ShARe/CLEF Task 1, we use the clinical notes from the CE task in 2010 i2b2/VA on account of 1) the data from 2010 i2b2/VA being easier to access and parse, 2) 2013 ShARe/CLEF containing disjoint entities and hence requiring more complicated tagging schemes. 
 Question: where did they obtain the annotated clinical notes from?
Output: Extractive
The attention function BIBREF11 is used to compute the similarity score between passages and questions as: INLINEFORM2 
 Question: Do they use attention?
Output: Yes-no
Further, we employ BLEU score to evaluate the accuracy of generations BIBREF21, and the number of distinct n-gram to evaluate the diversity of generations BIBREF6.  
 Question: How do they measure the diversity of inferences?
Output: Abstractive
From the chart, it is clear that there is a higher chance of fake news coming from unverified accounts. Notice that accounts spreading viral tweets with fake news have, on average, a larger ratio of friends/followers.  Figure FIGREF24 shows that, in contrast to other kinds of viral tweets, those containing fake news were created more recently. 
 Question: What are the characteristics of the accounts that spread fake news?
Output: Abstractive
To evaluate the performance of our approach, we used a subset of the SNIPS BIBREF12 dataset, which is readily available in RASA nlu format.  We use accuracy of intent and entity recognition as our task and metric.  
 Question: How are their changes evaluated?
Output: Abstractive
The answer styles considered in the experiments corresponded to the two tasks. The NLG task requires a well-formed answer that is an abstractive summary of the question and ten passages, averaging 16.6 words. The Q&A task also requires an abstractive answer but prefers a more concise answer than the NLG task, averaging 13.1 words, where many of the answers do not contain the context of the question. 
 Question: What is an "answer style"?
Output: Abstractive
However, we found that explicitly initializing these embeddings with vectors pre-trained over a large collection of unlabeled data significantly improved performance (see Section "Effects of initialized embeddings and corrupt-sampling schemes" ). 
 Question: What is the new initialization method proposed in this paper?
Output: Abstractive
Active learning sharply increases the performance of iteratively trained machine learning models by selectively determining which unlabeled samples should be annotated.  
 Question: What is active learning?
Output: Abstractive
Comparing Eq.DISPLAY_FORM14 with Eq.DISPLAY_FORM22, we can see that Eq.DISPLAY_FORM14 is actually a soft form of $F1$, using a continuous $p$ rather than the binary $\mathbb {I}( p_{i1}>0.5)$. This gap isn't a big issue for balanced datasets, but is extremely detrimental if a big proportion of training examples are easy-negative ones: easy-negative examples can easily dominate training since their probabilities can be pushed to 0 fairly easily. Meanwhile, the model can hardly distinguish between hard-negative examples and positive ones, which has a huge negative effect on the final F1 performance.

To address this issue, we propose to multiply the soft probability $p$ with a decaying factor $(1-p)$, changing Eq.DISPLAY_FORM22 to the following form:

One can think $(1-p_{i1})$ as a weight associated with each example, which changes as training proceeds. The intuition of changing $p_{i1}$ to $(1-p_{i1}) p_{i1}$ is to push down the weight of easy examples. For easy examples whose probability are approaching 0 or 1, $(1-p_{i1}) p_{i1}$ makes the model attach significantly less focus to them. Figure FIGREF23 gives gives an explanation from the perspective in derivative: the derivative of $\frac{(1-p)p}{1+(1-p)p}$ with respect to $p$ approaches 0 immediately after $p$ approaches 0, which means the model attends less to examples once they are correctly classified.

A close look at Eq.DISPLAY_FORM14 reveals that it actually mimics the idea of focal loss (FL for short) BIBREF16 for object detection in vision. Focal loss was proposed for one-stage object detector to handle foreground-background tradeoff encountered during training. It down-weights the loss assigned to well-classified examples by adding a $(1-p)^{\beta }$ factor, leading the final loss to be $(1-p)^{\beta }\log p$. 
 Question: How are weights dynamically adjusted?
Output: Extractive
French and Russian, and Arabic can be regarded as high resource languages whereas Hindi has far less data and can be considered as low resource. 
 Question: Is the system tested on low-resource languages?
Output: Yes-no
In addition to it, we extract three hand-crafted polarity scores, which are minimum, mean, and maximum polarity scores, from each review. These polarity scores of words are computed as in (DISPLAY_FORM4). For example, if a review consists of five words, it would have five polarity scores and we utilise only three of these sentiment scores as mentioned. Lastly, we concatenate these three scores to the averaged word vector per review.

That is, each review is represented by the average word vector of its constituent word embeddings and three supervised scores. We then feed these inputs into the SVM approach. The flowchart of our framework is given in Figure FIGREF11. When combining the unsupervised features, which are word vectors created on a word basis, with supervised three scores extracted on a review basis, we have better state-of-the-art results. 
 Question: Which hand-crafted features are combined with word2vec?
Output: Extractive
It is worth mentioning that the collected texts contain a large quantity of errors of several types: orthographic, syntactic, code-switched words (i.e. words not in the required language), jokes, etc. Hence, the original written sentences have been processed in order to produce “cleaner” versions, in order to make the data usable for some research purposes (e.g. to train language models, to extract features for proficiency assessment, ...). 
 Question: Are any of the utterances ungrammatical?
Output: Yes-no
These are four conjunctively written Nguni languages (zul, xho, nbl, ssw), Afrikaans (afr) and English (eng), three disjunctively written Sotho languages (nso, sot, tsn), as well as tshiVenda (ven) and Xitsonga (tso). The Nguni languages are similar to each other and harder to distinguish. The same is true of the Sotho languages. Similar languages are to each other are:
- Nguni languages: zul, xho, nbl, ssw
- Sotho languages: nso, sot, tsn 
 Question: Which languages are similar to each other?
Output: Extractive
We collected the dataset with 1900 dialogs and 8533 turns.  
 Question: What is the average length of dialog?
Output: Abstractive
The final result submitted in public leaderboard is 0.73019 and in private leaderboard is 0.58455. It is quite different in bad way. That maybe is the result of the model too overfit on train set tuning on public test set. 
 Question: What is private dashboard?
Output: Abstractive
In Section SECREF5 , we discuss the resulting data collection of questions, texts and answers via crowdsourcing on Amazon Mechanical Turk (henceforth MTurk). 
 Question: what crowdsourcing platform was used?
Output: Extractive
Figure FIGREF29 shows the $\text{MPAs}$ of the proposed DP-LSTM and vanilla LSTM for comparison.  
 Question: Is the model compared against a linear regression baseline?
Output: Yes-no
The current state-of-the-art approach BIBREF14 , BIBREF15 uses maximum entropy and CRF models with a combination of language model and hand-crafted features to predict if each character in the hashtag is the beginning of a new word. 
 Question: What current state of the art method was used for comparison?
Output: Extractive
Using the LDA model, each person in the dataset is with a topic probability vector $X_i$ . Assume $x_{ik}\in X_{i}$ denotes the likelihood that the $\emph {i}^{th}$ tweet account favors $\emph {k}^{th}$ topic in the dataset. Our topic based features can be calculated as below. Global Outlier Standard Score measures the degree that a user's tweet content is related to a certain topic compared to the other users. Local Outlier Standard Score measures the degree of interest someone shows to a certain topic by considering his own homepage content only. Three baseline classification methods: Support Vector Machines (SVM), Adaboost, and Random Forests are adopted to evaluate our extracted features. 
 Question: How do they detect spammers?
Output: Abstractive
The corpus consists of 53 documents, which contain an average number of 156.1 sentences per document, each with 19.55 tokens on average. 
 Question: How many documents are in the new corpus?
Output: Extractive
The dataset needed to train the W2V was obtained using the information extracted from a dump of the Italian Wikipedia (dated 2019.04.01), from the main categories of Italian Google News (WORLD, NATION, BUSINESS, TECHNOLOGY, ENTERTAINMENT, SPORTS, SCIENCE, HEALTH) and from some anonymized chats between users and a customer care chatbot (Laila). The dataset (composed of 2.6 GB of raw text) includes $421\,829\,960$ words divided into $17\,305\,401$ sentences. All the words were then transformed to lowercase (to avoid a double presence) finally producing a vocabulary of $618\,224$ words. 
 Question: What is the dataset used as input to the Word2Vec algorithm?
Output: Abstractive
In this paper, we introduce a novel policy model to output multiple actions per turn (called multi-act), generating a sequence of tuples and expanding agents' expressive power. Each tuple is defined as $(\textit {continue}, \textit {act}, \textit {slots})$, where continue indicates whether to continue or stop producing new acts, act is an act type (e.g., inform or request), and slots is a set of slots (names) associated with the current act type. Correspondingly, a novel decoder (Figure FIGREF5) is proposed to produce such sequences. Each tuple is generated by a cell called gated Continue Act Slots (gCAS, as in Figure FIGREF7), which is composed of three sequentially connected gated units handling the three components of the tuple. This decoder can generate multi-acts in a double recurrent manner BIBREF18.  
 Question: What is specific to gCAS cell?
Output: Abstractive
This paper explores the potential of predicting a user's industry -the aggregate of enterprises in a particular field- by identifying industry indicative text in social media.  
 Question: What do they mean by a person's industry?
Output: Extractive
We compare the above model to a similar model, where rather than explicitly represent $K$ features as input, we have $K$ features in the form of a genre embedding, i.e. we learn a genre specific embedding for each of the gothic, scifi, and philosophy genres, as studied in BIBREF8 and BIBREF7. 
 Question: Is this style generator compared to some baseline?
Output: Yes-no
It is true that the above-mentioned associated caption sentences for each concept-set are human-written and do describe scenes that cover all given concepts. However, they are created under specific contexts (i.e. an image or a video) and thus might be less representative for common sense. To better measure the quality and interpretability of generative reasoners, we need to evaluate them with scenes and rationales created by using concept-sets only as the signals for annotators.

We collect more human-written scenes for each concept-set in dev and test set through crowd-sourcing via the Amazon Mechanical Turk platform. Each input concept-set is annotated by at least three different humans. The annotators are also required to give sentences as the rationales, which further encourage them to use common sense in creating their scenes. 
 Question: Are the sentences in the dataset written by humans who were shown the concept-sets?
Output: Yes-no
In this study we use transcripts and results of Oxford-style debates from the public debate series “Intelligence Squared Debates” (IQ2 for short). 
 Question: what debates dataset was used?
Output: Extractive
Since it is infeasible to manually annotate all of the comments, we process this dataset with the goal of extracting threads that involve suspected trolling attempts and the direct responses to them.  We had two human annotators who were trained on snippets (i.e., (suspected trolling attempt, responses) pairs) taken from 200 conversations and were allowed to discuss their findings. After this training stage, we asked them to independently label the four aspects for each snippet.  
 Question: Do they use a crowdsourcing platform for annotation?
Output: Yes-no
To test this hypothesis we first trained UG-WGAN in English, Chinese and German following the procedure described in Section "UG-WGAN" . For this experiment we trained UG-WGAN on the English and Russian language following the procedure described in Section "UG-WGAN" .  To do so we trained 6 UG-WGAN models on the following languages: English, Russian, Arabic, Chinese, German, Spanish, French. 
 Question: What are the languages they consider in this paper?
Output: Abstractive
The Wikipedia dataset consists of articles from English Wikipedia, with quality class labels assigned by the Wikipedia community. 
 Question: Which languages do they use?
Output: Extractive
We evaluated SDNet on CoQA dataset 
 Question: Is the model evaluated on other datasets?
Output: Yes-no
Hashtag prediction for social media has been addressed earlier, for example in BIBREF15 , BIBREF16 . BIBREF15 also use a neural architecture, but compose text embeddings from a lookup table of words. 
 Question: Is this hashtag prediction task an established task, or something new?
Output: Abstractive
Task 1: Quora Duplicate Question Pair Detection Task 2: Ranking questions in Bing's People Also Ask 
 Question: On which tasks do they test their conflict method?
Output: Extractive
If we take the best psr ensemble trained on CBT as a baseline, improving the model architecture as in BIBREF6 , BIBREF7 , BIBREF8 , BIBREF9 , BIBREF10 , BIBREF11 , continuing to use the original CBT training data, lead to improvements of INLINEFORM0 and INLINEFORM1 absolute on named entities and common nouns respectively. By contrast, inflating the training dataset provided a boost of INLINEFORM2 while using the same model. 
 Question: How large are the improvements of the Attention-Sum Reader model when using the BookTest dataset?
Output: Extractive
Our system significantly out-performed the baseline on the joint task, as the baseline scored significantly lower for the gender task than for the variety task. 
 Question: On which task does do model do best?
Output: Abstractive
We would like to thank the organizers of the WASSA-2017 Shared Task on Emotion Intensity, for providing the data, the guidelines and timely support. 
 Question: what dataset was used?
Output: Extractive
We used a dataset provided by Shortir, an Indonesian news aggregator and summarizer company. The dataset contains roughly 20K news articles. Each article has the title, category, source (e.g., CNN Indonesia, Kumparan), URL to the original article, and an abstractive summary which was created manually by a total of 2 native speakers of Indonesian 
 Question: Did they use a crowdsourcing platform for the summaries?
Output: Yes-no
Baseline Experiments
In this section, we briefly describe a baseline and evaluation scripts that we release, with a detailed documentation, along with the corpus. 
 Question: What baseline system is proposed?
Output: Abstractive
In the second round, we collected 293 annotations from 12 annotators. After Korektor, there are 4262 unique sentences (including 150 seed sentences) that form the COSTRA 1.0 dataset. 
 Question: How are possible sentence transformations represented in dataset, as new sentences?
Output: Abstractive
In terms of speed, our system was able to lemmatize 7.4 million words on a personal laptop in almost 2 minutes compared to 2.5 hours for MADAMIRA, i.e. 75 times faster.  
 Question: How was speed measured?
Output: Abstractive
Since the present study focuses on Spanish tweets, all tweets from the English datasets were translated into Spanish.  
 Question: What other languages did they translate the data from?
Output: Extractive
Amazon Reviews Dataset BIBREF24 is a large dataset with millions of reviews from different product categories. For our experiments, we consider a subset of 20000 reviews from the domains Cell Phones and Accessories(C), Clothing and Shoes(S), Home and Kitchen(H) and Tools and Home Improvement(T). Out of 20000 reviews, 10000 are positive and 10000 are negative. We use 12800 reviews for training, 3200 reviews for validation and 4000 reviews for testing from each domain. 
 Question: For the purposes of this paper, how is something determined to be domain specific knowledge?
Output: Abstractive
Specifically, Roget's Thesaurus BIBREF38 , BIBREF39 is used to derive the concepts and concept word-groups to be used as the external lexical resource for our proposed method. By using the concept word-groups, we have trained the GloVe algorithm with the proposed modification given in Section SECREF4 on a snapshot of English Wikipedia measuring 8GB in size, with the stop-words filtered out. 
 Question: Do they report results only on English data?
Output: Yes-no
Our model encodes the information from audio and text sequences using dual RNNs and then combines the information from these sources using a feed-forward neural model to predict the emotion class. 
 Question: How do they combine audio and text sequences in their RNN?
Output: Extractive