niryuu/sni-each-converted huggingface dataset:
Construction of the auxiliary sentence
For simplicity, we mainly describe our method with TABSA as an example.

We consider the following four methods to convert the TABSA task into a sentence pair classification task:

The sentence we want to generate from the target-aspect pair is a question, and the format needs to be the same. For example, for the set of a target-aspect pair (LOCATION1, safety), the sentence we generate is “what do you think of the safety of location - 1 ?"

For the NLI task, the conditions we set when generating sentences are less strict, and the form is much simpler. The sentence created at this time is not a standard sentence, but a simple pseudo-sentence, with (LOCATION1, safety) pair as an example: the auxiliary sentence is: “location - 1 - safety".

For QA-B, we add the label information and temporarily convert TABSA into a binary classification problem ( INLINEFORM0 ) to obtain the probability distribution. At this time, each target-aspect pair will generate three sequences such as “the polarity of the aspect safety of location - 1 is positive", “the polarity of the aspect safety of location - 1 is negative", “the polarity of the aspect safety of location - 1 is none". We use the probability value of INLINEFORM1 as the matching score. For a target-aspect pair which generates three sequences ( INLINEFORM2 ), we take the class of the sequence with the highest matching score for the predicted category.

The difference between NLI-B and QA-B is that the auxiliary sentence changes from a question to a pseudo-sentence. The auxiliary sentences are: “location - 1 - safety - positive", “location - 1 - safety - negative", and “location - 1 - safety - none".

After we construct the auxiliary sentence, we can transform the TABSA task from a single sentence classification task to a sentence pair classification task. As shown in Table TABREF19 , this is a necessary operation that can significantly improve the experimental results of the TABSA task. 
 Question: How do they generate the auxiliary sentence?
Output: The sentence we want to generate from the target-aspect pair is a question, and the format needs to be the same. For the NLI task, the conditions we set when generating sentences are less strict, and the form is much simpler. For QA-B, we add the label information and temporarily convert TABSA into a binary classification problem ( INLINEFORM0 ) to obtain the probability distribution auxiliary sentence changes from a question to a pseudo-sentence
We try 3 variants of each training set to fine-tune our models: (i) the original one in English (Orig), (ii) an English paraphrase of it generated through back-translation using Spanish or Finnish as pivot (BT-ES and BT-FI), and (iii) a machine translated version in Spanish or Finnish (MT-ES and MT-FI). 
 Question: What languages do they use in their experiments?
Output: English Spanish Finnish
However, probably the most surprising result in the field is Zipf's law, which states that if one ranks words by their frequency in a large text, the resulting rank frequency distribution is approximately a power law, for all languages BIBREF0, BIBREF11. Another interesting characterization of texts is the Heaps-Herdan law, which describes how the vocabulary -that is, the set of different words- grows with the size of a text, the number of which, empirically, has been found to grow as a power of the text size BIBREF18, BIBREF19. It is worth noting that it has been argued that this law is a consequence Zipf's law. BIBREF20, BIBREF21 
 Question: How do Zipf and Herdan-Heap's laws differ?
Output: Zipf's law describes change of word frequency rate, while Heaps-Herdan describes different word number in large texts (assumed that Hepas-Herdan is consequence of Zipf's)
Results and Analysis 
 Question: what quantitative analysis is done?
Output: Answer with content missing: (Evaluation section) Given that in CLIR the primary goal is to get a better ranked list of documents against a translated query, we only report Mean Average Precision (MAP).
Using an encoder-decoder backbone, our model may be regarded as an extension of the constituent parsing model of BIBREF18 as shown in Figure FIGREF4. The difference is that in our model both constituent and dependency parsing share the same token representation and shared self-attention layers and each has its own individual Self-Attention Layers and subsequent processing layers. Our model includes four modules: token representation, self-attention encoder, constituent and dependency parsing decoder. Constituent Parsing Decoder Dependency Parsing Decoder 
 Question: What are the models used to perform constituency and dependency parsing?
Output: token representation self-attention encoder, Constituent Parsing Decoder  Dependency Parsing Decoder
In this experiment set, our BERT implementation is compared to several systems that participated in the MEDDOCAN challenge: a CRF classifier BIBREF18, a spaCy entity recogniser BIBREF18, and NLNDE BIBREF12, the winner of the shared task and current state of the art for sensitive information detection and classification in Spanish clinical text.  However, attending to the obtained results, BERT remains only 0.3 F1-score points behind, and would have achieved the second position among all the MEDDOCAN shared task competitors. Taking into account that only 3% of the gold labels remain incorrectly annotated, the task can be considered almost solved, and it is not clear if the differences among the systems are actually significant, or whether they stem from minor variations in initialisation or a long-tail of minor labelling inconsistencies. 
 Question: Does BERT reach the best performance among all the algorithms compared?
Output: No
The seq2seq model with global attention gives the best results with an average target BLEU score of 29.65 on the style transfer dataset, compared with an average target BLEU score of 26.97 using the seq2seq model with pointer networks. 
 Question: What is best BLEU score of language style transfer authors got?
Output: seq2seq model with global attention gives the best results with an average target BLEU score of 29.65
The annotator carried out all annotation. 
 Question: How many annotators tagged each tweet?
Output: One
We find that BIBREF27 zhao2018unsupervised make use of a set of discrete variables that define high-level attributes of a response. Although they interpret meanings of the learned discrete latent variables by clustering data according to certain classes (e.g. dialog acts), such latent variables still have no exact meanings. In our model, we connect each latent variable with a word in the vocabulary, thus each latent variable has an exact semantic meaning. Besides, they focus on multi-turn dialogue generation and presented an unsupervised discrete sentence representation learning method learned from the context while our concentration is primarily on single-turn dialogue generation with no context information. 
 Question: How does discrete latent variable has an explicit semantic meaning to improve the CVAE on short-text conversation?
Output: we connect each latent variable with a word in the vocabulary, thus each latent variable has an exact semantic meaning.
Long-short Term Hybrid Memory and Multi-attention Block. Long-short Term Hybrid Memory (LSTHM) is an extension of the Long-short Term Memory (LSTM) by reformulating the memory component to carry hybrid information.  In the LSTHM model, we seek to build a memory mechanism for each modality which in addition to storing view-specific dynamics, is also able to store the cross-view dynamics that are important for that modality. This allows the memory to function in a hybrid manner. 
 Question: What is the difference between Long-short Term Hybrid Memory and LSTMs?
Output: Long-short Term Hybrid Memory (LSTHM) is an extension of the Long-short Term Memory (LSTM)
We used the public tool, word2vec, released by Mikolov-2013 to obtain the word embeddings. Their neural network approach is similar to the feed-forward neural networks BIBREF5 , BIBREF6 . In the Skip-gram model architecture we used, we have chosen 200 as the dimension of the obtained word vectors. 
 Question: What type and size of word embeddings were used?
Output: word2vec 200 as the dimension of the obtained word vectors
Finally, a comparison was made between the Skip-gram model W10N20 obtained at the 50th epoch and the other two W2V in Italian present in the literature (BIBREF9 and BIBREF10). The first test (Table TABREF15) was performed considering all the analogies present, and therefore evaluating as an error any analogy that was not executable (as it related to one or more words absent from the vocabulary).

As it can be seen, regardless of the metric used, our model has significantly better results than the other two models, both overall and within the two macro-areas. 
 Question: Are the word embeddings evaluated?
Output: Yes
Spelling correction tests described in literature have tended to focus on one approach applied to a specific corpus. Limited examples include works on spellchecking mammography reports and tweets BIBREF7 , BIBREF4 . These works emphasized the importance of tailoring correction systems to specific problems of corpora they are applied to. For example, mammography reports suffer from poor typing, which in this case is a repetitive work done in relative hurry. Tweets, on the other hand, tend to contain emoticons and neologisms that can trick solutions based on rules and dictionaries, such as LanguageTool. The latter is, by itself, fairly well suited for Polish texts, since a number of extensions to the structure of this application was inspired by problems with morphology of Polish language BIBREF3 . 
 Question: Which specific error correction solutions have been proposed for specialized corpora in the past?
Output: spellchecking mammography reports and tweets BIBREF7 , BIBREF4
A data collection process involving location neutral keywords used by gang members, with an expanded search of their retweet, friends and follower networks, led to identifying 400 authentic gang member profiles on Twitter. Our study discovered that the text in their tweets and profile descriptions, their emoji use, their profile images, and music interests embodied by links to YouTube music videos, can help a classifier distinguish between gang and non-gang member profiles. 
 Question: How is the ground truth of gang membership established in this dataset?
Output:  text in their tweets and profile descriptions, their emoji use, their profile images, and music interests embodied by links to YouTube music videos, can help a classifier distinguish between gang and non-gang member profiles
The model consists of one layer of LSTM followed by three dense layers. The LSTM layer uses a dropout value of 0.2.  
 Question: Do they use dropout?
Output: Yes
We show that by determining and integrating heterogeneous set of features from different modalities – aesthetic features from posted images (colorfulness, hue variance, sharpness, brightness, blurriness, naturalness), choice of profile picture (for gender, age, and facial expression), the screen name, the language features from both textual content and profile's description (n-gram, emotion, sentiment), and finally sociability from ego-network, and user engagement – we can reliably detect likely depressed individuals in a data set of 8,770 human-annotated Twitter users. 
 Question: What is the source of the textual data? 
Output: Users' tweets
Across all 26 datasets, RCRN outperforms not only standard BiLSTMs but also 3L-BiLSTMs which have approximately equal parameterization. 
 Question: Does their model have more parameters than other models?
Output: approximately equal parameterization
is the standard word counting method whereby the feature vector represents the term frequency of the words in a sentence. is similar to BoW, except that it is derived by the counting of the words in the sentence weighted by individual word's term-frequency and inverse-document-frequency BIBREF31 . is derived from clustering of words-embeddings with k-means into 5000 clusters, and follow by BoW representation of the words in 5000 clusters. 
 Question: What other non-neural baselines do the authors compare to? 
Output: bag of words, tf-idf, bag-of-means
Our Reddit data was gathered using Reddit's public API, collecting the most recent jokes. Every time the scraper ran, it also updated the upvote score of the previously gathered jokes. This data collection occurred every hour through the months of March and April 2019. Since the data was already split into body and punchline sections from Reddit, we created separate datasets containing the body of the joke exclusively and the punchline of the joke exclusively. Additionally, we created a dataset that combined the body and punchline together.

Some sample jokes are shown in Table 1, above. The distribution of joke scores varies wildly, ranging from 0 to 136,354 upvotes. We found that there is a major jump between the 0-200 upvote range and the 200 range and onwards, with only 6% of jokes scoring between 200-20,000. We used this natural divide as the cutoff to decide what qualified as a funny joke, giving us 13884 not-funny jokes and 2025 funny jokes. 
 Question: How they evaluate if joke is humorous or not?
Output: The distribution of joke scores varies wildly, ranging from 0 to 136,354 upvotes. We found that there is a major jump between the 0-200 upvote range and the 200 range and onwards, with only 6% of jokes scoring between 200-20,000. We used this natural divide as the cutoff to decide what qualified as a funny joke, giving us 13884 not-funny jokes and 2025 funny jokes.
Finally, the morphosyntactic annotation was performed automatically, which may lead to errors, especially in the case of noisy web text. 
 Question: Did they use a crowdsourcing platform for annotations?
Output: No
A document $d$ can be defined as a finite sequence of terms (independent textual entities within a document, for example, words), namely $d=(t_1,t_2,\dots ,t_n)$. 
 Question: What representations for textual documents do they use?
Output: finite sequence of terms
Thus we consider the task of answer generation for TweetQA and we use several standard metrics for natural language generation to evaluate QA systems on our dataset, namely we consider BLEU-1 BIBREF16 , Meteor BIBREF17 and Rouge-L BIBREF18 in this paper. 
 Question: What evaluation metrics do they use?
Output: BLEU-1 Meteor  Rouge-L
Axelrod's causal mapping method comprises a set of conventions to graphically represent networks of causes and effects (the nodes in a network) as well as the qualitative aspects of this relation (the network’s directed edges, notably assertions of whether the causal linkage is positive or negative). 
 Question: What are the causal mapping methods employed?
Output: Axelrod's causal mapping method
While a host of input modalities have been considered in other NLG tasks, such as text summarization BIBREF24 , image captioning BIBREF25 and table-to-text generation BIBREF26 , traditional QG mainly focused on textual inputs, especially declarative sentences, explained by the original application domains of question answering and education, which also typically featured textual inputs.

Recently, with the growth of various QA applications such as Knowledge Base Question Answering (KBQA) BIBREF27 and Visual Question Answering (VQA) BIBREF28 , NQG research has also widened the spectrum of sources to include knowledge bases BIBREF29 and images BIBREF10 . 
 Question: What are all the input modalities considered in prior work in question generation?
Output: Textual inputs, knowledge bases, and images.
The selection model selects the best answer from the set $\lbrace a_i\rbrace _{i=1}^N$ observed during the interaction by predicting the difference of the F1 score to the average F1 of all variants. We train another neural network to pick the best answer from the candidates. We frame the task as binary classification, distinguishing between above and below average performance. In training, we compute the F1 score of the answer for every instance. If the rewrite produces an answer with an F1 score greater than the average score of the other rewrites the instance is assigned a positive label. 
 Question: how are multiple answers from multiple reformulated questions aggregated?
Output: The selection model selects the best answer from the set $\lbrace a_i\rbrace _{i=1}^N$ observed during the interaction by predicting the difference of the F1 score to the average F1 of all variants.
Pre-trained English NER model We construct the English NER system following BIBREF7 . This system uses a bidirectional LSTM as a character-level language model to take context information for word embedding generation.  For pre-trained English NER system, we use the default NER model of Flair. 
 Question: Which pre-trained English NER model do they use?
Output: Bidirectional LSTM based NER model of Flair
In addition to the traditional VAE structure, we introduces an extra context-aware latent variable in CWVAE to learn the event background knowledge. In the pretrain stage, CWVAE is trained on an auxiliary dataset (consists of three narrative story corpora and contains rich event background knowledge), to learn the event background information by using the context-aware latent variable. Subsequently, in the finetune stage, CWVAE is trained on the task-specific dataset to adapt the event background information to each specific aspect of If-Then inferential target (e.g., intents, reactions, etc.). 
 Question: How does the context-aware variational autoencoder learn event background information?
Output:  CWVAE is trained on an auxiliary dataset to learn the event background information by using the context-aware latent variable.  Then, in finetute stage, CWVAE is trained on the task-specific dataset to adapt the event background information to each specific aspect of If-Then inferential target.
In this work, we propose to generate unanswerable questions by editing an answerable question and conditioning on the corresponding paragraph that contains the answer. So the generated unanswerable questions are more lexically similar and relevant to the context. Moreover, by using the answerable question as a prototype and its answer span as a plausible answer, the generated examples can provide more discriminative training signal to the question answering model. 
 Question: Does their approach require a dataset of unanswerable questions mapped to similar answerable questions?
Output: Yes
These models define parametrized similarity scoring functions $: Q\times T\rightarrow \mathbb {R}$ , where $Q$ is the set of natural language questions and $T$ is the set of paraphrases of logical forms. 
 Question: Does a neural scoring function take both the question and the logical form as inputs?
Output: Yes
Due to the high cost of pre-training BERT language model, we directly adopt parameters pre-trained by Google in Chinese general corpus. The named entity recognition is applied on both pathology report texts and query texts. 
 Question: What data is the language model pretrained on?
Output: Chinese general corpus
This method, which we refer to as progressive dynamic hurdles (PDH), allows models that are consistently performing well to train for more steps. It begins as ordinary tournament selection evolutionary architecture search with early stopping, with each child model training for a relatively small $s_0$ number of steps before being evaluated for fitness. However, after a predetermined number of child models, $m$ , have been evaluated, a hurdle, $h_0$ , is created by calculating the the mean fitness of the current population. For the next $m$ child models produced, models that achieve a fitness greater than $h_0$ after $s_0$ train steps are granted an additional $s_1$ steps of training and then are evaluated again to determine their final fitness. Once another $m$ models have been considered this way, another hurdle, $h_1$ , is constructed by calculating the mean fitness of all members of the current population that were trained for the maximum number of steps. For the next $m$ child models, training and evaluation continues in the same fashion, except models with fitness greater than $m$0 after $m$1 steps of training are granted an additional $m$2 number of train steps, before being evaluated for their final fitness. This process is repeated until a satisfactory number of maximum training steps is reached. 
 Question: what is the proposed Progressive Dynamic Hurdles method?
Output: allows models that are consistently performing well to train for more steps
We propose the following challenge to the community: We must develop formal definition and evaluation for faithfulness that allows us the freedom to say when a method is sufficiently faithful to be useful in practice.

We note two possible approaches to this end:

Across models and tasks: The degree (as grayscale) of faithfulness at the level of specific models and tasks. Perhaps some models or tasks allow sufficiently faithful interpretation, even if the same is not true for others.

For example, the method may not be faithful for some question-answering task, but faithful for movie review sentiment, perhaps based on various syntactic and semantic attributes of those tasks.

Across input space: The degree of faithfulness at the level of subspaces of the input space, such as neighborhoods of similar inputs, or singular inputs themselves. If we are able to say with some degree of confidence whether a specific decision's explanation is faithful to the model, even if the interpretation method is not considered universally faithful, it can be used with respect to those specific areas or instances only. 
 Question: What faithfulness criteria does they propose?
Output: Across models and tasks: The degree (as grayscale) of faithfulness at the level of specific models and tasks. Across input space: The degree of faithfulness at the level of subspaces of the input space, such as neighborhoods of similar inputs, or singular inputs themselves.
Since it is infeasible to manually annotate all of the comments, we process this dataset with the goal of extracting threads that involve suspected trolling attempts and the direct responses to them.  We had two human annotators who were trained on snippets (i.e., (suspected trolling attempt, responses) pairs) taken from 200 conversations and were allowed to discuss their findings. After this training stage, we asked them to independently label the four aspects for each snippet.  
 Question: Do they use a crowdsourcing platform for annotation?
Output: No
We evaluate performance of our models on the SQUAD BIBREF16 dataset (denoted $\mathcal {S}$ ). We use the same split as that of BIBREF4 , where a random subset of 70,484 instances from $\mathcal {S}\ $ are used for training ( ${\mathcal {S}}^{tr}$ ), 10,570 instances for validation ( ${\mathcal {S}}^{val}$ ), and 11,877 instances for testing ( ${\mathcal {S}}^{te}$ ). 
 Question: Which datasets are used to train this model?
Output: SQUAD
In this paper, in spite of previous statements, we present a system that uses rule-based and dictionary-based methods combined (in a way we prefer to call resource-based).  
 Question: What does their system consist of?
Output: rule-based and dictionary-based methods
We evaluate a multi-task approach and three algorithms that explicitly model the task dependencies. 
 Question: Will these findings be robust through different datasets and different question answering algorithms?
Output: Yes
Gazetteers: 6)Contributor first names; 7)Contributor last names; 8)Contributor types ("soprano", "violinist", etc.); 9)Classical work types ("symphony", "overture", etc.); 10)Musical instruments; 11)Opus forms ("op", "opus"); 12)Work number forms ("no", "number"); 13)Work keys ("C", "D", "E", "F" , "G" , "A", "B", "flat", "sharp"); 14)Work Modes ("major", "minor", "m"). 
 Question: What task-specific features are used?
Output: 6)Contributor first names 7)Contributor last names 8)Contributor types ("soprano", "violinist", etc.) 9)Classical work types ("symphony", "overture", etc.) 10)Musical instruments 11)Opus forms ("op", "opus") 12)Work number forms ("no", "number") 13)Work keys ("C", "D", "E", "F" , "G" , "A", "B", "flat", "sharp") 14)Work Modes ("major", "minor", "m")
As can be seen, the proposed DSC loss outperforms the best baseline results by a large margin, i.e., outperforming BERT-tagger by +1.86 in terms of F1 score on CTB5, +1.80 on CTB6 and +2.19 on UD1.4. 
 Question: What are method's improvements of F1 w.r.t. baseline BERT tagger for Chinese POS datasets?
Output: +1.86 in terms of F1 score on CTB5 +1.80 on CTB6 +2.19 on UD1.4
We use Waseem and Hovy BIBREF5, Davidson et al. BIBREF9, and Waseem et al. BIBREF10 as baselines and compare the results with our different fine-tuning strategies using pre-trained BERTbase model.  
 Question: What baseline is used?
Output: Waseem and Hovy BIBREF5 Davidson et al. BIBREF9 Waseem et al. BIBREF10
The regressor and classifier used Support Vector Regression (SVR) and Support Vector Classification (SVC) respectively. The table includes a neural regressor (NNR) and a neural classifier (NNC). The neural classifier is trained in two set ups: “NNC top 5” uses classification labels as described in Section SECREF3, and “NNC SU4 F1” uses the regression labels, that is, the ROUGE-SU4 F1 scores of each sentence. 
 Question: What classification approaches were experimented for this task?
Output: NNC SU4 F1 NNC top 5 Support Vector Classification (SVC)
Questions are gathered from anonymized, aggregated queries to the Google search engine. Queries that are likely to be yes/no questions are heuristically identified: we found selecting queries where the first word is in a manually constructed set of indicator words and are of sufficient length, to be effective.

Questions are only kept if a Wikipedia page is returned as one of the first five results, in which case the question and Wikipedia page are given to a human annotator for further processing.

Annotators label question/article pairs in a three-step process. First, they decide if the question is good, meaning it is comprehensible, unambiguous, and requesting factual information. This judgment is made before the annotator sees the Wikipedia page. Next, for good questions, annotators find a passage within the document that contains enough information to answer the question. Annotators can mark questions as “not answerable" if the Wikipedia article does not contain the requested information. Finally, annotators mark whether the question's answer is “yes" or “no". Annotating data in this manner is quite expensive since annotators need to search entire Wikipedia documents for relevant evidence and read the text carefully. 
 Question: how was the dataset built?
Output: Questions are gathered from anonymized, aggregated queries to the Google search engine. Queries that are likely to be yes/no questions are heuristically identified: we found selecting queries where the first word is in a manually constructed set of indicator words and are of sufficient length, to be effective. Questions are only kept if a Wikipedia page is returned as one of the first five results, in which case the question and Wikipedia page are given to a human annotator for further processing. Annotators label question/article pairs in a three-step process. First, they decide if the question is good, meaning it is comprehensible, unambiguous, and requesting factual information. This judgment is made before the annotator sees the Wikipedia page. Next, for good questions, annotators find a passage within the document that contains enough information to answer the question. Annotators can mark questions as “not answerable" if the Wikipedia article does not contain the requested information. Finally, annotators mark whether the question's answer is “yes" or “no"
This step consists of generating a query out of the claim and querying a search engine (here, we experiment with Google and Bing) in order to retrieve supporting documents. Rather than querying the search engine with the full claim (as on average, a claim is two sentences long), we generate a shorter query following the lessons highlighted in BIBREF0 . We rank the words by means of tf-idf. We compute the idf values on a 2015 Wikipedia dump and the English Gigaword. BIBREF0 suggested that a good way to perform high-quality search is to only consider the verbs, the nouns and the adjectives in the claim; thus, we exclude all words in the claim that belong to other parts of speech. Moreover, claims often contain named entities (e.g., names of persons, locations, and organizations); hence, we augment the initial query with all the named entities from the claim's text. We use IBM's AlchemyAPI to identify named entities. Ultimately, we generate queries of 5–10 tokens, which we execute against a search engine. We then collect the snippets and the URLs in the results, skipping any result that points to a domain that is considered unreliable. 
 Question: How are the potentially relevant text fragments identified?
Output:  Generate a query out of the claim and querying a search engine, rank the words by means of TF-IDF, use IBM's AlchemyAPI to identify named entities, generate queries of 5–10 tokens, which execute against a search engine, and collect the snippets and the URLs in the results, skipping any result that points to a domain that is considered unreliable.
In order to train a neural g2p system, one needs a large quantity of pronunciation data. A standard dataset for g2p is the Carnegie Mellon Pronouncing Dictionary BIBREF12 . However, that is a monolingual English resource, so it is unsuitable for our multilingual task. Instead, we use the multilingual pronunciation corpus collected by deri2016grapheme for all experiments. This corpus consists of spelling–pronunciation pairs extracted from Wiktionary. It is already partitioned into training and test sets. Corpus statistics are presented in Table TABREF10 In addition to the raw IPA transcriptions extracted from Wiktionary, the corpus provides an automatically cleaned version of transcriptions. Cleaning is a necessary step because web-scraped data is often noisy and may be transcribed at an inconsistent level of detail. The data cleaning used here attempts to make the transcriptions consistent with the phonemic inventories used in Phoible BIBREF4 .  
 Question: what datasets did they use?
Output: the Carnegie Mellon Pronouncing Dictionary BIBREF12 the multilingual pronunciation corpus collected by deri2016grapheme  ranscriptions extracted from Wiktionary
One way to analyze the model is to compute model gradients with respect to input features BIBREF26, BIBREF25. Figure FIGREF37 shows that in this particular example, the most important model inputs are verbs possibly associated with the entity butter, in addition to the entity's mentions themselves. It further shows that the model learns to extract shallow clues of identifying actions exerted upon only the entity being tracked, regardless of other entities, by leveraging verb semantics. 
 Question: What evidence do they present that the model attends to shallow context clues?
Output: Using model gradients with respect to input features they presented that the most important model inputs are verbs associated with entities which shows that the model attends to shallow context clues
For de-identification tasks, the three metrics we will use to evaluate the performance of our architecture are Precision, Recall and INLINEFORM0 score as defined below. 
 Question: What evaluation metrics do they use?
Output: Precision, Recall and INLINEFORM0 score
We initiated crawling with 100 questions randomly selected from different topics so that different genre of questions can be covered. The crawling of the questions follow a BFS pattern through the related question links. We obtained 822,040 unique questions across 80,253 different topics with a total of 1,833,125 answers to these questions. 
 Question: Does the experiments focus on a specific domain?
Output: No
TOEFL listening comprehension test is for human English learners whose native language is not English. This paper reports how today's machine can perform with such a test.

The listening comprehension task considered here is highly related to Spoken Question Answering (SQA) BIBREF0 , BIBREF1 .  
 Question: What is the new task proposed in this work?
Output:  listening comprehension task
For the other techniques, we extract paragraphs containing any word from a predetermined list of LGTBQ terms (shown in Table TABREF19) 
 Question: How do they identify discussions of LGBTQ people in the New York Times?
Output: act paragraphs containing any word from a predetermined list of LGTBQ terms
Therefore, in our analysis, we focus on the chain features related to the phenomena of shining through and explicitation. 
 Question: Which coreference phenomena are analyzed?
Output: shining through explicitation
The functional dissimilarity score was computed using sentences from the test set in CoNLL 2017 Universal Dependencies task BIBREF20 for the relevant languages with the provided UPOS sequences. We computed the nearest neighbours experiment for all languages in the training data for the above models. 
 Question: Which evaluation metrics do they use for language modelling?
Output:  functional dissimilarity score nearest neighbours experiment
With the purpose of comparing the proposed model with a popular state-of-the-art approach for image classification, for the LabelMe dataset, the following baseline was introduced:

Bosch 2006 (mv): This baseline is similar to one in BIBREF33 . The authors propose the use of pLSA to extract the latent topics, and the use of k-nearest neighbor (kNN) classifier using the documents' topics distributions. For this baseline, unsupervised LDA is used instead of pLSA, and the labels from the different annotators for kNN (with INLINEFORM0 ) are aggregated using majority voting (mv).

The results obtained by the different approaches for the LabelMe data are shown in Fig. FIGREF94 , where the svi version is using mini-batches of 200 documents.

Analyzing the results for the Reuters-21578 and LabelMe data, we can observe that MA-sLDAc outperforms all the baselines, with slightly better accuracies for the batch version, especially in the Reuters data. Interestingly, the second best results are consistently obtained by the multi-annotator approaches, which highlights the need for accounting for the noise and biases of the answers of the different annotators. Both the batch and the stochastic variational inference (svi) versions of the proposed model (MA-sLDAc) are compared with the following baselines:

[itemsep=0.02cm]

LDA + LogReg (mv): This baseline corresponds to applying unsupervised LDA to the data, and learning a logistic regression classifier on the inferred topics distributions of the documents. The labels from the different annotators were aggregated using majority voting (mv). Notice that, when there is a single annotator label per instance, majority voting is equivalent to using that label for training. This is the case of the 20-Newsgroups' simulated annotators, but the same does not apply for the experiments in Section UID89 .

LDA + Raykar: For this baseline, the model of BIBREF21 was applied using the documents' topic distributions inferred by LDA as features.

LDA + Rodrigues: This baseline is similar to the previous one, but uses the model of BIBREF9 instead.

Blei 2003 (mv): The idea of this baseline is to replicate a popular state-of-the-art approach for document classification. Hence, the approach of BIBREF0 was used. It consists of applying LDA to extract the documents' topics distributions, which are then used to train a SVM. Similarly to the previous approach, the labels from the different annotators were aggregated using majority voting (mv).

sLDA (mv): This corresponds to using the classification version of sLDA BIBREF2 with the labels obtained by performing majority voting (mv) on the annotators' answers. 
 Question: what are the state of the art approaches?
Output: Bosch 2006 (mv) LDA + LogReg (mv) LDA + Raykar LDA + Rodrigues Blei 2003 (mv) sLDA (mv)
We defined the reward as being 1 for successfully completing the task, and 0 otherwise. A discount of $0.95$ was used to incentivize the system to complete dialogs faster rather than slower, yielding return 0 for failed dialogs, and $G = 0.95^{T-1}$ for successful dialogs, where $T$ is the number of system turns in the dialog.   0.95^{T-1} reward  0.95^{T-1}  
 Question: What is the reward model for the reinforcement learning appraoch?
Output: reward 1 for successfully completing the task, with a discount by the number of turns, and reward 0 when fail
Although such an approach has been used in different studies during feature engineering, the selection of word vectors and the number of clusters remain a trial-end-error procedure.  
 Question: Which other hyperparameters, other than number of clusters are typically evaluated in this type of research?
Output: selection of word vectors
For the sentiment analysis, we look at our problem in a multi-label setting, our two labels being sentiment polarity and the candidate/category in consideration. 
 Question: How many label options are there in the multi-label task?
Output:  two labels
SCA BIBREF5 softly augments a randomly chosen word in a sentence by its contextual mixture of multiple related words, i.e., replacing the one-hot representation of a word by a distribution provided by a language model over the vocabulary. 
 Question: How does soft contextual data augmentation work?
Output: softly augments a randomly chosen word in a sentence by its contextual mixture of multiple related words replacing the one-hot representation of a word by a distribution provided by a language model over the vocabulary
The baseline approach is based on BIBREF20 . It divides the task of interpreting commands for behavioral navigation into two steps: path generation, and path verification. For path generation, this baseline uses a standard sequence-to-sequence model augmented with an attention mechanism, similar to BIBREF23 , BIBREF6 . For path verification, the baseline uses depth-first search to find a route in the graph that matches the sequence of predicted behaviors. If no route matches perfectly, the baseline changes up to three behaviors in the predicted sequence to try to turn it into a valid path. 
 Question: What baselines did they compare their model with?
Output: the baseline where path generation uses a standard sequence-to-sequence model augmented with attention mechanism and path verification uses depth-first search
The most intuitive way to represent syntactic information is to use individual dependency relations directly, like dependency head and dependency relation label, denoted as Dep and Rel for short. In order to preserve the structural information of dependency trees as much as possible, we take the syntactic path between candidate arguments and predicates in dependency trees as linguistic knowledge. Referring to BIBREF9, we use the Tree-based Position Feature (TPF) as Dependency Path (DepPath) and use the Shortest Dependency Path (SDP) as Relation Path (RelPath). 
 Question: What different approaches of encoding syntactic information authors present?
Output: dependency head and dependency relation label, denoted as Dep and Rel for short Tree-based Position Feature (TPF) as Dependency Path (DepPath) Shortest Dependency Path (SDP) as Relation Path (RelPath)
The output of a system with the target words in the predicted order is compared to the gold ranking of the DURel data set. As the metric to assess how well the model's output fits the gold ranking Spearman's $\rho $ was used. The higher Spearman's rank-order correlation the better the system's performance. 
 Question: How is evaluation performed?
Output: As the metric to assess how well the model's output fits the gold ranking Spearman's $\rho $ was used
Features. We argue that different kinds of features like the sentiment of the text, morality, and other text-based features are critical to detect the nonfactual Twitter accounts by utilizing their occurrence during reporting the news in an account's timeline. We employ a rich set of features borrowed from previous works in fake news, bias, and rumors detection BIBREF0, BIBREF1, BIBREF8, BIBREF9.

[leftmargin=4mm]

Emotion: We build an emotions vector using word occurrences of 15 emotion types from two available emotional lexicons. We use the NRC lexicon BIBREF10, which contains $\sim $14K words labeled using the eight Plutchik's emotions BIBREF11. The other lexicon is SentiSense BIBREF12 which is a concept-based affective lexicon that attaches emotional meanings to concepts from the WordNet lexical database. It has $\sim $5.5 words labeled with emotions from a set of 14 emotional categories We use the categories that do not exist in the NRC lexicon.

Sentiment: We extract the sentiment of the tweets by employing EffectWordNet BIBREF13, SenticNet BIBREF14, NRC BIBREF10, and subj_lexicon BIBREF15, where each has the two sentiment classes, positive and negative.

Morality: Features based on morality foundation theory BIBREF16 where words are labeled in one of the following 10 categories (care, harm, fairness, cheating, loyalty, betrayal, authority, subversion, sanctity, and degradation).

Style: We use canonical stylistic features, such as the count of question marks, exclamation marks, consecutive characters and letters, links, hashtags, users' mentions. In addition, we extract the uppercase ratio and the tweet length.

Words embeddings: We extract words embeddings of the words of the tweet using $Glove\-840B-300d$ BIBREF17 pretrained model. The tweet final representation is obtained by averaging its words embeddings. 
 Question: What features are extracted?
Output: Sentiment Morality Style Words embeddings
Our contribution leverages pre-trained word embeddings (GloVe, trained on wikipedia+gigaword corpus), the DepecheMood affective lexicon, and convolutional neural networks. 
 Question: What embeddings do they use?
Output: GloVe
Our evaluation comprises of a rich set of 19 different algorithms to recommend tags for e-books, which we group into (i) popularity-based, (ii) similarity-based (i.e., using content information), and (iii) hybrid approaches. 
 Question: what algorithms did they use?
Output: popularity-based similarity-based hybrid
Traditional text-to-speech (TTS) systems are composed of complex pipelines BIBREF0 , these often include accoustic frontends, duration model, acoustic prediction model and vocoder models. The architecture of our model utilizes RNN-based Seq2Seq model for generating mel spectrogram from text. The architecture is similar to that of Tacotron 2 BIBREF4  Direct comparison of model parameters between ours and the open-source tacotron 2, our model contains 4.5 million parameters, whereas the Tacotron 2 contains around 13 million parameters with default setting. By helping our model learn attention alignment faster, we can afford to use a smaller overall model to achieve similar quality speech quality. 
 Question: Do they reduce the number of parameters in their architecture compared to other direct text-to-speech models?
Output: Yes
The dataset comprises a total of 353 conversations from 40 speakers (11 nurses, 16 patients, and 13 caregivers) with consent to the use of anonymized data for research. In Section SECREF16 , we build templates and expression pools using linguistic analysis followed by manual verification. 
 Question: How big is their created dataset?
Output: 353 conversations from 40 speakers (11 nurses, 16 patients, and 13 caregivers) we build templates and expression pools using linguistic analysis
Our model encodes the information from audio and text sequences using dual RNNs and then combines the information from these sources using a feed-forward neural model to predict the emotion class. 
 Question: How do they combine audio and text sequences in their RNN?
Output: combines the information from these sources using a feed-forward neural model
We apply Han to such conversations in a sequential manner by feeding each user turn to Han as they occur, to determine if the conversation should escalate. If so, the user will be transferred to a live chat representative to continue the conversation. The user must wait for the human representative to review the IVA chat history and resume the failed task. 
 Question: How do they gather human reviews?
Output: human representative to review the IVA chat history and resume the failed task
we follow Lapata2005Automatic to measure coherence as sentence similarity 
 Question: Did the authors evaluate their system output for coherence?
Output: Yes
Experiments ::: Baselines
We evaluate MPAD against multiple state-of-the-art baseline models, including hierarchical ones, to enable fair comparison with the hierarchical MPAD variants.

doc2vec BIBREF37. Doc2vec (or paragraph vector) is an extension of word2vec that learns vectors for documents in a fully unsupervised manner. Document embeddings are then fed to a logistic regression classifier.

CNN BIBREF38. The convolutional neural network architecture, well-known in computer vision, is applied to text. There is one spatial dimension and the word embeddings are used as channels (depth dimensions).

DAN BIBREF39. The Deep Averaging Network passes the unweighted average of the embeddings of the input words through multiple dense layers and a final softmax.

Tree-LSTM BIBREF40 is a generalization of the standard LSTM architecture to constituency and dependency parse trees.

DRNN BIBREF41. Recursive neural networks are stacked and applied to parse trees.

LSTMN BIBREF42 is an extension of the LSTM model where the memory cell is replaced by a memory network which stores word representations.

C-LSTM BIBREF43 combines convolutional and recurrent neural networks. The region embeddings provided by a CNN are fed to a LSTM.

SPGK BIBREF44 also models documents as word co-occurrence networks. It computes a graph kernel that compares shortest paths extracted from the word co-occurrence networks and then uses a SVM to categorize documents.

WMD BIBREF45 is an application of the well-known Earth Mover's Distance to text. A k-nearest neighbor classifier is used.

S-WMD BIBREF46 is a supervised extension of the Word Mover's Distance.

Semantic-CNN BIBREF47. Here, a CNN is applied to semantic units obtained by clustering words in the embedding space.

LSTM-GRNN BIBREF26 is a hierarchical model where sentence embeddings are obtained with a CNN and a GRU-RNN is fed the sentence representations to obtain a document vector.

HN-ATT BIBREF27 is another hierarchical model, where the same encoder architecture (a bidirectional GRU-RNN) is used for both sentences and documents, with different parameters. A self-attention mechanism is applied to the RNN annotations at each level. 
 Question: What is the state-of-the-art system?
Output: doc2vec  CNN DAN Tree-LSTM DRNN LSTMN C-LSTM SPGK WMD S-WMD Semantic-CNN LSTM-GRNN HN-ATT
Because of this, we do not claim that this dataset can be considered a ground truth. 
 Question: How is the ground truth for fake news established?
Output: Ground truth is not established in the paper
The only requirement is that the model accepts as input, an embedding layer (for entities and relations). If a model fulfills this requirement (which a large number of neural models on knowledge graphs do), we can just use Dolores embeddings as a drop-in replacement. We just initialize the corresponding embedding layer with Dolores embeddings. 
 Question: Is fine-tuning required to incorporate these embeddings into existing models?
Output: No