niryuu/sni-each-converted huggingface dataset:
We collect utterances from the $\mathbf {C}$hinese $\mathbf {A}$rtificial $\mathbf {I}$ntelligence $\mathbf {S}$peakers (CAIS), and annotate them with slot tags and intent labels. The training, validation and test sets are split by the distribution of intents, where detailed statistics are provided in the supplementary material. Since the utterances are collected from speaker systems in the real world, intent labels are partial to the PlayMusic option. We adopt the BIOES tagging scheme for slots instead of the BIO2 used in the ATIS, since previous studies have highlighted meaningful improvements with this scheme BIBREF30 in the sequence labeling field. 
 Question: What is the source of the CAIS dataset?
Output: the $\mathbf {C}$hinese $\mathbf {A}$rtificial $\mathbf {I}$ntelligence $\mathbf {S}$peakers (CAIS)
Three datasets annotated with emotions are commonly used for the development and evaluation of emotion detection systems, namely the Affective Text dataset, the Fairy Tales dataset, and the ISEAR dataset. A summary is provided in Table TABREF8 , which also shows, in the bottom row, what role each dataset has in our experiments: apart from the development portion of the Affective Text, which we used to develop our models (Section SECREF4 ), all three have been used as benchmarks for our evaluation. 
 Question: Which existing benchmarks did they compare to?
Output: Affective Text Fairy Tales ISEAR
For the proposed model, we denote INLINEFORM0 parameterized by INLINEFORM1 as a neural-based feature encoder that maps documents from both domains to a shared feature space, and INLINEFORM2 parameterized by INLINEFORM3 as a fully connected layer with softmax activation serving as the sentiment classifier. We have left the feature encoder INLINEFORM0 unspecified, for which, a few options can be considered. In our implementation, we adopt a one-layer CNN structure from previous works BIBREF22 , BIBREF4 , as it has been demonstrated to work well for sentiment classification tasks. 
 Question: What is the architecture of the model?
Output: one-layer CNN structure from previous works BIBREF22 , BIBREF4
We use the unshuffled version of the French OSCAR corpus, which amounts to 138GB of uncompressed text and 32.7B SentencePiece tokens. 
 Question: What is CamemBERT trained on?
Output: unshuffled version of the French OSCAR corpus
According to the categories mentioned in BIBREF5 , irony can be classified into 3 classes: verbal irony by means of a polarity contrast, the sentences containing expression whose polarity is inverted between the intended and the literal evaluation; other types of verbal irony, the sentences that show no polarity contrast between the literal and intended meaning but are still ironic; and situational irony, the sentences that describe situations that fail to meet some expectations. As ironies in the latter two categories are obscure and hard to understand, we decide to only focus on ironies in the first category in this work.   Because of the lack of previous work and baselines on irony generation, we implement our model based on style transfer.  
 Question: What are the difficulties in modelling the ironic pattern?
Output: obscure and hard to understand  lack of previous work and baselines on irony generation
Experiments ::: Baselines
To comprehensively evaluate our AGDT, we compare the AGDT with several competitive models. 
 Question: Is the model evaluated against other Aspect-Based models?
Output: Yes
Internally, the ASR system maintains a rich hypothesis space in the form of speech lattices or confusion networks (cnets). 
 Question: What is a word confusion network?
Output: It is a network used to encode speech lattices to maintain a rich hypothesis space.
To control the quality, we ensured that a single annotator annotates maximum 120 headlines (this protects the annotators from reading too many news headlines and from dominating the annotations). Secondly, we let only annotators who geographically reside in the U.S. contribute to the task.

We test the annotators on a set of $1,100$ test questions for the first phase (about 10% of the data) and 500 for the second phase. Annotators were required to pass 95%. 
 Question: How is quality of annotation measured?
Output: Annotators went through various phases to make sure their annotations did not deviate from the mean.
Later, BIBREF8 introduced an RNN with an external stack memory to learn simple context-free languages, such as $a^n b^m$ , $a^nb^ncb^ma^m$ , and $a^{n+m} b^n c^m$ . Similar studies BIBREF15 , BIBREF16 , BIBREF17 , BIBREF10 , BIBREF11 have explored the existence of stable counting mechanisms in simple RNNs, which would enable them to learn various context-free and context-sensitive languages BIBREF9 , on the other hand, proposed a variant of Long Short-Term Memory (LSTM) networks to learn two context-free languages, $a^n b^n$ , $a^n b^m B^m A^n$ , and one strictly context-sensitive language, $a^n b^n c^n$ . 
 Question: How do they get the formal languages?
Output: These are well-known formal languages some of which was used in the literature to evaluate the learning capabilities of RNNs.
Compared with 2015, our QA-SRL gold yielded 1593 annotations, with 989 core and 604 adjuncts, while theirs yielded 1315 annotations, 979 core and 336 adjuncts.  
 Question: How much more coverage is in the new dataset?
Output: 278 more annotations
Both the distributional hypothesis itself and Tugendhat's interpretation of Frege's work are examples of holistic approaches to meaning, where the meaning of the whole determines the meaning of parts. 
 Question: How does Frege's holistic and functional approach to meaning relates to general distributional hypothesis?
Output: interpretation of Frege's work are examples of holistic approaches to meaning
We use the Bayesian model of garg2012unsupervised as our base monolingual model. The semantic roles are predicate-specific. 
 Question: What does an individual model consist of?
Output: Bayesian model of garg2012unsupervised as our base monolingual model
In this section we present two datasets used in our experiments: The NowThisNews dataset, collected for the purpose of this paper, and The BreakingNews dataset BIBREF4 , publicly available dataset of news articles.

contains 4090 posts with associated videos from NowThisNews Facebook page collected between 07/2015 and 07/2016. For each post we collected its title and the number of views of the corresponding video, which we consider our popularity metric. Due to a fairly lengthy data collection process, we decided to normalize our data by first grouping posts according to their publication month and then labeling the posts for which the popularity metric exceeds the median monthly value as popular, the remaining part as unpopular. 
 Question: Where do they obtain the news videos from?
Output: NowThisNews Facebook page
However, recent advances in Deep Learning techniques have shown that the NER task can benefit from the use of neural architectures, such as biLSTM-networks BIBREF3 , BIBREF4 . We use the implementation proposed in BIBREF24 for conducting three different experiments. 
 Question: Which machine learning algorithms did the explore?
Output: biLSTM-networks
A total of 2,50,000 tweets over a period of August 31st, 2015 to August 25th,2016 on Microsoft are extracted from twitter API BIBREF15 . The news on twitter about Microsoft and tweets regarding the product releases were also included. Stock opening and closing prices of Microsoft from August 31st, 2015 to August 25th, 2016 are obtained from Yahoo! Finance BIBREF16 . 
 Question: What dataset is used to train the model?
Output: 2,50,000 tweets Stock opening and closing prices of Microsoft from August 31st, 2015 to August 25th, 2016
This section reports the results of the experiments conducted on two data sets for evaluating the performances of wDTW and wTED against other baseline methods. We denote the following distance/similarity measures. WMD: The Word Mover's Distance introduced in Section SECREF1 . VSM: The similarity measure introduced in Section UID12 . PV-DTW: PV-DTW is the same as Algorithm SECREF21 except that the distance between two paragraphs is not based on Algorithm SECREF20 but rather computed as INLINEFORM0 where INLINEFORM1 is the PV embedding of paragraph INLINEFORM2 . PV-TED: PV-TED is the same as Algorithm SECREF23 except that the distance between two paragraphs is not based on Algorithm SECREF20 but rather computed as INLINEFORM0 . 
 Question: Which are the state-of-the-art models?
Output: WMD VSM PV-DTW PV-TED
For the evaluation of summaries we use the standard ROGUE metric. For comparison with previous AMR based summarization methods, we report the Recall, Precision and INLINEFORM0 scores for ROGUE-1. 
 Question: Which evaluation methods are used?
Output: Quantitative evaluation methods using ROUGE, Recall, Precision and F1.
The semi-parametric models were trained on 32 GPUs with each replica split over 2 GPUs, one to train the translation model and the other for computing the CSTM. 
 Question: Does their combination of a non-parametric retrieval and neural network get trained end-to-end?
Output: Yes
The previously mentioned datasets are all in English 
 Question: For which languages most of the existing MRC datasets are created?
Output: English
LUWAK is implemented in pure JavaScript code, and it uses the LocalStorage of a web browser. 
 Question: What programming language is the tool written in?
Output: JavaScript
We characterize the evaluation of Emotionally-Aware Chatbot into two different parts, qualitative and quantitative assessment. Qualitative assessment will focus on assessing the functionality of the software, while quantitative more focus on measure the chatbots' performance with a number. Based on our investigation of several previous studies, we found that most of the works utilized ISO 9241 to assess chatbots' quality by focusing on the usability aspect. This aspect can be grouped into three focuses, including efficiency, effectiveness, and satisfaction, concerning systems' performance to achieve the specified goals. In automatic evaluation, some studies focus on evaluating the system at emotion level BIBREF15 , BIBREF28 . Therefore, some common metrics such as precision, recall, and accuracy are used to measure system performance, compared to the gold label. This evaluation is similar to emotion classification tasks such as previous SemEval 2018 BIBREF32 and SemEval 2019 . Other studies also proposed to use perplexity to evaluate the model at the content level (to determine whether the content is relevant and grammatical) BIBREF14 , BIBREF39 , BIBREF28 . This evaluation metric is widely used to evaluate dialogue-based systems which rely on probabilistic approach BIBREF61 . Another work by BIBREF14 used BLEU to evaluate the machine response and compare against the gold response (the actual response), although using BLEU to measure conversation generation task is not recommended by BIBREF62 due to its low correlation with human judgment. This evaluation involves human judgement to measure the chatbots' performance, based on several criteria. BIBREF15 used three annotators to rate chatbots' response in two criteria, content (scale 0,1,2) and emotion (scale 0,1). Content is focused on measuring whether the response is natural acceptable and could plausible produced by a human. This metric measurement is already adopted and recommended by researchers and conversation challenging tasks, as proposed in BIBREF38 .  
 Question: How are EAC evaluated?
Output: Qualitatively through efficiency, effectiveness and satisfaction aspects and quantitatively through metrics such as precision, recall, accuracy, BLEU score and even human judgement.
Our model got the first position in the German sub-task with a macro F1 score of 0.62. 
 Question: What is the performance of the model for the German sub-task A?
Output: macro F1 score of 0.62
We consider three tasks representing a broad selection of natural language understanding scenarios: paper acceptance prediction based on the PeerRead data set BIBREF2, Named Entity Recognition (NER) based on the Broad Twitter Corpus BIBREF3, and author stance prediction based on the RumEval-19 data set BIBREF6.  
 Question: What are three challenging tasks authors evaluated their sequentially aligned representations?
Output: paper acceptance prediction Named Entity Recognition (NER) author stance prediction
Our model yields the best results on both UAS and LAS metrics of all languages except the Japanese. As for Japanese, our model gives unsatisfactory results because the original treebank was written in Roman phonetic characters instead of hiragana, which is used by both common Japanese writing and our pre-trained embeddings. Despite this, our model overall still gives 1.0% higher average UAS and LAS than the previous best parser, BIAF. Although both BIAF and STACKPTR parsers have achieved relatively high parsing accuracies on the 12 languages and have all UAS higher than 90%, our model achieves state-of-the-art results in all languages for both UAS and LAS. Overall, our model reports more than 1.0% higher average UAS than STACKPTR and 0.3% higher than BIAF. 
 Question: What are performance compared to former models?
Output: model overall still gives 1.0% higher average UAS and LAS than the previous best parser, BIAF our model reports more than 1.0% higher average UAS than STACKPTR and 0.3% higher than BIAF
if there is an error in the graph, the triple is likely to be inconsistent with its neighborhood, and thus the model should put least trust on this triple. In other words, the error triple should have the least influence on the model's prediction of the training data. Here, we demonstrate another potential use of adversarial modifications: finding erroneous triples in the knowledge graph. Intuitively, if there is an error in the graph, the triple is likely to be inconsistent with its neighborhood, and thus the model should put least trust on this triple. In other words, the error triple should have the least influence on the model's prediction of the training data. Formally, to find the incorrect triple $\langle s^{\prime }, r^{\prime }, o\rangle $ in the neighborhood of the train triple $\langle s, r, o\rangle $ , we need to find the triple $\langle s^{\prime },r^{\prime },o\rangle $ that results in the least change $\Delta _{(s^{\prime },r^{\prime })}(s,r,o)$ when removed from the graph. 
 Question: How is this approach used to detect incorrect facts?
Output: if there is an error in the graph, the triple is likely to be inconsistent with its neighborhood, and thus the model should put least trust on this triple. In other words, the error triple should have the least influence on the model's prediction of the training data.
The Sentence corpus (S) consists of domain-targeted $~$ 80K sentences and 280 GB of plain text extracted from web pages used by BIBREF6 aristo2016:combining. We use the text corpora (S) from BIBREF6 aristo2016:combining to build our tuple KB.  We take the top 200 hits, run Open IE v4, and aggregate the resulting tuples over all $a \in A$ and over all questions in $Q_\mathit {tr}$ to create the tuple KB (T). 
 Question: What was the textual source to which OpenIE was applied?
Output: domain-targeted $~$ 80K sentences and 280 GB of plain text extracted from web pages used by BIBREF6 aristo2016:combining
Our Recurrent +ELMo model uses the language model from BIBREF9 to provide contextualized embeddings to the baseline model outlined above, as recommended by the authors.

Our OpenAI GPT model fine-tunes the 12 layer 768 dimensional uni-directional transformer from BIBREF27 , which has been pre-trained as a language model on the Books corpus BIBREF36 . 
 Question: did they use other pretrained language models besides bert?
Output: Yes
We use a benchmark dataset created by ceccarelli2013learning from the CoNLL 2003 data.  
 Question: What is the benchmark dataset?
Output: a benchmark dataset created by ceccarelli2013learning from the CoNLL 2003 data
To develop the dataset, we extracted 200 sentences from Sorani Kurdish books of grades one to three of the primary school in the Kurdistan Region of Iraq. We randomly created 2000 sentences from the extracted sentences. 
 Question: What is the size of the dataset?
Output: 2000 sentences
BERT-ADA BIBREF33 is a domain-adapted BERT-based model proposed for the APC task, which fine-tuned the BERT-BASE model on task-related corpus. This model obtained state-of-the-art accuracy on the Laptops dataset. We build a joint model for the multi-task of ATE and APC based on the BERT-BASE model. After optimizing the model parameters according to the empirical result, the joint model based on BERT-BASE achieved hopeful performance on all three datasets and even surpassed other proposed BERT based improved models on some datasets, such as BERT-PT, AEN-BERT, SDGCN-BERT, and so on. 
 Question: What was state of the art on SemEval-2014 task4 Restaurant and Laptop dataset?
Output: BERT-ADA BERT-PT, AEN-BERT, SDGCN-BERT
Figure FIGREF5 shows that E-BERT performs comparable to BERT and ERNIE on unfiltered LAMA. 
 Question: Which of the two ensembles yields the best performance?
Output: Answer with content missing: (Table 2) CONCAT ensemble
The goal of an language model is to assign meaningful probabilities to a sequence of words. Given a set of tokens $\mathbf {X}=(x_1,....,x_T)$, where $T$ is the length of a sequence, our task is to estimate the joint conditional probability $P(\mathbf {X})$ which is

were $(x_{1}, \ldots , x_{i-1})$ is the context. An Intrinsic evaluation of the performance of Language Models is perplexity (PPL) which is defined as the inverse probability of the set of the tokens and taking the $T^{th}$ root were $T$ is the number of tokens We propose an approximation of the joint probability as,

This type of approximations has been previously explored with Bi-directional RNN LM's BIBREF9 but not for deep transformer models. We therefore, define a pseudo-perplexity score from the above approximated joint probability. 
 Question: How is pseudo-perplexity defined?
Output: Answer with content missing: (formulas in selection): Pseudo-perplexity is perplexity where conditional joint probability is approximated.
Compared with the BERT-BASE model, BERT-SPC significantly improves the accuracy and F1 score of aspect polarity classification. 
 Question: How much better is performance of the proposed model compared to the state of the art in these various experiments?
Output: significantly improves the accuracy and F1 score of aspect polarity classification
Through experiments on two large complex cross-domain datasets, SParC BIBREF2 and CoSQL BIBREF6, we carefully compare and analyze the performance of different context modeling methods.  
 Question: What two large datasets are used for evaluation?
Output: SParC BIBREF2 and CoSQL BIBREF6
First, workers were given a seed qualitative relation q+/-( INLINEFORM0 ) in the domain, expressed in English (e.g., “If a surface has more friction, then an object will travel slower”), and asked to enter two objects, people, or situations to compare. They then created a question, guided by a large number of examples, and were encouraged to be imaginative and use their own words. Second, the LFs were elicited using a novel technique of reverse-engineering them from a set of follow-up questions, without exposing workers to the underlying formalism. This is possible because of the constrained space of LFs. Referring to LF templates (1) and (2) earlier (Section SECREF13 ), these questions are as follows:

From this information, we can deduce the target LF ( INLINEFORM0 is the complement of INLINEFORM1 , INLINEFORM2 , we arbitrarily set INLINEFORM3 =world1, hence all other variables can be inferred). 
 Question: How do they obtain the logical forms of their questions in their dataset?
Output:  workers were given a seed qualitative relation asked to enter two objects, people, or situations to compare created a question, guided by a large number of examples LFs were elicited using a novel technique of reverse-engineering them from a set of follow-up questions
Table TABREF16 shows the confusion matrices for facial and audio emotion recognition on our complete AMMER data set and Table TABREF17 shows the results per class for each method, including facial and audio data and micro and macro averages. The classification from facial expressions yields a macro-averaged $\text{F}_1$ score of 33 % across the three emotions joy, insecurity, and annoyance (P=0.31, R=0.35). 
 Question: How is face and audio data analysis evaluated?
Output: confusion matrices $\text{F}_1$ score
In order to obtain insights about why the more fine-grained bi-sense emoji embedding helps in understanding the complexed sentiments behind tweets, we visualize the attention weights for ATT-E-LSTM and MATT-BiE-LSTM for comparison. Therefore, we construct the new input INLINEFORM0 to each LSTM unit by concatenating the original word embedding and the attention vector in Equation EQREF21 to distribute the senti-emoji information to each step. This model is called Multi-level Attention-based LSTM with Bi-sense Emoji Embedding (MATT-BiE-LSTM) Attention-based LSTM with emojis:We also use the word-emoji embedding to calculate the emoji-word attention following Equation EQREF20 and EQREF21 , and the only difference is that we replace the attention-derived senti-emoji embedding with the pre-trained word-emoji embedding by fasttext, denoted as ATT-E-LSTM. In Figure FIGREF27 (a), the ATT-E-LSTM model (baseline) assigns relatively more weights on the word “no” and “pressure”, while MATT-BiE-LSTM attends mostly on the word “happy” and “lovely”. The different attention distributions suggest that the proposed senti-emoji embedding is capable of recognizing words with strong sentiments that are closely related to the true sentiment even with the presence of words with conflicting sentiments, such as “pressure” and “happy”. while ATT-E-LSTM tends to pick up all sentimental words which could raise confusions. The senti-emoji embedding is capable of extracting representations of complexed semantics and sentiments which help guide the attentions even in cases when the word sentiment and emoji sentiment are somewhat contradictory to each other. From Figure FIGREF27 (b) and (c) we can observe that the ATT-E-LSTM assigns more weights on the sentiment-irrelevant words than the MATT-BiE-LSTM such as “hoodies”, “wait” and “after”, indicating that the proposed model is more robust to irrelevant words and concentrates better on important words. Because of the senti-emoji embedding obtained through bi-sense emoji embedding and the sentence-level LSTM encoding on the text input (described in Section SECREF13 ), we are able to construct a more robust embedding based on the semantic and sentiment information from the whole context compared to the word-emoji embedding used in ATT-E-LSTM which takes only word-level information into account. 
 Question: What evidence does visualizing the attention give to show that it helps to obtain a more robust understanding of semantics and sentiments?
Output: The different attention distributions suggest that the proposed senti-emoji embedding is capable of recognizing words with strong sentiments that are closely related to the true sentiment even with the presence of words with conflicting sentiments
In the proposed model, in addition to the co-occurrence edges, we link two nodes (words) if the corresponding word embedding representation is similar. 
 Question: Do the use word embeddings alone or they replace some previous features of the model with word embeddings?
Output: They use it as addition to previous model - they add new edge between words if word embeddings are similar.
While manually analyzing the raw dataset, we noticed that looking at the tweet one has replied to or has quoted, provides significant contextual information. We call these, “context tweets". As humans can better understand a tweet with the reference of its context, our assumption is that computers also benefit from taking context tweets into account in detecting abusive language.

As shown in the examples below, (2) is labeled abusive due to the use of vulgar language. However, the intention of the user can be better understood with its context tweet (1).

(1) I hate when I'm sitting in front of the bus and somebody with a wheelchair get on.

INLINEFORM0 (2) I hate it when I'm trying to board a bus and there's already an as**ole on it.

Similarly, context tweet (3) is important in understanding the abusive tweet (4), especially in identifying the target of the malice.

(3) Survivors of #Syria Gas Attack Recount `a Cruel Scene'.

INLINEFORM0 (4) Who the HELL is “LIKE" ING this post? Sick people....

Huang et al. huang2016modeling used several attributes of context tweets for sentiment analysis in order to improve the baseline LSTM model. However, their approach was limited because the meta-information they focused on—author information, conversation type, use of the same hashtags or emojis—are all highly dependent on data.

In order to avoid data dependency, text sequences of context tweets are directly used as an additional feature of neural network models. We use the same baseline model to convert context tweets to vectors, then concatenate these vectors with outputs of their corresponding labeled tweets. More specifically, we concatenate max-pooled layers of context and labeled tweets for the CNN baseline model. As for RNN, the last hidden states of context and labeled tweets are concatenated. 
 Question: What additional features and context are proposed?
Output: using tweets that one has replied or quoted to as contextual information
These actions consist of the following components:

[leftmargin=*]

Co-Reference Resolution: To support multi-turn interactions, it is sometimes necessary to use co-reference resolution techniques for effective retrieval. Query Generation: This component generates a query based on the past user-system interactions. Retrieval Model: This is the core ranking component that retrieves documents or passages from a large collection. Result Generation: The retrieved documents can be too long to be presented using some interfaces. 
 Question: What are the different modules in Macaw?
Output: Co-Reference Resolution Query Generation Retrieval Model Result Generation
Our AV in-cabin data-set includes 30 hours of multimodal data collected from 30 passengers (15 female, 15 male) in 20 rides/sessions. 10 types of passenger intents are identified and annotated as: Set/Change Destination, Set/Change Route (including turn-by-turn instructions), Go Faster, Go Slower, Stop, Park, Pull Over, Drop Off, Open Door, and Other (turn music/radio on/off, open/close window/trunk, change AC/temp, show map, etc.). Relevant slots are identified and annotated as: Location, Position/Direction, Object, Time-Guidance, Person, Gesture/Gaze (this, that, over there, etc.), and None. In addition to utterance-level intent types and their slots, word-level intent keywords are annotated as Intent as well. We obtained 1260 unique utterances having commands to AMIE from our in-cabin data-set. We expanded this data-set via Amazon Mechanical Turk and ended up with 3347 utterances having intents. The annotations for intents and slots are obtained on the transcribed utterances by majority voting of 3 annotators. 
 Question: What are the supported natural commands?
Output: Set/Change Destination Set/Change Route Go Faster Go Slower Stop Park Pull Over Drop Off Open Door Other
We developed an AdaBoost-based classifier to detect our new fake reviews, consisting of 200 shallow decision trees (depth 2). 
 Question: What kind of model do they use for detection?
Output: AdaBoost-based classifier
We collected the dataset with 1900 dialogs and 8533 turns.  
 Question: What is the average length of dialog?
Output: 4.49 turns
Both the audio and the transcript are de-identified (by removing the identifying information) with digital zeros and [de-identified] tags, respectively. 
 Question: Is the data de-identified?
Output: Yes
Our transference model extends the original transformer model to multi-encoder based transformer architecture. The transformer architecture BIBREF12 is built solely upon such attention mechanisms completely replacing recurrence and convolutions. The transformer uses positional encoding to encode the input and output sequences, and computes both self- and cross-attention through so-called multi-head attentions, which are facilitated by parallelization. We use multi-head attention to jointly attend to information at different positions from different representation subspaces. 
 Question: Which algorithm is used in the UDS-DFKI system?
Output: Our transference model extends the original transformer model to multi-encoder based transformer architecture. The transformer architecture BIBREF12 is built solely upon such attention mechanisms completely replacing recurrence and convolutions.
The dataset (composed of 2.6 GB of raw text) includes $421\,829\,960$ words divided into $17\,305\,401$ sentences. 
 Question: How big is dataset used to train Word2Vec for the Italian Language?
Output: $421\,829\,960$ words divided into $17\,305\,401$ sentences
As in the work of E. Tong et al. ( BIBREF9 ), we pre-train word embeddings using a skip-gram model BIBREF4 applied to unlabeled data from escort ads, however, we go further by analyzing the emojis' embeddings and thereby expand the trafficking lexicon. 
 Question: Do they use pretrained word embeddings?
Output: Yes
We evaluate the proposed models on three conversationet dataset such as MojiTalk BIBREF16, PersonaChat BIBREF11, Empathetic-Dialogues BIBREF26. 
 Question: What three conversational datasets are used for evaluation?
Output: MojiTalk  PersonaChat  Empathetic-Dialogues
SVM: We define 3 sets of features to characterize each question. The first is a simple bag-of-words set of features over the question (SVM-BOW), the second is bag-of-words features of the question as well as length of the question in words (SVM-BOW + LEN), and lastly we extract bag-of-words features, length of the question in words as well as part-of-speech tags for the question (SVM-BOW + LEN + POS). This results in vectors of 200, 201 and 228 dimensions respectively, which are provided to an SVM with a linear kernel. No-Answer Baseline (NA) : Most of the questions we receive are difficult to answer in a legally-sound way on the basis of information present in the privacy policy. We establish a simple baseline to quantify the effect of identifying every question as unanswerable. Word Count Baseline : To quantify the effect of using simple lexical matching to answer the questions, we retrieve the top candidate policy sentences for each question using a word count baseline BIBREF53, which counts the number of question words that also appear in a sentence. We include the top 2, 3 and 5 candidates as baselines. Human Performance: We pick each reference answer provided by an annotator, and compute the F1 with respect to the remaining references, as described in section 4.2.1. Each reference answer is treated as the prediction, and the remaining n-1 answers are treated as the gold reference. The average of the maximum F1 across all reference answers is computed as the human baseline. 
 Question: Were other baselines tested to compare with the neural baseline?
Output: SVM No-Answer Baseline (NA)  Word Count Baseline Human Performance
The corpus is freely available at the following link 
 Question: Is this dataset publicly available?
Output: Yes
We draw our data from news publications, wine reviews, and Reddit, which in addition to large volume, also let us characterize binomials in new ways, and analyze differences in binomial orderings across communities and over time.  We develop new metrics for the agreement of binomial orderings across communities and the movement of binomial orderings over time. Using subreddits as communities, these metrics reveal variations in orderings, some of which suggest cultural change influencing language. We develop a null model to determine how much variation in binomial orderings we might expect across communities and across time, if binomial orderings were randomly ordered according to global asymmetry values.  
 Question: How is order of binomials tracked across time?
Output: draw our data from news publications, wine reviews, and Reddit develop new metrics for the agreement of binomial orderings across communities and the movement of binomial orderings over time  develop a null model to determine how much variation in binomial orderings we might expect across communities and across time
In terms of the future development of this technology, we emphasize the importance of answering four questions: 1) How to introduce unsupervised pre-training into NLG tasks with cross-modal context? 2) How to design a generic pre-training algorithm to fit a wide range of NLG tasks? 3) How to reduce the computing resources required for large-scale pre-training? 4) What aspect of knowledge do the pre-trained models provide for better language generation? 
 Question: Which future direction in NLG are discussed?
Output: 1) How to introduce unsupervised pre-training into NLG tasks with cross-modal context? 2) How to design a generic pre-training algorithm to fit a wide range of NLG tasks? 3) How to reduce the computing resources required for large-scale pre-training? 4) What aspect of knowledge do the pre-trained models provide for better language generation?
We evaluate an RNN model which uses bidirectionally summed GRU memory cells BIBREF18 and uses the final states as embeddings; a CNN model which uses sentence-max-pooled convolutional filters as embeddings BIBREF19 ; an RNN-CNN model which puts the CNN on top of per-token GRU outputs rather than the word embeddings BIBREF20 ; and an attn1511 model inspired by BIBREF20 that integrates the RNN-CNN model with per-word attention to build hypothesis-specific evidence embeddings. We also report the baseline results of avg mean of word embeddings in the sentence with projection matrix and DAN Deep Averaging Network model that employs word-level dropout and adds multiple nonlinear transformations on top of the averaged embeddings BIBREF21 . 
 Question: what were the baselines?
Output: RNN model CNN model  RNN-CNN model attn1511 model Deep Averaging Network model avg mean of word embeddings in the sentence with projection matrix
As shown in Table TABREF3 , we collect Amazon review keywords for 2,896 e-books (publishers: Kiwi, Rowohlt, Fischer, and Droemer), which leads to 33,663 distinct review keywords and on average 30 keyword assignments per e-book. 
 Question: how large is the vocabulary?
Output: 33,663
We evaluate the proposed methods in 8 languages, showing a significant ability to learn from partial data. We additionally experiment with initializing CBL with domain-specific instance-weighting schemes, showing mixed results. In the process, we use weighted variants of popular NER models, showing strong performance in both non-neural and neural settings. Finally, we show experiments in a real-world setting, by employing non-speakers to manually annotate romanized Bengali text. We experiment on 8 languages. Four languages – English, German, Spanish, Dutch – come from the CoNLL 2002/2003 shared tasks BIBREF21, BIBREF22. These are taken from newswire text, and have labelset of Person, Organization, Location, Miscellaneous.

The remaining four languages come from the LORELEI project BIBREF23. These languages are: Amharic (amh: LDC2016E87), Arabic (ara: LDC2016E89), Hindi (hin: LDC2017E62), and Somali (som: LDC2016E91). These come from a variety of sources including discussion forums, newswire, and social media.  
 Question: Which languages are evaluated?
Output: Bengali English, German, Spanish, Dutch Amharic Arabic Hindi Somali
Traditional text-to-speech (TTS) systems are composed of complex pipelines BIBREF0 , these often include accoustic frontends, duration model, acoustic prediction model and vocoder models. Neural text-to-speech systems have garnered large research interest in the past 2 years. The first to fully explore this avenue of research was Google's tacotron BIBREF1 system. The architecture of our model utilizes RNN-based Seq2Seq model for generating mel spectrogram from text. The architecture is similar to that of Tacotron 2 BIBREF4 In the original Tacotron 2, the attention mechanism used was location sensitive attention BIBREF12 combined the original additive Seq2Seq BIBREF7 Bahdanau attention.

We propose to replace this attention with the simpler query-key attention from transformer model Following the logic above, we utilize a similar method from BIBREF6 that adds an additional guided attention loss to the overall loss objective, which acts to help the attention mechanism become monotoic as early as possible. As seen from FIGREF24 , an attention loss mask, INLINEFORM0 , is created applies a loss to force the attention alignment, INLINEFORM1 , to be nearly diagonal. 
 Question: Which modifications do they make to well-established Seq2seq architectures?
Output: Replacing attention mechanism to query-key attention, and adding a loss to make the attention mask as diagonal as possible
We work with a dataset composed of 3,206 news articles, each labeled real or fake, with a perfect 50/50 split between 1,603 real and fake articles, respectively. 
 Question: What is the size of the dataset?
Output: 3,206
Multimodal representation.
We combined textual and image representations in two simple ways. The first method is concatenation of the text and image representation (concat). Before concatenation we applied the L2 normalization to each of the modalities. The second method it to learn a common space for the two modalities before concatenation (project).

The projection of each modality learns a space of $d$-dimensions, so that $h_{1}, h_{2} \in \mathbb {R}^{d}$. Once the multimodal representation is produced ($h_{m}$) for the left and right pairs, vectors are directly plugged into the regression layers. Projections are learned end-to-end with the regression layers and the MSE as loss function. 
 Question: What multimodal representations are used in the experiments?
Output: The second method it to learn a common space for the two modalities before concatenation (project) The first method is concatenation of the text and image representation (concat)
To show the effectiveness of our approach, we show results on the SICK dataset BIBREF1, a common benchmark for logic-based NLI, and find MonaLog to be competitive with more complicated logic-based approaches (many of which require full semantic parsing and more complex logical machinery). 
 Question: Do they beat current state-of-the-art on SICK?
Output: No
We measure how contextual a word representation is using three different metrics: self-similarity, intra-sentence similarity, and maximum explainable variance. 
 Question: What experiments are proposed to test that upper layers produce context-specific embeddings?
Output: They measure self-similarity, intra-sentence similarity and maximum explainable variance of the embeddings in the upper layers.
In this paper, we study verifiable robustness, i.e., providing a certificate that for a given network and test input, no attack or perturbation under the specification can change predictions, using the example of text classification tasks, Stanford Sentiment Treebank (SST) BIBREF15 and AG News BIBREF16.  
 Question: Which dataset do they use?
Output: Stanford Sentiment Treebank (SST) BIBREF15 and AG News BIBREF16
A dialog turn from one speaker may not only be a direct response to the other speaker's query, but also likely to be a continuation of his own previous statement. Thus, when modeling turn $k$ in a dialog, we propose to connect the last RNN state of turn $k-2$ directly to the starting RNN state of turn $k$ , instead of letting it to propagate through the RNN for turn $k-1$ . 
 Question: How long of dialog history is captured?
Output: two previous turns
In this work, we look at a wide range of attributes and report prediction results on 62 demographic attributes. 
 Question: How many demographic attributes they try to predict?
Output: 62
Conditional Random Fields
Conditional Random Fields (CRF) BIBREF10 are a standard approach when dealing with sequential data in the context of sequence labeling. BiLSTM-CRF
Prior to the emergence of deep neural language models, BiLSTM-CRF models BIBREF11 had achieved state-of-the-art results for the task of sequence labeling. Multi-Task Learning
Multi-Task Learning (MTL) BIBREF15 has become popular with the progress in deep learning. BioBERT
Deep neural language models have recently evolved to a successful method for representing text. In particular, Bidirectional Encoder Representations from Transformers (BERT) outperformed previous state-of-the-art methods by a large margin on various NLP tasks BIBREF17. 
 Question: What baselines do they introduce?
Output: Conditional Random Fields BiLSTM-CRF Multi-Task Learning BioBERT
We evaluate the proposed transfer learning techniques in two non-English language pairs of WMT 2019 news translation tasks: French$\rightarrow $German and German$\rightarrow $Czech. 
 Question: Are experiments performed with any other pair of languages, how did proposed method perform compared to other models?
Output: No
Following previous work BIBREF13, BIBREF12, BIBREF10, BIBREF17, BIBREF9, BIBREF7, we choose SemCor3.0 as training corpus, which is the largest corpus manually annotated with WordNet sense for WSD. 
 Question: Is SemCor3.0 reflective of English language data in general?
Output: Yes
We provide three pairs of short/long datasets from different domains (movies and restaurants) and from different languages (English and Korean) suitable for the task: Mov_en, Res_en, and Mov_ko. Most of the datasets are from previous literature and are gathered differently The Mov_en datasets are gathered from different websites; the short dataset consists of hand-picked sentences by BIBREF19 from document-level reviews from the Rotten Tomatoes website, while the long dataset consists of reviews from the IMDB website obtained by BIBREF20. 
 Question: What dierse domains and languages are present in new datasets?
Output: movies  restaurants English  Korean
According to our observation, if a vernacular paragraph contains more poetic images used in classical literature, its generated poem usually achieves higher score. We also observed that poems generated from descriptive paragraphs achieve higher scores than from logical or philosophical paragraphs. 
 Question: What are some guidelines in writing input vernacular so model can generate 
Output:  if a vernacular paragraph contains more poetic images used in classical literature, its generated poem usually achieves higher score poems generated from descriptive paragraphs achieve higher scores than from logical or philosophical paragraphs
The data presented in this paper was collected and validated via Mozilla's Common Voice initiative. Using either the Common Voice website or iPhone app, contributors record their voice by reading sentences displayed on the screen (see Figure (FIGREF5)). The recordings are later verified by other contributors using a simple voting system. Shown in Figure (FIGREF6), this validation interface has contributors mark $<$audio,transcript$>$ pairs as being either correct (up-vote) or incorrect (down-vote). 
 Question: What crowdsourcing platform is used for data collection and data validation?
Output: the Common Voice website  iPhone app
Our multi-task BERT models involve six different Arabic classification tasks. Author profiling and deception detection in Arabic (APDA). LAMA+DINA Emotion detection. Sentiment analysis in Arabic tweets. 
 Question: What are the tasks used in the mulit-task learning setup?
Output: Author profiling and deception detection in Arabic LAMA+DINA Emotion detection Sentiment analysis in Arabic tweets
We use a siamese neural network, shown to perform state-of-the-art few-shot learning BIBREF11, as our baseline model. We modify the original to account for sequential data, with each twin composed of an embedding layer, a Long-Short Term Memory (LSTM) BIBREF12 layer, and a feed-forward layer with Rectified Linear Unit (ReLU) activations. 
 Question: What were the baselines?
Output: Siamese neural network consisting of an embedding layer, a LSTM layer and a feed-forward layer with ReLU activations
Moreover, QA and QG have probabilistic correlation as both tasks relate to the joint probability between $q$ and $a$ . Given a question-answer pair $\langle q, a \rangle $ , the joint probability $P(q, a)$ can be computed in two equivalent ways.

$$P(q, a) = P(a) P(q|a) = P(q)P(a|q)$$ (Eq. 1)

The conditional distribution $P(q|a)$ is exactly the QG model, and the conditional distribution $P(a|q)$ is closely related to the QA model. Existing studies typically learn the QA model and the QG model separately by minimizing their own loss functions, while ignoring the probabilistic correlation between them.

Based on these considerations, we introduce a training framework that exploits the duality of QA and QG to improve both tasks. There might be different ways of exploiting the duality of QA and QG. In this work, we leverage the probabilistic correlation between QA and QG as the regularization term to influence the training process of both tasks. Specifically, the training objective of our framework is to jointly learn the QA model parameterized by $\theta _{qa}$ and the QG model parameterized by $\theta _{qg}$ by minimizing their loss functions subject to the following constraint.

$$P_a(a) P(q|a;\theta _{qg}) = P_q(q)P(a|q;\theta _{qa})$$ (Eq. 3)

$P_a(a)$ and $P_q(q)$ are the language models for answer sentences and question sentences, respectively. Overall, the framework includes three components, namely a QA model, a QG model and a regularization term that reflects the duality of QA and QG. The QA specific objective aims to minimize the loss function $l_{qa}(f_{qa}(a,q;\theta _{qa}), label)$ , where $label$ is 0 or 1 that indicates whether $a$ is the correct answer of $q$ or not. For each correct question-answer pair, the QG specific objective is to minimize the following loss function,

$$l_{qg}(q, a) = -log P_{qg}(q|a;\theta _{qg})$$ (Eq. 6)

where $a$ is the correct answer of $q$ . The third objective is the regularization term which satisfies the probabilistic duality constrains as given in Equation 3 . Specifically, given a correct $\langle q, a \rangle $ pair, we would like to minimize the following loss function,

$$ \nonumber l_{dual}(a,q;\theta _{qa}, \theta _{qg}) &= [logP_a(a) + log P(q|a;\theta _{qg}) \\ & - logP_q(q) - logP(a|q;\theta _{qa})]^2$$ (Eq. 9)

where $P_a(a)$ and $P_q(q)$ are marginal distributions, which could be easily obtained through language model. 
 Question: What does "explicitly leverages their probabilistic correlation to guide the training process of both models" mean?
Output: The framework jointly learns parametrized QA and QG models subject to the constraint in equation 2. In more detail, they minimize QA and QG loss functions, with a third dual loss for regularization.
It should also be noted that signals can be recognized even when the model misclassifies relations, since ${\Delta }_s$ does not rely on correct classification: it merely quantifies the contribution of a word in context toward the correct label's score. If we examine the influence of each word on the score of the correct relation, that impact should and does still correlate with human judgments based on what the system may tag as the second or third best class to choose. 
 Question: Where does proposed metric overlap with juman judgement?
Output: influence of each word on the score of the correct relation, that impact should and does still correlate with human judgments
We choose Zulu (ZUL), a Bantoid language. In contrast to the first two, it is strongly prefixing. 
 Question: What is an example of a prefixing language?
Output: Zulu
Although the competition proposes two different scenarios, in fact, both are guided by the snomed ct ontology —for subtask 1, entities must be identified with offsets and mapped to a predefined set of four classes (PROTEINAS, NORMALIZABLES, NO_NORMALIZABLES and UNCLEAR); for subtask 2, a list of all snomed ct ids (sctid) for entities occurring in the text must be given, which has been called concept indexing by the shared task organizers. 
 Question: What are the two PharmaCoNER subtasks?
Output: Entity identification with offset mapping and concept indexing