niryuu/sni-each-converted huggingface dataset:
Even though this corpus has incorrect sentences and their emotional labels, they lack their respective corrected sentences, necessary for the training of our model. In order to obtain this missing information, we outsource native English speakers from an unbiased and anonymous platform, called Amazon Mechanical Turk (MTurk) BIBREF19, which is a paid marketplace for Human Intelligence Tasks (HITs). We use this platform to create tasks for native English speakers to format the original incorrect tweets into correct sentences. Some examples are shown in Table TABREF12. The dataset used to evaluate the models' performance is the Chatbot Natural Language Unerstanding (NLU) Evaluation Corpus, introduced by Braun et al. BIBREF20 to test NLU services. It is a publicly available benchmark and is composed of sentences obtained from a German Telegram chatbot used to answer questions about public transport connections. The dataset has two intents, namely Departure Time and Find Connection with 100 train and 106 test samples, shown in Table TABREF18. Even though English is the main language of the benchmark, this dataset contains a few German station and street names. 
 Question: Do they report results only on English datasets?
Output: Yes-no
Evaluation is done using the development set, consisting of 22 documents and 1112 PIE candidates, and the test set, which consists of 23 documents and 1127 PIE candidates. For each method the best set of parameters and/or options is determined using the development set, after which the best variant by F1-score of each method is evaluated on the test set.

Since these documents in the corpus are exhaustively annotated for PIEs (see Section SECREF40), we can calculate true and false positives, and false negatives, and thus precision, recall and F1-score. 
 Question: Are PIEs extracted automatically subjected to human evaluation?
Output: Yes-no
Before designing this NLP toolkit, we conducted a survey among engineers and identified a spectrum of three typical personas. 
 Question: How do the authors evidence the claim that many engineers find it a big overhead to choose from multiple frameworks, models and optimization techniques?
Output: Abstractive
To train our model, we generated a dataset of 20,000 demonstrated 7 DOF trajectories (6 robot joints and 1 gripper dimension) in our simulated environment together with a sentence generator capable of creating natural task descriptions for each scenario. In order to create the language generator, we conducted an human-subject study to collect sentence templates of a placement task as well as common words and synonyms for each of the used features. By utilising these data, we are able to generate over 180,000 unique sentences, depending on the generated scenario. To test our model, we generated 500 new scenario testing each of the three features to identify the correct target among other bowls.  
 Question: Does proposed end-to-end approach learn in reinforcement or supervised learning manner?
Output: Abstractive
Inspired by the line of work reported in these studies, we propose to use modified objective functions for a different purpose: learning more interpretable dense word embeddings. By doing this, we aim to incorporate semantic information from an external lexical resource into the word embedding so that the embedding dimensions are aligned along predefined concepts. This alignment is achieved by introducing a modification to the embedding learning process. In our proposed method, which is built on top of the GloVe algorithm BIBREF2 , the cost function for any one of the words of concept word-groups is modified by the introduction of an additive term to the cost function.  
 Question: What is the additive modification to the objective function?
Output: Abstractive
We test the existing debiasing approaches, CDA and REG, as well but since BIBREF5 reported that results fluctuate substantially with different REG regularization coefficients, we perform hyperparameter tuning and report the best results in Table TABREF12 . 
 Question: which existing strategies are compared?
Output: Extractive
Inspired by the psycholinguistic studies BIBREF5 , BIBREF4 , BIBREF7 proposed a semi-character based RNN (ScRNN) that processes a sentence of words with misspelled characters, predicting the correct words at each step. Let $s = \lbrace w_1, w_2, \dots , w_n\rbrace $ denote the input sentence, a sequence of constituent words $w_i$ . Each input word ( $w_i$ ) is represented by concatenating (i) a one hot vector of the first character ( $\mathbf {w_{i1}}$ ); (ii) a one hot representation of the last character ( $\mathbf {w_{il}}$ , where $l$ is the length of word $w_i$ ); and (iii) a bag of characters representation of the internal characters ( $\sum _{j=2}^{l-1}\mathbf {w_{ij}})$ . ScRNN treats the first and the last characters individually, and is agnostic to the ordering of the internal characters. Each word, represented accordingly, is then fed into a BiLSTM cell. At each sequence step, the training target is the correct corresponding word (output dimension equal to vocabulary size), and the model is optimized with cross-entropy loss. 
 Question: What is a semicharacter architecture?
Output: Abstractive
To help the human representative quickly determine the cause of the escalation, we generate a visualization of the user's turns using the attention weights to highlight the turns influential in the escalation decision. 
 Question: Do they explain model predictions solely on attention weights?
Output: Yes-no
Pre-trained English NER model We construct the English NER system following BIBREF7 . This system uses a bidirectional LSTM as a character-level language model to take context information for word embedding generation.  For pre-trained English NER system, we use the default NER model of Flair. 
 Question: Which pre-trained English NER model do they use?
Output: Abstractive
Accordingly, marcheggiani2017 presented a neural model putting syntax aside for dependency-based SRL and obtain favorable results, which overturns the inherent belief that syntax is indispensable in SRL task BIBREF11 . 
 Question: Are there syntax-agnostic SRL models before?
Output: Yes-no
In this section we discuss the state of the art on conversational systems in three perspectives: types of interactions, types of architecture, and types of context reasoning. ELIZA BIBREF11 was one of the first softwares created to understand natural language processing.  Right after ELIZA came PARRY, developed by Kenneth Colby, who is psychiatrist at Stanford University in the early 1970s. A.L.I.C.E. (Artificial Linguistic Internet Computer Entity) BIBREF12 appeared in 1995 but current version utilizes AIML, an XML language designed for creating stimulus-response chat robots BIBREF13 . Cleverbot (1997-2014) is a chatbot developed by the British AI scientist Rollo Carpenter.  
 Question: What is the state of the art described in the paper?
Output: Extractive
Starting with this list, we can locate the profile page for a user, and subsequently extract additional information, which includes fields such as name, email, occupation, industry, and so forth. We also generate two maps that delineate the gender distribution in the dataset. Our dataset provides mappings between location, profile information, and language use, which we can leverage to generate maps that reflect demographic, linguistic, and psycholinguistic properties of the population represented in the dataset. 
 Question: Which demographic dimensions of people do they obtain?
Output: Extractive
For PHASE-ONE, we randomly shuffled and divided the data (1913 signals from 14 individuals) into train (80%), development (10%) and test sets (10%). In PHASE-TWO, in order to perform a fair comparison with the previous methods reported on the same dataset, we perform a leave-one-subject out cross-validation experiment using the best settings we learn from PHASE-ONE. 
 Question: How many electrodes were used on the subject in EEG sessions?
Output: Extractive
For each object, $O_i$ in the input Open IE tuple $(S; P; O_1; O_2 \ldots )$ , we add a triple $(S; P; O_i)$ to this table. 
 Question: Are the OpenIE extractions all triples?
Output: Yes-no
We decided to use sentences involving at least one race- or gender-associated word. The sentences were intended to be short and grammatically simple. We also wanted some sentences to include expressions of sentiment and emotion, since the goal is to test sentiment and emotion systems. 
 Question: What criteria are used to select the 8,640 English sentences?
Output: Abstractive
We used the relation classification dataset of the SemEval 2010 task 8 BIBREF8 . It consists of sentences which have been manually labeled with 19 relations (9 directed relations and one artificial class Other). 8,000 sentences have been distributed as training set and 2,717 sentences served as test set. 
 Question: Which dataset do they train their models on?
Output: Extractive
Then, we created test sets with varying levels of perturbation operations - $\lbrace 20\%,40\%,60\%\rbrace $. 
 Question: Are recurrent neural networks trained on perturbed data?
Output: Yes-no
GhostVLAD is an extension of the NetVLAD approach, which we discussed in the previous section. GhostVLAD works exactly similar to NetVLAD except it adds Ghost clusters along with the NetVLAD clusters. So, now we will have a K+G number of clusters instead of K clusters. The Ghost clusters are added to map any noisy or irrelevant content into ghost clusters and are not included during the feature aggregation stage, as shown in Figure 1 (Right side). 
 Question: What is the GhostVLAD approach?
Output: Extractive
We evaluate our approaches on two public Chinese span-extraction machine reading comprehension datasets: CMRC 2018 (simplified Chinese) BIBREF8 and DRCD (traditional Chinese) BIBREF9. The statistics of the two datasets are listed in Table TABREF29. 
 Question: Is this a span-based (extractive) QA task?
Output: Yes-no
Subsequently, an LSTM decoder generates the characters in the output word form using encoder states and an attention mechanism. 
 Question: Does the model have attention?
Output: Yes-no
For this knowledge, we create a knowledge base by counting how many times a predicate-argument tuple appears in a corpus and use the resulted number to represent the preference strength. Specifically, we use the English Wikipedia as the base corpus for such counting. 
 Question: What is the source of external knowledge?
Output: Abstractive
We started by answering always YES (in batch 2 and 3) to get the baseline performance. For batch 4 we used entailment. 
 Question: What was the baseline model?
Output: Extractive
It has been shown that one can significantly increase the semantic information carried by a NER system when we successfully linking entities from a deep learning method to the related entities from a knowledge base BIBREF26 , BIBREF27 . Redirection: For the Wikidata linking element, we recognize that the lookup will be constrained by the most common lookup name for each entity.  
 Question: How do they combine a deep learning model with a knowledge base?
Output: Abstractive
The unlabeled set of development data was used in the training of both the UBM and the i-vector extractor. 
 Question: Do they single out a validation set from the fixed SRE training set?
Output: Yes-no
First, we propose a class of recurrent-like neural networks for NLP tasks that satisfy the differential equation DISPLAYFORM0

where DISPLAYFORM0

and where INLINEFORM0 and INLINEFORM1 are learned functions. INLINEFORM2 corresponds to traditional RNNs, with INLINEFORM3 . For INLINEFORM4 , this takes the form of RNN cells with either nested internal memories or dependencies that extend temporally beyond the immediately previous hidden state.  
 Question: What novel class of recurrent-like networks is proposed?
Output: Abstractive
Embedding: We developed different variations of our models with a simple lookup table embeddings learned from scratch and using high-performance contextual embeddings, which are ELMo BIBREF11, BERT BIBREF16 and ClinicalBERT BIBREF13 (trained and provided by the authors). 
 Question: What embeddings are used?
Output: Extractive
One of the major drawbacks of SCRF models is their high computational cost. In our experiments, the CTC model is around 3-4 times faster than the SRNN model that uses the same RNN encoder. To cut down the computational cost, we investigated if CTC can be used to pretrain the RNN encoder to speed up the training of the joint model. Figure 2 shows the convergence curves of the joint model with and without CTC pretraining, and we see pretraining indeed improves the convergence speed of the joint model. 
 Question: Can SCRF be used to pretrain the model?
Output: Yes-no
However, all previous IE benchmarks BIBREF18 are too small to train neural network models typically used in QA, and thus we need to build a large benchmark. Therefore, we build a large scale benchmark named QA4IE benchmark which consists of 293K Wikipedia articles and 2M golden relation triples with 636 different relation types. We manually find 148 relations which can be projected to a WikiData relation out of 2064 DBpedia relations. 
 Question: Was this benchmark automatically created from an existing dataset?
Output: Yes-no
This step entails counting occurrences of all words in the training corpus and sorting them in order of decreasing occurrence. As mentioned, the vocabulary is taken to be the INLINEFORM0 most frequently occurring words, that occur at least some number INLINEFORM1 times. It is implemented in Spark as a straight-forward map-reduce job. 
 Question: Do they perform any morphological tokenization?
Output: Yes-no
We recruited 100 crowdworkers on Amazon Mechanical Turk (AMT) and measured completion times and accuracies for typing randomly sampled sentences from the Yelp corpus.  
 Question: How many participants were trying this communication game?
Output: Extractive
We constructed our seed lexicon consisting of 15 positive words and 15 negative words, as shown in Section SECREF27.  
 Question: How big is seed lexicon used for training?
Output: Abstractive
The Block Zoo is an open framework, and more modules can be added in the future. Embedding Layer: Word/character embedding and extra handcrafted feature embedding such as pos-tagging are supported. Neural Network Layers: Block zoo provides common layers like RNN, CNN, QRNN BIBREF2 , Transformer BIBREF3 , Highway network, Encoder Decoder architecture, etc. Furthermore, attention mechanisms are widely used in neural networks. Thus we also support multiple attention layers, such as Linear/Bi-linear Attention, Full Attention BIBREF4 , Bidirectional attention flow BIBREF5 , etc. Meanwhile, regularization layers such as Dropout, Layer Norm, Batch Norm, etc are also supported for improving generalization ability. Loss Function: Besides of the loss functions built in PyTorch, we offer more options such as Focal Loss BIBREF6 .

Metrics: For classification task, AUC, Accuracy, Precision/Recall, F1 metrics are supported. For sequence labeling task, F1/Accuracy are supported. For knowledge distillation task, MSE/RMSE are supported. For MRC task, ExactMatch/F1 are supported. 
 Question: What neural network modules are included in NeuronBlocks?
Output: Extractive
We pit our model against the following baselines: 1) SVM with unigram, bigram, and trigram features, which is a standard yet rather strong classifier for text features; 2) SVM with average word embedding, where a document is represented as a continuous representation by averaging the embeddings of the composite words; 3) SVM with average transformed word embeddings (the INLINEFORM0 in equation EQREF6 ), where a document is represented as a continuous representation by averaging the transformed embeddings of the composite words; 4) two mature deep learning models on text classification, CNN BIBREF3 and Recurrent Convolutional Neural Networks (RCNN) BIBREF0 , where the hyperparameters are based on their work; 5) the above SVM and deep learning models with comment information;  
 Question: What are the baselines?
Output: Extractive
The SnapCaptions dataset is composed of 10K user-generated image (snap) and textual caption pairs where named entities in captions are manually labeled by expert human annotators (entity types: PER, LOC, ORG, MISC). 
 Question: Which types of named entities do they recognize?
Output: Extractive
In the first task, participants were instructed to read the sentences naturally, without any specific task other than comprehension. Participants were told to read the sentences normally without any special instructions. 
 Question: What is a normal reading paradigm?
Output: Extractive
We thus try to remove the language-specific information from the representations by centering the representations of sentences in each language so that their average lies at the origin of the vector space. 
 Question: Are language-specific and language-neutral components disjunctive?
Output: Yes-no
The CRF module achieved the best result on the Thai sentence segmentation task BIBREF8 ; therefore, we adopt the Bi-LSTM-CRF model as our baseline.  
 Question: Which deep learning architecture do they use for sentence segmentation?
Output: Extractive
Moreover, because English does not mark grammatical gender, approaches developed for English are not transferable to morphologically rich languages that exhibit gender agreement BIBREF8 . 
 Question: Why does not the approach from English work on other languages?
Output: Abstractive
RKS approach proposed in BIBREF21, BIBREF22, explicitly maps data vectors to a space where linear separation is possible. The RKS method provides an approximate kernel function via explicit mapping. 
 Question: What is the Random Kitchen Sink approach?
Output: Abstractive
To develop the dataset, we extracted 200 sentences from Sorani Kurdish books of grades one to three of the primary school in the Kurdistan Region of Iraq. We randomly created 2000 sentences from the extracted sentences. 
 Question: How was the dataset collected?
Output: Abstractive
In this work, we consider words that can indicate the characteristics of the neighbor words as contextual keywords and develop an approach to generate features from the automatically extracted contextual keywords. 
 Question: What contextual features are used?
Output: Abstractive
To verify our assumption that target encoding and orthogonal regularization help to boost the diversity of generated sequences, we use two metrics, one quantitative and one qualitative, to measure diversity of generation. First, we simply calculate the average unique predictions produced by both INLINEFORM0 and INLINEFORM1 in experiments shown in Section SECREF36 .  In this example there are 29 ground truth phrases. Neither of the models is able to generate all of the keyphrases, but it is obvious that the predictions from INLINEFORM0 all start with “test”, while predictions from INLINEFORM1 are diverse. 
 Question: How is keyphrase diversity measured?
Output: Extractive
In order to make the corpus collection easier and faster, we adopted a semi-automatic procedure based on sequential neural models BIBREF19, BIBREF20. Instead, since tokens are transcribed at morpheme level, we split Arabish tokens into characters, and Arabic tokens into morphemes, and we treated each token itself as a sequence. Our model learns thus to map Arabish characters into Arabic morphemes. With this model we automatically transcribed into Arabic morphemes, roughly, 5,000 additional tokens, corresponding to the second annotation block.  Manual transcription plus a 
 Question: How does the semi-automatic construction process work?
Output: Abstractive
Vanilla ST baseline: The vanilla ST BIBREF9 has only a speech encoder and a decoder. It is trained from scratch on the ST-TED corpus.

Pre-training baselines: We conduct three pre-training baseline experiments: 1) encoder pre-training, in which the ST encoder is initialized from an ASR model; 2) decoder pre-training, in which the ST decoder is initialized from an MT model; and 3) encoder-decoder pre-training, where both the encoder and decoder are pre-trained. The ASR model has the same architecture with vanilla ST model, trained on the mixture of ST-TED and TED-LIUM2 corpus. The MT model has a text encoder and decoder with the same architecture of which in TCEN. It is first trained on WMT data (out-of-domain) and then fine-tuned on in-domain data.

Multi-task baselines: We also conduct three multi-task baseline experiments including one-to-many setting, many-to-one setting, and many-to-many setting. In the first two settings, we train the model with $\alpha _{st}=0.75$ while $\alpha _{asr}=0.25$ or $\alpha _{mt}=0.25$. For many-to-many setting, we use $\alpha _{st}=0.6, \alpha _{asr}=0.2$ and $\alpha _{mt}=0.2$.. For MT task, we use only in-domain data.

Many-to-many+pre-training: We train a many-to-many multi-task model where the encoders and decoders are derived from pre-trained ASR and MT models. Triangle+pre-train: BIBREF18 DBLP:conf/naacl/AnastasopoulosC18 proposed a triangle multi-task strategy for speech translation. Their model solves the subnet waste issue by concatenating an ST decoder to an ASR encoder-decoder model. Notably, their ST decoder can consume representations from the speech encoder as well as the ASR decoder. For a fair comparison, the speech encoder and the ASR decoder are initialized from the pre-trained ASR model. The Triangle model is fine-tuned under their multi-task manner. 
 Question: What are the baselines?
Output: Extractive
The dataset comprises a total of 353 conversations from 40 speakers (11 nurses, 16 patients, and 13 caregivers) with consent to the use of anonymized data for research. In Section SECREF16 , we build templates and expression pools using linguistic analysis followed by manual verification. 
 Question: How big is their created dataset?
Output: Extractive
The challenge is addressed as follows: given a natural language input sequence describing the scene, such as a piece of a story coming from a transcript, the goal is to infer which action is most likely to happen next. 
 Question: Do they literally just treat this as "predict the next spell that appears in the text"?
Output: Yes-no
For the evaluation of performance of the proposed method on the NLI task, SNLI BIBREF22 and MultiNLI BIBREF23 datasets are used. We use Quora Question Pairs dataset BIBREF24 in evaluating the performance of our method on the PI task. In evaluating sentiment classification performance, the Stanford Sentiment Treebank (SST) BIBREF25 is used. 
 Question: Which datasets were used?
Output: Extractive
Paraphrases can be obtained by translating an English string into a foreign language and then back-translating it into English.  
 Question: It looks like learning to paraphrase questions, a neural scoring model and a answer selection model cannot be trained end-to-end. How are they trained?
Output: Abstractive
The text and image encodings were combined by concatenation, which resulted in a feature vector of 4,864 dimensions. This multimodal representation was afterward fed as input into a multi-layer perceptron (MLP) with two hidden layer of 100 neurons with a ReLU activation function. 
 Question: Is the dataset multimodal?
Output: Yes-no
Our final survey based classification results showed an accuracy of 96% in detecting PTSD and mean squared error of 1.2 in estimating its intensity given we have four intensity, No PTSD, Low Risk PTSD, Moderate Risk PTSD and High Risk PTSD with a score of 0, 1, 2 and 3 respectively.  
 Question: How is the intensity of the PTSD established?
Output: Abstractive
Dataset Quality Analysis ::: Inter-Annotator Agreement (IAA)
To estimate dataset consistency across different annotations, we measure F1 using our UA metric with 5 generators per predicate. Dataset Quality Analysis ::: Dataset Assessment and Comparison
We assess both our gold standard set and the recent Dense set against an integrated expert annotated sample of 100 predicates.  Dataset Quality Analysis ::: Agreement with PropBank Data
It is illuminating to observe the agreement between QA-SRL and PropBank (CoNLL-2009) annotations BIBREF7.  
 Question: How was quality measured?
Output: Abstractive
Active learning methods can generally be described into two parts: a learning engine and a selection engine BIBREF28 . The learning engine is essentially a classifier, which is mainly used for training of classification problems. The selection engine is based on the sampling strategy, which chooses samples that need to be relabeled by annotators from unlabeled data. Then, relabeled samples are added to training set for classifier to re-train, thus continuously improving the accuracy of the classifier. In this paper, a CRF-based segmenter and a scoring model are employed as learning engine and selection engine, respectively. 
 Question: How does the active learning model work?
Output: Abstractive
Knowledge Base Question Answering (KBQA) systems answer questions by obtaining information from KB tuples BIBREF0 , BIBREF1 , BIBREF2 , BIBREF3 , BIBREF4 , BIBREF5 . 
 Question: What is te core component for KBQA?
Output: Extractive
Despite this, we found that the training of our embeddings was not considerably slower than the training of order-2 equivalents such as SGNS. Explicitly, our GPU trained CBOW vectors (using the experimental settings found below) in 3568 seconds, whereas training CP-S and JCP-S took 6786 and 8686 seconds respectively. 
 Question: Do they measure computation time of their factorizations compared to other word embeddings?
Output: Yes-no
Using the distribution of the individual words in a category, we can compile distributions for the entire category, and therefore generate maps for these word categories.  
 Question: Do they build a model to automatically detect demographic, lingustic or psycological dimensons of people?
Output: Yes-no
Furthermore, the gains in WER over the baseline are significantly larger for the Density Ratio method than for Shallow Fusion, with up to 28% relative reduction in WER (17.5% $\rightarrow $ 12.5%) compared to up to 17% relative reduction (17.5% $\rightarrow $ 14.5%) for Shallow Fusion, in the no fine-tuning scenario. 
 Question: What metrics are used for evaluation?
Output: Abstractive
Given that there is no public dataset available with financial intents in Portuguese, we have employed the incremental approach to create our own training set for the Intent Classifier. We have created domain-specific word vectors by considering a set 246,945 documents, corresponding to of 184,001 Twitter posts and and 62,949 news articles, all related to finance . 
 Question: What datasets are used?
Output: Abstractive
In its general form, a type-logical grammar consists of following components: 
 Question: Does Grail accept Prolog inputs?
Output: Yes-no
The baseline is a hierarchical phrase-based system BIBREF29 with a 4-gram language model, with feature weights tuned using MIRA BIBREF30 . Therefore, an appropriate additional baseline would be to mark translation rules with these indicator functions but without the scores, akin to identifying rules with phrases in them (Baseline + SegOn). 
 Question: Which translation systems do they compare against?
Output: Extractive
BIBREF17: The Lemming model is a log-linear model that performs joint morphological tagging and lemmatization.  
 Question: What were the non-neural baselines used for the task?
Output: Abstractive
We then train a paragraph vector model using the Document to Vector (Doc2Vec) framework BIBREF7 on the whole set (13 million) of preprocessed text records, although training on smaller sets (1 million) also produces good results. 
 Question: Which text embedding methodologies are used?
Output: Extractive
Another approach for generating query and image representations is treating images as a black box. Given the statistics of our dataset (3B query, image pairs with 220M unique queries and 900M unique images), we know that different queries co-occur with the same images. Intuitively, if a query $q_1$ co-occurs with many of the same images as query $q_2$ , then $q_1$ and $q_2$ are likely to be semantically similar, regardless of the visual content of the shared images. Thus, we can use a method that uses only co-occurrence statistics to better understand how well we can capture relationships between queries. 
 Question: Could you learn such embedding simply from the image annotations and without using visual information?
Output: Yes-no
We evaluate our model on a publicly available dataset, KARA ONE BIBREF17 , composed of multimodal data for stimulus-based, imagined and articulated speech state corresponding to 7 phonemic/syllabic ( /iy/, /piy/, /tiy/, /diy/, /uw/, /m/, /n/ ) as well as 4 words(pat, pot, knew and gnaw). The dataset consists of 14 participants, with each prompt presented 11 times to each individual.  
 Question: What data was presented to the subjects to elicit event-related responses?
Output: Extractive
Each game's video ranges from 30 to 50 minutes in length which contains image and chat data linked to the specific timestamp of the game. 
 Question: What is the average length of the recordings?
Output: Abstractive
To better analyze model generalization to an unseen, new domain as well as model leveraging the out-of-domain sources, we propose a new architecture which is an extension of the ARED model. In order to better select, aggregate and control the semantic information, a Refinement Adjustment LSTM-based component (RALSTM) is introduced to the decoder side. The proposed model can learn from unaligned data by jointly training the sentence planning and surface realization to produce natural language sentences.  
 Question: What is the difference of the proposed model with a standard RNN encoder-decoder?
Output: Abstractive
Despite its usefulness, linked entities extracted from ELS's have issues because of low precision rates BIBREF11 and design challenges in training datasets BIBREF12 . First, the extracted entities may be ambiguous. Second, the linked entities may also be too common to be considered an entity.  
 Question: Why are current ELS's not sufficiently effective?
Output: Abstractive
Similarly the agents utterances can be clustered to identify system responses. However, we argue that rather than treating user utterances and agents responses in an isolated manner, there is merit in jointly clustering them. There is adjacency information of these utterances that can be utilized to identify better user intents and system responses. 
 Question: Do they study frequent user responses to help automate modelling of those?
Output: Yes-no
For instance, we observed that winter words were together and far away from summer ones. Week days were grouped as well and far from week-end days. 
 Question: What geometric properties do embeddings display?
Output: Abstractive
we take the CORD-19 dataset BIBREF2, which contains over 45,000 scholarly articles, including over 33,000 with full text, about COVID-19, SARS-CoV-2, and related coronaviruses. We develop sentence classification methods to identify all sentences narrating radiological findings from COVID-19.  We conduct experiments to verify the effectiveness of our method. From the CORD-19 dataset, our method successfully discovers a set of clinical findings that are closely related with COVID-19. 
 Question: How large is the collection of COVID-19 literature?
Output: Extractive
For the competition, we provided 1,000 labeled dialogues from each dataset for training, and 240 unlabeled dialogues from each dataset for evaluation.  
 Question: How large is the first dataset?
Output: Abstractive
Since we do not have an end-to-end dataset, the generated English poem may not work well with Shakespeare style transfer as shown in Figure FIGREF12 for "Starry Night" with a low average content score. This happens when the style transfer dataset does not have similar words in the training set of sentences. A solution would be to expand the style transfer dataset, for a better representation of the poem data. 
 Question: What limitations do the authors demnostrate of their model?
Output: Extractive
The annotation process was a trial-and-error, with cycles composed of annotation, discussing confusing entities, updating the annotation guide schematic and going through the corpus section again to correct entities following guide changes. 
 Question: Did they experiment with the corpus?
Output: Yes-no
Three annotators, A1-A3, mark each tweet in the Dataset H as drunk or sober. 
 Question: Is the data acquired under distant supervision verified by humans at any stage?
Output: Yes-no
One distinguishing feature of educational topics is their breadth of sub-topics and points of view, as they attract researchers, practitioners, parents, students, or policy-makers. We assume that this diversity leads to the linguistic variability of the education topics and thus represents a challenge for NLP. 
 Question: What challenges do different registers and domains pose to this task?
Output: Extractive
We leveraged an existing, annotated Twitter dataset that was constructed based on a hierarchical model of depression-related symptoms BIBREF12 , BIBREF13 . The dataset contains 9,473 annotations for 9,300 tweets. Each tweet is annotated as no evidence of depression (e.g., “Citizens fear an economic depression") or evidence of depression (e.g., “depressed over disappointment").  
 Question: Do they evaluate only on English datasets?
Output: Yes-no
The data set we evaluate on in this work is WMT English-French NewsTest2014, which has 380M words of parallel training data and a 3003 sentence test set. The NewsTest2013 set is used for validation. 
 Question: Do they only test on one dataset?
Output: Yes-no
Our Transformer Transducer model architecture has 18 audio and 2 label encoder layers. Every layer is identical for both audio and label encoders. The details of computations in a layer are shown in Figure FIGREF10 and Table TABREF11. All the models for experiments presented in this paper are trained on 8x8 TPU with a per-core batch size of 16 (effective batch size of 2048). 
 Question: Does model uses pretrained Transformer encoders?
Output: Yes-no
2) Answers are formulated by domain experts with legal training.  
 Question: Are the experts comparable to real-world users?
Output: Yes-no
We introduce a new model for difficulty prediction combining learned representations induced via a pre-trained `universal' sentence encoder BIBREF6 , and a sentence encoder learned from scratch for this task. In total, this resulted in 57,505 and 2,428 sentences in the train and test set abstracts, respectively. 
 Question: How much data is needed to train the task-specific encoder?
Output: Abstractive
According the distribution of the sentiment score, the sentiment on Tesla is slightly skewed towards positive during the testing period. 
 Question: Do the authors give any examples of major events which draw the public's attention and the impact they have on stock price?
Output: Yes-no
Thus, SIM has many fewer parameters than existing dialogue state tracking models. To compensate for the exclusion of slot-specific parameters, we incorporate better feature representation of user utterance and dialogue states using syntactic information and convolutional neural networks (CNN).  
 Question: How do they prevent the model complexity increasing with the increased number of slots?
Output: Abstractive
The term `propaganda' derives from propagare in post-classical Latin, as in “propagation of the faith" BIBREF1, and thus has from the beginning been associated with an intentional and potentially multicast communication; only later did it become a pejorative term. It was pragmatically defined in the World War II era as “the expression of an opinion or an action by individuals or groups deliberately designed to influence the opinions or the actions of other individuals or groups with reference to predetermined ends" BIBREF2. 
 Question: How is "propaganda" defined for the purposes of this study?
Output: Extractive
We model the ASP placement task as a successor of the AEP task. For all the `relevant' news entity pairs, the task is to determine the correct entity section. Each section in a Wikipedia entity page represents a different topic. For example, Barack Obama has the sections `Early Life', `Presidency', `Family and Personal Life' etc. Article-Section Ground-truth. The dataset consists of the triple INLINEFORM0 , where INLINEFORM1 , where we assume that INLINEFORM2 has already been determined as relevant. We therefore have a multi-class classification problem where we need to determine the section of INLINEFORM3 where INLINEFORM4 is cited.  
 Question: How do they determine the exact section to use the input article?
Output: Abstractive
The current configuration of Aristo comprises of eight solvers, described shortly, each of which attempts to answer a multiple choice question. To study particular phenomena and develop solvers, the project has created larger datasets to amplify and study different problems, resulting in 10 new datasets and 5 large knowledge resources for the community.

The solvers can be loosely grouped into:

Statistical and information retrieval methods

Reasoning methods

Large-scale language model methods Over the life of the project, the relative importance of the methods has shifted towards large-scale language methods. The field of NLP has advanced substantially with the advent of large-scale language models such as ELMo (BID6), ULMFit (BID37), GPT (BID38), BERT (BID7), and RoBERTa (BID8). These models are trained to perform various language prediction tasks such as predicting a missing word or the next sentence, using large amounts of text (e.g., BERT was trained on Wikipedia + the Google Book Corpus of 10,000 books). They can also be fine-tuned to new language prediction tasks, such as question-answering, and have been remarkably successful in the few months that they have been available. We apply BERT to multiple choice questions by treating the task as classification: Given a question $q$ with answer options $a_{i}$ and optional background knowledge $K_{i}$, we provide it to BERT as:

[CLS] $K_i$ [SEP] $q$ [SEP] $a_{i}$ [SEP] The AristoBERT solver uses three methods to apply BERT more effectively. First, we retrieve and supply background knowledge along with the question when using BERT. This provides the potential for BERT to “read” that background knowledge and apply it to the question, although the exact nature of how it uses background knowledge is more complex and less interpretable. Second, we fine-tune BERT using a curriculum of several datasets, including some that are not science related. Finally, we ensemble different variants of BERT together. 
 Question: Is Aristo just some modern NLP model (ex. BERT) finetuned od data specific for this task?
Output: Yes-no
This work also contributes a new dataset of INLINEFORM0 pairs of free-form natural language instructions and high-level navigation plans. This dataset was collected through Mechanical Turk using 100 simulated environments with a corresponding topological map and, to the best of our knowledge, it is the first of its kind for behavioral navigation. The dataset opens up opportunities to explore data-driven methods for grounding navigation commands into high-level motion plans. While the dataset was collected with simulated environments, no structure was imposed on the navigation instructions while crowd-sourcing data. Thus, many instructions in our dataset are ambiguous. Moreover, the order of the behaviors in the instructions is not always the same. For instance, a person said “turn right and advance” to describe part of a route, while another person said “go straight after turning right” in a similar situation. The high variability present in the natural language descriptions of our dataset makes the problem of decoding instructions into behaviors not trivial. See Appendix A of the supplementary material for additional details on our data collection effort. 
 Question: Did the collection process use a WoZ method?
Output: Yes-no
We use OpenNMT BIBREF24 as the implementation of the NMT system for all experiments BIBREF5 . PBMT-R is a phrase-based method with a reranking post-processing step BIBREF18 . Hybrid performs sentence splitting and deletion operations based on discourse representation structures, and then simplifies sentences with PBMT-R BIBREF25 . SBMT-SARI BIBREF19 is syntax-based translation model using PPDB paraphrase database BIBREF26 and modifies tuning function (using SARI). Dress is an encoder-decoder model coupled with a deep reinforcement learning framework, and the parameters are chosen according to the original paper BIBREF20 . 
 Question: what state of the art methods did they compare with?
Output: Extractive
In future work we also intend to add these types of studies to the ERP predictions.

Discussion 
 Question: What datasets are used?
Output: Abstractive
convertible.pl: implementing DCG rules for 1st and 3rd steps in the three-steps conversion, as well as other rules including lexicon. 
 Question: What DCGs are used?
Output: Abstractive
BIBREF12 found in a Chinese corpus that the word label "End" has a better performance than "Begin". This motivates us to carry out a backward greedy search over each sentence's label sequence to identify word boundaries. If two words segmented in a sentence are identified as nouns, and one word is immediately before the other, we assemble their boundaries, creating a new word candidate for entity recognition. This strategy has the advantage to find named entities with long word length. It also reduces the influence caused by different segmentation criteria. 
 Question: What boundary assembling method is used?
Output: Extractive
We use the following cheap ways to generate pseudo-source texts:

copy: in this setting, the source side is a mere copy of the target-side data. copy-marked: another way to integrate copies without having to deal with OOVs is to augment the source vocabulary with a copy of the target vocabulary.  copy-dummies: instead of using actual copies, we replace each word with “dummy” tokens.  
 Question: what data simulation techniques were introduced?
Output: Extractive
We conducted experiments on the Amazon reviews dataset BIBREF9, which is a benchmark dataset in the cross-domain sentiment analysis field. This dataset contains Amazon product reviews of four different product domains: Books (B), DVD (D), Electronics (E), and Kitchen (K) appliances. Each review is originally associated with a rating of 1-5 stars and is encoded in 5,000 dimensional feature vectors of bag-of-words unigrams and bigrams.

Experiment ::: Dataset and Task Design ::: Binary-Class.
From this dataset, we constructed 12 binary-class cross-domain sentiment analysis tasks: B$\rightarrow $D, B$\rightarrow $E, B$\rightarrow $K, D$\rightarrow $B, D$\rightarrow $E, D$\rightarrow $K, E$\rightarrow $B, E$\rightarrow $D, E$\rightarrow $K, K$\rightarrow $B, K$\rightarrow $D, K$\rightarrow $E. Following the setting of previous works, we treated a reviews as class `1' if it was ranked up to 3 stars, and as class `2' if it was ranked 4 or 5 stars. For each task, $\mathcal {D}_S$ consisted of 1,000 examples of each class, and $\mathcal {D}_T$ consists of 1500 examples of class `1' and 500 examples of class `2'. In addition, since it is reasonable to assume that $\mathcal {D}_T$ can reveal the distribution of target domain data, we controlled the target domain testing dataset to have the same class ratio as $\mathcal {D}_T$. Using the same label assigning mechanism, we also studied model performance over different degrees of $\rm {P}(\rm {Y})$ shift, which was evaluated by the max value of $\rm {P}_S(\rm {Y}=i)/\rm {P}_T(\rm {Y}=i), \forall i=1, \cdots , L$. Please refer to Appendix C for more detail about the task design for this study.

Experiment ::: Dataset and Task Design ::: Multi-Class.
We additionally constructed 12 multi-class cross-domain sentiment classification tasks. Tasks were designed to distinguish reviews of 1 or 2 stars (class 1) from those of 4 stars (class 2) and those of 5 stars (class 3). For each task, $\mathcal {D}_S$ contained 1000 examples of each class, and $\mathcal {D}_T$ consisted of 500 examples of class 1, 1500 examples of class 2, and 1000 examples of class 3. Similarly, we also controlled the target domain testing dataset to have the same class ratio as $\mathcal {D}_T$. 
 Question: Which sentiment analysis tasks are addressed?
Output: Abstractive
Modern Standard Arabic (MSA) and Classical Arabic (CA) have two types of vowels, namely long vowels, which are explicitly written, and short vowels, aka diacritics, which are typically omitted in writing but are reintroduced by readers to properly pronounce words. We show that our model achieves a case ending error rate (CEER) of 3.7% for MSA and 2.5% for CA. For MSA, this CEER is more than 60% lower than other state-of-the-art systems such as Farasa and the RDI diacritizer, which are trained on the same dataset and achieve CEERs of 10.7% and 14.4% respectively.  
 Question: what are the previous state of the art?
Output: Extractive
We construct three datasets based on IMDB reviews and Yelp reviews. The IMDB dataset is binarised and split into a training and test set, each with 25K reviews (2K reviews from the training set are reserved for development). For Yelp, we binarise the ratings, and create 2 datasets, where we keep only reviews with $\le $ 50 tokens (yelp50) and $\le $200 tokens (yelp200). 
 Question: What datasets do they use?
Output: Extractive
We employ precision, recall, F1 and accuracy for evaluation metrics. 
 Question: what evaluation metrics are reported?
Output: Extractive
In this paper, we have shown that both human and machine translation can alter superficial patterns in data, which requires reconsidering previous findings in cross-lingual transfer learning. 
 Question: Does the professional translation or the machine translation introduce the artifacts?
Output: Yes-no
Results form Table TABREF31 may give an idea that INLINEFORM0 is just an scaled INLINEFORM1 . While it is true that they show a linear correlation, INLINEFORM2 may produce a different system ranking than INLINEFORM3 given the integral multi-reference principle it follows. However, what we consider the most profitable about INLINEFORM4 is the twofold inclusion of all available references it performs. First, the construction of INLINEFORM5 to provide a more inclusive reference against to whom be evaluated and then, the computation of INLINEFORM6 , which scales the result depending of the agreement between references. We showed how INLINEFORM0 is an inclusive metric which not only evaluates the performance of a system against all references, but also takes into account the agreement between them.  
 Question: What makes it a more reliable metric?
Output: Abstractive
The goal of an language model is to assign meaningful probabilities to a sequence of words. Given a set of tokens $\mathbf {X}=(x_1,....,x_T)$, where $T$ is the length of a sequence, our task is to estimate the joint conditional probability $P(\mathbf {X})$ which is

were $(x_{1}, \ldots , x_{i-1})$ is the context. An Intrinsic evaluation of the performance of Language Models is perplexity (PPL) which is defined as the inverse probability of the set of the tokens and taking the $T^{th}$ root were $T$ is the number of tokens We propose an approximation of the joint probability as,

This type of approximations has been previously explored with Bi-directional RNN LM's BIBREF9 but not for deep transformer models. We therefore, define a pseudo-perplexity score from the above approximated joint probability. 
 Question: How is pseudo-perplexity defined?
Output: Abstractive