z-uo/qasper-squad huggingface dataset:
Paper: A Sentiment Analysis of Breast Cancer Treatment Experiences and Healthcare Perceptions Across Twitter
 We evaluated tweet sentiments with hedonometrics, BIBREF21 , BIBREF22 , using LabMT, a labeled set of 10,000 frequently occurring words rated on a `happiness' scale by individuals contracted through Amazon Mechanical Turk, a crowd-sourced survey tool. These happiness scores helped quantify the average emotional rating of text by totaling the scores from applicable words and normalizing by their total frequency. Hence, the average happiness score, INLINEFORM0 , of a corpus with INLINEFORM1 words in common with LabMT was computed with the weighted arithmetic mean of each word's frequency, INLINEFORM2 , and associated happiness score, INLINEFORM3 : DISPLAYFORM0 
Question: What machine learning and NLP methods were used to sift tweets relevant to breast cancer experiences?
Answer: ML  logistic regression classifier combined with a Convolutional Neural Network (CNN) to identify self-reported diagnostic tweets.
NLP methods:  tweet conversion to numeric word vector,  removing tweets containing hyperlinks, removing "retweets", removing all tweets containing horoscope indicators,  lowercasing and  removing punctuation.
Paper: Joint Event and Temporal Relation Extraction with Shared Representations and Structured Prediction
The extraction of temporal relations among events is an important natural language understanding (NLU) task that can benefit many downstream tasks such as question answering, information retrieval, and narrative generation. The task can be modeled as building a graph for a given text, whose nodes represent events and edges are labeled with temporal relations correspondingly. Figure FIGREF1 illustrates such a graph for the text shown therein. The nodes assassination, slaughtered, rampage, war, and Hutu are the candidate events, and different types of edges specify different temporal relations between them: assassination is BEFORE rampage, rampage INCLUDES slaughtered, and the relation between slaughtered and war is VAGUE. Since “Hutu” is actually not an event, a system is expected to annotate the relations between “Hutu” and all other nodes in the graph as NONE (i.e., no relation).
Question: What kind of events do they extract?
Paper: Joint Event and Temporal Relation Extraction with Shared Representations and Structured Prediction
In this section we briefly summarize the existing work on event extraction and temporal relation extraction. To the best of our knowledge, there is no prior work on joint event and relation extraction, so we will review joint entity and relation extraction works instead.
Question: Is this the first paper to propose a joint model for event and temporal relation extraction?
Paper: Joint Event and Temporal Relation Extraction with Shared Representations and Structured Prediction
In this section we first provide an overview of our neural SSVM model, and then describe each component in our framework in detail (i.e., the multi-tasking neural scoring module, and how inference and learning are performed). We denote the set of all possible relation labels (including NONE) as $\mathcal {R}$, all event candidates (both events and non-events) as $\mathcal {E}$, and all relation candidates as $\mathcal {E}\mathcal {E}$.
Question: What datasets were used for this work?
Paper: Empirical Gaussian priors for cross-lingual transfer learning
The people of the world speak about 6,900 different languages. Open-source off-the-shelf natural language processing (NLP) toolboxes like OpenNLP and CoreNLP cover only 6–7 languages, and we have sufficient labeled training data for inducing models for about 20–30 languages. In other words, supervised sequence learning algorithms are not sufficient to induce POS models for but a small minority of the world's languages.
Question: What languages did they experiment with?
Paper: Variational Transformers for Diverse Response Generation
Convolutional and fully-attentional feed-forward architectures, such as Transformers BIBREF0, have emerged as effective alternatives to RNNs BIBREF1 in wide range of NLP tasks. These architectures remove the computational temporal dependency during the training and effectively address the long-standing vanishing gradients problem of recurrent models by processing all inputs simultaneously. Notably, transformers apply a fully attention strategy, where each token in the sequence is informed by other tokens via a self-attention mechanism. It acts as an effectively global receptive field across the whole sequences which absence in RNNs. Despite the powerful modeling capability of trasnformers, they often fail to model one-to-many relation in dialogue response generation tasks BIBREF2 due to their deterministic nature. As a result, they generate dull and generic response (e.g., “I am not sure"), especially with greedy and beam search, which are widely used in other sequence modeling tasks. There have been attempts to generate diverse and informative dialogue responses by incorporating latent variable(s) into the RNN encoder-decoder architecture. In particular BIBREF2 adapt a conditional variational autoencoder (CVAE) to capture discourse-level variations of dialogue, while BIBREF3 and BIBREF4 integrates latent variables in the hidden states of the RNN decoder. However, the inherently sequential computation of aforementioned models limit the efficiency for large scale training.
Question: What approach performs better in experiments global latent or sequence of fine-grained latent variables?
Answer: PPL: SVT
Diversity: GVT
Embeddings Similarity: SVT
Human Evaluation: SVT
Paper: Variational Transformers for Diverse Response Generation
Conversational systems has been widely studied BIBREF5, BIBREF6, BIBREF7, BIBREF8. Compare to rule-based systems BIBREF5, BIBREF6, sequence-to-sequence conversation models achieve superior performance in terms of scalable training and generalization ability BIBREF7. However, it has been pointed out that encoder-decoder models tend to generate generic and repetitive responses like “I am sorry" BIBREF9. To address this issue, there have been three main lines of work. The first is adding additional information (e.g., persona) as input to guild model generate more informative responses BIBREF10, BIBREF11. The second modifies the learning objective to promote more diverse generation BIBREF9, and the third integrates stochastic latent variables into Seq2Seq models by using the CVAE framework BIBREF12, BIBREF2. Our work comes within this third line introducing a novel model, the Variational Transformer, to improve dialogue response generation.
Question: What baselines other than standard transformers are used in experiments?
Paper: Variational Transformers for Diverse Response Generation
Many works have attempted to combine CVAEs with encoder-decoder architectures for sequence generation tasks. BIBREF13 propose a variational encoder-decoder model for neural machine translation, while BIBREF14 apply variational recurrent neural networks (VRNN) BIBREF15 for text summarization. BIBREF2 and BIBREF16 explore incorporating meta features into CVAE framework in dialogue response generation tasks. BIBREF3 and BIBREF4 propose variational autoregressive decoders which enhanced by highly multi-modal latent variables to capture the high variability in dialogue responses. BIBREF17 further augment variational autoregressive decoders with dynamic memory networks for improving generation quality. We unify the previous successful ideas of CVAE, and explore the combinations of CVAE and Transformer.
Question: What three conversational datasets are used for evaluation?
Paper: Czech Text Processing with Contextual Embeddings: POS Tagging, Lemmatization, Parsing and NER
Recently, a novel way of computing word embeddings has been proposed. Instead of computing one word embedding for each word which sums over all its occurrences, ignoring the appropriate word meaning in various contexts, the contextualized embeddings are computed for each word occurrence, taking into account the whole sentence. Three ways of computing such contextualized embeddings have been proposed: ELMo BIBREF0, BERT BIBREF1 and Flair BIBREF2, along with precomputed models.
Question: What previous approaches did this method outperform?
Paper: Czech Text Processing with Contextual Embeddings: POS Tagging, Lemmatization, Parsing and NER
As for the Prague Dependency Treebank (PDT) BIBREF4, most of the previous works are non-neural systems with one exception of BIBREF5 who hold the state of the art for Czech POS tagging and lemmatization, achieved with the recurrent neural network (RNN) using end-to-end trainable word embeddings and character-level word embeddings. Otherwise, Spoustová et al. (2009) BIBREF6 used an averaged perceptron for POS tagging. For parsing the PDT, Holan and Zabokrtský (2006) BIBREF7 and Novák and Žabokrtský (2007) BIBREF8 used a combination of non-neural parsing techniques .
Question: How big is the Universal Dependencies corpus?
Paper: Czech Text Processing with Contextual Embeddings: POS Tagging, Lemmatization, Parsing and NER
The Prague Dependency Treebank 3.5 BIBREF4 is a 2018 edition of the core Prague Dependency Treebank. The Prague Dependency Treebank 3.5 contains the same texts as the previous versions since 2.0, and is divided into train, dtest, and etest subparts, where dtest is used as a development set and etest as a test set. The dataset consists of several layers – the morphological m-layer is the largest and contains morphological annotations (POS tags and lemmas), the analytical a-layer contains labeled dependency trees, and the t-layer is the smallest and contains tectogrammatical trees. The statistics of PDT 3.5 sizes is presented in Table TABREF7.
Question: What data is the Prague Dependency Treebank built on?
Paper: Czech Text Processing with Contextual Embeddings: POS Tagging, Lemmatization, Parsing and NER
The Universal Dependencies project BIBREF10 seeks to develop cross-linguistically consistent treebank annotation of morphology and syntax for many languages. We evaluate the Czech PDT treebank of UD 2.3 BIBREF18, which is an automated conversion of PDT 3.5 a-layer to Universal Dependencies annotation. The original POS tags are used to generate UPOS (universal POS tags), XPOS (language-specific POS tags, in this case the original PDT tags), and Feats (universal morphological features). The UD lemmas are the raw textual lemmas, so the discriminative numeric suffix of PDT is dropped. The dependency trees are converted according to the UD guidelines, adapting both the unlabeled trees and the dependency labels.
Question: What data is used to build the embeddings?
Paper: Language Transfer for Early Warning of Epidemics from Social Media
The spread of influenza is a major health concern. Without appropriate preventative measures, this can escalate to an epidemic, causing high levels of mortality. A potential route to early detection is to analyse statements on social media platforms to identify individuals who have reported experiencing symptoms of the illness. These numbers can be used as a proxy to monitor the spread of the virus.
Question: How big is dataset used for fine-tuning model for detection of red flag medical symptoms in individual statements?
Paper: Language Transfer for Early Warning of Epidemics from Social Media
Previously, authors have created multilingual models which should allow transfer between languages by aligning models BIBREF0 or embedding spaces BIBREF1, BIBREF2. An alternative is translation of a high-resource language into the target low-resource language; for instance, BIBREF3 combined translation with subsequent selective correction by active learning of uncertain words and phrases believed to describe entities, to create a labelled dataset for named entity recognition.
Question: Is there any explanation why some choice of language pair is better than the other?
Paper: Sentence Encoders on STILTs: Supplementary Training on Intermediate Labeled-data Tasks
It has become clear over the last year that pretraining sentence encoder neural networks on unsupervised tasks, such as language modeling, then fine-tuning them on individual target tasks, can yield significantly better target task performance than could be achieved using target task training data alone BIBREF1 , BIBREF0 , BIBREF2 . Large-scale unsupervised pretraining in experiments like these seems to produce pretrained sentence encoders with substantial knowledge of the target language (which, so far, is generally English). These works have shown that a mostly task-agnostic, one-size-fits-all approach to fine-tuning a large pretrained model with a thin output layer for a given task can achieve results superior to individually optimized models.
Question: Is the new model evaluated on the tasks that BERT and ELMo are evaluated on?
Paper: Sentence Encoders on STILTs: Supplementary Training on Intermediate Labeled-data Tasks
 BIBREF8 compare several pretraining tasks for syntactic target tasks, and find that language model pretraining reliably performs well. BIBREF9 investigate the architectural choices behind ELMo-style pretraining with a fixed encoder, and find that the precise choice of encoder architecture strongly influences training speed, but has a relatively small impact on performance. In an publicly-available ICLR 2019 submission, BIBREF10 compare a variety of tasks for pretraining in an ELMo-style setting with no encoder fine-tuning. They conclude that language modeling generally works best among candidate single tasks for pretraining, but show some cases in which a cascade of a model pretrained on language modeling followed by another model pretrained on tasks like MNLI can work well. The paper introducing BERT BIBREF2 briefly mentions encouraging results in a direction similar to ours: One footnote notes that unpublished experiments show “substantial improvements on RTE from multi-task training with MNLI”.
Question: Does the additional training on supervised tasks hurt performance in some tasks?
Paper: Back Attention Knowledge Transfer for Low-resource Named Entity Recognition
Named entity recognition (NER) is a sequence tagging task that extracts the continuous tokens into specified classes, such as person names, organizations and locations. Current state-of-the-art approaches for NER usually base themselves on long short-term memory recurrent neural networks (LSTM RNNs) and a subsequent conditional random field (CRF) to predict the sequence labels BIBREF0 . Performances of neural NER methods are compromised if the training data are not enough BIBREF1 . This problem is severe for many languages due to a lack of labeled datasets, e.g., German and Spanish. In comparison, NER on English is well developed and there exist abundant labeled data for training purpose. Therefore, in this work, we regard English as a high-resource language, while other languages, even Chinese, as low-resource languages.
Question: Which translation system do they use to translate to English?
Answer: Attention-based translation model with convolution sequence to sequence model
Paper: Back Attention Knowledge Transfer for Low-resource Named Entity Recognition
In this section, we will introduce the BAN in three parts. Our model is based on the mainstream NER model BIBREF5 , using BiLSTM-CRF as the basic network structure. Given a sentence INLINEFORM0 and corresponding labels INLINEFORM1 , where INLINEFORM2 denotes the INLINEFORM3 th token and INLINEFORM4 denotes the INLINEFORM5 th label. The NER task is to estimate the probability INLINEFORM6 . Figure FIGREF1 shows the main architecture of our model.
Question: Which languages do they work with?
Paper: Back Attention Knowledge Transfer for Low-resource Named Entity Recognition
Attention-base translation model We use the system of BIBREF6 , a convolutional sequence to sequence model. It divides translation process into two steps. First, in the encoder step, given an input sentence INLINEFORM0 of length INLINEFORM1 , INLINEFORM2 represents each word as word embedding INLINEFORM3 . After that, we obtain the absolute position of input elements INLINEFORM4 . Both vectors are concatenated to get input sentence representations INLINEFORM5 . Similarly, output elements INLINEFORM6 generated from decoder network have the same structure. A convolutional neural network (CNN) is used to get the hidden state of the sentence representation from left to right. Second, in the decoder step, attention mechanism is used in each CNN layer. In order to acquire the attention value, we combine the current decoder state INLINEFORM7 with the embedding of previous decoder output value INLINEFORM8 : DISPLAYFORM0 
Question: Which pre-trained English NER model do they use?
Answer: Bidirectional LSTM based NER model of Flair
Paper: Multilingual Graphemic Hybrid ASR with Massive Data Augmentation
It can be challenging to build high-accuracy automatic speech recognition (ASR) systems in real world due to the vast language diversity and the requirement of extensive manual annotations on which the ASR algorithms are typically built. Series of research efforts have thus far been focused on guiding the ASR of a target language by using the supervised data from multiple languages.
Question: How much training data is required for each low-resource language?
Paper: Multilingual Graphemic Hybrid ASR with Massive Data Augmentation
In this section we first briefly describe our deployed ASR architecture based on the weighted finite-state transducers (WFSTs) outlined in BIBREF26. Then we present its extension to multilingual training. Lastly, we discuss its language-independent decoding and language-specific decoding.
Question: What are the best within-language data augmentation methods?
Paper: Multilingual Graphemic Hybrid ASR with Massive Data Augmentation
In the ASR framework of a hybrid BLSTM-HMM, the decoding graph can be interpreted as a composed WFST of cascade $H \circ C \circ L \circ G$. Acoustic models, i.e. BLSTMs, produce acoustic scores over context-dependent HMM (i.e. triphone) states. A WFST $H$, which represents the HMM set, maps the triphone states to context-dependent phones.
Question: How much of the ASR grapheme set is shared between languages?
Answer: Little overlap except common basic Latin alphabet and that Hindi and Marathi languages use same script.
Paper: HateMonitors: Language Agnostic Abuse Detection in Social Media
In social media, abusive language denotes a text which contains any form of unacceptable language in a post or a comment. Abusive language can be divided into hate speech, offensive language and profanity. Hate speech is a derogatory comment that hurts an entire group in terms of ethnicity, race or gender. Offensive language is similar to derogatory comment, but it is targeted towards an individual. Profanity refers to any use of unacceptable language without a specific target. While profanity is the least threatening, hate speech has the most detrimental effect on the society.
Question: What is the performance of the model for the German sub-task A?
Paper: HateMonitors: Language Agnostic Abuse Detection in Social Media
Analyzing abusive language in social media is a daunting task. Waseem et al. BIBREF2 categorizes abusive language into two sub-classes – hate speech and offensive language. In their analysis of abusive language, Classifying abusive language into these two subtypes is more challenging due to the correlation between offensive language and hate speech BIBREF3. Nobata et al. BIBREF4 uses predefined language element and embeddings to train a regression model. With the introduction of better classification models BIBREF5, BIBREF6 and newer features BIBREF7, BIBREF3, BIBREF8, the research in hate and offensive speech detection has gained momentum.
Question: Is the model tested for language identification?
Paper: HateMonitors: Language Agnostic Abuse Detection in Social Media
The dataset at HASOC 2019 were given in three languages: Hindi, English, and German. Dataset in Hindi and English had three subtasks each, while German had only two subtasks. We participated in all the tasks provided by the organisers and decided to develop a single model that would be language agnostic. We used the same model architecture for all the three languages.
Question: Is the model compared to a baseline model?
Paper: HateMonitors: Language Agnostic Abuse Detection in Social Media
We present the statistics for HASOC dataset in Table TABREF5. From the table, we can observe that the dataset for the German language is highly unbalanced, English and Hindi are more or less balanced for sub-task A. For sub-task B German dataset is balanced but others are unbalanced. For sub-task C both the datasets are highly unbalanced.
Question: What are the languages used to test the model?
Answer: Hindi, English and German (German task won)
Paper: Fast Multi-language LSTM-based Online Handwriting Recognition
In this paper we discuss online handwriting recognition: Given a user input in the form of an ink, i.e. a list of touch or pen strokes, output the textual interpretation of this input. A stroke is a sequence of points INLINEFORM0 with position INLINEFORM1 and timestamp INLINEFORM2 .
Question: Which language has the lowest error rate reduction?
Answer: thai
Paper: Fast Multi-language LSTM-based Online Handwriting Recognition
Our handwriting recognition model draws its inspiration from research aimed at building end-to-end transcription models in the context of handwriting recognition BIBREF24 , optical character recognition BIBREF21 , and acoustic modeling in speech recognition BIBREF18 . The model architecture is constructed from common neural network blocks, i.e. bidirectional LSTMs and fully-connected layers (Figure FIGREF12 ). It is trained in an end-to-end manner using the CTC loss BIBREF24 .
Question: What datasets did they use?
Paper: BERT has a Moral Compass: Improvements of ethical and moral values of machines
There is a broad consensus that artificial intelligence (AI) research is progressing steadily, and that its impact on society is likely to increase. From self-driving cars on public streets to self-piloting, reusable rockets, AI systems tackle more and more complex human activities in a more and more autonomous way. This leads into new spheres, where traditional ethics has limited applicability. Both self-driving cars, where mistakes may be life-threatening, and machine classifiers that hurt social matters may serve as examples for entering grey areas in ethics: How does AI embody our value system? Can AI systems learn human ethical judgements? If not, can we contest the AI system?
Question: Do they report results only on English data?
Paper: BERT has a Moral Compass: Improvements of ethical and moral values of machines
In this section, we review our assumptions, in particular what we mean by moral choices, and the required background, following closely BIBREF0.
Question: What is the Moral Choice Machine?
Paper: BERT has a Moral Compass: Improvements of ethical and moral values of machines
Word-based approaches such as WEAT or Verb Extraction are comparatively simple. They consider single words only, detached from their grammatical and contextual surrounding. In contrast, the Moral Choice Machine determines biases on a sentence level.
Question: How is moral bias measured?
Answer: Answer with content missing: (formula 1) bias(q, a, b) = cos(a, q) − cos(b, q)
Bias is calculated as substraction of cosine similarities of question and some answer for two opposite answers.
Paper: BERT has a Moral Compass: Improvements of ethical and moral values of machines
As BIBREF0 (BIBREF0) showed the question/answer template is an appropriate method to extract moral biases. However as BIBREF13 (BIBREF13) showed, one is also able to even adapt the model's bias, e.g. debias the model's gender bias. They describe that the first step for debiasing word embeddings is to identify a direction (or, more generally, a subspace) of the embedding that captures the bias.
Question: What sentence embeddings were used in the previous Jentzsch paper?