z-uo/qasper-squad huggingface dataset:
Paper: Recent Advances in Neural Question Generation
Question Generation (QG) concerns the task of “automatically generating questions from various inputs such as raw text, database, or semantic representation" BIBREF0 . People have the ability to ask rich, creative, and revealing questions BIBREF1 ; e.g., asking Why did Gollum betray his master Frodo Baggins? after reading the fantasy novel The Lord of the Rings. How can machines be endowed with the ability to ask relevant and to-the-point questions, given various inputs? This is a challenging, complementary task to Question Answering (QA). Both QA and QG require an in-depth understanding of the input source and the ability to reason over relevant contexts. But beyond understanding, QG additionally integrates the challenges of Natural Language Generation (NLG), i.e., generating grammatically and semantically correct questions.
Question: Do they cover data augmentation papers?
Paper: Recent Advances in Neural Question Generation
For the sake of clean exposition, we first provide a broad overview of QG by conceptualizing the problem from the perspective of the three introduced aspects: (1) its learning paradigm, (2) its input modalities, and (3) the cognitive level it involves. This combines past research with recent trends, providing insights on how NQG connects to traditional QG research.
Question: What is the latest paper covered by this survey?
Answer: Kim et al. (2019)
Paper: Recent Advances in Neural Question Generation
QG research traditionally considers two fundamental aspects in question asking: “What to ask” and “How to ask”. A typical QG task considers the identification of the important aspects to ask about (“what to ask”), and learning to realize such identified aspects as natural language (“how to ask”). Deciding what to ask is a form of machine understanding: a machine needs to capture important information dependent on the target application, akin to automatic summarization. Learning how to ask, however, focuses on aspects of the language quality such as grammatical correctness, semantically preciseness and language flexibility.
Question: Do they survey visual question generation work?
Paper: Recent Advances in Neural Question Generation
Question generation is an NLG task for which the input has a wealth of possibilities depending on applications. While a host of input modalities have been considered in other NLG tasks, such as text summarization BIBREF24 , image captioning BIBREF25 and table-to-text generation BIBREF26 , traditional QG mainly focused on textual inputs, especially declarative sentences, explained by the original application domains of question answering and education, which also typically featured textual inputs.
Question: Do they survey multilingual aspects?
Paper: Recent Advances in Neural Question Generation
Finally, we consider the required cognitive process behind question asking, a distinguishing factor for questions BIBREF32 . A typical framework that attempts to categorize the cognitive levels involved in question asking comes from Bloom's taxonomy BIBREF33 , which has undergone several revisions and currently has six cognitive levels: Remembering, Understanding, Applying, Analyzing, Evaluating and Creating BIBREF32 .
Question: What learning paradigms do they cover in this survey?
Answer: Considering "What" and "How" separately versus jointly optimizing for both.
Paper: Recent Advances in Neural Question Generation
As QG can be regarded as a dual task of QA, in principle any QA dataset can be used for QG as well. However, there are at least two corpus-related factors that affect the difficulty of question generation. The first is the required cognitive level to answer the question, as we discussed in the previous section. Current NQG has achieved promising results on datasets consisting mainly of shallow factoid questions, such as SQuAD BIBREF36 and MS MARCO BIBREF38 . However, the performance drops significantly on deep question datasets, such as LearningQ BIBREF8 , shown in Section "Generation of Deep Questions" . The second factor is the answer type, i.e., the expected form of the answer, typically having four settings: (1) the answer is a text span in the passage, which is usually the case for factoid questions, (2) human-generated, abstractive answer that may not appear in the passage, usually the case for deep questions, (3) multiple choice question where question and its distractors should be jointly generated, and (4) no given answer, which requires the model to automatically learn what is worthy to ask. The design of NQG system differs accordingly.
Question: What are all the input modalities considered in prior work in question generation?
Answer: Textual inputs, knowledge bases, and images.
Paper: Recent Advances in Neural Question Generation
Although the datasets are commonly shared between QG and QA, it is not the case for evaluation: it is challenging to define a gold standard of proper questions to ask. Meaningful, syntactically correct, semantically sound and natural are all useful criteria, yet they are hard to quantify. Most QG systems involve human evaluation, commonly by randomly sampling a few hundred generated questions, and asking human annotators to rate them on a 5-point Likert scale. The average rank or the percentage of best-ranked questions are reported and used for quality marks.
Question: Do they survey non-neural methods for question generation?
Paper: Open Named Entity Modeling from Embedding Distribution
Named Entity Recognition is a major natural language processing task that recognizes the proper labels such as LOC (Location), PER (Person), ORG (Organization), etc. Like words or phrase, being a sort of language constituent, named entities also benefit from better representation for better processing. Continuous word representations, known as word embeddings, well capture semantic and syntactic regularities of words BIBREF0 and perform well in monolingual NE recognition BIBREF1 , BIBREF2 . Word embeddings also exhibit isomorphism structure across languages BIBREF3 . On account of these characteristics above, we attempt to utilize word embeddings to improve NE recognition for resource-poor languages with the help of richer ones. The state-of-the-art cross-lingual NE recognition methods are mainly based on annotation projection methods according to parallel corpora, translations BIBREF4 , BIBREF5 , BIBREF6 , BIBREF7 and Wikipedia methods BIBREF8 , BIBREF9 , BIBREF10 , BIBREF11 .
Question: What is their model?
Paper: Open Named Entity Modeling from Embedding Distribution
Seok BIBREF2 proposed that similar words are more likely to occupy close spatial positions, since their word embeddings carries syntactical and semantical informative clues. For an intuitive understanding, they listed the nearest neighbors of words included in the PER and ORG tags under cosine similarity metric. To empirically verify this observation and explore the performance of this property in Euclidean space , we list Top-5 nearest neighbors under Euclidean distance metric in Table 1 and illustrate a standard t-SNE BIBREF12 2- $D$ projection of the embeddings of three entity types with a sample of 500 words for each type.
Question: Do they evaluate on NER data sets?
Paper: Efficient Twitter Sentiment Classification using Subjective Distant Supervision
A lot of work has been done in the field of Twitter sentiment analysis till date. Sentiment analysis has been handled as a Natural Language Processing task at many levels of granularity. Most of these techniques use Machine Learning algorithms with features such as unigrams, n-grams, Part-Of-Speech (POS) tags. However, the training datasets are often very large, and hence with such a large number of features, this process requires a lot of computation power and time. The following question arises: What to do if we do not have resources that provide such a great amount of computation power? The existing solution to this problem is to use a smaller sample of the dataset. For sentiment analysis, if we train the model using a smaller randomly chosen sample, then we get low accuracy [16, 17]. In this paper, we propose a novel technique to sample tweets for building a sentiment classification model so that we get higher accuracy than the state-of-the-art baseline method, namely Distant Supervision, using a smaller set of tweets. Our model has lower computation time and higher accuracy compared to baseline model.
Question: What previously proposed methods is this method compared against?
Paper: Efficient Twitter Sentiment Classification using Subjective Distant Supervision
There has been a large amount of prior research in sentiment analysis of tweets. Read [10] shows that using emoticons as labels for positive and sentiment is effective for reducing dependencies in machine learning techniques. Alec Go [1] used Naive Bayes, SVM, and MaxEnt classifiers to train their model. This, as mentioned earlier, is our baseline model. Our model builds on this and achieves higher accuracy on a much smaller training dataset.
Question: How is effective word score calculated?
Paper: Efficient Twitter Sentiment Classification using Subjective Distant Supervision
Subjectivity refers to how someone's judgment is shaped by personal opinions and feelings instead of outside influences. An objective perspective is one that is not influenced by emotions, opinions, or personal feelings - it is a perspective based in fact, in things quantifiable and measurable. A subjective perspective is one open to greater interpretation based on personal feeling, emotion, aesthetics, etc.
Question: How is tweet subjectivity measured?
Paper: Dynamic Memory Networks for Visual and Textual Question Answering
Neural network based methods have made tremendous progress in image and text classification BIBREF0 , BIBREF1 . However, only recently has progress been made on more complex tasks that require logical reasoning. This success is based in part on the addition of memory and attention components to complex neural networks. For instance, memory networks BIBREF2 are able to reason over several facts written in natural language or (subject, relation, object) triplets. Attention mechanisms have been successful components in both machine translation BIBREF3 , BIBREF4 and image captioning models BIBREF5 .
Question: Why is supporting fact supervision necessary for DMN?
Paper: Dynamic Memory Networks for Visual and Textual Question Answering
We begin by outlining the DMN for question answering and the modules as presented in BIBREF6 .
Question: What does supporting fact supervision mean?
Paper: Dynamic Memory Networks for Visual and Textual Question Answering
We propose and compare several modeling choices for two crucial components: input representation, attention mechanism and memory update. The final DMN+ model obtains the highest accuracy on the bAbI-10k dataset without supporting facts and the VQA dataset BIBREF8 . Several design choices are motivated by intuition and accuracy improvements on that dataset.
Question: What changes they did on input module?
Paper: Dynamic Memory Networks for Visual and Textual Question Answering
In the DMN specified in BIBREF6 , a single GRU is used to process all the words in the story, extracting sentence representations by storing the hidden states produced at the end of sentence markers. The GRU also provides a temporal component by allowing a sentence to know the content of the sentences that came before them. Whilst this input module worked well for bAbI-1k with supporting facts, as reported in BIBREF6 , it did not perform well on bAbI-10k without supporting facts (Sec. "Model Analysis" ).
Question: What improvements they did for DMN?
Paper: Dynamic Memory Networks for Visual and Textual Question Answering
To apply the DMN to visual question answering, we introduce a new input module for images. The module splits an image into small local regions and considers each region equivalent to a sentence in the input module for text. The input module for VQA is composed of three parts, illustrated in Fig. 3 : local region feature extraction, visual feature embedding, and the input fusion layer introduced in Sec. "Input Module for Text QA" .
Question: How does the model circumvent the lack of supporting facts during training?
Paper: Dynamic Memory Networks for Visual and Textual Question Answering
The episodic memory module, as depicted in Fig. 4 , retrieves information from the input facts $\overleftrightarrow{F} = [\overleftrightarrow{f_1}, \hdots , \overleftrightarrow{f_N}]$ provided to it by focusing attention on a subset of these facts. We implement this attention by associating a single scalar value, the attention gate $g^t_i$ , with each fact $\overleftrightarrow{f}_i$ during pass $t$ . This is computed by allowing interactions between the fact and both the question representation and the episode memory state. 
Question: Does the DMN+ model establish state-of-the-art ?
Paper: Low-Level Linguistic Controls for Style Transfer and Content Preservation
All text has style, whether it be formal or informal, polite or aggressive, colloquial, persuasive, or even robotic. Despite the success of style transfer in image processing BIBREF0, BIBREF1, there has been limited progress in the text domain, where disentangling style from content is particularly difficult.
Question: Is this style generator compared to some baseline?
Paper: Low-Level Linguistic Controls for Style Transfer and Content Preservation
Following in the footsteps of machine translation, style transfer in text has seen success by using parallel data. BIBREF5 use modern translations of Shakespeare plays to build a modern-to-Shakespearan model. BIBREF6 compile parallel data for formal and informal sentences, allowing them to successfully use various machine translation techniques. While parallel data may work for very specific styles, the difficulty of finding parallel texts dramatically limits this approach.
Question: How they perform manual evaluation, what is criteria?
Paper: Low-Level Linguistic Controls for Style Transfer and Content Preservation
There has been a decent amount of work on this approach in the past few years BIBREF7, BIBREF8, mostly focusing on variations of an encoder-decoder framework in which style is modeled as a monolithic style embedding. The main obstacle is often to disentangle style and content. However, it remains a challenging problem.
Question: What metrics are used for automatic evaluation?
Paper: Low-Level Linguistic Controls for Style Transfer and Content Preservation
Several papers have worked on controlling style when generating sentences from restaurant meaning representations BIBREF11, BIBREF12. In each of these cases, the diversity in outputs is quite small given the constraints of the meaning representation, style is often constrained to interjections (like “yeah”), and there is no original style from which to transfer.
Question: How they know what are content words?
Paper: Low-Level Linguistic Controls for Style Transfer and Content Preservation
Style, in literary research, is anything but a stable concept, but it nonetheless has a long tradition of study in the digital humanities. In a remarkably early quantitative study of literature, BIBREF14 charts sentence-level stylistic attributes specific to a number of novelists. Half a century later, BIBREF15 builds on earlier work in information theory by BIBREF16, and defines a literary text as consisting of two “materials": “the vocabulary, and some structural properties, the style, of its author."
Question: How they model style as a suite of low-level linguistic controls, such as frequency of pronouns, prepositions, and subordinate clause constructions?
Paper: Fusing Visual, Textual and Connectivity Clues for Studying Mental Health
0pt*0*0
Question: Do they report results only on English data?
Paper: Fusing Visual, Textual and Connectivity Clues for Studying Mental Health
Depression is a highly prevalent public health challenge and a major cause of disability worldwide. Depression affects 6.7% (i.e., about 16 million) Americans each year . According to the World Mental Health Survey conducted in 17 countries, on average, about 5% of people reported having an episode of depression in 2011 BIBREF0 . Untreated or under-treated clinical depression can lead to suicide and other chronic risky behaviors such as drug or alcohol addiction.
Question: What insights into the relationship between demographics and mental health are provided?
Paper: Fusing Visual, Textual and Connectivity Clues for Studying Mental Health
Mental Health Analysis using Social Media:
Question: What model is used to achieve 5% improvement on F1 for identifying depressed individuals on Twitter?
Paper: Fusing Visual, Textual and Connectivity Clues for Studying Mental Health
Self-disclosure clues have been extensively utilized for creating ground-truth data for numerous social media analytic studies e.g., for predicting demographics BIBREF36 , BIBREF41 , and user's depressive behavior BIBREF46 , BIBREF47 , BIBREF48 . For instance, vulnerable individuals may employ depressive-indicative terms in their Twitter profile descriptions. Others may share their age and gender, e.g., "16 years old suicidal girl"(see Figure FIGREF15 ). We employ a huge dataset of 45,000 self-reported depressed users introduced in BIBREF46 where a lexicon of depression symptoms consisting of 1500 depression-indicative terms was created with the help of psychologist clinician and employed for collecting self-declared depressed individual's profiles. A subset of 8,770 users (24 million time-stamped tweets) containing 3981 depressed and 4789 control users (that do not show any depressive behavior) were verified by two human judges BIBREF46 . This dataset INLINEFORM0 contains the metadata values of each user such as profile descriptions, followers_count, created_at, and profile_image_url.
Question: How do this framework facilitate demographic inference from social media?
Answer: Demographic information is predicted using weighted lexicon of terms.
Paper: Fusing Visual, Textual and Connectivity Clues for Studying Mental Health
We now provide an in-depth analysis of visual and textual content of vulnerable users.
Question: What types of features are used from each data type?
Paper: Fusing Visual, Textual and Connectivity Clues for Studying Mental Health
We leverage both the visual and textual content for predicting age and gender.
Question: How is the data annotated?
Answer: The data are self-reported by Twitter users and then verified by two human experts.
Paper: Fusing Visual, Textual and Connectivity Clues for Studying Mental Health
We use the above findings for predicting depressive behavior. Our model exploits early fusion BIBREF32 technique in feature space and requires modeling each user INLINEFORM0 in INLINEFORM1 as vector concatenation of individual modality features. As opposed to computationally expensive late fusion scheme where each modality requires a separate supervised modeling, this model reduces the learning effort and shows promising results BIBREF75 . To develop a generalizable model that avoids overfitting, we perform feature selection using statistical tests and all relevant ensemble learning models. It adds randomness to the data by creating shuffled copies of all features (shadow feature), and then trains Random Forest classifier on the extended data. Iteratively, it checks whether the actual feature has a higher Z-score than its shadow feature (See Algorithm SECREF6 and Figure FIGREF45 ) BIBREF76 .
Question: Where does the information on individual-level demographics come from?
Answer: From Twitter profile descriptions of the users.
Paper: Incorporating Sememes into Chinese Definition Modeling
Chinese definition modeling is the task of generating a definition in Chinese for a given Chinese word. This task can benefit the compilation of dictionaries, especially dictionaries for Chinese as a foreign language (CFL) learners.
Question: Is there an online demo of their system?
Paper: Incorporating Sememes into Chinese Definition Modeling
The definition modeling task is to generate an explanatory sentence for the interpreted word. For example, given the word “旅馆” (hotel), a model should generate a sentence like this: “给旅行者提供食宿和其他服务的地方” (A place to provide residence and other services for tourists). Since distributed representations of words have been shown to capture lexical syntax and semantics, it is intuitive to employ word embeddings to generate natural language definitions.
Question: Do they perform manual evaluation?
Paper: Incorporating Sememes into Chinese Definition Modeling
The baseline model BIBREF3 is implemented with a recurrent neural network based encoder-decoder framework. Without utilizing the information of sememes, it learns a probabilistic mapping $P(y | x)$ from the word $x$ to be defined to a definition $y = [y_1, \dots , y_T ]$ , in which $y_t$ is the $t$ -th word of definition $y$ .
Question: Do they compare against Noraset et al. 2017?
Paper: Incorporating Sememes into Chinese Definition Modeling
Our proposed model aims to incorporate sememes into the definition modeling task. Given the word to be defined $x$ and its corresponding sememes $s=[s_1, \dots , s_N ]$ , we define the probability of generating the definition $y=[y_1, \dots , y_t ]$ as: 
Question: What is a sememe?
Paper: RobBERT: a Dutch RoBERTa-based Language Model
The advent of neural networks in natural language processing (NLP) has significantly improved state-of-the-art results within the field. While recurrent neural networks (RNNs) and long short-term memory networks (LSTMs) initially dominated the field, recent models started incorporating attention mechanisms and then later dropped the recurrent part and just kept the attention mechanisms in so-called transformer models BIBREF0. This latter type of model caused a new revolution in NLP and led to popular language models like GPT-2 BIBREF1, BIBREF2 and ELMo BIBREF3. BERT BIBREF4 improved over previous transformer models and recurrent networks by allowing the system to learn from input text in a bidirectional way, rather than only from left-to-right or the other way around. This model was later re-implemented, critically evaluated and improved in the RoBERTa model BIBREF5.
Question: What data did they use?
Paper: RobBERT: a Dutch RoBERTa-based Language Model
Transformer models have been successfully used for a wide range of language tasks. Initially, transformers were introduced for use in machine translation, where they vastly improved state-of-the-art results for English to German in an efficient manner BIBREF0. This transformer model architecture resulted in a new paradigm in NLP with the migration from sequence-to-sequence recurrent neural networks to transformer-based models by removing the recurrent component and only keeping attention. This cornerstone was used for BERT, a transformer model that obtained state-of-the-art results for eleven natural language processing tasks, such as question answering and natural language inference BIBREF4. BERT is pre-trained with large corpora of text using two unsupervised tasks. The first task is word masking (also called the Cloze task BIBREF9 or masked language model (MLM)), where the model has to guess which word is masked in certain position in the text. The second task is next sentence prediction. This is done by predicting if two sentences are subsequent in the corpus, or if they are randomly sampled from the corpus. These tasks allowed the model to create internal representations about a language, which could thereafter be reused for different language tasks. This architecture has been shown to be a general language model that could be fine-tuned with little data in a relatively efficient way for a very distinct range of tasks and still outperform previous architectures BIBREF4.
Question: What is the state of the art?
Paper: RobBERT: a Dutch RoBERTa-based Language Model
This section describes the data and training regime we used to train our Dutch RoBERTa-based language model called RobBERT.
Question: What language tasks did they experiment on?