z-uo/qasper-squad huggingface dataset:
Paper: Minimally Supervised Learning of Affective Events Using Discourse Relations
Affective events BIBREF0 are events that typically affect people in positive or negative ways. For example, getting money and playing sports are usually positive to the experiencers; catching cold and losing one's wallet are negative. Understanding affective events is important to various natural language processing (NLP) applications such as dialogue systems BIBREF1, question-answering systems BIBREF2, and humor recognition BIBREF3. In this paper, we work on recognizing the polarity of an affective event that is represented by a score ranging from $-1$ (negative) to 1 (positive).
Question: What is the seed lexicon?
Answer: a vocabulary of positive and negative predicates that helps determine the polarity score of an event
Paper: Minimally Supervised Learning of Affective Events Using Discourse Relations
Learning affective events is closely related to sentiment analysis. Whereas sentiment analysis usually focuses on the polarity of what are described (e.g., movies), we work on how people are typically affected by events. In sentiment analysis, much attention has been paid to compositionality. Word-level polarity BIBREF5, BIBREF6, BIBREF7 and the roles of negation and intensification BIBREF8, BIBREF6, BIBREF9 are among the most important topics. In contrast, we are more interested in recognizing the sentiment polarity of an event that pertains to commonsense knowledge (e.g., getting money and catching cold).
Question: What are the results?
Answer: Using all data to train: AL -- BiGRU achieved 0.843 accuracy, AL -- BERT achieved 0.863 accuracy, AL+CA+CO -- BiGRU achieved 0.866 accuracy, AL+CA+CO -- BERT achieved 0.835, accuracy, ACP -- BiGRU achieved 0.919 accuracy, ACP -- BERT achived 0.933, accuracy, ACP+AL+CA+CO -- BiGRU achieved 0.917 accuracy, ACP+AL+CA+CO -- BERT achieved 0.913 accuracy. 
Using a subset to train: BERT achieved 0.876 accuracy using ACP (6K), BERT achieved 0.886 accuracy using ACP (6K) + AL, BiGRU achieved 0.830 accuracy using ACP (6K), BiGRU achieved 0.879 accuracy using ACP (6K) + AL + CA + CO.
Paper: Minimally Supervised Learning of Affective Events Using Discourse Relations
Question: How are relations used to propagate polarity?
Answer: based on the relation between events, the suggested polarity of one event can determine the possible polarity of the other event 
cause relation: both events in the relation should have the same polarity; concession relation: events should have opposite polarity
Paper: Minimally Supervised Learning of Affective Events Using Discourse Relations
Question: How big is the Japanese data?
Answer: 7000000 pairs of events were extracted from the Japanese Web corpus, 529850 pairs of events were extracted from the ACP corpus
The ACP corpus has around 700k events split into positive and negative polarity
Paper: Minimally Supervised Learning of Affective Events Using Discourse Relations
Our method requires a very small seed lexicon and a large raw corpus. We assume that we can automatically extract discourse-tagged event pairs, $(x_{i1}, x_{i2})$ ($i=1, \cdots $) from the raw corpus. We refer to $x_{i1}$ and $x_{i2}$ as former and latter events, respectively. As shown in Figure FIGREF1, we limit our scope to two discourse relations: Cause and Concession.
Question: What are labels available in dataset for supervision?
Paper: Minimally Supervised Learning of Affective Events Using Discourse Relations
The seed lexicon matches (1) the latter event but (2) not the former event, and (3) their discourse relation type is Cause or Concession. If the discourse relation type is Cause, the former event is given the same score as the latter. Likewise, if the discourse relation type is Concession, the former event is given the opposite of the latter's score. They are used as reference scores during training.
Question: How big are improvements of supervszed learning results trained on smalled labeled data enhanced with proposed approach copared to basic approach?
Answer: 3%
Paper: Minimally Supervised Learning of Affective Events Using Discourse Relations
The seed lexicon matches neither the former nor the latter event, and their discourse relation type is Cause. We assume the two events have the same polarities.
Question: How does their model learn using mostly raw data?
Answer: by exploiting discourse relations to propagate polarity from seed predicates to final sentiment polarity
Paper: Minimally Supervised Learning of Affective Events Using Discourse Relations
The seed lexicon matches neither the former nor the latter event, and their discourse relation type is Concession. We assume the two events have the reversed polarities.
Question: How big is seed lexicon used for training?
Answer: 30 words
Paper: Minimally Supervised Learning of Affective Events Using Discourse Relations
Using AL, CA, and CO data, we optimize the parameters of the polarity function $p(x)$. We define a loss function for each of the three types of event pairs and sum up the multiple loss functions.
Question: How large is raw corpus used for training?
Paper: PO-EMO: Conceptualization, Annotation, and Modeling of Aesthetic Emotions in German and English Poetry
1.1em
Question: Does the paper report macro F1?
Paper: PO-EMO: Conceptualization, Annotation, and Modeling of Aesthetic Emotions in German and English Poetry
1.1.1em
Question: How is the annotation experiment evaluated?
Paper: PO-EMO: Conceptualization, Annotation, and Modeling of Aesthetic Emotions in German and English Poetry
1.1.1.1em
Question: What are the aesthetic emotions formalized?
Paper: Community Identity and User Engagement in a Multi-Community Landscape
“If each city is like a game of chess, the day when I have learned the rules, I shall finally possess my empire, even if I shall never succeed in knowing all the cities it contains.”
Question: Do they report results only on English data?
Paper: Community Identity and User Engagement in a Multi-Community Landscape
A community's identity derives from its members' common interests and shared experiences BIBREF15 , BIBREF20 . In this work, we structure the multi-community landscape along these two key dimensions of community identity: how distinctive a community's interests are, and how dynamic the community is over time.
Question: How do the various social phenomena examined manifest in different types of communities?
Answer: Dynamic communities have substantially higher rates of monthly user retention than more stable communities. More distinctive communities exhibit moderately higher monthly retention rates than more generic communities. There is also a strong positive relationship between a community's dynamicity and the average number of months that a user will stay in that community - a short-term trend observed for monthly retention translates into longer-term engagement and suggests that long-term user retention might be strongly driven by the extent to which a community continually provides novel content.
Paper: Community Identity and User Engagement in a Multi-Community Landscape
In order to illustrate the diversity within the multi-community space, and to provide an intuition for the underlying structure captured by the proposed typology, we first examine a few example communities and draw attention to some key social dynamics that occur within them.
Question: What patterns do they observe about how user engagement varies with the characteristics of a community?
Paper: Community Identity and User Engagement in a Multi-Community Landscape
Our approach follows the intuition that a distinctive community will use language that is particularly specific, or unique, to that community. Similarly, a dynamic community will use volatile language that rapidly changes across successive windows of time. To capture this intuition automatically, we start by defining word-level measures of specificity and volatility. We then extend these word-level primitives to characterize entire comments, and the community itself.
Question: How did the select the 300 Reddit communities for comparison?
Answer: They selected all the subreddits from January 2013 to December 2014 with at least 500 words in the vocabulary and at least 4 months of the subreddit's history. They also removed communities with the bulk of the contributions are in foreign language.
They collect subreddits from January 2013 to December 2014,2 for which there are at
least 500 words in the vocabulary used to estimate the measures,
in at least 4 months of the subreddit’s history. They compute our measures over the comments written by users in a community in time windows of months, for each sufficiently active month, and manually remove communities where the bulk of the contributions are in a foreign language.
Paper: Community Identity and User Engagement in a Multi-Community Landscape
Having described these word-level measures, we now proceed to establish the primary axes of our typology:
Question: How do the authors measure how temporally dynamic a community is?
Paper: Community Identity and User Engagement in a Multi-Community Landscape
We now explain how our typology can be applied to the particular setting of Reddit, and describe the overall behaviour of our linguistic axes in this context.
Question: How do the authors measure how distinctive a community is?
Paper: Question Answering based Clinical Text Structuring Using Pre-trained Language Model
Clinical text structuring (CTS) is a critical task for fetching medical research data from electronic health records (EHRs), where structural patient medical data, such as whether the patient has specific symptoms, diseases, or what the tumor size is, how far from the tumor is cut at during the surgery, or what the specific laboratory test result is, are obtained. It is important to extract structured data from clinical text because bio-medical systems or bio-medical researches greatly rely on structured data but they cannot obtain them directly. In addition, clinical text often contains abundant healthcare information. CTS is able to provide large-scale extracted structured data for enormous down-stream clinical researches.
Question: What data is the language model pretrained on?
Paper: Question Answering based Clinical Text Structuring Using Pre-trained Language Model
Clinical text structuring is a final problem which is highly related to practical applications. Most of existing studies are case-by-case. Few of them are developed for the general purpose structuring task. These studies can be roughly divided into three categories: rule and dictionary based methods, task-specific end-to-end methods and pipeline methods.
Question: What baselines is the proposed model compared against?
Paper: Question Answering based Clinical Text Structuring Using Pre-trained Language Model
Recently, some works focused on pre-trained language representation models to capture language information from text and then utilizing the information to improve the performance of specific natural language processing tasks BIBREF24, BIBREF25, BIBREF26, BIBREF27 which makes language model a shared model to all natural language processing tasks. Radford et al. BIBREF24 proposed a framework for fine-tuning pre-trained language model. Peters et al. BIBREF25 proposed ELMo which concatenates forward and backward language models in a shallow manner. Devlin et al. BIBREF26 used bidirectional Transformers to model deep interactions between the two directions. Yang et al. BIBREF27 replaced the fixed forward or backward factorization order with all possible permutations of the factorization order and avoided using the [MASK] tag which causes pretrain-finetune discrepancy that BERT is subject to.
Question: How is the clinical text structuring task defined?
Answer: CTS is extracting structural data from medical research data (unstructured). Authors define QA-CTS task that aims to discover most related text from original text.
Paper: Question Answering based Clinical Text Structuring Using Pre-trained Language Model
Given a sequence of paragraph text $X=<x_1, x_2, ..., x_n>$, clinical text structuring (CTS) can be regarded to extract or generate a key-value pair where key $Q$ is typically a query term such as proximal resection margin and value $V$ is a result of query term $Q$ according to the paragraph text $X$.
Question: What are the specific tasks being unified?
Paper: Question Answering based Clinical Text Structuring Using Pre-trained Language Model
In this section, we present an effective model for the question answering based clinical text structuring (QA-CTS). As shown in Fig. FIGREF8, paragraph text $X$ is first passed to a clinical named entity recognition (CNER) model BIBREF12 to capture named entity information and obtain one-hot CNER output tagging sequence for query text $I_{nq}$ and paragraph text $I_{nt}$ with BIEOS (Begin, Inside, End, Outside, Single) tag scheme. $I_{nq}$ and $I_{nt}$ are then integrated together into $I_n$. Meanwhile, the paragraph text $X$ and query text $Q$ are organized and passed to contextualized representation model which is pre-trained language model BERT BIBREF26 here to obtain the contextualized representation vector $V_s$ of both text and query. Afterwards, $V_s$ and $I_n$ are integrated together and fed into a feed forward network to calculate the start and end index of answer-related text. Here we define this calculation problem as a classification for each word to be the start or end word.
Question: Is all text in this dataset a question, or are there unrelated sentences in between questions?
Answer: the dataset consists of pathology reports including sentences and questions and answers about tumor size and resection margins so it does include additional sentences
Paper: Question Answering based Clinical Text Structuring Using Pre-trained Language Model
For any clinical free-text paragraph $X$ and query $Q$, contextualized representation is to generate the encoded vector of both of them. Here we use pre-trained language model BERT-base BIBREF26 model to capture contextual information.
Question: How many questions are in the dataset?
Answer: 2,714
Paper: Question Answering based Clinical Text Structuring Using Pre-trained Language Model
Since BERT is trained on general corpus, its performance on biomedical domain can be improved by introducing biomedical domain-specific features. In this paper, we introduce clinical named entity information into the model.
Question: What is the perWhat are the tasks evaluated?
Paper: Question Answering based Clinical Text Structuring Using Pre-trained Language Model
There are two ways to integrate two named entity information vectors $I_{nt}$ and $I_{nq}$ or hidden contextualized representation $V_s$ and named entity information $I_n$, where $I_n = [I_{nt}; I_{nq}]$. The first one is to concatenate them together because they have sequence output with a common dimension. The second one is to transform them into a new hidden representation. For the concatenation method, the integrated representation is described as follows.
Question: Are there privacy concerns with clinical data?
Paper: Question Answering based Clinical Text Structuring Using Pre-trained Language Model
The final step is to use integrated representation $H_i$ to predict the start and end index of answer-related text. Here we define this calculation problem as a classification for each word to be the start or end word. We use a feed forward network (FFN) to compress and calculate the score of each word $H_f$ which makes the dimension to $\left\langle l_s, 2\right\rangle $ where $l_s$ denotes the length of sequence.
Question: How they introduce domain-specific features into pre-trained language model?
Paper: Question Answering based Clinical Text Structuring Using Pre-trained Language Model
Two-stage training mechanism is previously applied on bilinear model in fine-grained visual recognition BIBREF31, BIBREF32, BIBREF33. Two CNNs are deployed in the model. One is trained at first for coarse-graind features while freezing the parameter of the other. Then unfreeze the other one and train the entire model in a low learning rate for fetching fine-grained features.
Question: How big is QA-CTS task dataset?
Paper: Question Answering based Clinical Text Structuring Using Pre-trained Language Model
In this section, we devote to experimentally evaluating our proposed task and approach. The best results in tables are in bold.
Question: How big is dataset of pathology reports collected from Ruijing Hospital?
Paper: Question Answering based Clinical Text Structuring Using Pre-trained Language Model
Our dataset is annotated based on Chinese pathology reports provided by the Department of Gastrointestinal Surgery, Ruijin Hospital. It contains 17,833 sentences, 826,987 characters and 2,714 question-answer pairs. All question-answer pairs are annotated and reviewed by four clinicians with three types of questions, namely tumor size, proximal resection margin and distal resection margin. These annotated instances have been partitioned into 1,899 training instances (12,412 sentences) and 815 test instances (5,421 sentences). Each instance has one or several sentences. Detailed statistics of different types of entities are listed in Table TABREF20.
Question: What are strong baseline models in specific tasks?
Paper: Progress and Tradeoffs in Neural Language Models
Deep learning has unquestionably advanced the state of the art in many natural language processing tasks, from syntactic dependency parsing BIBREF0 to named-entity recognition BIBREF1 to machine translation BIBREF2 . The same certainly applies to language modeling, where recent advances in neural language models (NLMs) have led to dramatically better approaches as measured using standard metrics such as perplexity BIBREF3 , BIBREF4 .
Question: What aspects have been compared between various language models?
Answer: Quality measures using perplexity and recall, and performance measured using latency and energy usage.
Paper: Progress and Tradeoffs in Neural Language Models
 BIBREF3 evaluate recent neural language models; however, their focus is not on the computational footprint of each model, but rather the perplexity. To further reduce perplexity, many neural language model extensions exist, such as continuous cache pointer BIBREF5 and mixture of softmaxes BIBREF6 . Since our focus is on comparing “core” neural and non-neural approaches, we disregard these extra optimizations techniques in all of our models.
Question: what classic language models are mentioned in the paper?
Paper: Progress and Tradeoffs in Neural Language Models
We conducted our experiments on Penn Treebank (PTB; BIBREF12 ) and WikiText-103 (WT103; BIBREF13 ). Preprocessed by BIBREF14 , PTB contains 887K tokens for training, 70K for validation, and 78K for test, with a vocabulary size of 10,000. On the other hand, WT103 comprises 103 million tokens for training, 217K for validation, and 245K for test, spanning a vocabulary of 267K unique tokens.
Question: What is a commonly used evaluation metric for language models?
Paper: Stay On-Topic: Generating Context-specific Fake Restaurant Reviews
Automatically generated fake reviews have only recently become natural enough to fool human readers. Yao et al. BIBREF0 use a deep neural network (a so-called 2-layer LSTM BIBREF1 ) to generate fake reviews, and concluded that these fake reviews look sufficiently genuine to fool native English speakers. They train their model using real restaurant reviews from yelp.com BIBREF2 . Once trained, the model is used to generate reviews character-by-character. Due to the generation methodology, it cannot be easily targeted for a specific context (meaningful side information). Consequently, the review generation process may stray off-topic. For instance, when generating a review for a Japanese restaurant in Las Vegas, the review generation process may include references to an Italian restaurant in Baltimore. The authors of BIBREF0 apply a post-processing step (customization), which replaces food-related words with more suitable ones (sampled from the targeted restaurant). The word replacement strategy has drawbacks: it can miss certain words and replace others independent of their surrounding words, which may alert savvy readers. As an example: when we applied the customization technique described in BIBREF0 to a review for a Japanese restaurant it changed the snippet garlic knots for breakfast with garlic knots for sushi).
Question: Which dataset do they use a starting point in generating fake reviews?
Paper: Stay On-Topic: Generating Context-specific Fake Restaurant Reviews
Fake reviews User-generated content BIBREF5 is an integral part of the contemporary user experience on the web. Sites like tripadvisor.com, yelp.com and Google Play use user-written reviews to provide rich information that helps other users choose where to spend money and time. User reviews are used for rating services or products, and for providing qualitative opinions. User reviews and ratings may be used to rank services in recommendations. Ratings have an affect on the outwards appearance. Already 8 years ago, researchers estimated that a one-star rating increase affects the business revenue by 5 – 9% on yelp.com BIBREF6 .
Question: Do they use a pretrained NMT model to help generating reviews?
Paper: Stay On-Topic: Generating Context-specific Fake Restaurant Reviews
We discuss the attack model, our generative machine learning method and controlling the generative process in this section.
Question: How does using NMT ensure generated reviews stay on topic?
Paper: Stay On-Topic: Generating Context-specific Fake Restaurant Reviews
Wang et al. BIBREF7 described a model of crowd-turfing attacks consisting of three entities: customers who desire to have fake reviews for a particular target (e.g. their restaurant) on a particular platform (e.g. Yelp), agents who offer fake review services to customers, and workers who are orchestrated by the agent to compose and post fake reviews.
Question: What kind of model do they use for detection?
Paper: Stay On-Topic: Generating Context-specific Fake Restaurant Reviews
We propose the use of NMT models for fake review generation. The method has several benefits: 1) the ability to learn how to associate context (keywords) to reviews, 2) fast training time, and 3) a high-degree of customization during production time, e.g. introduction of specific waiter or food items names into reviews.
Question: Does their detection tool work better than human detection?
Paper: Saliency Maps Generation for Automatic Text Summarization
Ever since the LIME algorithm BIBREF0 , "explanation" techniques focusing on finding the importance of input features in regard of a specific prediction have soared and we now have many ways of finding saliency maps (also called heat-maps because of the way we like to visualize them). We are interested in this paper by the use of such a technique in an extreme task that highlights questions about the validity and evaluation of the approach. We would like to first set the vocabulary we will use. We agree that saliency maps are not explanations in themselves and that they are more similar to attribution, which is only one part of the human explanation process BIBREF1 . We will prefer to call this importance mapping of the input an attribution rather than an explanation. We will talk about the importance of the input relevance score in regard to the model's computation and not make allusion to any human understanding of the model as a result.
Question: Which baselines did they compare?
Paper: Saliency Maps Generation for Automatic Text Summarization
We present in this section the baseline model from See et al. See2017 trained on the CNN/Daily Mail dataset. We reproduce the results from See et al. See2017 to then apply LRP on it.
Question: How many attention layers are there in their model?
Answer: one