z-uo/qasper-squad huggingface dataset:
Paper: Gated Convolutional Neural Networks for Domain Adaptation
With the advancement in technology and invention of modern web applications like Facebook and Twitter, users started expressing their opinions and ideologies at a scale unseen before. The growth of e-commerce companies like Amazon, Walmart have created a revolutionary impact in the field of consumer business. People buy products online through these companies and write reviews for their products. These consumer reviews act as a bridge between consumers and companies. Through these reviews, companies polish the quality of their services. Sentiment Classification (SC) is one of the major applications of Natural Language Processing (NLP) which aims to find the polarity of text. In the early stages BIBREF0 of text classification, sentiment classification was performed using traditional feature selection techniques like Bag-of-Words (BoW) BIBREF1 or TF-IDF. These features were further used to train machine learning classifiers like Naive Bayes (NB) BIBREF2 and Support Vector Machines (SVM) BIBREF3 . They are shown to act as strong baselines for text classification BIBREF4 . However, these models ignore word level semantic knowledge and sequential nature of text. Neural networks were proposed to learn distributed representations of words BIBREF5 . Skip-gram and CBOW architectures BIBREF6 were introduced to learn high quality word representations which constituted a major breakthrough in NLP. Several neural network architectures like recursive neural networks BIBREF7 and convolutional neural networks BIBREF8 achieved excellent results in text classification. Recurrent neural networks which were proposed for dealing sequential inputs suffer from vanishing BIBREF9 and exploding gradient problems BIBREF10 . To overcome this problem, Long Short Term Memory (LSTM) was introduced BIBREF11 .
Question: For the purposes of this paper, how is something determined to be domain specific knowledge?
Answer: reviews under distinct product categories are considered specific domain knowledge
Paper: Gated Convolutional Neural Networks for Domain Adaptation
Traditionally methods for tackling Domain Adaptation are lexicon based. Blitzer BIBREF19 used a pivot method to select features that occur frequently in both domains. It assumes that the selected pivot features can reliably represent the source domain. The pivots are selected using mutual information between selected features and the source domain labels. SFA BIBREF13 method argues that pivot features selected from source domain cannot attest a representation of target domain. Hence, SFA tries to exploit the relationship between domain-specific and domain independent words via simultaneously co-clustering them in a common latent space. SDA BIBREF14 performs Domain Adaptation by learning intermediate representations through auto-encoders. Yu BIBREF20 used two auxiliary tasks to help induce sentence embeddings that work well across different domains. These embeddings are trained using Convolutional Neural Networks (CNN).
Question: Does the fact that GCNs can perform well on this tell us that the task is simpler than previously thought?
Paper: Gated Convolutional Neural Networks for Domain Adaptation
In this section, we introduce a model based on Gated Convolutional Neural Networks for Domain Adaptation. We present the problem definition of Domain Adaptation, followed by the architecture of the proposed model.
Question: Are there conceptual benefits to using GCNs over more complex architectures like attention?
Paper: Deep contextualized word representations for detecting sarcasm and irony
Sarcastic and ironic expressions are prevalent in social media and, due to the tendency to invert polarity, play an important role in the context of opinion mining, emotion recognition and sentiment analysis BIBREF0 . Sarcasm and irony are two closely related linguistic phenomena, with the concept of meaning the opposite of what is literally expressed at its core. There is no consensus in academic research on the formal definition, both terms are non-static, depending on different factors such as context, domain and even region in some cases BIBREF1 .
Question: Do they evaluate only on English?
Paper: Deep contextualized word representations for detecting sarcasm and irony
Apart from the relevance for industry applications related to sentiment analysis, sarcasm and irony detection has received great traction within the NLP research community, resulting in a variety of methods, shared tasks and benchmark datasets. Computational approaches for the classification task range from rule-based systems BIBREF4 , BIBREF11 and statistical methods and machine learning algorithms such as Support Vector Machines BIBREF3 , BIBREF12 , Naive Bayes and Decision Trees BIBREF13 leveraging extensive feature sets, to deep learning-based approaches. In this context, BIBREF14 . delivered state-of-the-art results by using an intra-attentional component in addition to a recurrent neural network. Previous work such as the one by BIBREF15 had proposed a convolutional long-short-term memory network (CNN-LSTM-DNN) that also achieved excellent results. A comprehensive survey on automatic sarcasm detection was done by BIBREF16 , while computational irony detection was reviewed by BIBREF17 .
Question: What are the 7 different datasets?
Paper: Deep contextualized word representations for detecting sarcasm and irony
The wide spectrum of linguistic cues that can serve as indicators for sarcastic and ironic expressions has been usually exploited for automatic sarcasm or irony detection by modeling them in the form of binary features in traditional machine learning.
Question: What are the three different sources of data?
Paper: Deep contextualized word representations for detecting sarcasm and irony
We test our proposed approach for binary classification on either sarcasm or irony, on seven benchmark datasets retrieved from different media sources. Below we describe each dataset, please see Table TABREF1 below for a summary.
Question: What type of model are the ELMo representations used in?
Answer: A bi-LSTM with max-pooling on top of it
Paper: Deep contextualized word representations for detecting sarcasm and irony
Table TABREF2 summarizes our results. For each dataset, the top row denotes our baseline and the second row shows our best comparable model. Rows with FULL models denote our best single model trained with all the development available data, without any other preprocessing other than mentioned in the previous section. In the case of the Twitter datasets, rows indicated as AUG refer to our the models trained using the augmented version of the corresponding datasets.
Question: Which morphosyntactic features are thought to indicate irony or sarcasm?
Paper: SDNet: Contextualized Attention-based Deep Network for Conversational Question Answering
Traditional machine reading comprehension (MRC) tasks share the single-turn setting of answering a single question related to a passage. There is usually no connection between different questions and answers to the same passage. However, the most natural way humans seek answers is via conversation, which carries over context through the dialogue flow.
Question: Is the model evaluated on other datasets?
Paper: SDNet: Contextualized Attention-based Deep Network for Conversational Question Answering
In this section, we propose the neural model, SDNet, for the conversational question answering task, which is formulated as follows. Given a passage $\mathcal {C}$ , and history question and answer utterances $Q_1, A_1, Q_2, A_2, ..., Q_{k-1}, A_{k-1}$ , the task is to generate response $A_k$ given the latest question $Q_k$ . The response is dependent on both the passage and history utterances.
Question: Does the model incorporate coreference and entailment?
Paper: SDNet: Contextualized Attention-based Deep Network for Conversational Question Answering
Encoding layer encodes each token in passage and question into a fixed-length vector, which includes both word embeddings and contextualized embeddings. For contextualized embedding, we utilize the latest result from BERT BIBREF5 . Different from previous work, we fix the parameters in BERT model and use the linear combination of embeddings from different layers in BERT.
Question: Is the incorporation of context separately evaluated?
Paper: Phonetic Feedback for Speech Enhancement With and Without Parallel Speech Data
Typical speech enhancement techniques focus on local criteria for improving speech intelligibility and quality. Time-frequency prediction techniques use local spectral quality estimates as an objective function; time domain methods directly predict clean output with a potential spectral quality metric BIBREF0. Such techniques have been extremely successful in predicting a speech denoising function, but also require parallel clean and noisy speech for training. The trained systems implicitly learn the phonetic patterns of the speech signal in the coordinated output of time-domain or time-frequency units. However, our hypothesis is that directly providing phonetic feedback can be a powerful additional signal for speech enhancement. For example, many local metrics will be more attuned to high-energy regions of speech, but not all phones of a language carry equal energy in production (compare /v/ to /ae/).
Question: Which frozen acoustic model do they use?
Paper: Phonetic Feedback for Speech Enhancement With and Without Parallel Speech Data
Speech enhancement is a rich field of work with a huge variety of techniques. Spectral feature based enhancement systems have focused on masking approaches BIBREF2, and have gained popularity with deep learning techniques BIBREF3 for ideal ratio mask and ideal binary mask estimation BIBREF4.
Question: By how much does using phonetic feedback improve state-of-the-art systems?
Answer: Improved AECNN-T by 2.1 and AECNN-T-SM BY 0.9
Paper: Assessing the Efficacy of Clinical Sentiment Analysis and Topic Extraction in Psychiatric Readmission Risk Prediction
Psychotic disorders affect approximately 2.5-4% of the population BIBREF0 BIBREF1. They are one of the leading causes of disability worldwide BIBREF2 and are a frequent cause of inpatient readmission after discharge BIBREF3. Readmissions are disruptive for patients and families, and are a key driver of rising healthcare costs BIBREF4 BIBREF5. Assessing readmission risk is therefore critically needed, as it can help inform the selection of treatment interventions and implement preventive measures.
Question: What features are used?
Paper: Assessing the Efficacy of Clinical Sentiment Analysis and Topic Extraction in Psychiatric Readmission Risk Prediction
The corpus consists of a collection of 2,346 clinical notes (admission notes, progress notes, and discharge summaries), which amounts to 2,372,323 tokens in total (an average of 1,011 tokens per note). All the notes were written in English and extracted from the EHRs of 183 psychosis patients from McLean Psychiatric Hospital in Belmont, MA, all of whom had in their history at least one instance of 30-day readmission.
Question: Do they compare to previous models?
Paper: Assessing the Efficacy of Clinical Sentiment Analysis and Topic Extraction in Psychiatric Readmission Risk Prediction
The readmission risk prediction task was performed at the admission level. An admission consists of a collection of all the clinical notes for a given patient written by medical personnel between inpatient admission and discharge. Every admission was labeled as either `readmitted' (i.e. the patient was readmitted within the next 30 days of discharge) or `not readmitted'. Therefore, the classification task consists of creating a single feature representation of all the clinical notes belonging to one admission, plus the past medical history and demographic information of the patient, and establishing whether that admission will be followed by a 30-day readmission or not.
Question: How do they incorporate sentiment analysis?
Paper: Assessing the Efficacy of Clinical Sentiment Analysis and Topic Extraction in Psychiatric Readmission Risk Prediction
Structure features are features that were identified on the EHR using regular expression matching and include rating scores that have been reported in the psychiatric literature as correlated with increased readmission risk, such as Global Assessment of Functioning, Insight and Compliance:
Question: What is the dataset used?
Paper: Assessing the Efficacy of Clinical Sentiment Analysis and Topic Extraction in Psychiatric Readmission Risk Prediction
Unstructured features aim to capture the state of the patient in relation to seven risk factor domains (Appearance, Thought Process, Thought Content, Interpersonal, Substance Use, Occupation, and Mood) from the free-text narratives on the EHR. These seven domains have been identified as associated with readmission risk in prior work BIBREF14.
Question: How do they extract topics?
Paper: Attending to Characters in Neural Sequence Labeling Models
 This work is licenced under a Creative Commons Attribution 4.0 International Licence.
Question: How does this compare to simple interpolation between a word-level and a character-level language model?
Paper: Analysing Coreference in Transformer Outputs
In the present paper, we analyse coreference in the output of three neural machine translation systems (NMT) that were trained under different settings. We use a transformer architecture BIBREF0 and train it on corpora of different sizes with and without the specific coreference information. Transformers are the current state-of-the-art in NMT BIBREF1 and are solely based on attention, therefore, the kind of errors they produce might be different from other architectures such as CNN or RNN-based ones. Here we focus on one architecture to study the different errors produced only under different data configurations.
Question: What translationese effects are seen in the analysis?
Paper: Analysing Coreference in Transformer Outputs
Coreference is related to cohesion and coherence. The latter is the logical flow of inter-related ideas in a text, whereas cohesion refers to the text-internal relationship of linguistic elements that are overtly connected via lexico-grammatical devices across sentences BIBREF13. As stated by BIBREF14, this connectedness of texts implies dependencies between sentences. And if these dependencies are neglected in translation, the output text no longer has the property of connectedness which makes a sequence of sentences a text. Coreference expresses identity to a referent mentioned in another textual part (not necessarily in neighbouring sentences) contributing to text connectedness. An addressee is following the mentioned referents and identifies them when they are repeated. Identification of certain referents depends not only on a lexical form, but also on other linguistic means, e.g. articles or modifying pronouns BIBREF15. The use of these is influenced by various factors which can be language-dependent (range of linguistic means available in grammar) and also context-independent (pragmatic situation, genre). Thus, the means of expressing reference differ across languages and genres. This has been shown by some studies in the area of contrastive linguistics BIBREF6, BIBREF3, BIBREF5. Analyses in cross-lingual coreference resolution BIBREF16, BIBREF17, BIBREF18, BIBREF19 show that there are still unsolved problems that should be addressed.
Question: What languages are seen in the news and TED datasets?
Paper: Analysing Coreference in Transformer Outputs
Differences between languages and genres in the linguistic means expressing reference are important for translation, as the choice of an appropriate referring expression in the target language poses challenges for both human and machine translation. In translation studies, there is a number of corpus-based works analysing these differences in translation. However, most of them are restricted to individual phenomena within coreference. For instance, BIBREF20 analyse abstract anaphors in English-German translations. To our knowledge, they do not consider chains. BIBREF21 in their contrastive analysis of potential coreference chain members in English-German translations, describe transformation patterns that contain different types of referring expressions. However, the authors rely on automatic tagging and parsing procedures and do not include chains into their analysis. The data used by BIBREF4 and BIBREF22 contain manual chain annotations. The authors focus on different categories of anaphoric pronouns in English-Czech translations, though not paying attention to chain features (e.g. their number or size).
Question: How are the coreference chain translations evaluated?
Paper: Analysing Coreference in Transformer Outputs
As explained in the introduction, several recent works tackle the automatic translation of pronouns and also coreference BIBREF25, BIBREF26 and this has, in part, motivated the creation of devoted shared tasks and test sets to evaluate the quality of pronoun translation BIBREF7, BIBREF27, BIBREF28, BIBREF29.
Question: How are the (possibly incorrect) coreference chains in the MT outputs annotated?
Paper: Analysing Coreference in Transformer Outputs
Our NMT systems are based on a transformer architecture BIBREF0 as implemented in the Marian toolkit BIBREF42 using the transformer big configuration.
Question: Which three neural machine translation systems are analyzed?
Paper: Analysing Coreference in Transformer Outputs
is trained with the concatenation of Common Crawl, Europarl, a cleaned version of Rapid and the News Commentary corpus. We oversample the latter in order to have a significant representation of data close to the news genre in the final corpus.
Question: Which coreference phenomena are analyzed?
Paper: Exploring Scholarly Data by Semantic Query on Knowledge Graph Embedding Space
In recent years, digital libraries have moved towards open science and open access with several large scholarly datasets being constructed. Most popular datasets include millions of papers, authors, venues, and other information. Their large size and heterogeneous contents make it very challenging to effectively manage, explore, and utilize these datasets. The knowledge graph has emerged as a universal data format for representing knowledge about entities and their relationships in such complicated data. The main part of a knowledge graph is a collection of triples, with each triple $ (h, t, r) $ denoting the fact that relation $ r $ exists between head entity $ h $ and tail entity $ t $. This can also be formalized as a labeled directed multigraph where each triple $ (h, t, r) $ represents a directed edge from node $ h $ to node $ t $ with label $ r $. Therefore, it is straightforward to build knowledge graphs for scholarly data by representing natural connections between scholarly entities with triples such as (AuthorA, Paper1, write) and (Paper1, Paper2, cite).
Question: What new interesting tasks can be solved based on the uncanny semantic structures of the embedding space?
Paper: Exploring Scholarly Data by Semantic Query on Knowledge Graph Embedding Space
Knowledge graph has gradually become the standard data format for heterogeneous and complicated datasets BIBREF4. There have been several attempts to build knowledge graph for scholarly data, either adopting the scholarly network directly BIBREF5, or deriving the knowledge graph from some similarity measures BIBREF6 BIBREF7, or constructing the knowledge graph from survey papers BIBREF8. However, they mostly focus on the data format or graph inference aspects of knowledge graph. In this paper, we instead focus on the knowledge graph embedding methods and especially the application of embedding vectors in data exploration.
Question: What are the uncanny semantic structures of the embedding space?
Paper: Exploring Scholarly Data by Semantic Query on Knowledge Graph Embedding Space
For a more in depth survey of knowledge graph embedding methods, please refer to BIBREF0, which defines their architecture, categorization, and interaction mechanisms. In this paper, we only focus on the semantic structures of the state-of-the-art model CP$ _h $ BIBREF2, which is an extension of CP BIBREF9.
Question: What is the general framework for data exploration by semantic queries?
Paper: Exploring Scholarly Data by Semantic Query on Knowledge Graph Embedding Space
The most popular word embedding models in recent years are word2vec variants such as word2vec skipgram BIBREF3, which predicts the context-words $ c_i $ independently given the target-word $ w $, that is:
Question: What data exploration is supported by the analysis of these semantic structures?
Paper: NumNet: Machine Reading Comprehension with Numerical Reasoning
Machine reading comprehension (MRC) aims to infer the answer to a question given the document. In recent years, researchers have proposed lots of MRC models BIBREF0, BIBREF1, BIBREF2, BIBREF3 and these models have achieved remarkable results in various public benchmarks such as SQuAD BIBREF4 and RACE BIBREF5. The success of these models is due to two reasons: (1) Multi-layer architectures which allow these models to read the document and the question iteratively for reasoning; (2) Attention mechanisms which would enable these models to focus on the part related to the question in the document.
Question: what are the existing models they compared with?
Paper: Contextualized Word Embeddings Enhanced Event Temporal Relation Extraction for Story Understanding
Event temporal relation understanding is a major component of story/narrative comprehension. It is an important natural language understanding (NLU) task with broad applications to downstream tasks such as story understanding BIBREF0 , BIBREF1 , BIBREF2 , question answering BIBREF3 , BIBREF4 , and text summarization BIBREF5 , BIBREF6 .
Question: Do they report results only on English data?
Paper: Contextualized Word Embeddings Enhanced Event Temporal Relation Extraction for Story Understanding
We investigate both neural network-based models and traditional feature-based models. We briefly introduce them in this section.
Question: What conclusions do the authors draw from their detailed analyses?
Paper: Contextualized Word Embeddings Enhanced Event Temporal Relation Extraction for Story Understanding
is created by annotating 1600 sentences of 320 five-sentence stories sampled from ROCStories BIBREF7 dataset. CaTeRS contains both temporal and causal relations in an effort to understand and predict commonsense relations between events.
Question: Do the BERT-based embeddings improve results?
Paper: Contextualized Word Embeddings Enhanced Event Temporal Relation Extraction for Story Understanding
CAEVO consists of both linguistic-rule-based sieves and feature-based trainable sieves. We train CAEVO sieves with our train set and evaluate them on both dev and test sets. CAEVO is an end-to-end system that automatically annotates both events and relations. In order to resolve label annotation mismatch between CAEVO and our gold data, we create our own final input files to CAEVO system. Default parameter settings are used when running the CAEVO system.
Question: What were the traditional linguistic feature-based models?
Paper: Contextualized Word Embeddings Enhanced Event Temporal Relation Extraction for Story Understanding
Table TABREF25 contains the best hyper-parameters and Table TABREF26 contains micro-average F1 scores for both datasets on dev and test sets. We only consider positive pairs, i.e. correct predictions on NONE pairs are excluded for evaluation. In general, the baseline model CAEVO is outperformed by both NN models, and NN model with BERT embedding achieves the greatest performance. We now provide more detailed analysis and discussion for each dataset.
Question: What type of baseline are established for the two datasets?