z-uo/qasper-squad huggingface dataset:
Paper: Bias in Semantic and Discourse Interpretation
Bias is generally considered to be a negative term: a biased story is seen as one that perverts or subverts the truth by offering a partial or incomplete perspective on the facts. But bias is in fact essential to understanding: one cannot interpret a set of facts—something humans are disposed to try to do even in the presence of data that is nothing but noise [38]—without relying on a bias or hypothesis to guide that interpretation. Suppose someone presents you with the sequence INLINEFORM0 and tells you to guess the next number. To make an educated guess, you must understand this sequence as instantiating a particular pattern; otherwise, every possible continuation of the sequence will be equally probable for you. Formulating a hypothesis about what pattern is at work will allow you to predict how the sequence will play out, putting you in a position to make a reasonable guess as to what comes after 3. Formulating the hypothesis that this sequence is structured by the Fibonacci function (even if you don't know its name), for example, will lead you to guess that the next number is 5; formulating the hypothesis that the sequence is structured by the successor function but that every odd successor is repeated once will lead you to guess that it is 3. Detecting a certain pattern allows you to determine what we will call a history: a set of given entities or eventualities and a set of relations linking those entities together. The sequence of numbers INLINEFORM1 and the set of relation instances that the Fibonacci sequence entails as holding between them is one example of a history. Bias, then, is the set of features, constraints, and assumptions that lead an interpreter to select one history—one way of stitching together a set of observed data—over another.
Question: What factors contribute to interpretive biases according to this research?
Paper: Bias in Semantic and Discourse Interpretation
In this paper, we propose a program for research on bias. We will show how to model various types of bias as well as the way in which bias leads to the selection of a history for a set of data, where the data might be a set of nonlinguistic entities or a set of linguistically expressed contents. In particular, we'll look at what people call “unbiased” histories. For us these also involve a bias, what we call a “truth seeking bias”. This is a bias that gets at the truth or acceptably close to it. Our model can show us what such a bias looks like. And we will examine the question of whether it is possible to find such a truth oriented bias for a set of facts, and if so, under what conditions. Can we detect and avoid biases that don't get at the truth but are devised for some other purpose?
Question: Which interpretative biases are analyzed in this paper?
Paper: A Swiss German Dictionary: Variation in Speech and Writing
Swiss German refers to any of the German varieties that are spoken in about two thirds of Switzerland BIBREF0. Besides at least one of those dialectal varieties, Swiss German people also master standard (or 'High') German which is taught in school as the official language of communication.
Question: How many words are coded in the dictionary?
Paper: A Swiss German Dictionary: Variation in Speech and Writing
This dictionary complements previously developed resources for Swiss German, which share some common information. Spontaneous noisy writing has already been recorded in text corpora BIBREF1, BIBREF2, BIBREF3, some of which are also normalized. These resources contain relatively large lexicons of words used in context, but they do not contain any information about pronunciation. The features of speech are represented in other resources, such as BIBREF4, BIBREF5, BIBREF6, which, on the other hand, contain relatively small lexicons (small set of words known to vary across dialects). The ArchiMob corpus does contain a large lexicon of speech and writing (Dieth transcription), but the spoken part is available in audio sources only, without phonetic transcription.
Question: Is the model evaluated on the graphemes-to-phonemes task?
Paper: QuaRel: A Dataset and Models for Answering Questions about Qualitative Relationships
Many natural language tasks require recognizing and reasoning with qualitative relationships. For example, we may read about temperatures rising (climate science), a drug dose being increased (medicine), or the supply of goods being reduced (economics), and want to reason about the effects. Qualitative story problems, of the kind found in elementary exams (e.g., Figure FIGREF1 ), form a natural example of many of these linguistic and reasoning challenges, and is the target of this work.
Question: How does the QuaSP+Zero model work?
Paper: QuaRel: A Dataset and Models for Answering Questions about Qualitative Relationships
There has been rapid progress in question-answering (QA), spanning a wide variety of tasks and phenomena, including factoid QA BIBREF3 , entailment BIBREF4 , sentiment BIBREF5 , and ellipsis and coreference BIBREF6 . Our contribution here is the first dataset specifically targeted at qualitative relationships, an important category of language that has been less explored. While questions requiring reasoning about qualitative relations sometimes appear in other datasets, e.g., BIBREF7 , our dataset specifically focuses on them so their challenges can be studied.
Question: Which off-the-shelf tools do they use on QuaRel?
Paper: QuaRel: A Dataset and Models for Answering Questions about Qualitative Relationships
We first describe our framework for representing questions and the knowledge to answer them. Our dataset, described later, includes logical forms expressed in this language.
Question: How do they obtain the logical forms of their questions in their dataset?
Paper: QuaRel: A Dataset and Models for Answering Questions about Qualitative Relationships
We use a simple representation of qualitative relationships, leveraging prior work in qualitative reasoning BIBREF0 . Let INLINEFORM0 be the set of properties relevant to the question set's domain (e.g., smoothness, friction, speed). Let INLINEFORM1 be a set of qualitative values for property INLINEFORM2 (e.g., fast, slow). For the background knowledge about the domain itself (a qualitative model), following BIBREF0 Forbus1984QualitativePT, we use the following predicates: [vskip=1mm,leftmargin=5mm] q+(property1, property2)
Question: Do all questions in the dataset allow the answers to pick from 2 options?
Paper: Natural Language Interactions in Autonomous Vehicles: Intent Detection and Slot Filling from Passenger Utterances
One of the exciting yet challenging areas of research in Intelligent Transportation Systems is developing context-awareness technologies that can enable autonomous vehicles to interact with their passengers, understand passenger context and situations, and take appropriate actions accordingly. To this end, building multi-modal dialogue understanding capabilities situated in the in-cabin context is crucial to enhance passenger comfort and gain user confidence in AV interaction systems. Among many components of such systems, intent recognition and slot filling modules are one of the core building blocks towards carrying out successful dialogue with passengers. As an initial attempt to tackle some of those challenges, this study introduce in-cabin intent detection and slot filling models to identify passengers' intent and extract semantic frames from the natural language utterances in AV. The proposed models are developed by leveraging User Experience (UX) grounded realistic (ecologically valid) in-cabin dataset. This dataset is generated with naturalistic passenger behaviors, multiple passenger interactions, and with presence of a Wizard-of-Oz (WoZ) agent in moving vehicles with noisy road conditions.
Question: What is shared in the joint model?
Paper: Natural Language Interactions in Autonomous Vehicles: Intent Detection and Slot Filling from Passenger Utterances
Long Short-Term Memory (LSTM) networks BIBREF0 are widely-used for temporal sequence learning or time-series modeling in Natural Language Processing (NLP). These neural networks are commonly employed for sequence-to-sequence (seq2seq) and sequence-to-one (seq2one) modeling problems, including slot filling tasks BIBREF1 and utterance-level intent classification BIBREF2 , BIBREF3 which are well-studied for various application domains. Bidirectional LSTMs (Bi-LSTMs) BIBREF4 are extensions of traditional LSTMs which are proposed to improve model performance on sequence classification problems even further. Jointly modeling slot extraction and intent recognition BIBREF2 , BIBREF5 is also explored in several architectures for task-specific applications in NLP. Using Attention mechanism BIBREF6 , BIBREF7 on top of RNNs is yet another recent break-through to elevate the model performance by attending inherently crucial sub-modules of given input. There exist various architectures to build hierarchical learning models BIBREF8 , BIBREF9 , BIBREF10 for document-to-sentence level, and sentence-to-word level classification tasks, which are highly domain-dependent and task-specific.
Question: Are the intent labels imbalanced in the dataset?
Paper: Sentiment Analysis of Citations Using Word2vec
The evolution of scientific ideas happens when old ideas are replaced by new ones. Researchers usually conduct scientific experiments based on the previous publications. They either take use of others work as a solution to solve their specific problem, or they improve the results documented in the previous publications by introducing new solutions. I refer to the former as positive citation and the later negative citation. Citation sentence examples with different sentiment polarity are shown in Table TABREF2 .
Question: What kernels are used in the support vector machines?
Paper: Sentiment Analysis of Citations Using Word2vec
Mikolov et al. introduced word2vec technique BIBREF3 that can obtain word vectors by training text corpus. The idea of word2vec (word embeddings) originated from the concept of distributed representation of words BIBREF5 . The common method to derive the vectors is using neural probabilistic language model BIBREF6 . Word embeddings proved to be effective representations in the tasks of sentiment analysis BIBREF4 , BIBREF7 , BIBREF8 and text classification BIBREF9 . Sadeghian and Sharafat BIBREF10 extended word embeddings to sentence embeddings by averaging the word vectors in a sentiment review statement. Their results showed that word embeddings outperformed the bag-of-words model in sentiment classification. In this work, I are aiming at evaluating word embeddings for sentiment analysis of citations. The research questions are:
Question: What dataset is used?
Paper: Sentiment Analysis of Citations Using Word2vec
The SentenceModel provided by LingPipe was used to segment raw text into its constituent sentences . The data I used to train the vectors has noise. For example, there are incomplete sentences mistakenly detected (e.g. Publication Year.). To address this issue, I eliminated sentences with less than three words.
Question: What metrics are considered?
Paper: Modeling Coherence for Neural Machine Translation with Dynamic and Topic Caches
In the literature, several cache-based translation models have been proposed for conventional statistical machine translation, besides traditional n-gram language models and neural language models. In this section, we will first introduce related work in cache-based language models and then in translation models.
Question: Did the authors evaluate their system output for coherence?
Paper: Modeling Coherence for Neural Machine Translation with Dynamic and Topic Caches
In this section, we briefly describe the NMT model taken as a baseline. Without loss of generality, we adopt the NMT architecture proposed by bahdanau2015neural, with an encoder-decoder neural network.
Question: What evaluations did the authors use on their system?
Answer: BLEU scores, exact matches of words in both translations and topic cache, and cosine similarities of adjacent sentences for coherence.
Paper: Indiscapes: Instance Segmentation Networks for Layout Parsing of Historical Indic Manuscripts
The collection and analysis of historical document images is a key component in the preservation of culture and heritage. Given its importance, a number of active research efforts exist across the world BIBREF0, BIBREF1, BIBREF2, BIBREF3, BIBREF4, BIBREF5. In this paper, we focus on palm-leaf and early paper documents from the Indian sub-continent. In contrast with modern or recent era documents, such manuscripts are considerably more fragile, prone to degradation from elements of nature and tend to have a short shelf life BIBREF6, BIBREF7, BIBREF8. More worryingly, the domain experts who can decipher such content are small in number and dwindling. Therefore, it is essential to access the content within these documents before it is lost forever.
Question: What accuracy does CNN model achieve?
Answer: Combined per-pixel accuracy for character line segments is 74.79
Paper: Indiscapes: Instance Segmentation Networks for Layout Parsing of Historical Indic Manuscripts
A number of research groups have invested significant efforts in the creation and maintenance of annotated, publicly available historical manuscript image datasets BIBREF10, BIBREF11, BIBREF0, BIBREF1, BIBREF2, BIBREF3, BIBREF12. Other collections contain character-level and word-level spatial annotations for South-East Asian palm-leaf manuscripts BIBREF9, BIBREF4, BIBREF13. In these latter set of works, annotations for lines are obtained by considering the polygonal region formed by union of character bounding boxes as a line. While studies on Indic palm-leaf and paper-based manuscripts exist, these are typically conducted on small and often, private collections of documents BIBREF14, BIBREF15, BIBREF16, BIBREF17, BIBREF18, BIBREF19, BIBREF20. No publicly available large-scale, annotated dataset of historical Indic manuscripts exists to the best of our knowledge. In contrast with existing collections, our proposed dataset contains a much larger diversity in terms of document type (palm-leaf and early paper), scripts and annotated layout elements (see Tables TABREF5,TABREF8). An additional level of complexity arises from the presence of multiple manuscript pages within a single image (see Fig. FIGREF1).
Question: How many documents are in the Indiscapes dataset?
Answer: 508
Paper: Indiscapes: Instance Segmentation Networks for Layout Parsing of Historical Indic Manuscripts
The Indic manuscript document images in our dataset are obtained from two sources. The first source is the publicly available Indic manuscript collection from University of Pennsylvania's Rare Book and Manuscript Library BIBREF39, also referred to as Penn-in-Hand (PIH). From the $2{,}880$ Indic manuscript book-sets, we carefully curated 193 manuscript images for annotation. Our curated selection aims to maximize the diversity of the dataset in terms of various attributes such as the extent of document degradation, script language, presence of non-textual elements (e.g. pictures, tables) and number of lines. Some images contain multiple manuscript pages stacked vertically or horizontally (see bottom-left image in Figure FIGREF1). The second source for manuscript images in our dataset is Bhoomi, an assorted collection of 315 images sourced from multiple Oriental Research Institutes and libraries across India. As with the first collection, we chose a subset intended to maximize the overall diversity of the dataset. However, this latter set of images are characterized by a relatively inferior document quality, presence of multiple languages and from a layout point of view, predominantly contain long, closely and irregularly spaced text lines, binding holes and degradations (Figure FIGREF1). Though some document images contain multiple manuscripts, we do not attempt to split the image into multiple pages. While this poses a challenge for annotation and automatic image parsing, retaining such images in the dataset eliminates manual/semi-automatic intervention. As our results show, our approach can successfully handle such multi-page documents, thereby making it truly an end-to-end system.
Question: What language(s) are the manuscripts written in?
Paper: Semantic Document Distance Measures and Unsupervised Document Revision Detection
It is a common habit for people to keep several versions of documents, which creates duplicate data. A scholarly article is normally revised several times before being published. An academic paper may be listed on personal websites, digital conference libraries, Google Scholar, etc. In major corporations, a document typically goes through several revisions involving multiple editors and authors. Users would benefit from visualizing the entire history of a document. It is worthwhile to develop a system that is able to intelligently identify, manage and represent revisions. Given a collection of text documents, our study identifies revision relationships in a completely unsupervised way. For each document in a corpus we only use its content and the last modified timestamp. We assume that a document can be revised by many users, but that the documents are not merged together. We consider collaborative editing as revising documents one by one.
Question: What metrics are used to evaluation revision detection?
Paper: Semantic Document Distance Measures and Unsupervised Document Revision Detection
The two requirements for a document INLINEFORM0 being a revision of another document INLINEFORM1 are that INLINEFORM2 has been created later than INLINEFORM3 and that the content of INLINEFORM4 is similar to (has been modified from) that of INLINEFORM5 . More specifically, given a corpus INLINEFORM6 , for any two documents INLINEFORM7 , we want to find out the yes/no revision relationship of INLINEFORM8 and INLINEFORM9 , and then output all such revision pairs.
Question: How large is the Wikipedia revision dump dataset?
Paper: Semantic Document Distance Measures and Unsupervised Document Revision Detection
In this section, we first introduce the classic VSM model, the word2vec model, DTW and TED. We next demonstrate how to combine the above components to construct our semantic document distance measures: wDTW and wTED. We also discuss the implementation of our revision detection system.
Question: What are simulated datasets collected?
Answer: There are 6 simulated datasets collected which is initialised with a corpus of size 550 and simulated by generating new documents from Wikipedia extracts and replacing existing documents
Paper: Semantic Document Distance Measures and Unsupervised Document Revision Detection
VSM represents a set of documents as vectors of identifiers. The identifier of a word used in this work is the tf-idf weight. We represent documents as tf-idf vectors, and thus the similarity of two documents can be described by the cosine distance between their vectors. VSM has low algorithm complexity but cannot represent the semantics of words since it is based on the bag-of-words assumption.
Question: Which are the state-of-the-art models?
Paper: Multi-Task Bidirectional Transformer Representations for Irony Detection
The proliferation of social media has provided a locus for use, and thereby collection, of figurative and creative language data, including irony BIBREF0. According to the Merriam-Webster online dictionary, irony refers to “the use of word to express something other than and especially the opposite of the literal meaning." A complex, controversial, and intriguing linguistic phenomenon, irony has been studied in disciplines such as linguistics, philosophy, and rhetoric. Irony detection also has implications for several NLP tasks such as sentiment analysis, hate speech detection, fake news detection, etc BIBREF0. Hence, automatic irony detection can potentially improve systems designed for each of these tasks. In this paper, we focus on learning irony. More specifically, we report our work submitted to the FIRE 2019 Arabic irony detection task (IDAT@FIRE2019). We focus our energy on an important angle of the problem–the small size of training data.
Question: Why is being feature-engineering free an advantage?
Paper: Multi-Task Bidirectional Transformer Representations for Irony Detection
Question: Where did this model place in the final evaluation of the shared task?
Paper: Multi-Task Bidirectional Transformer Representations for Irony Detection
For our baseline, we use gated recurrent units (GRU) BIBREF4, a simplification of long-short term memory (LSTM) BIBREF5, which in turn is a variation of recurrent neural networks (RNNs). A GRU learns based on the following:
Question: What in-domain data is used to continue pre-training?
Paper: Multi-Task Bidirectional Transformer Representations for Irony Detection
BERT BIBREF1 is based on the Transformer BIBREF6, a network architecture that depends solely on encoder-decoder attention. The Transformer attention employs a function operating on queries, keys, and values. This attention function maps a query and a set of key-value pairs to an output, where the output is a weighted sum of the values. Encoder of the Transformer in BIBREF6 has 6 attention layers, each of which is composed of two sub-layers: (1) multi-head attention where queries, keys, and values are projected h times into linear, learned projections and ultimately concatenated; and (2) fully-connected feed-forward network (FFN) that is applied to each position separately and identically. Decoder of the Transformer also employs 6 identical layers, yet with an extra sub-layer that performs multi-head attention over the encoder stack. The architecture of BERT BIBREF1 is a multi-layer bidirectional Transformer encoder BIBREF6. It uses masked language models to enable pre-trained deep bidirectional representations, in addition to a binary next sentence prediction task captures context (i.e., sentence relationships). More information about BERT can be found in BIBREF1.
Question: What dialect is used in the Google BERT model and what is used in the task data?
Paper: Multi-Task Bidirectional Transformer Representations for Irony Detection
In multi-task learning (MTL), a learner uses a number of (usually relevant) tasks to improve performance on a target task BIBREF2, BIBREF3. The MTL setup enables the learner to use cues from various tasks to improve the performance on the target task. MTL also usually helps regularize the model since the learner needs to find representations that are not specific to a single task, but rather more general. Supervised learning with deep neural networks requires large amounts of labeled data, which is not always available. By employing data from additional tasks, MTL thus practically augments training data to alleviate need for large labeled data. Many researchers achieve state-of-the-art results by employing MTL in supervised learning settings BIBREF7, BIBREF8. In specific, BERT was successfully used with MTL. Hence, we employ multi-task BERT (following BIBREF8). For our training, we use the same pre-trained BERT-Base Multilingual Cased model as the initial checkpoint. For this MTL pre-training of BERT, we use the same afore-mentioned single-task BERT parameters. We now describe our data.
Question: What are the tasks used in the mulit-task learning setup?
Paper: Evaluating Rewards for Question Generation Models
Posing questions about a document in natural language is a crucial aspect of the effort to automatically process natural language data, enabling machines to ask clarification questions BIBREF0 , become more robust to queries BIBREF1 , and to act as automatic tutors BIBREF2 .
Question: What human evaluation metrics were used in the paper?
Answer: rating questions on a scale of 1-5 based on fluency of language used and relevance of the question to the context