z-uo/qasper-squad huggingface dataset:
Paper: Propagate-Selector: Detecting Supporting Sentences for Question Answering via Graph Neural Networks
Understanding texts and being able to answer a question posed by a human is a long-standing goal in the artificial intelligence field. Given the rapid advancement of neural network-based models and the availability of large-scale datasets, such as SQuAD BIBREF0 and TriviaQA BIBREF1, researchers have begun to concentrate on building automatic question-answering (QA) systems. One example of such a system is called the machine-reading question-answering (MRQA) model, which provides answers to questions from given passages BIBREF2, BIBREF3, BIBREF4.
Question: How much better performance of proposed model compared to answer-selection models?
Paper: Propagate-Selector: Detecting Supporting Sentences for Question Answering via Graph Neural Networks
Previous researchers have also investigated neural network-based models for MRQA. One line of inquiry employs an attention mechanism between tokens in the question and passage to compute the answer span from the given text BIBREF12, BIBREF3. As the task scope was extended from specific- to open-domain QA, several models have been proposed to select a relevant paragraph from the text to predict the answer span BIBREF13, BIBREF14. However, none of these methods have addressed reasoning over multiple sentences.
Question: How are some nodes initially connected based on text structure?
Paper: Katecheo: A Portable and Modular System for Multi-Topic Question Answering
When people interact with chatbots, smart speakers or digital assistants (e.g., Siri), one of their primary modes of interaction is information retrieval BIBREF0 . Thus, those that build dialog systems often have to tackle the problem of question answering.
Question: how many domains did they experiment with?
Answer: 2
Paper: Katecheo: A Portable and Modular System for Multi-Topic Question Answering
Katecheo is partially inspired by the work of BIBREF1 on DrQA. That previously developed method has two primary phases of question answering: document retrieval and reading comprehension. Together these functionalities enable open domain question answering. However, many dialog systems are not completely open domain. For example, developers might want to create a chatbot that has targeted conversations about restaurant reservations and movie times. It would be advantageous for such a chatbot to answer questions about food and entertainment, but the developers might not want to allow the conversation to stray into other topics.
Question: what pretrained models were used?
Paper: Transductive Learning with String Kernels for Cross-Domain Text Classification
Question: What domains are contained in the polarity classification dataset?
Paper: Transductive Learning with String Kernels for Cross-Domain Text Classification
Question: How long is the dataset?
Answer: 8000
Paper: Transductive Learning with String Kernels for Cross-Domain Text Classification
Transfer learning (or domain adaptation) aims at building effective classifiers for a target domain when the only available labeled training data belongs to a different (source) domain. Domain adaptation techniques can be roughly divided into graph-based methods BIBREF1 , BIBREF29 , BIBREF9 , BIBREF30 , probabilistic models BIBREF3 , BIBREF4 , knowledge-based models BIBREF14 , BIBREF31 , BIBREF11 and joint optimization frameworks BIBREF12 . The transfer learning methods from the literature show promising results in a variety of real-world applications, such as image classification BIBREF12 , text classification BIBREF13 , BIBREF16 , BIBREF3 , polarity classification BIBREF1 , BIBREF29 , BIBREF4 , BIBREF6 , BIBREF30 and others BIBREF32 .
Question: What machine learning algorithms are used?
Paper: Transductive Learning with String Kernels for Cross-Domain Text Classification
In recent years, methods based on string kernels have demonstrated remarkable performance in various text classification tasks BIBREF35 , BIBREF36 , BIBREF22 , BIBREF19 , BIBREF10 , BIBREF17 , BIBREF26 . String kernels represent a way of using information at the character level by measuring the similarity of strings through character n-grams. Lodhi et al. BIBREF35 used string kernels for document categorization, obtaining very good results. String kernels were also successfully used in authorship identification BIBREF22 . More recently, various combinations of string kernels reached state-of-the-art accuracy rates in native language identification BIBREF19 and Arabic dialect identification BIBREF17 . Interestingly, string kernels have been used in cross-domain settings without any domain adaptation, obtaining impressive results. For instance, Ionescu et al. BIBREF19 have employed string kernels in a cross-corpus (and implicitly cross-domain) native language identification experiment, improving the state-of-the-art accuracy by a remarkable INLINEFORM0 . Giménez-Pérez et al. BIBREF10 have used string kernels for single-source and multi-source polarity classification. Remarkably, they obtain state-of-the-art performance without using knowledge from the target domain, which indicates that string kernels provide robust results in the cross-domain setting without any domain adaptation. Ionescu et al. BIBREF17 obtained the best performance in the Arabic Dialect Identification Shared Task of the 2017 VarDial Evaluation Campaign BIBREF37 , with an improvement of INLINEFORM1 over the second-best method. It is important to note that the training and the test speech samples prepared for the shared task were recorded in different setups BIBREF37 , or in other words, the training and the test sets are drawn from different distributions. Different from all these recent approaches BIBREF19 , BIBREF10 , BIBREF17 , we use unlabeled data from the target domain to significantly increase the performance of string kernels in cross-domain text classification, particularly in English polarity classification.
Question: What is a string kernel?
Answer: String kernel is a technique that uses character n-grams to measure the similarity of strings
Paper: Towards an Unsupervised Entrainment Distance in Conversational Speech using Deep Neural Networks
Vocal entrainment is an established social adaptation mechanism. It can be loosely defined as one speaker's spontaneous adaptation to the speaking style of the other speaker. Entrainment is a fairly complex multifaceted process and closely associated with many other mechanisms such as coordination, synchrony, convergence etc. While there are various aspects and levels of entrainment BIBREF0 , there is also a general agreement that entrainment is a sign of positive behavior towards the other speaker BIBREF1 , BIBREF2 , BIBREF3 . High degree of vocal entrainment has been associated with various interpersonal behavioral attributes, such as high empathy BIBREF4 , more agreement and less blame towards the partner and positive outcomes in couple therapy BIBREF5 , and high emotional bond BIBREF6 . A good understanding of entrainment provides insights to various interpersonal behaviors and facilitates the recognition and estimation of these behaviors in the realm of Behavioral Signal Processing BIBREF7 , BIBREF8 . Moreover, it also contributes to the modeling and development of `human-like' spoken dialog systems or conversational agents.
Question: Which dataset do they use to learn embeddings?
Paper: Towards an Unsupervised Entrainment Distance in Conversational Speech using Deep Neural Networks
We use two datasets in this work: the training is done on the Fisher Corpus English Part 1 (LDC2004S13) BIBREF15 and testing on the Suicide Risk Assessment corpus BIBREF16 , along with Fisher.
Question: How do they correlate NED with emotional bond levels?
Answer: They compute Pearson’s correlation between NED measure for patient-to-therapist and patient-perceived emotional bond rating and NED measure for therapist-to-patient and patient-perceived emotional bond rating
Paper: Named Entity Recognition with Partially Annotated Training Data
Most modern approaches to NLP tasks rely on supervised learning algorithms to learn and generalize from labeled training data. While this has proven successful in high-resource scenarios, this is not realistic in many cases, such as low-resource languages, as the required amount of training data just doesn't exist. However, partial annotations are often easy to gather.
Question: What was their F1 score on the Bengali NER corpus?
Answer: 52.0%
Paper: Named Entity Recognition with Partially Annotated Training Data
The supervision paradigm in this paper, partial supervision, falls broadly under the category of semi-supervision BIBREF0, and is closely related to weak supervision BIBREF1 and incidental supervision BIBREF2, in the sense that data is constructed through some noisy process. However, all of the most related work shares a key difference from ours: reliance on a small amount of fully annotated data in addition to the noisy data.
Question: Which languages are evaluated?
Paper: End-to-End Speech Recognition: A review for the French Language
Automatic Speech Recognition (ASR) has traditionally used Hidden Markov Models (HMM), describing temporal variability, combined with Gaussian Mixture Models (GMM), computing emission probabilities from HMM states, to model and map acoustic features to phones. In recent years, the introduction of deep neural networks replacing GMM for acoustic modeling showed huge improvements compared to previous state-of-the-art systems BIBREF0, BIBREF1. However, building and training such systems can be complex and a lot of preprocessing steps are involved. Traditional ASR systems are also factorized in several modules, the acoustic model representing only one of them along with lexicon and language models.
Question: Which model have the smallest Character Error Rate and which have the smallest Word Error Rate?
Paper: End-to-End Speech Recognition: A review for the French Language
The CTC BIBREF5 can be seen as a direct translation of conventional HMM-DNN ASR systems into lexicon-free systems. Thus, the CTC follows the general ASR formulation, training the model to maximize $P(Y|X)$ the probability distribution over all possible label sequences: Y = arg Y A* p(Y|X) Here, $X$ denotes the observations, $Y$ is a sequence of acoustic units of length $L$ such that $Y = \lbrace y_{l} \in \, \mathcal {A} | l = 1, ..., L\rbrace $, where $\mathcal {A}$ is an alphabet containing all distinct units. As in traditional HMM-DNN systems, the CTC model makes conditional independence assumptions between output predictions at different time steps given aligned inputs and it uses the probabilistic chain rule to factorize the posterior distribution $p(Y|X)$ into three distributions (i.e. framewise posterior distribution, transition probability and prior distribution of units). However, unlike HMM-based models, the framewise posterior distribution is defined here as a framewise acoustic unit sequence $B$ with an additional blank label ${<}blank{>}$ such as $B = \lbrace b_{t} \in \, \mathcal {A}\, \cup \, {<}blank{>} | t = 1, ..., T\rbrace $. p(Y|X) = b=1Bt=1T p(bt | bt-1, Y) p(bt|X)pctc(Y|X) p(Y)
Question: What will be in focus for future work?
Paper: End-to-End Speech Recognition: A review for the French Language
As opposed to CTC, the attention-based approach BIBREF7, BIBREF8 does not assume conditional independence between predictions at different time steps and does not marginalize over all alignments. Thus the posterior distribution $p(Y|X)$ is directly computed by picking a soft alignment between each output step and every input step as follows: patt(Y|X) = l=1U p(yl | y1, ..., yl-1, X) Here $p(y_{l}|y_{1},...,y_{l-1}, X)$, – our attention-based objective function –, is obtained according to a probability distribution, typically a softmax, applied to the linear projection of the output of a recurrent neural network (or long-short term memory network), called decoder, such as: p(yl|y1,...,yl-1, X) = softmax(lin(RNN())) The decoder output is conditioned by the previous output $y_{l-1}$, a hidden vector $d_{l-1}$ and a context vector $c_{l}$. Here $d_{l-1}$ denotes the high level representation (i.e. hidden states) of the decoder at step $l-1$, encoding the target input, and $c_{l}$ designate the context – or symbol-wise vector in our case – for decoding step $l$, which is computed as the sum of the complete high representation $h$ of another recurrent neural network, encoding the source input $X$, weighted by $\alpha $ the attention weight: cl = s=1S l, s hs , l, s = (et, s)s'=1S (el, s') where $e_{t}$, also referred to as energy, measures how well the inputs around position $s$ and the output at position $l$ match, given the decoder states at decoding step $l-1$ and $h$ the encoder states for input $X$. In the following, we report the standard content-based mechanism and its location-aware variant which takes into account the alignment produced at the previous step using convolutional features: el, s = {ll content-based:
Question: Which acoustic units are more suited to model the French language?
Paper: End-to-End Speech Recognition: A review for the French Language
The RNN transducer architecture was first introduced by Graves and al. BIBREF9 to address the main limitation of the proposed CTC network: it cannot model interdependencies as it assumes conditional independence between predictions at different time steps.
Question: What are the existing end-to-end ASR approaches for the French language?
Paper: Analyzing Word Translation of Transformer Layers
Neural Machine Translation (NMT) has achieved great success in the last few years BIBREF0, BIBREF1, BIBREF2. The popular Transformer BIBREF2 model, which outperforms previous RNN/CNN based translation models BIBREF0, BIBREF1, is based on multi-layer self-attention networks and can be paralleled effectively.
Question: How much is decoding speed increased by increasing encoder and decreasing decoder depth?
Paper: MedDialog: A Large-scale Medical Dialogue Dataset
Telemedicine refers to the practice of delivering patient care remotely, where doctors provide medical consultations to patients using HIPAA compliant video-conferencing tools. As an important complement to traditional face-to-face medicine practiced physically in hospitals and clinics, telemedicine has a number of advantages. First, it increases access to care. For people living in medically under-served communities (e.g., rural areas) that are in shortage of clinicians, telemedicine enables them to receive faster and cheaper care compared with traveling over a long distance to visit a clinician. Second, it reduces healthcare cost. In a study by Jefferson Health, it is shown that diverting patients from emergency departments with telemedicine can save more than $1,500 per visit. Third, telemedicine can improve quality of care. The study in BIBREF0 shows that telemedicine patients score lower for depression, anxiety, and stress, and have 38% fewer hospital admissions. Other advantages include improving patient engagement and satisfaction, improving provider satisfaction, etc. Please refer to BIBREF1 for a more comprehensive review.
Question: Did they experiment on this dataset?
Paper: MedDialog: A Large-scale Medical Dialogue Dataset
The MedDialog dataset contains 1,145,231 consultations between patients and doctors. The total number of utterances is 3,959,333: 2,179,008 from doctors and 1,780,325 from patients. Each consultation consists of three parts: (1) description of patient's medical condition and history; (2) conversation between patient and doctor; (3) (optional) diagnosis and treatment suggestions given by the doctor. In the description of patient's medical condition and history, the following fields are included: present disease, detailed description of present disease, what help is needed from the doctor, how long the disease has been, medications, allergies, and past disease. Figure FIGREF3 shows an exemplar consultation. In the conversation, there are cases that multiple consecutive utterances are from the same person (either doctor or patient) and these utterances were posted at different time points. If we combine consecutive utterances from the same person into a single one, there are 3,209,660 utterances: 1,981,844 from doctors and 1,227,816 from patients. The data is crawled from haodf.com, which is an online platform of healthcare services, including medical consultation, scheduling appointment with doctors, etc.
Question: What language are the conversations in?
Paper: MedDialog: A Large-scale Medical Dialogue Dataset
Large number of conversations and utterances. To our best knowledge, MedDialog is the largest medical dialogue dataset. It has about 1.1 million conversations and 4 million utterances.
Question: How did they annotate the dataset?
Paper: MedDialog: A Large-scale Medical Dialogue Dataset
The language is Chinese, which is not easy for non-Chinese-speaking researchers to work on.
Question: What annotations are in the dataset?
Paper: Learning to Automatically Generate Fill-In-The-Blank Quizzes
With the advent of the Web 2.0, regular users were able to share, remix and distribute content very easily. As a result of this process, the Web became a rich interconnected set of heterogeneous data sources. Being in a standard format, it is suitable for many tasks involving knowledge extraction and representation. For example, efforts have been made to design games with the purpose of semi-automating a wide range of knowledge transfer tasks, such as educational quizzes, by leveraging on this kind of data.
Question: What is the size of the dataset?
Answer: 300,000 sentences with 1.5 million single-quiz questions
Paper: Learning to Automatically Generate Fill-In-The-Blank Quizzes
The problem of fill-in-the-blank question generation has been studied in the past by several authors. Perhaps the earlies approach is by BIBREF1 , who proposed a cloze question generation system which focuses on distractor generation using search engines to automatically measure English proficiency. In the same research line, we also find the work of BIBREF2 , BIBREF3 and BIBREF4 . In this context, the work of BIBREF10 probably represents the first effort in applying machine learning techniques for multiple-choice cloze question generation. The authors propose an approach that uses conditional random fields BIBREF11 based on hand-crafted features such as word POS tags.
Question: What language platform does the data come from?
Paper: Learning to Automatically Generate Fill-In-The-Blank Quizzes
We formalize the problem of automatic fill-on-the-blanks quiz generation using two different perspectives. These are designed to match with specific machine learning schemes that are well-defined in the literature. In both cases. we consider a training corpus of INLINEFORM0 pairs INLINEFORM1 where INLINEFORM2 is a sequence of INLINEFORM3 tokens and INLINEFORM4 is an index that indicates the position that should be blanked inside INLINEFORM5 .
Question: Which two schemes are used?
Paper: Fast Domain Adaptation for Neural Machine Translation
Due to the fact that Neural Machine Translation (NMT) is reaching comparable or even better performance compared to the traditional statistical machine translation (SMT) models BIBREF0 , BIBREF1 , it has become very popular in the recent years BIBREF2 , BIBREF3 , BIBREF4 . With the great success of NMT, new challenges arise which have already been address with reasonable success in traditional SMT. One of the challenges is domain adaptation. In a typical domain adaptation setup such as ours, we have a large amount of out-of-domain bilingual training data for which we already have a trained neural network model (baseline). Given only an additional small amount of in-domain data, the challenge is to improve the translation performance on the new domain without deteriorating the performance on the general domain significantly. One approach one might take is to combine the in-domain data with the out-of-domain data and train the NMT model from scratch. However, there are two main problems with that approach. First, training a neural machine translation system on large data sets can take several weeks and training a new model based on the combined training data is time consuming. Second, since the in-domain data is relatively small, the out-of-domain data will tend to dominate the training data and hence the learned model will not perform as well on the in-domain test data. In this paper, we reuse the already trained out-of-domain system and continue training only on the small portion of in-domain data similar to BIBREF5 . While doing this, we adapt the parameters of the neural network model to the new domain. Instead of relying completely on the adapted (further-trained) model and over fitting on the in-domain data, we decode using an ensemble of the baseline model and the adapted model which tends to perform well on the in-domain data without deteriorating the performance on the baseline general domain.
Question: How many examples do they have in the target domain?
Answer: Around 388k examples, 194k from tst2013 (in-domain) and 194k from newstest2014 (out-of-domain)
Paper: The Grail theorem prover: Type theory for syntax and semantics
This chapter describes a series of tools for developing and testing type-logical grammars. The Grail family of theorem provers have been designed to work with a variety of modern type-logical frameworks, including multimodal type-logical grammars BIBREF0 , NL $_{cl}$ BIBREF1 , the Displacement calculus BIBREF2 and hybrid type-logical grammars BIBREF3 .
Question: Does Grail accept Prolog inputs?
Paper: The Grail theorem prover: Type theory for syntax and semantics
Type-logical grammars are a family of grammar formalisms built on a foundation of logic and type theory. Type-logical grammars originated when BIBREF4 introduced his Syntactic Calculus (called the Lambek calculus, L, by later authors). Though Lambek built on the work of BIBREF5 , BIBREF6 and others, Lambek's main innovation was to cast the calculus as a logic, giving a sequent calculus and showing decidability by means of cut elimination. This combination of linguistic and computational applications has proved very influential.
Question: What formalism does Grail use?
Paper: A Joint Model for Question Answering and Question Generation
Question answering (QA) is the task of automatically producing an answer to a question given a corresponding document. It not only provides humans with efficient access to vast amounts of information, but also acts as an important proxy task to assess machine literacy via reading comprehension. Thanks to the recent release of several large-scale machine comprehension/QA datasets BIBREF0 , BIBREF1 , BIBREF2 , BIBREF3 , BIBREF4 , the field has undergone significant advancement, with an array of neural models rapidly approaching human parity on some of these benchmarks BIBREF5 , BIBREF6 , BIBREF7 . However, previous models do not treat QA as a task of natural language generation (NLG), but of pointing to an answer span within a document.
Question: Which components of QA and QG models are shared during training?
Paper: A Joint Model for Question Answering and Question Generation
Joint-learning on multiple related tasks has been explored previously BIBREF17 , BIBREF18 . In machine translation, for instance, BIBREF18 demonstrated that translation quality clearly improves over models trained with a single language pair when the attention mechanism in a neural translation model is shared and jointly trained on multiple language pairs.
Question: How much improvement does jointly learning QA and QG give, compared to only training QA?
Paper: Word Embeddings via Tensor Factorization
Word embeddings have been used to improve the performance of many NLP tasks including language modelling BIBREF1 , machine translation BIBREF2 , and sentiment analysis BIBREF3 . The broad applicability of word embeddings to NLP implies that improvements to their quality will likely have widespread benefits for the field.
Question: Do they test their word embeddings on downstream tasks?