z-uo/qasper-squad huggingface dataset:
Paper: Principles for Developing a Knowledge Graph of Interlinked Events from News Headlines on Twitter
Several successful efforts have led to publishing huge RDF (Resource Description Framework) datasets on Linked Open Data (LOD) such as DBpedia BIBREF0 and LinkedGeoData BIBREF1 . However, these sources are limited to either structured or semi-structured data. So far, a significant portion of the Web content consists of textual data from social network feeds, blogs, news, logs, etc. Although the Natural Language Processing (NLP) community has developed approaches to extract essential information from plain text (e.g., BIBREF2 , BIBREF3 , BIBREF4 ), there is convenient support for knowledge graph construction. Further, several lexical analysis based approaches extract only a limited form of metadata that is inadequate for supporting applications such as question answering systems. For example, the query “Give me the list of reported events by BBC and CNN about the number of killed people in Yemen in the last four days”, about a recent event (containing restrictions such as location and time) poses several challenges to the current state of Linked Data and relevant information extraction techniques. The query seeks “fresh” information (e.g., last four days) whereas the current version of Linked Data is encyclopedic and historical, and does not contain appropriate information present in a temporally annotated data stream. Further, the query specifies provenance (e.g., published by BBC and CNN) that might not always be available on Linked Data. Crucially, the example query asks about a specific type of event (i.e., reports of war caused killing people) with multiple arguments (e.g., in this case, location argument occurred in Yemen). In spite of recent progress BIBREF5 , BIBREF6 , BIBREF7 , there is still no standardized mechanism for (i) selecting background data model, (ii) recognizing and classifying specific event types, (iii) identifying and labeling associated arguments (i.e., entities as well as relations), (iv) interlinking events, and (v) representing events. In fact, most of the state-of-the-art solutions are ad hoc and limited. In this paper, we provide a systematic pipeline for developing knowledge graph of interlinked events. As a proof-of-concept, we show a case study of headline news on Twitter. The main contributions of this paper include:
Question: Which news organisations are the headlines sourced from?
Paper: Meta Relational Learning for Few-Shot Link Prediction in Knowledge Graphs
A knowledge graph is composed by a large amount of triples in the form of $(head\; entity,\, relation,\, tail\; entity)$ ( $(h, r, t)$ in short), encoding knowledge and facts in the world. Many KGs have been proposed BIBREF0 , BIBREF1 , BIBREF2 and applied to various applications BIBREF3 , BIBREF4 , BIBREF5 .
Question: What meta-information is being transferred?
Answer: high-order representation of a relation, loss gradient of relation meta
Paper: Meta Relational Learning for Few-Shot Link Prediction in Knowledge Graphs
One target of MetaR is to learn the representation of entities fitting the few-shot link prediction task and the learning framework is inspired by knowledge graph embedding methods. Furthermore, using loss gradient as one kind of meta information is inspired by MetaNet BIBREF12 and MAML BIBREF13 which explore methods for few-shot learning by meta-learning. From these two points, we regard knowledge graph embedding and meta-learning as two main kinds of related work.
Question: What datasets are used to evaluate the approach?
Answer: NELL-One, Wiki-One
Paper: SimplerVoice: A Key Message&Visual Description Generator System for Illiteracy
Illiteracy has been one of the most serious pervasive problems all over the world. According to the U. S. Department of Education, the National Center for Education Statistics, approximately 32 million adults in the United States are not able to read, which is about 14% of the entire adult population BIBREF0 . Additionally, 44% of the 2.4 million students in the U. S. federally funded adult education programs are English as a second language (ESL) students, and about 185,000 of them are at the lowest ESL level, beginning literacy BIBREF1 . While low-literate adults lack the ability to read and to understand text, particularly, the low-literate ESL adult learners also face the dual challenge of developing basic literacy skills which includes decoding, comprehending, and producing print, along with English proficiency, represent different nationalities and cultural backgrounds BIBREF2 . Hence, illiteracy is shown as a significant barrier that results in a person's struggling in every aspect of his or her daily life activity.
Question: Does their solution involve connecting images and text?
Paper: SimplerVoice: A Key Message&Visual Description Generator System for Illiteracy
In the field of ABE and SLA, researchers have conducted a number of studies to assist low-literate learners in their efforts to acquire literacy and language skills by reading interventions, and providing specific instructions through local education agencies, community colleges and educational organizations BIBREF3 , BIBREF1 .
Question: Which model do they use to generate key messages?
Paper: Modelling Semantic Categories using Conceptual Neighborhood
Vector space embeddings are commonly used to represent entities in fields such as machine learning (ML) BIBREF0, natural language processing (NLP) BIBREF1, information retrieval (IR) BIBREF2 and cognitive science BIBREF3. An important point, however, is that such representations usually represent both individuals and categories as vectors BIBREF4, BIBREF5, BIBREF6. Note that in this paper, we use the term category to denote natural groupings of individuals, as it is used in cognitive science, with individuals referring to the objects from the considered domain of discourse. For example, the individuals carrot and cucumber belong to the vegetable category. We use the term entities as an umbrella term covering both individuals and categories.
Question: What experiments they perform to demonstrate that their approach leads more accurate region based representations?
Paper: Modelling Semantic Categories using Conceptual Neighborhood
In distributional semantics, categories are frequently modelled as vectors. For example, BIBREF14 study the problem of deciding for a word pair $(i,c)$ whether $i$ denotes an instance of the category $c$, which they refer to as instantiation. They treat this problem as a binary classification problem, where e.g. the pair (AAAI, conference) would be a positive example, while (conference, AAAI) and (New York, conference) would be negative examples. Different from our setting, their aim is thus essentially to model the instantiation relation itself, similar in spirit to how hypernymy has been modelled in NLP BIBREF15, BIBREF16. To predict instantiation, they use a simple neural network model which takes as input the word vectors of the input pair $(i,c)$. They also experiment with an approach that instead models a given category as the average of the word vectors of its known instances and found that this led to better results.
Question: How they indentify conceptual neighbours?
Paper: The Transference Architecture for Automatic Post-Editing
The performance of state-of-the-art MT systems is not perfect, thus, human interventions are still required to correct machine translated texts into publishable quality translations BIBREF0. Automatic post-editing (APE) is a method that aims to automatically correct errors made by MT systems before performing actual human post-editing (PE) BIBREF1, thereby reducing the translators' workload and increasing productivity BIBREF2. APE systems trained on human PE data serve as MT post-processing modules to improve the overall performance. APE can therefore be viewed as a 2nd-stage MT system, translating predictable error patterns in MT output to their corresponding corrections. APE training data minimally involves MT output ($mt$) and the human post-edited ($pe$) version of $mt$, but additionally using the source ($src$) has been shown to provide further benefits BIBREF3, BIBREF4, BIBREF5.
Question: What experiment result led to conclussion that reducing the number of layers of the decoder does not matter much?
Paper: The Transference Architecture for Automatic Post-Editing
Recent advances in APE research are directed towards neural APE, which was first proposed by Pal:2016:ACL and junczysdowmunt-grundkiewicz:2016:WMT for the single-source APE scenario which does not consider $src$, i.e. $mt \rightarrow pe$. In their work, junczysdowmunt-grundkiewicz:2016:WMT also generated a large synthetic training dataset through back translation, which we also use as additional training data. Exploiting source information as an additional input can help neural APE to disambiguate corrections applied at each time step; this naturally leads to multi-source APE ($\lbrace src, mt\rbrace \rightarrow pe$). A multi-source neural APE system can be configured either by using a single encoder that encodes the concatenation of $src$ and $mt$ BIBREF9 or by using two separate encoders for $src$ and $mt$ and passing the concatenation of both encoders' final states to the decoder BIBREF10. A few approaches to multi-source neural APE were proposed in the WMT 2017 APE shared task. Junczysdowmunt:2017:WMT combine both $mt$ and $src$ in a single neural architecture, exploring different combinations of attention mechanisms including soft attention and hard monotonic attention. Chatterjee-EtAl:2017:WMT2 built upon the two-encoder architecture of multi-source models BIBREF10 by means of concatenating both weighted contexts of encoded $src$ and $mt$. Varis-bojar:2017:WMT compared two multi-source models, one using a single encoder with concatenation of $src$ and $mt$ sentences, and a second one using two character-level encoders for $mt$ and $src$ along with a character-level decoder.
Question: How much is performance hurt when using too small amount of layers in encoder?
Answer: comparing to the results from reducing the number of layers in the decoder, the BLEU score was 69.93 which is less than 1% in case of test2016 and in case of test2017 it was less by 0.2 %. In terms of TER it had higher score by 0.7 in case of test2016 and 0.1 in case of test2017.
Paper: The Transference Architecture for Automatic Post-Editing
We propose a multi-source transformer model called transference ($\lbrace src,mt\rbrace _{tr} \rightarrow pe$, Figure FIGREF1), which takes advantage of both the encodings of $src$ and $mt$ and attends over a combination of both sequences while generating the post-edited sentence. The second encoder, $enc_{src \rightarrow mt}$, makes use of the first encoder $enc_{src}$ and a sub-encoder $enc_{mt}$ for considering $src$ and $mt$. Here, the $enc_{src}$ encoder and the $dec_{pe}$ decoder are equivalent to the original transformer for neural MT. Our $enc_{src \rightarrow mt}$ follows an architecture similar to the transformer's decoder, the difference being that no masked multi-head self-attention is used to process $mt$.
Question: What was previous state of the art model for automatic post editing?
Paper: Emerging Language Spaces Learned From Massively Multilingual Corpora
Our primary goal is to learn meaning representations of sentences and sentence fragments by looking at the distributional information that is available in parallel corpora of human translations. The basic idea is to use translations into other languages as “semantic mirrors” of the original text, assuming that they represent the same meaning but with different symbols, wordings and linguistic structures. For this we discard any meaning diversions that may happen in translation due to target audience adaptation or other processes that may influence the semantics of the translated texts. We also assume that the material can be divided into meaningful and self-contained units, Bible verses in our case, and focus on the global data-driven model that hopefully can cope with instances that violate our assumptions.
Question: What neural machine translation models can learn in terms of transfer learning?
Answer: Multilingual Neural Machine Translation Models
Paper: An Annotation Scheme of A Large-scale Multi-party Dialogues Dataset for Discourse Parsing and Machine Comprehension
There are more and more NLP scholars focusing on the research of multi-party dialogues, such as multi-party dialogues discourse parsing and multi-party meeting summarization BIBREF0, BIBREF1, BIBREF2, BIBREF3, BIBREF4, BIBREF5. However, the scale of the STAC dataset has limited the research of discourse parsing for multi-party dialogues. On the other hand, as we know, there is no literature working on machine reading comprehension for multi-party dialogues. Considering the connection between the relevance between machine reading comprehension and discourse parsing, we annotate the dataset for two tasks for multi-party dialogues understanding.
Question: Did they experiment on the proposed task?
Paper: An Annotation Scheme of A Large-scale Multi-party Dialogues Dataset for Discourse Parsing and Machine Comprehension
Our dataset derives from the large scale multi-party dialogues dataset the Ubuntu Chat Corpus BIBREF6. The Ubuntu dataset is a large scale multi-party dialogues corpus.
Question: Is annotation done manually?
Paper: An Annotation Scheme of A Large-scale Multi-party Dialogues Dataset for Discourse Parsing and Machine Comprehension
This section will explain how to annotate discourse structure in multi-party dialogues.
Question: How large is the proposed dataset?
Paper: Unsupervised Learning of Style-sensitive Word Vectors
Analyzing and generating natural language texts requires the capturing of two important aspects of language: what is said and how it is said. In the literature, much more attention has been paid to studies on what is said. However, recently, capturing how it is said, such as stylistic variations, has also proven to be useful for natural language processing tasks such as classification, analysis, and generation BIBREF1 , BIBREF2 , BIBREF3 .
Question: How large is the dataset?
Paper: Unsupervised Learning of Style-sensitive Word Vectors
The key idea is to extend the continuous bag of words (CBOW) BIBREF0 by distinguishing nearby contexts and wider contexts under the assumption that a style persists throughout every single utterance in a dialog. We elaborate on it in this section.
Question: How is the dataset created?
Paper: Bayesian Sparsification of Recurrent Neural Networks
Recurrent neural networks (RNNs) are among the most powerful models for natural language processing, speech recognition, question-answering systems and other problems with sequential data BIBREF2 , BIBREF3 , BIBREF4 , BIBREF5 , BIBREF6 . For complex tasks such as machine translation BIBREF5 or speech recognition BIBREF3 modern RNN architectures incorporate a huge number of parameters. To use these models on portable devices with limited memory, for instance, smartphones, the model compression is desired. High compression level may also lead to an acceleration of RNNs. In addition, compression regularizes RNNs and helps to avoid overfitting.
Question: What is binary variational dropout?
Paper: Towards a Robust Deep Neural Network in Text Domain A Survey
Nowadays, DNNs have solved masses of significant practical problems in various areas like computer vision BIBREF0 , BIBREF1 , audio BIBREF2 , BIBREF3 , natural language processing (NLP) BIBREF4 , BIBREF5 etc. Due to the great success, systems based on DNN are widely deployed in physical world, including some sensitive security tasks. However, Szegedy et al. BIBREF6 found an interesting fact that a crafted input with small perturbations could easily fool DNN models. This kind of inputs is called adversarial examples. Certainly, with the development of theory and practice, the definitions of adversarial examples BIBREF6 , BIBREF7 , BIBREF8 , BIBREF9 are varied. But these definitions have two cores in common. One is that the perturbations are small and the ability of fooling DNN models is the other. It naturally raises a question why adversarial examples exist in DNNs. The reason why they are vulnerable to adversarial examples is probably because of DNNs’ linear nature. Goodfellow et al. BIBREF7 then gave this explanation after adversarial examples arose. Researchers therefore treat adversarial examples as a security problem and pay much attention to works of adversarial attacks and defenses BIBREF10 , BIBREF11 .
Question: Which strategies show the most promise in deterring these attacks?
Paper: Deep Contextualized Acoustic Representations For Semi-Supervised Speech Recognition
Current state-of-the-art models for speech recognition require vast amounts of transcribed audio data to attain good performance. In particular, end-to-end ASR models are more demanding in the amount of training data required when compared to traditional hybrid models. While obtaining a large amount of labeled data requires substantial effort and resources, it is much less costly to obtain abundant unlabeled data.
Question: What are baseline models on WSJ eval92 and LibriSpeech test-clean?
Paper: Deep Neural Machine Translation with Linear Associative Unit
Neural Machine Translation (NMT) is an end-to-end learning approach to machine translation which has recently shown promising results on multiple language pairs BIBREF0 , BIBREF1 , BIBREF2 , BIBREF3 , BIBREF4 , BIBREF5 , BIBREF6 , BIBREF7 . Unlike conventional Statistical Machine Translation (SMT) systems BIBREF8 , BIBREF9 , BIBREF10 , BIBREF11 , BIBREF12 which consist of multiple separately tuned components, NMT aims at building upon a single and large neural network to directly map input text to associated output text. Typical NMT models consists of two recurrent neural networks (RNNs), an encoder to read and encode the input text into a distributed representation and a decoder to generate translated text conditioned on the input representation BIBREF13 , BIBREF14 .
Question: Do they use the same architecture as LSTM-s and GRUs with just replacing with the LAU unit?
Paper: Graph Neural Networks with Generated Parameters for Relation Extraction
Recent years, graph neural networks (GNNs) have been applied to various fields of machine learning, including node classification BIBREF0 , relation classification BIBREF1 , molecular property prediction BIBREF2 , few-shot learning BIBREF3 , and achieve promising results on these tasks. These works have demonstrated GNNs' strong power to process relational reasoning on graphs.
Question: So this paper turns unstructured text inputs to parameters that GNNs can read?
Paper: On the emergence of syntactic structures: quantifying and modelling duality of patterning
In a seminal paper, Charles Hockett BIBREF0 identified duality of patterning as one of the core design features of human language. A language exhibits duality of patterning when it is organized at two distinct levels. At a first level, meaningless forms (typically referred to as phonemes) are combined into meaningful units (henceforth this property will be referred to as combinatoriality). For example, the English forms /k/, /a/, and /t/ are combined in different ways to obtain the three words /kat/, /akt/, and /tak/ (respectively written 'cat', 'act' and 'tack'). Because the individual forms in them are meaningless, these words have no relation in meaning in spite of being made of the same forms. This is a very important property, thanks to which all of the many words of the English lexicon can be obtained by relatively simple combinations of about forty phonemes. If phonemes had individual meaning, this degree of compactness would not be possible. At a second level, meaningful units (typically referred to as morphemes) are composed into larger units, the meaning of which is related to the individual meaning of the composing units (henceforth this property will be referred to as compositionality). For example, the meaning of the word 'boyfriend' is related to the meaning of the words 'boy' and 'friend' which composed it. The compositional level includes syntax as well. For example, the meaning of the sentence 'cats eat fishes' is related to the meaning of the words 'cats', 'eat', and 'fishes'. In this paper, for the sake of simplicity, we focus exclusively on the lexicon level. This has to be considered as a first step towards the comprehension of the emergence of complex structures in languages.
Question: What other models are compared to the Blending Game?
Paper: On the emergence of syntactic structures: quantifying and modelling duality of patterning
In this section we quantify the notion of duality of patterning as observed in real languages in order to provide suitable measures for the combinatoriality and compositionality.
Question: What empirical data are the Blending Game predictions compared to?
Paper: TArC: Incrementally and Semi-Automatically Collecting a Tunisian Arabish Corpus
Arabish is the romanization of Arabic Dialects (ADs) used for informal messaging, especially in social networks. This writing system provides an interesting ground for linguistic research, computational as well as sociolinguistic, mainly due to the fact that it is a spontaneous representation of the ADs, and because it is a linguistic phenomenon in constant expansion on the web. Despite such potential, little research has been dedicated to Tunisian Arabish (TA). In this paper we describe the work we carried to develop a flexible and multi-purpose TA resource. This will include a TA corpus, together with some tools that could be useful for analyzing the corpus and for its extension with new data.
Question: How does the semi-automatic construction process work?
Answer: Automatic transcription of 5000 tokens through sequential neural models trained on the annotated part of the corpus
Paper: TArC: Incrementally and Semi-Automatically Collecting a Tunisian Arabish Corpus
In this section, we provide an overview of work done on automatic processing of TUN and TA. As briefly outlined above, many studies on TUN and TA aim at solving the lack of standard orthography. The first Conventional Orthography for Dialectal Arabic (CODA) was for Egyptian Arabic BIBREF2 and it was used by bies2014transliteration for Egyptian Arabish transliteration into Arabic script. The CODA version for TUN (CODA TUN) was developed by DBLP:conf/lrec/ZribiBMEBH14, and was used in many studies, like boujelbane2015traitements. Such work presents a research on automatic word recognition in TUN. Narrowing down to the specific field of TA, CODA TUN was used in masmoudi2015arabic to realize a TA-Arabic script conversion tool, implemented with a rule-based approach. The most extensive CODA is CODA*, a unified set of guidelines for 28 Arab city dialects BIBREF0. For the present research, CODA* is considered the most convenient guideline to follow due to its extensive applicability, which will support comparative studies of corpora in different ADs. As we already mentioned, there are few NLP tools available for Arabish processing in comparison to the amount of NLP tools realized for Arabic. Considering the lack of spelling conventions for Arabish, previous effort has focused on automatic transliteration from Arabish to Arabic script, e.g. chalabi2012romanized, darwish2013arabizi, and al2014automatic. These three work are based on a character-to-character mapping model that aims at generating a range of alternative words that must then be selected through a linguistic model. A different method is presented in younes2018sequence, in which the authors present a sequence-to-sequence-based approach for TA-Arabic characters transliteration in both directions BIBREF3, BIBREF4.
Question: Does the paper report translation accuracy for an automatic translation model for Tunisian to Arabish words?