z-uo/qasper-squad huggingface dataset:
Paper: Canonicalizing Knowledge Base Literals
Ontology-based knowledge bases (KBs) like DBpedia BIBREF0 are playing an increasingly important role in domains such knowledge management, data analysis and natural language understanding. Although they are very valuable resources, the usefulness and usability of such KBs is limited by various quality issues BIBREF1 , BIBREF2 , BIBREF3 . One such issue is the use of string literals (both explicitly typed and plain literals) instead of semantically typed entities; for example in the triple $\langle $ River_Thames, passesArea, “Port Meadow, Oxford" $\rangle $ . This weakens the KB as it does not capture the semantics of such literals. If, in contrast, the object of the triple were an entity, then this entity could, e.g., be typed as Wetland and Park, and its location given as Oxford. This problem is pervasive and hence results in a significant loss of information: according to statistics from Gunaratna et al. BIBREF4 in 2016, the DBpedia property dbp:location has over 105,000 unique string literals that could be matched with entities. Besides DBpedia, such literals can also be found in some other KBs from encyclopedias (e.g., zhishi.me BIBREF5 ), in RDF graphs transformed from tabular data (e.g., LinkedGeoData BIBREF6 ), in aligned or evolving KBs, etc. 
Question: What is the reasoning method that is used?
Paper: Canonicalizing Knowledge Base Literals
In this study we consider a knowledge base (KB) that includes both ontological axioms that induce (at least) a hierarchy of semantic types (i.e., classes), and assertions that describe concrete entities (individuals). Each such assertion is assumed to be in the form of an RDF triple $\langle s,p,o \rangle $ , where $s$ is an entity, $p$ is a property and $o$ can be either an entity or a literal (i.e., a typed or untyped data value such as a string or integer).
Question: What KB is used in this work?
Paper: Canonicalizing Knowledge Base Literals
The technical framework for the classification problem is shown in Fig. 1 . It involves three main steps: (i) candidate class extraction; (ii) model training and prediction; and (iii) literal typing and canonicalization.
Question: What's the precision of the system?
Answer: 0.8320 on semantic typing, 0.7194 on entity matching
Paper: Towards Personalized Dialog Policies for Conversational Skill Discovery
Modern speech-based assistants, such as Amazon Alexa, Google Home, Microsoft Cortana, and Apple Siri, enable users to complete daily tasks such as shopping, setting reminders, and playing games using voice commands. Such human-like interfaces create a rich experience for users by enabling them to complete many tasks hands- and eyes-free in a conversational manner. Furthermore, these services offer tools to enable developers and customers to create custom voice experiences (skills) and as a result extend the capabilities of the assistant. Amazon's Alexa Skills Kit BIBREF0, Google's Actions and Microsoft's Cortana Skills Kit are examples of such tools. As the number of skills (with potentially overlapping functionality) increases, it becomes more difficult for end users to find the skills that can address their request.
Question: How did they measure effectiveness?
Paper: BERT is Not a Knowledge Base (Yet): Factual Knowledge vs. Name-Based Reasoning in Unsupervised QA
Imagine that you have a friend who claims to know a lot of trivia. During a quiz, you ask them about the native language of actor Jean Marais. They correctly answer French. For a moment you are impressed, until you realize that Jean is a typical French name. So you ask the same question about Daniel Ceccaldi (another French actor, but with an Italian-sounding name). This time your friend says “Italian, I guess.” If this were a Question Answering (QA) benchmark, your friend would have achieved a respectable accuracy of 50%. Yet, their performance does not indicate factual knowledge about the native languages of actors. Rather, it shows that they are able to reason about the likely origins of peoples' names (see Table TABREF1 for more examples).
Question: Which of the two ensembles yields the best performance?
Answer: Answer with content missing: (Table 2) CONCAT ensemble
Paper: BERT is Not a Knowledge Base (Yet): Factual Knowledge vs. Name-Based Reasoning in Unsupervised QA
The LAMA (LAnguage Model Analysis) benchmark BIBREF1 is supposed to probe for “factual and commonsense knowledge” inherent in LMs. In this paper, we focus on LAMA-Google-RE and LAMA-T-REx BIBREF5, which are aimed at factual knowledge. Contrary to most previous works on QA, LAMA tests LMs as-is, without supervised finetuning.
Question: What are the two ways of ensembling BERT and E-BERT?
Paper: BERT is Not a Knowledge Base (Yet): Factual Knowledge vs. Name-Based Reasoning in Unsupervised QA
It is often possible to guess properties of an entity from its name, with zero factual knowledge of the entity itself. This is because entities are often named according to implicit or explicit rules (e.g., the cultural norms involved in naming a child, copyright laws for industrial products, or simply a practical need for descriptive names). LAMA makes guessing even easier by its limited vocabulary, which may only contain a few candidates for a particular entity type.
Question: How is it determined that a fact is easy-to-guess?
Paper: Concurrent Parsing of Constituency and Dependency
Constituent and dependency are two typical syntactic structure representation forms as shown in Figure FIGREF1, which have been well studied from both linguistic and computational perspective BIBREF0, BIBREF1. In earlier time, linguists and NLP researchers discussed how to encode lexical dependencies in phrase structures, like Tree-adjoining grammar (TAG) BIBREF2 and head-driven phrase structure grammar (HPSG) BIBREF3.
Question: How is dependency parsing empirically verified?
Paper: Concurrent Parsing of Constituency and Dependency
Using an encoder-decoder backbone, our model may be regarded as an extension of the constituent parsing model of BIBREF18 as shown in Figure FIGREF4. The difference is that in our model both constituent and dependency parsing share the same token representation and shared self-attention layers and each has its own individual Self-Attention Layers and subsequent processing layers. Our model includes four modules: token representation, self-attention encoder, constituent and dependency parsing decoder.
Question: How are different network components evaluated?
Paper: Concurrent Parsing of Constituency and Dependency
In our model, token representation $x_i$ is composed by character, word and part-of-speech (POS) embeddings. For character-level representation, we explore two types of encoders, CharCNNs BIBREF19, BIBREF20 and CharLSTM BIBREF18, as both types have been verified their effectiveness. For word-level representation, we concatenate randomly initialized and pre-trained word embeddings. We consider two ways to compose the final token representation, summing and concatenation, $x_i$=$x_{char}$+$x_{word}$+$x_{POS}$, $x_i$=[$x_{char}$;$x_{word}$;$x_{POS}$].
Question: What are the performances obtained for PTB and CTB?
Paper: Concurrent Parsing of Constituency and Dependency
The encoder in our model is adapted from BIBREF21 to factor explicit content and position information in the self-attention process BIBREF18. The input matrices $X = [x_1, x_2, \dots , x_n ]$ in which $x_i$ is concatenated with position embedding are transformed by a self-attention encoder. We factor the model between content and position information both in self-attention sub-layer and feed-forward network, whose setting details follow BIBREF18. We also try different numbers of shared self-attention layers in section SECREF15.
Question: What are the models used to perform constituency and dependency parsing?
Paper: Improving Deep Transformer with Depth-Scaled Initialization and Merged Attention
The capability of deep neural models of handling complex dependencies has benefited various artificial intelligence tasks, such as image recognition where test error was reduced by scaling VGG nets BIBREF0 up to hundreds of convolutional layers BIBREF1. In NLP, deep self-attention networks have enabled large-scale pretrained language models such as BERT BIBREF2 and GPT BIBREF3 to boost state-of-the-art (SOTA) performance on downstream applications. By contrast, though neural machine translation (NMT) gained encouraging improvement when shifting from a shallow architecture BIBREF4 to deeper ones BIBREF5, BIBREF6, BIBREF7, BIBREF8, the Transformer BIBREF9, a currently SOTA architecture, achieves best results with merely 6 encoder and decoder layers, and no gains were reported by BIBREF9 from further increasing its depth on standard datasets.
Question: Is the proposed layer smaller in parameters than a Transformer?
Paper: Named Entity Disambiguation for Noisy Text
Named Entity Disambiguation (NED) is the task of linking mentions of entities in text to a given knowledge base, such as Freebase or Wikipedia. NED is a key component in Entity Linking (EL) systems, focusing on the disambiguation task itself, independently from the tasks of Named Entity Recognition (detecting mention bounds) and Candidate Generation (retrieving the set of potential candidate entities). NED has been recognized as an important component in NLP tasks such as semantic parsing BIBREF0 .
Question: What is the new initialization method proposed in this paper?
Answer: They initialize their word and entity embeddings with vectors pre-trained over a large corpus of unlabeled data.
Paper: Named Entity Disambiguation for Noisy Text
We introduce WikilinksNED, a large-scale NED dataset based on text fragments from the web. Our dataset is derived from the Wikilinks corpus BIBREF14 , which was constructed by crawling the web and collecting hyperlinks (mentions) linking to Wikipedia concepts (entities) and their surrounding text (context). Wikilinks contains 40 million mentions covering 3 million entities, collected from over 10 million web pages.
Question: How was a quality control performed so that the text is noisy but the annotations are accurate?
Answer: The authors believe that the Wikilinks corpus  contains ground truth annotations while being noisy. They discard mentions that cannot have ground-truth verified by comparison with Wikipedia.
Paper: Question Asking as Program Generation
In active machine learning, a learner is able to query an oracle in order to obtain information that is expected to improve performance. Theoretical and empirical results show that active learning can speed acquisition for a variety of learning tasks BIBREF0 . Although impressive, most work on active machine learning has focused on relatively simple types of information requests (most often a request for a supervised label). In contrast, humans often learn by asking far richer questions which more directly target the critical parameters in a learning task. A human child might ask “Do all dogs have long tails?" or “What is the difference between cats and dogs?" BIBREF1 . A long term goal of artificial intelligence (AI) is to develop algorithms with a similar capacity to learn by asking rich questions. Our premise is that we can make progress toward this goal by better understanding human question asking abilities in computational terms BIBREF2 .
Question: Is it a neural model? How is it trained?
Answer: No, it is a probabilistic model trained by finding feature weights through gradient ascent
Paper: Event detection in Colombian security Twitter news using fine-grained latent topic analysis
Twitter is a social network that has been used worldwide as a means of news spreading. In fact, more than 85% of its users use Twitter to be updated with news, and do so on a daily basis BIBREF0. Users behaviour of this social network has been found efficient in electronic word-of-mouth processes BIBREF1, which is a key component for the quick spreading of breaking news. This would lead to think that news-related content occupies the majority of the tweets volume. However, on average, the proportion of news-related content to the total content of tweets is 1% worldwide, but increases dramatically (up to 15%) in countries in conflict BIBREF2. An extrapolation of these findings indicates that Colombia might have a high content of news-related tweets, since it is well-known that Colombia is one of the most violent countries in the world, and has been for decades BIBREF3.
Question: How do people engage in Twitter threads on different types of news?
Paper: Event detection in Colombian security Twitter news using fine-grained latent topic analysis
In this section, we describe the dataset used in our research, as well as the methods to perform fine-grained latent topic analysis to process all the data. The method is largely based on our previous work BIBREF17.
Question: How are the clusters related to security, violence and crime identified?
Paper: Customized Image Narrative Generation via Interactive Visual Question Generation and Answering
Recent advances in visual language field enabled by deep learning techniques have succeeded in bridging the gap between vision and language in a variety of tasks, ranging from describing the image BIBREF0 , BIBREF1 , BIBREF2 , BIBREF3 to answering questions about the image BIBREF4 , BIBREF5 . Such achievements were possible under the premise that there exists a set of ground truth references that are universally applicable regardless of the target, scope, or context. In real-world setting, however, image descriptions are prone to an infinitely wide range of variabilities, as different viewers may pay attention to different aspects of the image in different contexts, resulting in a variety of descriptions or interpretations. Due to its subjective nature, such diversity is difficult to obtain with conventional image description techniques.
Question: What are the features of used to customize target user interaction?
Paper: Ask to Learn: A Study on Curiosity-driven Question Generation
The growing interest in Machine Reading Comprehension (MRC) has sparked significant research efforts on Question Generation (QG), the dual task to Question Answering (QA). In QA, the objective is to produce an adequate response given a query and a text; conversely, for QG, the task is generally defined as generating relevant question given a source text, focusing on a specific answer span. To our knowledge, all works tackling QG have thus far focused exclusively on generating relevant questions which can be answered given the source text: for instance, given AAAI was founded in 1979 as input, a question likely to be automatically generated would be When was AAAI founded?, where the answer 1979 is a span of the input. Such questions are useful to evaluate reading comprehension for both machines BIBREF0, BIBREF1 and humans BIBREF2.
Question: How they evaluate quality of generated output?
Answer: Through human evaluation where they are asked to evaluate the generated output on a likert scale.
Paper: Ask to Learn: A Study on Curiosity-driven Question Generation
Deep learning models have been widely applied to text generation tasks such as machine translation BIBREF5, abstractive summarization BIBREF6 or dialog BIBREF7, providing significant gains in performance. The state of the art approaches are based on sequence to sequence models BIBREF8, BIBREF9. In recent years, significant research efforts have been directed to the tasks of Machine Reading Comprehension (MRC) and Question Answering (QA) BIBREF0, BIBREF10. The data used for tackling these tasks are usually composed of $\lbrace context, question, answer\rbrace $ triplets: given a context and the question, a model is trained to predict the answer.
Question: What automated metrics authors investigate?
Paper: Identifying Products in Online Cybercrime Marketplaces: A Dataset for Fine-grained Domain Adaptation
NLP can be extremely useful for enabling scientific inquiry, helping us to quickly and efficiently understand large corpora, gather evidence, and test hypotheses BIBREF0 , BIBREF1 . One domain for which automated analysis is particularly useful is Internet security: researchers obtain large amounts of text data pertinent to active threats or ongoing cybercriminal activity, for which the ability to rapidly characterize that text and draw conclusions can reap major benefits BIBREF2 , BIBREF3 . However, conducting automatic analysis is difficult because this data is out-of-domain for conventional NLP models, which harms the performance of both discrete models BIBREF4 and deep models BIBREF5 . Not only that, we show that data from one cybercrime forum is even out of domain with respect to another cybercrime forum, making this data especially challenging.
Question: What supervised models are experimented with?
Paper: Identifying Products in Online Cybercrime Marketplaces: A Dataset for Fine-grained Domain Adaptation
We consider several forums that vary in the nature of products being traded:
Question: Who annotated the data?
Paper: Identifying Products in Online Cybercrime Marketplaces: A Dataset for Fine-grained Domain Adaptation
We developed our annotation guidelines through six preliminary rounds of annotation, covering 560 posts. Each round was followed by discussion and resolution of every post with disagreements. We benefited from members of our team who brought extensive domain expertise to the task. As well as refining the annotation guidelines, the development process trained annotators who were not security experts. The data annotated during this process is not included in Table TABREF3 .
Question: What are the four forums the data comes from?
Answer: Darkode,  Hack Forums, Blackhat and Nulled.
Paper: Multi-Source Syntactic Neural Machine Translation
Neural machine translation (NMT) typically makes use of a recurrent neural network (RNN) -based encoder and decoder, along with an attention mechanism BIBREF0 , BIBREF1 , BIBREF2 , BIBREF3 . However, it has been shown that RNNs require some supervision to learn syntax BIBREF4 , BIBREF5 , BIBREF6 . Therefore, explicitly incorporating syntactic information into NMT has the potential to improve performance. This is particularly true for source syntax, which can improve the model's representation of the source language.
Question: How do they obtain parsed source sentences?
Paper: Multi-Source Syntactic Neural Machine Translation
Using linearized parse trees within sequential frameworks was first done in the context of neural parsing. vinyals2015grammar parsed using an attentional seq2seq model; they used linearized, unlexicalized parse trees on the target side and sentences on the source side. In addition, as in this work, they used an external parser to create synthetic parsed training data, resulting in improved parsing performance. choe2016parsing adopted a similar strategy, using linearized parses in an RNN language modeling framework.
Question: What kind of encoders are used for the parsed source sentence?
Paper: Multi-Source Syntactic Neural Machine Translation
Among the first proposals for using source syntax in NMT was that of luong2015multi, who introduced a multi-task system in which the source data was parsed and translated using a shared encoder and two decoders. More radical changes to the standard NMT paradigm have also been proposed. eriguchi2016tree introduced tree-to-sequence NMT; this model took parse trees as input using a tree-LSTM BIBREF10 encoder. bastings2017graph used a graph convolutional encoder in order to take labeled dependency parses of the source sentences into account. hashimoto2017neural added a latent graph parser to the encoder, allowing it to learn soft dependency parses while simultaneously learning to translate.
Question: Whas is the performance drop of their model when there is no parsed input?
Paper: Self-attention based end-to-end Hindi-English Neural Machine Translation
Machine Translation, which is a field of concentrate under common language preparing, focuses at deciphering normal language naturally utilizing machines. Information driven machine interpretation has turned into the overwhelming field of concentrate because of the availability of substantial parallel corpora. The primary target of information driven machine interpretation is to decipher concealed source language, given that the frameworks take in interpretation learning from sentence adjusted bi-lingual preparing information.
Question: How were their results compared to state-of-the-art?
Paper: Contextual Recurrent Units for Cloze-style Reading Comprehension
Neural network based approaches have become popular frameworks in many machine learning research fields, showing its advantages over traditional methods. In NLP tasks, two types of neural networks are widely used: Recurrent Neural Network (RNN) and Convolutional Neural Network (CNN).
Question: What supports the claim that injected CNN into recurent units will enhance ability of the model to catch local context and reduce ambiguities?
Paper: Contextual Recurrent Units for Cloze-style Reading Comprehension
Gated recurrent unit (GRU) has been proposed in the scenario of neural machine translations BIBREF0. It has been shown that the GRU has comparable performance in some tasks compared to the LSTM. Another advantage of GRU is that it has a simpler neural architecture than LSTM, showing a much efficient computation.
Question: How is CNN injected into recurent units?
Paper: Contextual Recurrent Units for Cloze-style Reading Comprehension
In this section, we will give a detailed introduction to our CRU model. Firstly, we will give a brief introduction to GRU BIBREF0 as preliminaries, and then three variants of our CRU model will be illustrated.
Question: Are there some results better than state of the art on these tasks?
Paper: Contextual Recurrent Units for Cloze-style Reading Comprehension
Gated Recurrent Unit (GRU) is a type of recurrent unit that models sequential data BIBREF0, which is similar to LSTM but is much simpler and computationally effective than the latter one. We will briefly introduce the formulation of GRU. Given a sequence $x = \lbrace x_1, x_2, ..., x_n\rbrace $, GRU will process the data in the following ways. For simplicity, the bias term is omitted in the following equations.
Question: Do experiment results show consistent significant improvement of new approach over traditional CNN and RNN models?
Paper: Contextual Recurrent Units for Cloze-style Reading Comprehension
By only modeling word-level representation may have drawbacks in representing the word that has different meanings when the context varies. Here is an example that shows this problem.
Question: What datasets are used for testing sentiment classification and reading comprehension?
Paper: Graph-based Filtering of Out-of-Vocabulary Words for Encoder-Decoder Models
Encoder-decoder models BIBREF0 are effective in tasks such as machine translation ( BIBREF1 , BIBREF1 ; BIBREF2 , BIBREF2 ) and grammatical error correction BIBREF3 . Vocabulary in encoder-decoder models is generally selected from the training corpus in descending order of frequency, and low-frequency words are replaced with an unknown word token <unk>. The so-called out-of-vocabulary (OOV) words are replaced with <unk> to not increase the decoder's complexity and to reduce noise. However, naive frequency-based OOV replacement may lead to loss of information that is necessary for modeling context in the encoder.
Question: So we do not use pre-trained embedding in this case?