z-uo/qasper-squad huggingface dataset:
Paper: A Parallel Corpus of Theses and Dissertations Abstracts
The availability of cross-language parallel corpora is one of the basis of current Statistical and Neural Machine Translation systems (e.g. SMT and NMT). Acquiring a high-quality parallel corpus that is large enough to train MT systems, specially NMT ones, is not a trivial task, since it usually demands human curating and correct alignment. In light of that, the automated creation of parallel corpora from freely available resources is extremely important in Natural Language Processing (NLP), enabling the development of accurate MT solutions. Many parallel corpora are already available, some with bilingual alignment, while others are multilingually aligned, with 3 or more languages, such as Europarl BIBREF0 , from the European Parliament, JRC-Acquis BIBREF1 , from the European Commission, OpenSubtitles BIBREF2 , from movies subtitles.
Question: What SMT models did they look at?
Paper: A Parallel Corpus of Theses and Dissertations Abstracts
In this section, we detail the information retrieved from CAPES website, the filtering process, the sentence alignment, and the evaluation experiments. An overview of the steps employed in this article is shown in Figure FIGREF1 .
Question: Which NMT models did they experiment with?
Paper: Casting a Wide Net: Robust Extraction of Potentially Idiomatic Expressions
Idiomatic expressions pose a major challenge for a wide range of applications in natural language processing BIBREF0. These include machine translation BIBREF1, BIBREF2, semantic parsing BIBREF3, sentiment analysis BIBREF4, and word sense disambiguation BIBREF5. Idioms show significant syntactic and morphological variability (e.g. beans being spilled for spill the beans), which makes them hard to find automatically. Moreover, their non-compositional nature makes idioms really hard to interpret, because their meaning is often very different from the meanings of the words that make them up. Hence, successful systems need not only be able to recognise idiomatic expressions in text or dialogue, but they also need to give a proper interpretation to them. As a matter of fact, current language technology performs badly on idiom understanding, a phenomenon that perhaps has not received enough attention.
Question: How big PIE datasets are obtained from dictionaries?
Paper: Casting a Wide Net: Robust Extraction of Potentially Idiomatic Expressions
The ambiguity of phrases like wake up and smell the coffee poses a terminological problem. Usually, these phrases are called idiomatic expressions, which is suitable when they are used in an idiomatic sense, but not so much when they are used in a literal sense. Therefore, we propose a new term: potentially idiomatic expressions, or PIEs for short. The term potentially idiomatic expression refers to those expressions which can have an idiomatic meaning, regardless of whether they actually have that meaning in a given context. So, see the light is a PIE in both `After another explanation, I finally saw the light' and `I saw the light of the sun through the trees', while it is an idiomatic expression in the first context, and a literal phrase in the latter context.
Question: What compleentary PIE extraction methods are used to increase reliability further?
Paper: Casting a Wide Net: Robust Extraction of Potentially Idiomatic Expressions
This section is structured so as to reflect the dual contribution of the present work. First, we discuss existing resources annotated for idiomatic expressions. Second, we discuss existing approaches to the automatic extraction of idioms.
Question: Are PIEs extracted automatically subjected to human evaluation?
Paper: Casting a Wide Net: Robust Extraction of Potentially Idiomatic Expressions
There are four sizeable sense-annotated PIE corpora for English: the VNC-Tokens Dataset BIBREF9, the Gigaword dataset BIBREF14, the IDIX Corpus BIBREF10, and the SemEval-2013 Task 5 dataset BIBREF15. An overview of these corpora is presented in Table TABREF7.
Question: What dictionaries are used for automatic extraction of PIEs?
Paper: Pivot-based Transfer Learning for Neural Machine Translation between Non-English Languages
Machine translation (MT) research is biased towards language pairs including English due to the ease of collecting parallel corpora. Translation between non-English languages, e.g., French$\rightarrow $German, is usually done with pivoting through English, i.e., translating French (source) input to English (pivot) first with a French$\rightarrow $English model which is later translated to German (target) with a English$\rightarrow $German model BIBREF0, BIBREF1, BIBREF2. However, pivoting requires doubled decoding time and the translation errors are propagated or expanded via the two-step process.
Question: Are experiments performed with any other pair of languages, how did proposed method perform compared to other models?
Paper: Pivot-based Transfer Learning for Neural Machine Translation between Non-English Languages
In this section, we first review existing approaches to leverage a pivot language in low-resource/zero-resource MT. They can be divided into three categories:
Question: Is pivot language used in experiments English or some other language?
Paper: Pivot-based Transfer Learning for Neural Machine Translation between Non-English Languages
Our methods are based on a simple transfer learning principle for NMT, adjusted to a usual data condition for non-English language pairs: lots of source-pivot and pivot-target parallel data, little (low-resource) or no (zero-resource) source-target parallel data. Here are the core steps of the plain transfer (Figure FIGREF10):
Question: What are multilingual models that were outperformed in performed experiment?
Paper: Image Captioning: Transforming Objects into Words
Image captioning—the task of providing a natural language description of the content within an image—lies at the intersection of computer vision and natural language processing. As both of these research areas are highly active and have experienced many recent advances, the progress in image captioning has naturally followed suit. On the computer vision side, improved convolutional neural network and object detection architectures have contributed to improved image captioning systems. On the natural language processing side, more sophisticated sequential models, such as attention-based recurrent neural networks, have similarly resulted in more accurate image caption generation.
Question: What are the common captioning metrics?
Paper: Semi-Supervised Methods for Out-of-Domain Dependency Parsing
[display] 1
Question: Which English domains do they evaluate on?
Paper: Rethinking Exposure Bias In Language Modeling
Likelihood-based language models with deep neural networks have been widely adopted to tackle language tasks BIBREF0, BIBREF1, BIBREF2, BIBREF3. By far, one of the most popular training strategies is teacher forcing, which derives from the general maximum likelihood estimation (MLE) principle BIBREF4. Under the teacher forcing schema, a model is trained to make predictions conditioned on ground-truth inputs. Although this strategy enables effective training of large neural networks, it is susceptible to aggravate exposure bias: a model may perform poorly at the inference stage, once its self-generated prefix diverges from the previously learned ground-truth data BIBREF5.
Question: What is the road exam metric?
Paper: Rethinking Exposure Bias In Language Modeling
As an early work to address exposure bias, BIBREF5 proposed a curriculum learning approach called scheduled sampling, which gradually replaces the ground-truth tokens with the model's own predictions while training. Later, BIBREF9 criticized this approach for pushing the model towards overfitting onto the corpus distribution based on the position of each token in the sequence, instead of learning about the prefix.
Question: What are the competing models?
Answer: TEACHER FORCING (TF), SCHEDULED SAMPLING (SS),  SEQGAN, RANKGAN, LEAKGAN.
Paper: Zero-Shot Relation Extraction via Reading Comprehension
Relation extraction systems populate knowledge bases with facts from an unstructured text corpus. When the type of facts (relations) are predefined, one can use crowdsourcing BIBREF0 or distant supervision BIBREF1 to collect examples and train an extraction model for each relation type. However, these approaches are incapable of extracting relations that were not specified in advance and observed during training. In this paper, we propose an alternative approach for relation extraction, which can potentially extract facts of new types that were neither specified nor observed a priori.
Question: How is the input triple translated to a slot-filling task?
Answer: The relation R(x,y) is mapped onto a question q whose answer is y
Paper: Multi-scale Octave Convolutions for Robust Speech Recognition
Deep convolutional neural networks (CNNs) with 2D convolutions and small kernels BIBREF1, have achieved state-of-the-art results for several speech recognition tasks BIBREF2, BIBREF3, BIBREF4, BIBREF5, BIBREF6. The accuracy of those models grows with their complexity, leading to redundant latent representations. Several approaches have been proposed in the literature to reduce this redundancy BIBREF7, BIBREF8, BIBREF9, BIBREF10, BIBREF11, and therefore to improve their efficiency.
Question: Is model compared against state of the art models on these datasets?
Paper: Multi-scale Octave Convolutions for Robust Speech Recognition
An octave convolutional layer BIBREF0 factorizes the output feature maps of a convolutional layer into two groups. The resolution of the low-frequency feature maps is reduced by an octave – height and width dimensions are divided by 2. In this work, we explore spatial reduction by up to 3 octaves – dividing by $2^t$, where $t=1,2,3$ – and for up to 4 groups. We refer to such a layer as a multi-octave convolutional (MultiOctConv) layer, and an example with three groups and reductions of one and two octaves is depicted in Fig. FIGREF1.
Question: How is octave convolution concept extended to multiple resolutions and octaves?
Paper: A Label Semantics Approach to Linguistic Hedges
The modelling of natural language relies on the idea that languages are compositional, i.e. that the meaning of a sentence is a function of the meanings of the words in the sentence, as proposed by BIBREF0 . Whether or not this principle tells the whole story, it is certainly important as we undoubtedly manage to create and understand novel combinations of words. Fuzzy set theory has long been considered a useful framework for the modelling of natural language expressions, as it provides a functional calculus for concept combination BIBREF1 , BIBREF2 .
Question: Does this paper address the variation among English dialects regarding these hedges?
Paper: Behavior Gated Language Models
Recurrent neural network language models (RNNLM) can theoretically model the word history over an arbitrarily long length of time and thus have been shown to perform better than traditional n-gram models BIBREF0. Recent prior work has continuously improved the performance of RNNLMs through hyper-parameter tuning, training optimization methods, and development of new network architectures BIBREF1, BIBREF2, BIBREF3, BIBREF4.
Question: On which dataset is model trained?
Paper: Behavior Gated Language Models
In this section, we first describe a typical RNN based language model which serves as a baseline for this study. Second, we introduce the proposed behavior prediction model for extracting behavioral information. Finally, the proposed architecture of the language model which incorporates the behavioral information through a gating mechanism is presented.
Question: How is module that analyzes behavioral state trained?
Answer: pre-trained to identify the presence of behavior from a sequence of word using the Couples Therapy Corpus
Paper: Open-World Knowledge Graph Completion
Knowledge Graphs (KGs) are a special type of information network that represents knowledge using RDF-style triples $\langle h$ , $r$ , $t\rangle $ , where $h$ represents some head entity and $r$ represents some relationship that connects $h$ to some tail entity $t$ . In this formalism a statement like “Springfield is the capital of Illinois” can be represented as $\langle $ Springfield, capitalOf, Illinois $\rangle $ . Recently, a variety of KGs, such as DBPedia BIBREF0 , and ConceptNet BIBREF1 , have been curated in the service of fact checking BIBREF2 , question answering BIBREF3 , entity linking BIBREF4 , and for many other tasks BIBREF5 . Despite their usefulness and popularity, KGs are often noisy and incomplete. For example, DBPedia, which is generated from Wikipedia's infoboxes, contains $4.6$ million entities, but half of these entities contain less than 5 relationships.
Question: Can the model add new relations to the knowledge graph, or just new entities?
Answer: The model does not add new relations to the knowledge graph.
Paper: The STEM-ECR Dataset: Grounding Scientific Entity References in STEM Scholarly Content to Authoritative Encyclopedic and Lexicographic Sources
1.1em
Question: How large is the dataset?
Paper: Using Gaussian Processes for Rumour Stance Classification in Social Media
There is an increasing need to interpret and act upon rumours spreading quickly through social media during breaking news, where new reports are released piecemeal and often have an unverified status at the time of posting. Previous research has posited the damage that the diffusion of false rumours can cause in society, and that corrections issued by news organisations or state agencies such as the police may not necessarily achieve the desired effect sufficiently quickly BIBREF0 , BIBREF1 . Being able to determine the accuracy of reports is therefore crucial in these scenarios. However, the veracity of rumours in circulation is usually hard to establish BIBREF2 , since as many views and testimonies as possible need to be assembled and examined in order to reach a final judgement. Examples of rumours that were later disproven, after being widely circulated, include a 2010 earthquake in Chile, where rumours of a volcano eruption and a tsunami warning in Valparaiso spawned on Twitter BIBREF3 . Another example is the England riots in 2011, where false rumours claimed that rioters were going to attack Birmingham's Children's Hospital and that animals had escaped from London Zoo BIBREF4 .
Question: Why is a Gaussian process an especially appropriate method for this classification problem?
Paper: Topic Spotting using Hierarchical Networks with Self Attention
0pt1ex1ex
Question: Do the authors do manual evaluation?
Paper: Learning Personalized End-to-End Goal-Oriented Dialog
There has been growing research interest in training dialog systems with end-to-end models BIBREF0 , BIBREF1 , BIBREF2 in recent years. These models are directly trained on past dialogs, without assumptions on the domain or dialog state structure BIBREF3 . One of their limitations is that they select responses only according to the content of the conversation and are thus incapable of adapting to users with different personalities. Specifically, common issues with such content-based models include: (i) the inability to adjust language style flexibly BIBREF4 ; (ii) the lack of a dynamic conversation policy based on the interlocutor's profile BIBREF5 ; and (iii) the incapability of handling ambiguities in user requests.
Question: What datasets did they use?
Paper: Determining the Scale of Impact from Denial-of-Service Attacks in Real Time Using Twitter
Denial of Service attacks are explicit attempts to stop legitimate users from accessing specific network systems BIBREF0. Attackers try to exhaust network resources like bandwidth, or server resources like CPU and memory. As a result, the targeted system slows down or becomes unusable BIBREF1. On-line service providers like Bank Of America, Facebook and Reddit are often the target of such attacks and the frequency and scale of those attacks has increased rapidly in recent years BIBREF2.
Question: Do twitter users tend to tweet about the DOS attack when it occurs? How much data supports this assumption?
Answer: The dataset contains about 590 tweets about DDos attacks.
Paper: Determining the Scale of Impact from Denial-of-Service Attacks in Real Time Using Twitter
Denial of Service (DoS) attacks are a major threat to Internet security, and detecting them has been a core task of the security community for more than a decade. There exists significant amount of prior work in this domain. BIBREF9, BIBREF10, BIBREF11 all introduced different methods to tackle this problem. The major difference between this work and previous ones are that instead of working on the data of the network itself, we use the reactions of users on social networks to identify an intrusion.
Question: What is the training and test data used?
Answer: Tweets related to a Bank of America DDos attack were used as training data. The test datasets contain tweets related to attacks to Bank of America, PNC and Wells Fargo.
Paper: Determining the Scale of Impact from Denial-of-Service Attacks in Real Time Using Twitter
Figure FIGREF4 outlines the entire pipeline of the model from preprocessing tweets to modeling them and finally detecting / ranking future tweets that are related to a DoS issue and measuring its severity.
Question: Was performance of the weakly-supervised model compared to the performance of a supervised model?
Paper: #MeTooMA: Multi-Aspect Annotations of Tweets Related to the MeToo Movement
Over the last couple of years, the MeToo movement has facilitated several discussions about sexual abuse. Social media, especially Twitter, was one of the leading platforms where people shared their experiences of sexual harassment, expressed their opinions, and also offered support to victims. A large portion of these tweets was tagged with a dedicated hashtag #MeToo, and it was one of the main trending topics in many countries. The movement was viral on social media and the hashtag used over 19 million times in a year.
Question: Do the tweets come from a specific region?
Paper: Introducing RONEC -- the Romanian Named Entity Corpus
Language resources are an essential component in entire R&D domains. From the humble but vast repositories of monolingual texts that are used by the newest language modeling approaches like BERT and GPT, to parallel corpora that allows our machine translation systems to inch closer to human performance, to the more specialized resources like WordNets that encode semantic relations between nodes, these resources are necessary for the general advancement of Natural Language Processing, which eventually evolves into real apps and services we are (already) taking for granted.
Question: Did they experiment with the corpus?
Paper: Introducing RONEC -- the Romanian Named Entity Corpus
We note that, while fragmentary, there are a few related language resources available, but none that specifically target named entities:
Question: What writing styles are present in the corpus?
Answer: current news, historical news, free time, sports, juridical news pieces, personal adverts, editorials.
Paper: Introducing RONEC -- the Romanian Named Entity Corpus
ROCO BIBREF4 is a Romanian journalistic corpus that contains approx. 7.1M tokens. It is rich in proper names, numerals and named entities. The corpus has been automatically annotated at word-level with morphosyntactic information (MSD annotations).
Question: How did they determine the distinct classes?
Paper: A General-Purpose Tagger with Convolutional Neural Networks
Recently, character composition models have shown great success in many NLP tasks, mainly because of their robustness in dealing with out-of-vocabulary (OOV) words by capturing sub-word informations. Among the character composition models, bidirectional long short-term memory (LSTM) models and convolutional neural networks (CNN) are widely applied in many tasks, e.g. part-of-speech (POS) tagging BIBREF0 , BIBREF1 , named entity recognition BIBREF2 , language modeling BIBREF3 , BIBREF4 , machine translation BIBREF5 and dependency parsing BIBREF6 , BIBREF7 .
Question: Do they jointly tackle multiple tagging problems?
Paper: A General-Purpose Tagger with Convolutional Neural Networks
Our proposed CNN tagger has two main components: the character composition model and the context encoding model. Both components are essentially CNN models, capturing different levels of information: the first CNN captures morphological information from character n-grams, the second one captures contextual information from word n-grams. Figure FIGREF2 shows a diagram of both models of the tagger.
Question: How many parameters does their CNN have?
Paper: A General-Purpose Tagger with Convolutional Neural Networks
The character composition model is similar to Yu:2017, where several convolution filters are used to capture character n-grams of different sizes. The outputs of each convolution filter are fed through a max pooling layer, and the pooling outputs are concatenated to represent the word.
Question: How do they confirm their model working well on out-of-vocabulary problems?
Paper: Towards Machine Comprehension of Spoken Content: Initial TOEFL Listening Comprehension Test by Machine
With the popularity of shared videos, social networks, online course, etc, the quantity of multimedia or spoken content is growing much faster beyond what human beings can view or listen to. Accessing large collections of multimedia or spoken content is difficult and time-consuming for humans, even if these materials are more attractive for humans than plain text information. Hence, it will be great if the machine can automatically listen to and understand the spoken content, and even visualize the key information for humans. This paper presents an initial attempt towards the above goal: machine comprehension of spoken content. In an initial task, we wish the machine can listen to and understand an audio story, and answer the questions related to that audio content. TOEFL listening comprehension test is for human English learners whose native language is not English. This paper reports how today's machine can perform with such a test.
Question: What approach does this work propose for the new task?
Paper: Towards Machine Comprehension of Spoken Content: Initial TOEFL Listening Comprehension Test by Machine
In this paper, we develop and propose a new task of machine comprehension of spoken content which was never mentioned before to our knowledge. We take TOEFL listening comprehension test as an corpus for this work. TOEFL is an English examination which tests the knowledge and skills of academic English for English learners whose native languages is not English. In this examination, the subjects would first listen to an audio story around five minutes and then answer several question according to that story. The story is related to the college life such as conversation between the student and the professor or a lecture in the class. Each question has four choices where only one is correct. An real example in the TOEFL examination is shown in Fig. 1 . The upper part is the manual transcription of a small part of the audio story. The questions and four choices are listed too. The correct choice to the question in Fig. 1 is choice A. The questions in TOEFL are not simple even for a human with relatively good knowledge because the question cannot be answered by simply matching the words in the question and in the choices with those in the story, and key information is usually buried by many irrelevant utterances. To answer the questions like “Why does the student go to professor's office?", the listeners have to understand the whole audio story and draw the inferences to answer the question correctly. As a result, this task is believed to be very challenging for the state-of-the-art spoken language understanding technologies.
Question: What is the new task proposed in this work?