z-uo/qasper-squad huggingface dataset:
Paper: Speakers account for asymmetries in visual perspective so listeners don't have to
Our success as a social species depends on our ability to understand, and be understood by, different communicative partners across different contexts. Theory of mind—the ability to represent and reason about others' mental states—is considered to be the key mechanism that supports such context-sensitivity in our everyday social interactions. Being able to reason about what others see, want, and think allows us to make more accurate predictions about their future behavior in different contexts and adjust our own behaviors accordingly BIBREF0 . Over the past two decades, however, there has been sustained debate over the extent to which adults actually make of use theory of mind in communication.
Question: Did participants behave unexpectedly?
Paper: Speakers account for asymmetries in visual perspective so listeners don't have to
How does an unscripted speaker change her communicative behavior when there is uncertainty about exactly what her partner can see? To address this question empirically, we randomly assigned participants to the roles of speaker and listener and paired them over the web to play an interactive communication task BIBREF57 .
Question: Was this experiment done in a lab?
Paper: Domain Adaptation via Teacher-Student Learning for End-to-End Speech Recognition
Recently, with the advancement of deep learning, great progress has been made in end-to-end (E2E) automatic speech recognition (ASR). With the goal of directly mapping a sequence of speech frames to a sequence of output tokens, an E2E ASR system incorporates the acoustic model, language model and pronunciation model of a conventional ASR system into a single deep neural network (DNN). The most dominant approaches for E2E ASR include connectionist temporal classification (CTC) BIBREF0, BIBREF1, recurrent neural network transducer (RNNT) BIBREF2 and attention-based encoder-decoder (AED) models BIBREF3, BIBREF4, BIBREF5.
Question: How long is new model trained on 3400 hours of data?
Paper: HAS-QA: Hierarchical Answer Spans Model for Open-domain Question Answering
Open-domain question answering (OpenQA) aims to seek answers for a broad range of questions from a large knowledge sources, e.g., structured knowledge bases BIBREF0 , BIBREF1 and unstructured documents from search engine BIBREF2 . In this paper we focus on the OpenQA task with the unstructured knowledge sources retrieved by search engine.
Question: How much does HAS-QA improve over baselines?
Paper: Question Answering and Question Generation as Dual Tasks
Question answering (QA) and question generation (QG) are two fundamental tasks in natural language processing BIBREF0 , BIBREF1 . Both tasks involve reasoning between a question sequence $q$ and an answer sentence $a$ . In this work, we take answer sentence selection BIBREF2 as the QA task, which is a fundamental QA task and is very important for many applications such as search engine and conversational bots. The task of QA takes a question sentence $q$ and a list of candidate answer sentences as the input, and finds the top relevant answer sentence from the candidate list. The task of QG takes a sentence $a$ as input, and generates a question sentence $q$ which could be answered by $a$ .
Question: What does "explicitly leverages their probabilistic correlation to guide the training process of both models" mean?
Answer: The framework jointly learns parametrized QA and QG models subject to the constraint in equation 2. In more detail, they minimize QA and QG loss functions, with a third dual loss for regularization.
Paper: Multimodal Word Distributions
To model language, we must represent words. We can imagine representing every word with a binary one-hot vector corresponding to a dictionary position. But such a representation contains no valuable semantic information: distances between word vectors represent only differences in alphabetic ordering. Modern approaches, by contrast, learn to map words with similar meanings to nearby points in a vector space BIBREF0 , from large datasets such as Wikipedia. These learned word embeddings have become ubiquitous in predictive tasks.
Question: How does this compare to contextual embedding methods?
Paper: Task-driven Visual Saliency and Attention-based Visual Question Answering
Visual question answering (VQA) comes as a classic task which combines visual and textual modal data into a unified system. Taking an image and a natural language question about it as input, a VQA system is supposed to output the corresponding natural language answer. VQA problem requires image and text understanding, common sense and knowledge inference. The solution of VQA problem will be a great progress in approaching the goal of Visual Turing Test, and is also conducive to tasks such as multi-modal retrieval, image captioning and accessibility facilities.
Question: Does the new system utilize pre-extracted bounding boxes and/or features?
Paper: Task-driven Visual Saliency and Attention-based Visual Question Answering
Saliency generally comes from contrasts between a pixel or an object and its surroundings, describing how outstanding it is. It could facilitate learning by focusing the most pertinent regions. Saliency detection methods mimic the human attention in psychology, including both bottom-up and top-down manners BIBREF15 . Typical saliency methods BIBREF16 , BIBREF17 are pixel- or object-oriented, which are not appropriate for VQA due to center bias and are difficulty in collecting large scale eye tracking data.
Question: To which previous papers does this work compare its results?
Paper: KagNet: Knowledge-Aware Graph Networks for Commonsense Reasoning
Human beings are rational and a major component of rationality is the ability to reason. Reasoning is the process of combining facts and beliefs to make new decisions BIBREF0 , as well as the ability to manipulate knowledge to draw inferences BIBREF1 . Commonsense reasoning utilizes the basic knowledge that reflects our natural understanding of the world and human behaviors, which is common to all humans.
Question: Do they consider other tasks?
Paper: Localized Flood DetectionWith Minimal Labeled Social Media Data Using Transfer Learning
There are various forms of a natural disaster such as flood, earthquake, volcano eruptions, storms, etc. but the flood is one of the lethal and prominent forms of natural disaster according to World Meteorological Organization (WMO) for most of the countries. National Weather Services (NWS) reported 28,826 flash floods events in the United States from October 2007 to October 2015 which resulted in 278 live loss and million-dollar worth crop and property damage BIBREF0. Monitoring and detecting floods in advance and proactively working towards saving peoples live and minimizing damage at the same time is amongst one of the most important tasks nowadays. In recent times, humans are extremely active on social media such as Twitter, Facebook, Youtube, Flickr, Instagram, etc. People use these platform extensively to share crucial information via message, photos and videos in real-time on social media for their interaction and information dissemination on every topic and acts as an active human sensor. It has been observed in the past few years via several case studies that social media also contributes significantly and being used extensively for crisis-related feeds BIBREF1 and extremely helpful in situation awareness towards crisis management BIBREF2, BIBREF3, BIBREF4. Emergency first responders agency, humanitarian organizations, city authorities and other end users are always looking for the right amount and content that would be helpful in the crisis scenarios but generally, social media provides an overwhelming amount of unlabeled data and it is very crucial to filter out the right kind of information using text classification. The advances in Artificial Intelligence (AI) which includes machine learning and Natural Language Processing (NLP) methods can track and focus on humanitarian relief process and extract meaningful insights from the huge amount of social media data generated regularly in a timely manner.
Question: What were the model's results on flood detection?
Paper: Localized Flood DetectionWith Minimal Labeled Social Media Data Using Transfer Learning
Growing active user base on social media and has been created a great opportunity for extracting crucial information in real-time for various events and topics. Social media is being vigorously used as the communication channel in the time of any crisis or any natural disaster in order to convey the actionable information to the emergency responders to help them by more situational awareness context so that they make a better decision for rescue operations, sending alerts, reaching out people right on time. There have been numerous works proposed related to crisis management using social media content which is discussed in the following section.
Question: What dataset did they use?
Paper: Applications of Online Deep Learning for Crisis Response Using Social Media Information
Emergency events such as natural or man-made disasters bring unique challenges for humanitarian response organizations. Particularly, sudden-onset crisis situations demand officials to make fast decisions based on minimum information available to deploy rapid crisis response. However, information scarcity during time-critical situations hinders decision-making processes and delays response efforts BIBREF0 , BIBREF1 .
Question: What exactly is new about this stochastic gradient descent algorithm?
Paper: Small and Practical BERT Models for Sequence Labeling
We discuss two core models for addressing sequence labeling problems and describe, for each, training them in a single-model multilingual setting: (1) the Meta-LSTM BIBREF0 , an extremely strong baseline for our tasks, and (2) a multilingual BERT-based model BIBREF1 .
Question: What codemixed language pairs are evaluated?
Paper: Small and Practical BERT Models for Sequence Labeling
The Meta-LSTM is the best-performing model of the CoNLL 2018 Shared Task BIBREF2 for universal part-of-speech tagging and morphological features. The model is composed of 3 LSTMs: a character-BiLSTM, a word-BiLSTM and a single joint BiLSTM which takes the output of the character and word-BiLSTMs as input. The entire model structure is referred to as Meta-LSTM.
Question: How do they compress the model?
Paper: Small and Practical BERT Models for Sequence Labeling
BERT is a transformer-based model BIBREF3 pretrained with a masked-LM task on millions of words of text. In this paper our BERT-based experiments make use of the cased multilingual BERT model available on GitHub and pretrained on 104 languages.
Question: What is the multilingual baseline?
Paper: Dialogue Act Recognition via CRF-Attentive Structured Network
Dialogue Act Recognition (DAR) is an essential problem in modeling and detecting discourse structure. The goal of DAR is to attach semantic labels to each utterance in a conversation and recognize the speaker's intention, which can be regarded as a sequence labeling task. Many applications have benefited from the use of automatic dialogue act recognition such as dialogue systems, machine translation, automatic speech recognition, topic identification and talking avatars BIBREF0 BIBREF1 BIBREF2 . One of the primary applications of DAR is to support task-oriented discourse agent system. Knowing the past utterances of DA can help ease the prediction of the current DA state, thus help to narrow the range of utterance generation topics for the current turn. For instance, the "Greeting" and "Farewell" acts are often followed with another same type utterances, the "Answer" act often responds to the former "Question" type utterance. Thus if we can correctly recognize the current dialogue act, we can easily predict the following utterance act and generate a corresponding response. Table 1 shows a snippet of the kind of discourse structure in which we are interested.
Question: Which features do they use?
Paper: Dialogue Act Recognition via CRF-Attentive Structured Network
In this section, we study the problem of dialogue act recognition from the viewpoint of extending rich CRF-attentive structural dependencies. We first present the hierarchical semantic inference with memory mechanism from three levels: word level, utterance level and conversation level. We then develop graphical structured attention to the linear chain conditional random field to fully utilize the contextual dependencies.
Question: By how much do they outperform state-of-the-art solutions on SWDA and MRDA?
Paper: Named Entity Recognition on Twitter for Turkish using Semi-supervised Learning with Word Embeddings
Microblogging environments, which allow users to post short messages, have gained increased popularity in the last decade. Twitter, which is one of the most popular microblogging platforms, has become an interesting platform for exchanging ideas, following recent developments and trends, or discussing any possible topic. Since Twitter has an enormously wide range of users with varying interests and sharing preferences, a significant amount of content is being created rapidly. Therefore, mining such platforms can extract valuable information. As a consequence, extracting information from Twitter has become a hot topic of research. For Twitter text mining, one popular research area is opinion mining or sentiment analysis, which is surely useful for companies or political parties to gather information about their services and products BIBREF0 . Another popular research area is content analysis, or more specifically topic modeling, which is useful for text classification and filtering applications on Twitter BIBREF1 . Moreover, event monitoring and trend analysis are also other examples of useful application areas on microblog texts BIBREF2 .
Question: What type and size of word embeddings were used?
Paper: Named Entity Recognition on Twitter for Turkish using Semi-supervised Learning with Word Embeddings
There are various important studies of NER on Twitter for English. Ritter-2011 presented a two-phase NER system for tweets, T-NER, using Conditional Random Fields (CRF) and including tweet-specific features. Liu-2011 proposed a hybrid NER approach based on K-Nearest Neighbors and linear CRF. Liu-2012 presented a factor graph-based method for NER on Twitter. Li-2012 described an unsupervised approach for tweets, called TwiNER. Bontcheva-2013 described an NLP pipeline for tweets, called TwitIE. Very recently, Cherry-2015 have shown the effectiveness of Brown clusters and word vectors on Twitter NER for English.
Question: What data was used to build the word embeddings?
Paper: BiSET: Bi-directional Selective Encoding with Template for Abstractive Summarization
Abstractive summarization aims to shorten a source article or paragraph by rewriting while preserving the main idea. Due to the difficulties in rewriting long documents, a large body of research on this topic has focused on paragraph-level article summarization. Among them, sequence-to-sequence models have become the mainstream and some have achieved state-of-the-art performance BIBREF0 , BIBREF1 , BIBREF2 . In general, the only available information for these models during decoding is simply the source article representations from the encoder and the generated words from the previous time steps BIBREF2 , BIBREF3 , BIBREF4 , while the previous words are also generated based on the article representations. Since natural language text is complicated and verbose in nature, and training data is insufficient in size to help the models distinguish important article information from noise, sequence-to-sequence models tend to deteriorate with the accumulation of word generation, e.g., they generate irrelevant and repeated words frequently BIBREF5 .
Question: How are templates discovered from training data?
Paper: Efficiency through Auto-Sizing: Notre Dame NLP's Submission to the WNGT 2019 Efficiency Task
The Transformer network BIBREF3 is a neural sequence-to-sequence model that has achieved state-of-the-art results in machine translation. However, Transformer models tend to be very large, typically consisting of hundreds of millions of parameters. As the number of parameters directly corresponds to secondary storage requirements and memory consumption during inference, using Transformer networks may be prohibitively expensive in scenarios with constrained resources. For the 2019 Workshop on Neural Generation of Text (WNGT) Efficiency shared task BIBREF0, the Notre Dame Natural Language Processing (NDNLP) group looked at a method of inducing sparsity in parameters called auto-sizing in order to reduce the number of parameters in the Transformer at the cost of a relatively minimal drop in performance.
Question: What is WNGT 2019 shared task?
Answer: efficiency task aimed  at reducing the number of parameters while minimizing drop in performance
Paper: Neural Network Translation Models for Grammatical Error Correction
Grammatical error correction (GEC) is a challenging task due to the variability of the type of errors and the syntactic and semantic dependencies of the errors on the surrounding context. Most of the grammatical error correction systems use classification and rule-based approaches for correcting specific error types. However, these systems use several linguistic cues as features. The standard linguistic analysis tools like part-of-speech (POS) taggers and parsers are often trained on well-formed text and perform poorly on ungrammatical text. This introduces further errors and limits the performance of rule-based and classification approaches to GEC. As a consequence, the phrase-based statistical machine translation (SMT) approach to GEC has gained popularity because of its ability to learn text transformations from erroneous text to correct text from error-corrected parallel corpora without any additional linguistic information. They are also not limited to specific error types. Currently, many state-of-the-art GEC systems are based on SMT or use SMT components for error correction BIBREF0 , BIBREF1 , BIBREF2 . In this paper, grammatical error correction includes correcting errors of all types, including word choice errors and collocation errors which constitute a large class of learners' errors.
Question: Do they use pretrained word representations in their neural network models?
Paper: Neural Network Translation Models for Grammatical Error Correction
In the past decade, there has been increasing attention on grammatical error correction in English, mainly due to the growing number of English as Second Language (ESL) learners around the world. The popularity of this problem in natural language processing research grew further through Helping Our Own (HOO) and the CoNLL shared tasks BIBREF5 , BIBREF6 , BIBREF7 , BIBREF8 . Most published work in GEC aimed at building specific classifiers for different error types and then use them to build hybrid systems BIBREF9 , BIBREF10 . One of the first approaches of using SMT for GEC focused on correction of countability errors of mass nouns (e.g., many informations INLINEFORM0 much information) BIBREF11 . They had to use an artificially constructed parallel corpus for training their SMT system. Later, the availability of large-scale error corrected data BIBREF12 further improved SMT-based GEC systems.
Question: How do they combine the two proposed neural network models?
Paper: Neural Network Translation Models for Grammatical Error Correction
In this paper, the task of grammatical error correction is formulated as a translation task from the language of `bad' English to the language of `good' English. That is, the source sentence is written by a second language learner and potentially contains grammatical errors, whereas the target sentence is the corrected fluent sentence. We use a phrase-based machine translation framework BIBREF18 for translation, which employs a log-linear model to find the best translation INLINEFORM0 given a source sentence INLINEFORM1 . The best translation is selected according to the following equation: INLINEFORM2 
Question: Which dataset do they evaluate grammatical error correction on?
Paper: Context-aware Deep Model for Entity Recommendation in Search Engine at Alibaba
Over the past few years, major commercial search engines have enriched and improved the user experience by proactively presenting related entities for a query along with the regular web search results. Figure FIGREF3 shows an example of Alibaba ShenMa search engine's entity recommendation results presented on the panel of its mobile search result page.
Question: How many users/clicks does their search engine have?
Paper: Rapid Classification of Crisis-Related Data on Social Networks using Convolutional Neural Networks
Time-critical analysis of social media data streams is important for many application areas. For instance, responders to humanitarian disasters (e.g., earthquake, flood) need information about the disasters to determine what help is needed and where. This information usually breaks out on social media before other sources. During the onset of a crisis situation, rapid analysis of messages posted on microblogging platforms such as Twitter can help humanitarian organizations like the United Nations gain situational awareness, learn about urgent needs of affected people at different locations, and decide on actions accordingly BIBREF0 , BIBREF1 .
Question: what was their baseline comparison?
Paper: Information-Theoretic Probing for Linguistic Structure
Neural networks are the backbone of modern state-of-the-art Natural Language Processing (NLP) systems. One inherent by-product of training a neural network is the production of real-valued representations. Many speculate that these representations encode a continuous analogue of discrete linguistic properties, e.g., part-of-speech tags, due to the networks' impressive performance on many NLP tasks BIBREF0. As a result of this speculation, one common thread of research focuses on the construction of probes, i.e., supervised models that are trained to extract the linguistic properties directly BIBREF0, BIBREF1, BIBREF2, BIBREF3, BIBREF4, BIBREF5. A syntactic probe, then, is a model for extracting syntactic properties, such as part-of-speech, from the representations BIBREF6.
Question: Was any variation in results observed based on language typology?
Answer: It is observed some variability - but not significant. Bert does not seem to gain much more syntax information than with type level information.
Paper: Information-Theoretic Probing for Linguistic Structure
Following hewitt-liang-2019-designing, we consider probes that examine syntactic knowledge in contextualized embeddings. These probes only consider a single token's embedding and try to perform the task using only that information. Specifically, in this work, we consider part-of-speech (POS) labeling: determining a word's part of speech in a given sentence. For example, we wish to determine whether the word love is a noun or a verb. This task requires the sentential context for success. As an example, consider the utterance “love is blind” where, only with the context, is it clear that love is a noun. Thus, to do well on this task, the contextualized embeddings need to encode enough about the surrounding context to correctly guess the POS.
Question: Does the work explicitly study the relationship between model complexity and linguistic structure encoding?
Paper: Detecting and Extracting Events from Text Documents
Among the several senses that The Oxford English Dictionary, the most venerable dictionary of English, provides for the word event, are the following.
Question: Which datasets are used in this work?
Paper: A Sketch-Based System for Semantic Parsing
Open domain semantic parsing aims to map natural language utterances to structured meaning representations. Recently, seq2seq based approaches have achieved promising performance by structure-aware networks, such as sequence-to-actionBIBREF0 and STAMPBIBREF1.
Question: Does the training dataset provide logical form supervision?