z-uo/qasper-squad huggingface dataset:
Paper: "How May I Help You?": Modeling Twitter Customer Service Conversations Using Fine-Grained Dialogue Acts
The need for real-time, efficient, and reliable customer service has grown in recent years. Twitter has emerged as a popular medium for customer service dialogue, allowing customers to make inquiries and receive instant live support in the public domain. In order to provide useful information to customers, agents must first understand the requirements of the conversation, and offer customers the appropriate feedback. While this may be feasible at the level of a single conversation for a human agent, automatic analysis of conversations is essential for data-driven approaches towards the design of automated customer support agents and systems.
Question: Do they evaluate only on English datasets?
Paper: "How May I Help You?": Modeling Twitter Customer Service Conversations Using Fine-Grained Dialogue Acts
Developing computational speech and dialogue act models has long been a topic of interest BIBREF5 , BIBREF6 , BIBREF7 , BIBREF8 , with researchers from many different backgrounds studying human conversations and developing theories around conversational analysis and interpretation on intent. Modern intelligent conversational BIBREF3 , BIBREF4 and dialogue systems draw principles from many disciplines, including philosophy, linguistics, computer science, and sociology. In this section, we describe relevant previous work on speech and dialogue act modeling, general conversation modeling on Twitter, and speech and dialogue act modeling of customer service in other data sources.
Question: Which patterns and rules are derived?
Paper: "How May I Help You?": Modeling Twitter Customer Service Conversations Using Fine-Grained Dialogue Acts
The underlying goal of this work is to show how a well-defined taxonomy of dialogue acts can be used to summarize semantic information in real-time about the flow of a conversation to derive meaningful insights into the success/failure of the interaction, and then to develop actionable rules to be used in automating customer service interactions. We focus on the customer service domain on Twitter, which has not previously been explored in the context of dialogue act classification. In this new domain, we can provide meaningful recommendations about good communicative practices, based on real data. Our methodology pipeline is shown in Figure FIGREF2 .
Question: How are customer satisfaction, customer frustration and overall problem resolution data collected?
Answer: By annotators on Amazon Mechanical Turk.
Paper: "How May I Help You?": Modeling Twitter Customer Service Conversations Using Fine-Grained Dialogue Acts
As described in the related work, the taxonomy of 12 acts to classify dialogue acts in an instant-messaging scenario, developed by Ivanovic in 2005, has been used by previous work when approaching the task of dialogue act classification for customer service BIBREF18 , BIBREF20 , BIBREF19 , BIBREF21 , BIBREF22 . The dataset used consisted of eight conversations from chat logs in the MSN Shopping Service (around 550 turns spanning around 4,500 words) BIBREF19 . The conversations were gathered by asking five volunteers to use the platform to inquire for help regarding various hypothetical situations (i.e. buying an item for someone) BIBREF19 . The process of selection of tags to develop the taxonomy, beginning with the 42 tags from the DAMSL set BIBREF0 , involved removing tags inappropriate for written text, and collapsing sets of tags into a more coarse-grained label BIBREF18 . The final taxonomy consists of the following 12 dialogue acts (sorted by frequency in the dataset): Statement (36%), Thanking (14.7%), Yes-No Question (13.9%), Response-Acknowledgement (7.2%), Request (5.9%), Open-Question (5.3%), Yes-Answer (5.1%), Conventional-Closing (2.9%), No-Answer (2.5%), Conventional-Opening (2.3%), Expressive (2.3%) and Downplayer (1.9%).
Question: Which Twitter customer service industries are investigated?
Paper: "How May I Help You?": Modeling Twitter Customer Service Conversations Using Fine-Grained Dialogue Acts
Given our taxonomy of fine-grained dialogue acts that expands upon previous work, we set out to gather annotations for Twitter customer service conversations.
Question: Which dialogue acts are more suited to the twitter domain?
Paper: Building a Neural Machine Translation System Using Only Synthetic Parallel Data
Given the data-driven nature of neural machine translation (NMT), the limited source-to-target bilingual sentence pairs have been one of the major obstacles in building competitive NMT systems. Recently, pseudo parallel data, which refer to the synthetic bilingual sentence pairs automatically generated by existing translation models, have reported promising results with regard to the data scarcity in NMT. Many studies have found that the pseudo parallel data combined with the real bilingual parallel corpus significantly enhance the quality of NMT models BIBREF0 , BIBREF1 , BIBREF2 . In addition, synthesized parallel data have played vital roles in many NMT problems such as domain adaptation BIBREF0 , zero-resource NMT BIBREF3 , and the rare word problem BIBREF4 .
Question: How many improvements on the French-German translation benchmark?
Answer: one
Paper: Building a Neural Machine Translation System Using Only Synthetic Parallel Data
Given a source sentence $x = (x_1, \ldots , x_m)$ and its corresponding target sentence $y= (y_1, \ldots , y_n)$ , the NMT aims to model the conditional probability $p(y|x)$ with a single large neural network. To parameterize the conditional distribution, recent studies on NMT employ the encoder-decoder architecture BIBREF7 , BIBREF8 , BIBREF9 . Thereafter, the attention mechanism BIBREF10 , BIBREF11 has been introduced and successfully addressed the quality degradation of NMT when dealing with long input sentences BIBREF12 .
Question: How do they align the synthetic data?
Paper: Building a Neural Machine Translation System Using Only Synthetic Parallel Data
In statistical machine translation (SMT), synthetic bilingual data have been primarily proposed as a means to exploit monolingual corpora. By applying a self-training scheme, the pseudo parallel data were obtained by automatically translating the source-side monolingual corpora BIBREF13 , BIBREF14 . In a similar but reverse way, the target-side monolingual corpora were also employed to build the synthetic parallel data BIBREF15 , BIBREF16 . The primary goal of these works was to adapt trained SMT models to other domains using relatively abundant in-domain monolingual data.
Question: Where do they collect the synthetic data?
Paper: Hateful People or Hateful Bots? Detection and Characterization of Bots Spreading Religious Hatred in Arabic Social Media
The analysis of social media content to understand online human behavior has gained significant importance in recent years BIBREF0 , BIBREF1 , BIBREF2 , BIBREF3 . However, a major limitation of the design of such analysis is that it often fails to account for content created by bots, which can significantly influence the messaging in social media. A social bot is an autonomous entity on social media that is typically engineered to pass as a human, often with the intent to manipulate online discourse BIBREF4 . Recent studies have shown that a significant majority of the social media content is generated by bots. For example, a six-week study by the Pew Research Center found that around two-thirds of all tweets with URL links were posted by likely bots BIBREF5 . As a result, the presence of bots can negatively impact the results of social media analysis and misinform our understanding of how humans interact within the online social space. In particular, any social media analysis that doesn't take into account the impact of bots is incomplete. While some bots can be beneficial (e.g., customer service chatbots), the focus in this work is on content-polluter bots that mimic human behavior online to spread falsified information BIBREF6 , create a false sense of public support BIBREF7 , and proliferate dangerous ideologies BIBREF8 , BIBREF9 .
Question: Do they analyze what type of content Arabic bots spread in comparison to English?
Paper: Hateful People or Hateful Bots? Detection and Characterization of Bots Spreading Religious Hatred in Arabic Social Media
In this section, we first discuss the main challenges encountered in analyzing Arabic language and social media content in general. We then survey prior research on online hate speech and bot detection and analysis.
Question: Do they propose a new model to better detect Arabic bots specifically?
Paper: SIM: A Slot-Independent Neural Model for Dialogue State Tracking
With the rapid development in deep learning, there is a recent boom of task-oriented dialogue systems in terms of both algorithms and datasets. The goal of task-oriented dialogue is to fulfill a user's requests such as booking hotels via communication in natural language. Due to the complexity and ambiguity of human language, previous systems have included semantic decoding BIBREF0 to project natural language input into pre-defined dialogue states. These states are typically represented by slots and values: slots indicate the category of information and values specify the content of information. For instance, the user utterance “can you help me find the address of any hotel in the south side of the city” can be decoded as $inform(area, south)$ and $request(address)$, meaning that the user has specified the value south for slot area and requested another slot address.
Question: How do they prevent the model complexity increasing with the increased number of slots?
Answer: They exclude slot-specific parameters and incorporate better feature representation of user utterance and dialogue states using syntactic information and convolutional neural networks (CNN).
Paper: SIM: A Slot-Independent Neural Model for Dialogue State Tracking
As outlined in BIBREF9, the dialogue state tracking task is formulated as follows: at each turn of dialogue, the user's utterance is semantically decoded into a set of slot-value pairs. There are two types of slots. Goal slots indicate the category, e.g. area, food, and the values specify the constraint given by users for the category, e.g. South, Mediterranean. Request slots refer to requests, and the value is the category that the user demands, e.g. phone, area. Each user's turn is thus decoded into turn goals and turn requests. Furthermore, to summarize the user's goals so far, the union of all previous turn goals up to the current turn is defined as joint goals.
Question: What network architecture do they use for SIM?
Paper: SIM: A Slot-Independent Neural Model for Dialogue State Tracking
To predict whether a slot-value pair should be included in the turn goals/requests, previous models BIBREF0, BIBREF5 usually define network components for each slot $s\in S$. This can be cumbersome when the ontology is large, and it suffers from the insufficient data problem: the labelled data for a single slot may not suffice to effectively train the parameters for the slot-specific neural networks structure.
Question: How do they measure model size?
Answer: By the number of parameters.
Paper: Transformer Transducer: A Streamable Speech Recognition Model with Transformer Encoders and RNN-T Loss
In the past few years, models employing self-attention BIBREF0 have achieved state-of-art results for many tasks, such as machine translation, language modeling, and language understanding BIBREF0, BIBREF1. In particular, large Transformer-based language models have brought gains in speech recognition tasks when used for second-pass re-scoring and in first-pass shallow fusion BIBREF2. As typically used in sequence-to-sequence transduction tasks BIBREF3, BIBREF4, BIBREF5, BIBREF6, BIBREF7, Transformer-based models attend over encoder features using decoder features, implying that the decoding has to be done in a label-synchronous way, thereby posing a challenge for streaming speech recognition applications. An additional challenge for streaming speech recognition with these models is that the number of computations for self-attention increases quadratically with input sequence size. For streaming to be computationally practical, it is highly desirable that the time it takes to process each frame remains constant relative to the length of the input. Transformer-based alternatives to RNNs have recently been explored for use in ASR BIBREF8, BIBREF9, BIBREF10, BIBREF11.
Question: Does model uses pretrained Transformer encoders?
Paper: Transformer Transducer: A Streamable Speech Recognition Model with Transformer Encoders and RNN-T Loss
In this paper, we present all experimental results with the RNN-T loss BIBREF13 for consistency, which performs similarly to the monotonic RNN-T loss BIBREF19 in our experiments.
Question: What was previous state of the art model?
Paper: Transformer Transducer: A Streamable Speech Recognition Model with Transformer Encoders and RNN-T Loss
The Transformer BIBREF0 is composed of a stack of multiple identical layers. Each layer has two sub-layers, a multi-headed attention layer and a feed-forward layer. Our multi-headed attention layer first applies $\mathrm {LayerNorm}$, then projects the input to $\mathrm {Query}$, $\mathrm {Key}$, and $\mathrm {Value}$ for all the heads BIBREF1. The attention mechanism is applied separately for different attention heads. The attention mechanism provides a flexible way to control the context that the model uses. For example, we can mask the attention score to the left of the current frame to produce output conditioned only on the previous state history. The weight-averaged $\mathrm {Value}$s for all heads are concatenated and passed to a dense layer. We then employ a residual connection on the normalized input and the output of the dense layer to form the final output of the multi-headed attention sub-layer (i.e. $\mathrm {LayerNorm}(x) + \mathrm {AttentionLayer}(\mathrm {LayerNorm}(x))$, where $x$ is the input to the multi-headed attention sub-layer). We also apply dropout on the output of the dense layer to prevent overfitting. Our feed-forward sub-layer applies $\mathrm {LayerNorm}$ on the input first, then applies two dense layers. We use $\mathrm {ReLu}$ as the activation for the first dense layer. Again, dropout to both dense layers for regularization, and a residual connection of normalized input and the output of the second dense layer (i.e. $\mathrm {LayerNorm}(x) + \mathrm {FeedForwardLayer}(\mathrm {LayerNorm}(x))$, where $x$ is the input to the feed-forward sub-layer) are applied. See Figure FIGREF10 for more details.
Question: What was previous state of the art accuracy on LibriSpeech benchmark?
Paper: Transformer Transducer: A Streamable Speech Recognition Model with Transformer Encoders and RNN-T Loss
We evaluated the proposed model using the publicly available LibriSpeech ASR corpus BIBREF23. The LibriSpeech dataset consists of 970 hours of audio data with corresponding text transcripts (around 10M word tokens) and an additional 800M word token text only dataset. The paired audio/transcript dataset was used to train T-T models and an LSTM-based baseline. The full 810M word tokens text dataset was used for standalone language model (LM) training. We extracted 128-channel logmel energy values from a 32 ms window, stacked every 4 frames, and sub-sampled every 3 frames, to produce a 512-dimensional acoustic feature vector with a stride of 30 ms. Feature augmentation BIBREF22 was applied during model training to prevent overfitting and to improve generalization, with only frequency masking ($\mathrm {F}=50$, $\mathrm {mF}=2$) and time masking ($\mathrm {T}=30$, $\mathrm {mT}=10$).
Question: How big is LibriSpeech dataset?
Paper: Learning General Purpose Distributed Sentence Representations via Large Scale Multi-task Learning
Transfer learning has driven a number of recent successes in computer vision and NLP. Computer vision tasks like image captioning BIBREF0 and visual question answering typically use CNNs pretrained on ImageNet BIBREF1 , BIBREF2 to extract representations of the image, while several natural language tasks such as reading comprehension and sequence labeling BIBREF3 have benefited from pretrained word embeddings BIBREF4 , BIBREF5 that are either fine-tuned for a specific task or held fixed.
Question: Which language(s) do they work with?
Paper: Learning General Purpose Distributed Sentence Representations via Large Scale Multi-task Learning
The problem of learning distributed representations of phrases and sentences dates back over a decade. For example, BIBREF19 present an additive and multiplicative linear composition function of the distributed representations of individual words. BIBREF20 combine symbolic and distributed representations of words using tensor products. Advances in learning better distributed representations of words BIBREF4 , BIBREF5 combined with deep learning have made it possible to learn complex non-linear composition functions of an arbitrary number of word embeddings using convolutional or recurrent neural networks (RNNs). A network's representation of the last element in a sequence, which is a non-linear composition of all inputs, is typically assumed to contain a squashed “summary” of the sentence. Most work in supervised learning for NLP builds task-specific representations of sentences rather than general-purpose ones.
Question: How do they evaluate their sentence representations?
Paper: Learning General Purpose Distributed Sentence Representations via Large Scale Multi-task Learning
Five out of the six tasks that we consider for multi-task learning are formulated as sequence-to-sequence problems BIBREF26 , BIBREF27 . Briefly, sequence-to-sequence models are a specific case of encoder-decoder models where the inputs and outputs are sequential. They directly model the conditional distribution of outputs given inputs INLINEFORM0 . The input INLINEFORM1 and output INLINEFORM2 are sequences INLINEFORM3 and INLINEFORM4 . The encoder produces a fixed length vector representation INLINEFORM5 of the input, which the decoder then conditions on to generate an output. The decoder is auto-regressive and breaks down the joint probability of outputs into a product of conditional probabilities via the chain rule: INLINEFORM6 
Question: Which model architecture do they for sentence encoding?
Answer: Answer with content missing: (Skip-thought vectors-Natural Language Inference paragraphs) The encoder for the current sentence and the decoders for the previous (STP) and next sentence (STN) are typically parameterized as separate RNNs
- RNN
Paper: Learning General Purpose Distributed Sentence Representations via Large Scale Multi-task Learning
 BIBREF31 present a simple one-to-many multi-task sequence-to-sequence learning model for NMT that uses a shared encoder for English and task-specific decoders for multiple target languages. BIBREF22 extend this by also considering many-to-one (many encoders, one decoder) and many-to-many architectures. In this work, we consider a one-to-many model since it lends itself naturally to the idea of combining inductive biases from different training objectives. The same bidirectional GRU encodes the input sentences from different tasks into a compressed summary INLINEFORM0 which is then used to condition a task-specific GRU to produce the output sentence.
Question: How many tokens can sentences in their model at most contain?
Paper: Learning General Purpose Distributed Sentence Representations via Large Scale Multi-task Learning
Our motivation for multi-task training stems from theoretical insights presented in BIBREF18 . We refer readers to that work for a detailed discussion of results, but the conclusions most relevant to this discussion are (i) that learning multiple related tasks jointly results in good generalization as measured by the number of training examples required per task; and (ii) that inductive biases learned on sufficiently many training tasks are likely to be good for learning novel tasks drawn from the same environment.
Question: Which training objectives do they combine?
Paper: Learning General Purpose Distributed Sentence Representations via Large Scale Multi-task Learning
Multi-task training with different data sources for each task stills poses open questions. For example: When does one switch to training on a different task? Should the switching be periodic? Do we weight each task equally? If not, what training ratios do we use?
Question: Which data sources do they use?
Answer: - En-Fr (WMT14)
- En-De (WMT15)
- Skipthought (BookCorpus)
- AllNLI (SNLI + MultiNLI)
- Parsing (PTB + 1-billion word)
Paper: How to Do Simultaneous Translation Better with Consecutive Neural Machine Translation?
Simultaneous translation BIBREF0, BIBREF1, BIBREF2, BIBREF3, BIBREF4, BIBREF5, the task of producing a partial translation of a sentence before the whole input sentence ends, is useful in many scenarios including outbound tourism, international summit and multilateral negotiations. Different from the consecutive translation in which translation quality alone matters, simultaneous translation trades off between translation quality and latency. The syntactic structure difference between the source and target language makes simultaneous translation more challenging. For example, when translating from a verb-final (SOV) language (e.g., Japanese) to a verb-media (SVO) language (e.g., English), the verb appears much later in the source sequence than in the target language. Some premature translations can lead to significant loss in quality BIBREF5.
Question: Has there been previous work on SNMT?
Paper: How to Do Simultaneous Translation Better with Consecutive Neural Machine Translation?
Given a set of source–target sentence pairs $\left\langle \mathbf {x}_m,\mathbf {y}^*_m\right\rangle _{m=1}^M$, a consecutive NMT model can be trained by maximizing the log-likelihood of the target sentence from its entire source side context:
Question: Which languages do they experiment on?
Paper: How to Do Simultaneous Translation Better with Consecutive Neural Machine Translation?
In SNMT, we receive streaming input tokens, and learn to translate them in real-time. We formulate simultaneous translation as two nested loops: the outer loop that updates an input buffer with newly observed source tokens and the inner loop that translates source tokens in the buffer updated at each outer step.
Question: What corpora is used?
Paper: A Sentiment Analysis of Breast Cancer Treatment Experiences and Healthcare Perceptions Across Twitter
Twitter has shown potential for monitoring public health trends, BIBREF1 , BIBREF2 , BIBREF3 , BIBREF4 , BIBREF5 , disease surveillance, BIBREF6 , and providing a rich online forum for cancer patients, BIBREF7 . Social media has been validated as an effective educational and support tool for breast cancer patients, BIBREF8 , as well as for generating awareness, BIBREF9 . Successful supportive organizations use social media sites for patient interaction, public education, and donor outreach, BIBREF10 . The advantages, limitations, and future potential of using social media in healthcare has been thoroughly reviewed, BIBREF11 . Our study aims to investigate tweets mentioning “breast” and “cancer" to analyze patient populations and selectively obtain content relevant to patient treatment experiences.
Question: Do the authors report results only on English datasets?
Paper: A Sentiment Analysis of Breast Cancer Treatment Experiences and Healthcare Perceptions Across Twitter
 Twitter provides a free streaming Application Programming Interface (API), BIBREF12 , for researchers and developers to mine samples of public tweets. Language processing and data mining, BIBREF13 , was conducted using the Python programming language. The free public API allows targeted keyword mining of up to 1% of Twitter's full volume at any given time, referred to as the `Spritzer Feed'.
Question: How were breast cancer related posts compiled from the Twitter streaming API?
Answer: By using  keywords `breast' AND `cancer' in tweet collecting process.