z-uo/qasper-squad huggingface dataset:
Paper: AandP: Utilizing Prolog for converting between active sentence and passive sentence with three-steps conversion
There are 12 tenses in English. Each tense has a specific structure for the sentence. If each tense is handled individually, it will be quite long and be not an optimal solution. Therefore, as my best observation, I found a solution which divides 12 English tenses into 4 groups (same color means same group) based on the number of auxiliary verbs in the active sentence. This solution is summarized in Figure FIGREF72, consisting of:
Question: Is there information about performance of these conversion methods?
Paper: AandP: Utilizing Prolog for converting between active sentence and passive sentence with three-steps conversion
The three-steps conversion consists of three steps:
Question: Are there some experiments performed in the paper?
Paper: Revealing the Dark Secrets of BERT
Over the past year, models based on the Transformer architecture BIBREF0 have become the de-facto standard for state-of-the-art performance on many natural language processing (NLP) tasks BIBREF1, BIBREF2. Their key feature is the self-attention mechanism that provides an alternative to conventionally used recurrent neural networks (RNN).
Question: How much is performance improved by disabling attention in certain heads?
Paper: Revealing the Dark Secrets of BERT
There have been several recent attempts to assess BERT's ability to capture structural properties of language. BIBREF5 demonstrated that BERT consistently assigns higher scores to the correct verb forms as opposed to the incorrect one in a masked language modeling task, suggesting some ability to model subject-verb agreement. BIBREF6 extended this work to using multiple layers and tasks, supporting the claim that BERT's intermediate layers capture rich linguistic information. On the other hand, BIBREF7 concluded that LSTMs generalize to longer sequences better, and are more robust with respect to agreement distractors, compared to Transformers.
Question: In which certain heads was attention disabled in experiments?
Paper: Revealing the Dark Secrets of BERT
We pose the following research questions:
Question: What handcrafter features-of-interest are used?
Paper: Revealing the Dark Secrets of BERT
In this section, we present the experiments conducted to address the above research questions.
Question: What subset of GLUE tasks is used?
Paper: Exploring Hierarchical Interaction Between Review and Summary for Better Sentiment Analysis
Sentiment analysis BIBREF0, BIBREF1 is a fundamental task in natural language processing. In particular, sentiment analysis of user reviews has wide applicationsBIBREF2, BIBREF3, BIBREF4, BIBREF5. In many review websites such as Amazon and IMDb, the user is allowed to give a summary in addition to their review. Summaries usually contain more abstract information about the review. As shown in Figure FIGREF3, two screenshots of reviews were taken from Amazon and IMDb websites, respectively. The user-written summaries of these reviews can be highly indicative of the final polarity. As a result, it is worth considering them together with the review itself for making sentiment classification.
Question: Do they predict the sentiment of the review summary?
Paper: Exploring Hierarchical Interaction Between Review and Summary for Better Sentiment Analysis
The majority of recent sentiment analysis models are based on either convolutional or recurrent neural networks to encode sequences BIBREF10, BIBREF11.
Question: What is the performance difference of using a generated summary vs. a user-written one?
Answer: 2.7 accuracy points
Paper: Exploring Hierarchical Interaction Between Review and Summary for Better Sentiment Analysis
In this section, we introduce our proposed model in details. We first give the problem formulation, followed by an overview of the proposed model, and explain each layer of our model in details, before finally giving the loss function and training methods.
Question: Which review dataset do they use?
Paper: Generaci\'on autom\'atica de frases literarias en espa\~nol
Los investigadores en Procesamiento de Lenguaje Natural (PLN) durante mucho tiempo han utilizado corpus constituidos por documentos enciclopédicos (notablemente Wikipedia), periodísticos (periódicos o revistas) o especializados (documentos legales, científicos o técnicos) para el desarrollo y pruebas de sus modelos BIBREF0, BIBREF1, BIBREF2.
Question: What evaluation metrics did they look at?
Answer: accuracy with standard deviation
Paper: Generaci\'on autom\'atica de frases literarias en espa\~nol
La generación de texto es una tarea relativamente clásica, que ha sido estudiada en diversos trabajos. Por ejemplo, BIBREF5 presentan un modelo basado en cadenas de Markov para la generación de texto en idioma polaco. Los autores definen un conjunto de estados actuales y calculan la probabilidad de pasar al estado siguiente. La ecuación (DISPLAY_FORM1) calcula la probabilidad de pasar al estado $X_{i}$ a partir de $X_{j}$,
Question: What datasets are used?
Paper: Language-Agnostic Syllabification with Neural Sequence Labeling
Words can be considered compositions of syllables, which in turn are compositions of phones. Phones are units of sound producible by the human vocal apparatus. Syllables play an important role in prosody and are influential components of natural language understanding, speech production, and speech recognition systems. Text-to-speech (TTS) systems can rely heavily on automatically syllabified phone sequences BIBREF0. One prominent example is Festival, an open source TTS system that relies on a syllabification algorithm to organize speech production BIBREF1.
Question: What are the datasets used for the task?
Answer: Datasets used are Celex (English, Dutch), Festival (Italian), OpenLexuque (French), IIT-Guwahati (Manipuri), E-Hitz (Basque)
Paper: Language-Agnostic Syllabification with Neural Sequence Labeling
Syllabification can be considered a sequence labeling task where each label delineates the existence or absence of a syllable boundary. As such, syllabification has much in common with well-researched topics such as part-of-speech tagging, named-entity recognition, and chunking BIBREF17. Neural networks have recently outpaced more traditional methods in sequence labeling tasks. These neural-based approaches are taking the place of HMMs, maximum entropy Markov models (MEMM), and conditional random fields (CRF) BIBREF18.
Question: What is the accuracy of the model for the six languages tested?
Answer: Authors report their best models have following accuracy: English CELEX (98.5%), Dutch CELEX (99.47%), Festival (99.990%), OpenLexique (100%), IIT-Guwahat (95.4%), E-Hitz (99.83%)
Paper: Language-Agnostic Syllabification with Neural Sequence Labeling
Recurrent neural networks (RNNs) differ from standard feed-forward neural networks in their treatment of input order; each element is processed given the context of the input that came before. RNNs operate on sequential data and can take many forms. Our network leverages the long short-term memory (LSTM) cell which is a prominent RNN variant capable of capturing long-term sequential dependencies BIBREF20. The gated memory cells of LSTM are an improvement over the standard RNN because the standard RNN is often biased toward short-term dependencies BIBREF21, BIBREF22. At each time step, the LSTM cell determines what information is important to introduce, to keep, and to output. This is done using an input gate, a forget gate, and an output gate shown in Fig. FIGREF5. LSTM operates in a single direction through time. This can be a limitation when a time step has both past dependency and future dependency. For example, a consonant sound may be the coda of a syllable earlier in the sequence or the onset of a syllable later in the sequence. Thus, processing a phonetic sequence in both the forward and backwards directions provides an improved context for assigning syllable boundaries. A bidirectional LSTM (BiLSTM) is formed when an LSTM moving forward through time is concatenated with an LSTM moving backward through time BIBREF23.
Question: Which models achieve state-of-the-art performances?
Answer: CELEX (Dutch and English) - SVM-HMM
Festival, E-Hitz and OpenLexique - Liang hyphenation
IIT-Guwahat - Entropy CRF
Paper: Language-Agnostic Syllabification with Neural Sequence Labeling
Convolutional neural networks (CNNs) are traditionally used in computer vision, but perform well in many text processing tasks that benefit from position-invariant abstractions BIBREF24, BIBREF25. These abstractions depend exclusively on local neighboring features rather than the position of features in a global structure. According to a comparative study by BIBREF26, BiLSTMs tend to outperform CNNs in sequential tasks such as POS tagging, but CNNs tend to outperform BiLSTMs in global relation detection tasks such as keyphrase matching for question answering. We use both the BiLSTM and the CNN in our network so that the strengths of each are incorporated. CNNs have been combined with BiLSTMs to perform state-of-the-art sequence tagging in both POS tagging and NER. BIBREF27 used BiLSTMs to process the word sequence while each word's character sequence was processed with CNNs to provide a second representation. In textual syllabification, the only input is the phone sequence.
Question: Is the LSTM bidirectional?
Paper: A Latent Morphology Model for Open-Vocabulary Neural Machine Translation
Neural machine translation (NMT) systems are conventionally trained based on the approach of maximizing the log-likelihood on a training corpus in order to learn distributed representations of words according to their sentence context, which is highly demanding in terms of training data as well as the network capacity. Under conditions of lexical sparsity, which may include the cases when the amount of training examples is insufficient to observe words in different context, and particularly in translation of morphologically-rich languages, where the same word can have exponentially many different surface realizations due to syntactic conditions, which are often rarely or ever observed in any set of collected examples, the model may suffer in learning accurate representations of words. The standard approach to overcome this limitation is to replace the word representations in the model with subword units that are shared among words, which are, in principle, more reliable as they are observed more frequently in varying context BIBREF0, BIBREF1. One drawback related to this approach, however, is that the estimation of the subword vocabulary relies on word segmentation methods optimized using corpus-dependent statistics, disregarding any linguistic notion and the translation objective, which may result in morphological errors during splitting, resulting in subword units that are semantically ambiguous as they might be used in far too many lexical contexts BIBREF2. Moreover, the words are generated predicting multiple subword units, which makes generalizing to unseen word forms more difficult, where some of the subword units that could be used to reconstruct a given word may be unlikely in the given context. To alleviate the sub-optimal effects of using explicit segmentation and generalize better to new morphological forms, recent studies explored the idea of extending the same approach to model translation directly at the level of characters BIBREF3, BIBREF4, which, in turn, have demonstrated the requirement of using comparably deeper networks, as the network would then need to learn longer distance grammatical dependencies BIBREF5.
Question: What are the three languages studied in the paper?
Paper: An Incremental Parser for Abstract Meaning Representation
Semantic parsing aims to solve the problem of canonicalizing language and representing its meaning: given an input sentence, it aims to extract a semantic representation of that sentence. Abstract meaning representation BIBREF0 , or AMR for short, allows us to do that with the inclusion of most of the shallow-semantic natural language processing (NLP) tasks that are usually addressed separately, such as named entity recognition, semantic role labeling and co-reference resolution. AMR is partially motivated by the need to provide the NLP community with a single dataset that includes basic disambiguation information, instead of having to rely on different datasets for each disambiguation problem. The annotation process is straightforward, enabling the development of large datasets. Alternative semantic representations have been developed and studied, such as CCG BIBREF1 , BIBREF2 and UCCA BIBREF3 .
Question: Do they use pretrained models as part of their parser?
Paper: An Incremental Parser for Abstract Meaning Representation
Similarly to dependency parsing, AMR parsing is partially based on the identification of predicate-argument structures. Much of the dependency parsing literature focuses on transition-based dependency parsing—an approach to parsing that scans the sentence from left to right in linear time and updates an intermediate structure that eventually ends up being a dependency tree.
Question: Which subtasks do they evaluate on?
Paper: Multilingual Speech Recognition with Corpus Relatedness Sampling
In recent years, Deep Neural Networks (DNNs) have been successfully applied to Automatic Speech Recognition (ASR) for many well-resourced languages including Mandarin and English BIBREF0 , BIBREF1 . However, only a small portion of languages have clean speech labeled corpus. As a result, there is an increasing interest in building speech recognition systems for low-resource languages. To address this issue, researchers have successfully exploited multilingual speech recognition models by taking advantage of labeled corpora in other languages BIBREF2 , BIBREF3 . Multilingual speech recognition enables acoustic models to share parameters across multiple languages, therefore low-resource acoustic models can benefit from rich resources.
Question: Do they test their approach on large-resource tasks?
Paper: Multilingual Speech Recognition with Corpus Relatedness Sampling
Multilingual speech recognition has explored various models to share parameters across languages in different ways. For example, parameters can be shared by using posterior features from other languages BIBREF5 , applying the same GMM components across different HMM states BIBREF6 , training shared hidden layers in DNNs BIBREF2 , BIBREF3 or LSTM BIBREF4 , using language independent bottleneck features BIBREF7 , BIBREF8 . Some models only share their hidden layers, but use separate output layers to predict their phones BIBREF2 , BIBREF3 . Other models have only one shared output layer to predict the universal phone set shared by all languages BIBREF9 , BIBREF10 , BIBREF11 . While those works proposed the multilingual models in different ways, few of them have explicitly exploited the relatedness across various languages and corpora. In contrast, our work computes the relatedness between different corpora using the embedding representations and exploits them efficiently.
Question: By how much do they, on average, outperform the baseline multilingual model on 16 low-resource tasks?
Paper: Multilingual Speech Recognition with Corpus Relatedness Sampling
In this section, we describe our approach to compute the corpus embedding and our Corpus Relatedness Sampling strategy.
Question: How do they compute corpus-level embeddings?
Paper: Achieving Verified Robustness to Symbol Substitutions via Interval Bound Propagation
Deep models have been shown to be vulnerable against adversarial input perturbations BIBREF0, BIBREF1. Small, semantically invariant input alterations can lead to drastic changes in predictions, leading to poor performance on adversarially chosen samples. Recent work BIBREF2, BIBREF3, BIBREF4 also exposed the vulnerabilities of neural NLP models, e.g. with small character perturbations BIBREF5 or paraphrases BIBREF6, BIBREF7. These adversarial attacks highlight often unintuitive model failure modes and present a challenge to deploying NLP models.
Question: Which dataset do they use?
Paper: Quantifying Similarity between Relations with Fact Distribution
Author contributions: Hao Zhu designed the research; Weize Chen prepared the data, and organized data annotation; Hao Zhu and Xu Han designed the experiments; Weize Chen performed the experiments; Hao Zhu, Weize Chen and Xu Han wrote the paper; Zhiyuan Liu and Maosong Sun proofread the paper. Zhiyuan Liu is the corresponding author.
Question: Which competitive relational classification models do they test?
Answer: For relation prediction they test TransE and for relation extraction they test position aware neural sequence model
Paper: Quantifying Similarity between Relations with Fact Distribution
Just as introduced in sec:introduction, we quantify the similarity between relations by their corresponding head-tail entity pair distributions. Consider the typical case that we have got numbers of facts, but they are still sparse among all facts in the real world. How could we obtain a well-generalized distribution over the whole space of possible triples beyond the training facts? This section proposes a method to parameterize such a distribution.
Question: Which tasks do they apply their method to?
Paper: Quantifying Similarity between Relations with Fact Distribution
A fact is a triple INLINEFORM0 , where INLINEFORM1 and INLINEFORM2 are called head and tail entities, INLINEFORM3 is the relation connecting them, INLINEFORM4 and INLINEFORM5 are the sets of entities and relations respectively. We consider a score function INLINEFORM6 maps all triples to a scalar value. As a special case, the function can be factorized into the sum of two parts: INLINEFORM7 . We use INLINEFORM8 to define the unnormalized probability. DISPLAYFORM0 
Question: Which knowledge bases do they use?
Paper: Quantifying Similarity between Relations with Fact Distribution
Here we introduce our special design of neural networks. For the first part and the second part, we implement the scoring functions introduced in eq:local-normalization as DISPLAYFORM0 
Question: How do they gather human judgements for similarity between relations?
Answer: By assessing similarity of 360 pairs of relations from a subset of Wikidata using an integer similarity score from 0 to 4
Paper: Quantifying Similarity between Relations with Fact Distribution
Now we discuss the method to perform training. In this paper, we consider joint training. By minimizing the loss function, we compute the model parameters INLINEFORM0 : DISPLAYFORM0 
Question: Which sampling method do they use to approximate similarity between the conditional probability distributions over entity pairs?
Paper: The emergent algebraic structure of RNNs and embeddings in NLP
Tremendous advances in natural language processing (NLP) have been enabled by novel deep neural network architectures and word embeddings. Historically, convolutional neural network (CNN) BIBREF0 , BIBREF1 and recurrent neural network (RNN) BIBREF2 , BIBREF3 topologies have competed to provide state-of-the-art results for NLP tasks, ranging from text classification to reading comprehension. CNNs identify and aggregate patterns with increasing feature sizes, reflecting our common practice of identifying patterns, literal or idiomatic, for understanding language; they are thus adept at tasks involving key phrase identification. RNNs instead construct a representation of sentences by successively updating their understanding of the sentence as they read new words, appealing to the formally sequential and rule-based construction of language. While both networks display great efficacy at certain tasks BIBREF4 , RNNs tend to be the more versatile, have emerged as the clear victor in, e.g., language translation BIBREF5 , BIBREF6 , BIBREF7 , and are typically more capable of identifying important contextual points through attention mechanisms for, e.g., reading comprehension BIBREF8 , BIBREF9 , BIBREF10 , BIBREF11 . With an interest in NLP, we thus turn to RNNs.
Question: What text classification task is considered?
Answer: To classify a text as belonging to one of the ten possible classes.
Paper: The emergent algebraic structure of RNNs and embeddings in NLP
We embedded words as vectors and used a uni-directional GRU connected to a dense layer to classify the account from which tweets may have originated. The embeddings and simple network were trained end-to-end to avoid imposing any artificial or heuristic constraints on the system.
Question: What novel class of recurrent-like networks is proposed?
Answer: A network, whose learned functions satisfy a certain equation. The  network contains RNN cells with either nested internal memories or dependencies that extend temporally beyond the immediately previous hidden state.
Paper: The emergent algebraic structure of RNNs and embeddings in NLP
We provide two points to motivate examining the potential algebraic properties of RNNs and their space of inputs in the context of NLP.
Question: Is there a formal proof that the RNNs form a representation of the group?
Paper: Single headed attention based sequence-to-sequence model for state-of-the-art results on Switchboard-300
Powerful neural networks have enabled the use of “end-to-end” speech recognition models that directly map a sequence of acoustic features to a sequence of words without conditional independence assumptions. Typical examples are attention based encoder-decoder BIBREF0 and recurrent neural network transducer models BIBREF1. Due to training on full sequences, an utterance corresponds to a single observation from the view point of these models; thus, data sparsity is a general challenge for such approaches, and it is believed that these models are effective only when sufficient training data is available. Indeed, many end-to-end speech recognition papers focus on LibriSpeech, which has 960 hours of training audio. Nevertheless, the best performing systems follow the traditional hybrid approach BIBREF2, outperforming attention based encoder-decoder models BIBREF3, BIBREF4, BIBREF5, BIBREF6, and when less training data is used, the gap between “end-to-end” and hybrid models is more prominent BIBREF3, BIBREF7. Several methods have been proposed to tackle data sparsity and overfitting problems; a detailed list can be found in Sec. SECREF2. Recently, increasingly complex attention mechanisms have been proposed to improve seq2seq model performance, including stacking self and regular attention layers and using multiple attention heads in the encoder and decoder BIBREF4, BIBREF8.
Question: How much bigger is Switchboard-2000 than Switchboard-300 database?
Answer: Switchboard-2000 contains 1700 more hours of speech data.
Paper: Single headed attention based sequence-to-sequence model for state-of-the-art results on Switchboard-300
In contrast to traditional hybrid models, where even recurrent networks are trained on randomized, aligned chunks of labels and features BIBREF10, BIBREF11, whole sequence models are more prone to memorizing the training samples. In order to improve generalization, many of the methods we investigate introduce additional noise, either directly or indirectly, to stochastic gradient descent (SGD) training to avoid narrow, local optima. The other techniques we study address the highly non-convex nature of training neural networks, ease the optimization process, and speed up convergence.
Question: How big is Switchboard-300 database?