UNIST-Eunchan/NLP-Paper-to-QA-Generation huggingface dataset:
Abstract: Developing speech technologies for low-resource languages has become a very active research field over the last decade. Among others, Bayesian models have shown some promising results on artificial examples but still lack of in situ experiments. Our work applies state-of-the-art Bayesian models to unsupervised Acoustic Unit Discovery (AUD) in a real low-resource language scenario. We also show that Bayesian models can naturally integrate information from other resourceful languages by means of informative prior leading to more consistent discovered units. Finally, discovered acoustic units are used, either as the 1-best sequence or as a lattice, to perform word segmentation. Word segmentation results show that this Bayesian approach clearly outperforms a Segmental-DTW baseline on the same corpus.

Introduction: Out of nearly 7000 languages spoken worldwide, current speech (ASR, TTS, voice search, etc.) technologies barely address 200 of them. Broadening ASR technologies to ideally all possible languages is a challenge with very high stakes in many areas and is at the heart of several fundamental research problems ranging from psycholinguistic (how humans learn to recognize speech) to pure machine learning (how to extract knowledge from unlabeled data). The present work focuses on the narrow but important problem of unsupervised Acoustic Unit Discovery (AUD). It takes place as the continuation of an ongoing effort to develop a Bayesian model suitable for this task, which stems from the seminal work of BIBREF0 later refined and made scalable in BIBREF1 . This model, while rather crude, has shown that it can provide a clustering accurate enough to be used in topic identification of spoken document in unknown languages BIBREF2 . It was also shown that this model can be further improved by incorporating a Bayesian "phonotactic" language model learned jointly with the acoustic units BIBREF3 . Finally, following the work in BIBREF4 it has been combined successfully with variational auto-encoders leading to a model combining the potential of both deep neural networks and Bayesian models BIBREF5 . The contribution of this work is threefold:

Question: By how much they outperform the baseline?
Answer: 18.08 percent points on F-score
Abstract: Identifying and extracting data elements such as study descriptors in publication full texts is a critical yet manual and labor-intensive step required in a number of tasks. In this paper we address the question of identifying data elements in an unsupervised manner. Specifically, provided a set of criteria describing specific study parameters, such as species, route of administration, and dosing regimen, we develop an unsupervised approach to identify text segments (sentences) relevant to the criteria. A binary classifier trained to identify publications that met the criteria performs better when trained on the candidate sentences than when trained on sentences randomly picked from the text, supporting the intuition that our method is able to accurately identify study descriptors.

Introduction: Support for this research was provided by a grant from the National Institute of Environmental Health Sciences (AES 16002-001), National Institutes of Health to Oak Ridge National Laboratory. This research was supported in part by an appointment to the Oak Ridge National Laboratory ASTRO Program, sponsored by the U.S. Department of Energy and administered by the Oak Ridge Institute for Science and Education. This manuscript has been authored by UT-Battelle, LLC under Contract No. DE-AC05-00OR22725 with the U.S. Department of Energy. The United States Government retains and the publisher, by accepting the article for publication, acknowledges that the United States Government retains a non-exclusive, paid-up, irrevocable, worldwide license to publish or reproduce the published form of this manuscript, or allow others to do so, for United States Government purposes. The Department of Energy will provide public access to these results of federally sponsored research in accordance with the DOE Public Access Plan.

Question: What is a study descriptor?
Answer: Study descriptor is a set of structured data elements extracted from a publication text that contains specific expert knowledge pertaining to domain topics.
Abstract: Named Entity Recognition (NER) is a fundamental Natural Language Processing (NLP) task to extract entities from unstructured data. The previous methods for NER were based on machine learning or deep learning. Recently, pre-training models have significantly improved performance on multiple NLP tasks. In this paper, firstly, we introduce the architecture and pre-training tasks of four common pre-training models: BERT, ERNIE, ERNIE2.0-tiny, and RoBERTa. Then, we apply these pre-training models to a NER task by fine-tuning, and compare the effects of the different model architecture and pre-training tasks on the NER task. The experiment results showed that RoBERTa achieved state-of-the-art results on the MSRA-2006 dataset.

Introduction: Named Entity Recognition (NER) is a basic and important task in Natural Language Processing (NLP). It aims to recognize and classify named entities, such as person names and location namesBIBREF0. Extracting named entities from unstructured data can benefit many NLP tasks, for example Knowledge Graph (KG), Decision-making Support System (DSS), and Question Answering system. Researchers used rule-based and machine learning methods for the NER in the early yearsBIBREF1BIBREF2. Recently, with the development of deep learning, deep neural networks have improved the performance of NER tasksBIBREF3BIBREF4. However, it may still be inefficient to use deep neural networks because the performance of these methods depends on the quality of labeled data in training sets while creating annotations for unstructured data is especially difficultBIBREF5. Therefore, researchers hope to find an efficient method to extract semantic and syntactic knowledge from a large amount of unstructured data, which is also unlabeled. Then, apply the semantic and syntactic knowledge to improve the performance of NLP task effectively. Recent theoretical developments have revealed that word embeddings have shown to be effective for improving many NLP tasks. The Word2Vec and Glove models represent a word as a word embedding, where similar words have similar word embeddingsBIBREF6. However, the Word2Vec and Glove models can not solve the problem of polysemy. Researchers have proposed some pre-training models, such as BERT, ERNIE, and RoBERTa, to learn contextualized word embeddings from unstructured text corpusBIBREF7BIBREF8BIBREF9. These models not only solve the problem of polysemy but also obtain more accurate word representations. Therefore, researchers pay more attention to how to apply these pre-training models to improve the performance of NLP tasks. The purpose of this paper is to introduce the structure and pre-training tasks of four common pre-trained models (BERT, ERNIE, ERNIE2.0-tiny, RoBERTa), and how to apply these models to a NER task by fine-tuning. Moreover, we also conduct experiments on the MSRA-2006 dataset to test the effects of different pre-training models on the NER task, and discuss the reasons for these results from the model architecture and pre-training tasks respectively.

Question: what evaluation metrics did they use?
Answer: Precision 
Recall
F1
Abstract: We present an architecture for information extraction from text that augments an existing parser with a character-level neural network. The network is trained using a measure of consistency of extracted data with existing databases as a form of noisy supervision. Our architecture combines the ability of constraint-based information extraction systems to easily incorporate domain knowledge and constraints with the ability of deep neural networks to leverage large amounts of data to learn complex features. Boosting the existing parser's precision, the system led to large improvements over a mature and highly tuned constraint-based production information extraction system used at Bloomberg for financial language text.

Introduction: Unstructured textual data is abundant in the financial domain (see e.g. Figure FIGREF2 ). This information is by definition not in a format that lends itself to immediate processing. Hence, information extraction is an essential step in business applications that require fast, accurate, and low-cost information processing. In the financial domain, these applications include the creation of time series databases for macroeconomic forecasting or financial analysis, as well as the real-time extraction of time series data to inform algorithmic trading strategies. Bloomberg has had information extraction systems for financial language text for nearly a decade. To meet the application domain's high accuracy requirements, marrying constraints with statistical models is often beneficial, see e.g. BIBREF0 , BIBREF1 . Many quantities appearing in information extraction problems are by definition constrained in the numerical values they can assume (e.g. unemployment numbers cannot be negative numbers, while changes in unemployment numbers can be negative). The inclusion of such constraints may significantly boost data efficiency. Constraints can be complex in nature, and may involve multiple entities belonging to an extraction candidate generated by the parser. At Bloomberg, we found the system for information extraction described in this paper especially useful to extract time series (TS) data. As an example, consider numerical relations of the form ts_tick_abs (TS symbol, numerical value), e.g. ts_tick_abs (US_Unemployment, 4.9%), or ts_tick_rel (TS symbol, change in num. value), e.g. ts_tick_abs (US_Unemployment, -0.2%).

Question: by how much did the system improve?
Answer: By more than 90%
Abstract: The task of word-level quality estimation (QE) consists of taking a source sentence and machine-generated translation, and predicting which words in the output are correct and which are wrong. In this paper, propose a method to effectively encode the local and global contextual information for each target word using a three-part neural network approach. The first part uses an embedding layer to represent words and their part-of-speech tags in both languages. The second part leverages a one-dimensional convolution layer to integrate local context information for each target word. The third part applies a stack of feed-forward and recurrent neural networks to further encode the global context in the sentence before making the predictions. This model was submitted as the CMU entry to the WMT2018 shared task on QE, and achieves strong results, ranking first in three of the six tracks.

Introduction: Quality estimation (QE) refers to the task of measuring the quality of machine translation (MT) system outputs without reference to the gold translations BIBREF0 , BIBREF1 . QE research has grown increasingly popular due to the improved quality of MT systems, and potential for reductions in post-editing time and the corresponding savings in labor costs BIBREF2 , BIBREF3 . QE can be performed on multiple granularities, including at word level, sentence level, or document level. In this paper, we focus on quality estimation at word level, which is framed as the task of performing binary classification of translated tokens, assigning “OK” or “BAD” labels. Early work on this problem mainly focused on hand-crafted features with simple regression/classification models BIBREF4 , BIBREF5 . Recent papers have demonstrated that utilizing recurrent neural networks (RNN) can result in large gains in QE performance BIBREF6 . However, these approaches encode the context of the target word by merely concatenating its left and right context words, giving them limited ability to control the interaction between the local context and the target word. In this paper, we propose a neural architecture, Context Encoding Quality Estimation (CEQE), for better encoding of context in word-level QE. Specifically, we leverage the power of both (1) convolution modules that automatically learn local patterns of surrounding words, and (2) hand-crafted features that allow the model to make more robust predictions in the face of a paucity of labeled data. Moreover, we further utilize stacked recurrent neural networks to capture the long-term dependencies and global context information from the whole sentence. We tested our model on the official benchmark of the WMT18 word-level QE task. On this task, it achieved highly competitive results, with the best performance over other competitors on English-Czech, English-Latvian (NMT) and English-Latvian (SMT) word-level QE task, and ranking second place on English-German (NMT) and German-English word-level QE task.

Question: How many layers of recurrent neural networks do they use for encoding the global context?
Answer: 8
Abstract: Experimenting with a dataset of approximately 1.6M user comments from a Greek news sports portal, we explore how a state of the art RNN-based moderation method can be improved by adding user embeddings, user type embeddings, user biases, or user type biases. We observe improvements in all cases, with user embeddings leading to the biggest performance gains.

Introduction: News portals often allow their readers to comment on articles, in order to get feedback, engage their readers, and build customer loyalty. User comments, however, can also be abusive (e.g., bullying, profanity, hate speech), damaging the reputation of news portals, making them liable to fines (e.g., when hosting comments encouraging illegal actions), and putting off readers. Large news portals often employ moderators, who are frequently overwhelmed by the volume and abusiveness of comments. Readers are disappointed when non-abusive comments do not appear quickly online because of moderation delays. Smaller news portals may be unable to employ moderators, and some are forced to shut down their comments. In previous work BIBREF0 , we introduced a new dataset of approx. 1.6M manually moderated user comments from a Greek sports news portal, called Gazzetta, which we made publicly available. Experimenting on that dataset and the datasets of Wulczyn et al. Wulczyn2017, which contain moderated English Wikipedia comments, we showed that a method based on a Recurrent Neural Network (rnn) outperforms detox BIBREF1 , the previous state of the art in automatic user content moderation. Our previous work, however, considered only the texts of the comments, ignoring user-specific information (e.g., number of previously accepted or rejected comments of each user). Here we add user embeddings or user type embeddings to our rnn-based method, i.e., dense vectors that represent individual users or user types, similarly to word embeddings that represent words BIBREF2 , BIBREF3 . Experiments on Gazzetta comments show that both user embeddings and user type embeddings improve the performance of our rnn-based method, with user embeddings helping more. User-specific or user-type-specific scalar biases also help to a lesser extent.

Question: How much gain in performance was obtained with user embeddings?
Answer: On test set RNN that uses user embedding has AUC of 80.53 compared to base RNN 79.24.
Abstract: This paper describes the Amobee sentiment analysis system, adapted to compete in SemEval 2017 task 4. The system consists of two parts: a supervised training of RNN models based on a Twitter sentiment treebank, and the use of feedforward NN, Naive Bayes and logistic regression classifiers to produce predictions for the different sub-tasks. The algorithm reached the 3rd place on the 5-label classification task (sub-task C).

Introduction: Sentiment detection is the process of determining whether a text has a positive or negative attitude toward a given entity (topic) or in general. Detecting sentiment on Twitter—a social network where users interact via short 140-character messages, exchanging information and opinions—is becoming ubiquitous. Sentiment in Twitter messages (tweets) can capture the popularity level of political figures, ideas, brands, products and people. Tweets and other social media texts are challenging to analyze as they are inherently different; use of slang, mis-spelling, sarcasm, emojis and co-mentioning of other messages pose unique difficulties. Combined with the vast amount of Twitter data (mostly public), these make sentiment detection on Twitter a focal point for data science research. SemEval is a yearly event in which teams compete in natural language processing tasks. Task 4 is concerned with sentiment analysis in Twitter; it contains five sub-tasks which include classification of tweets according to 2, 3 or 5 labels and quantification of sentiment distribution regarding topics mentioned in tweets; for a complete description of task 4 see BIBREF0 . This paper describes our system and participation in all sub-tasks of SemEval 2017 task 4. Our system consists of two parts: a recurrent neural network trained on a private Twitter dataset, followed by a task-specific combination of model stacking and logistic regression classifiers. The paper is organized as follows: section SECREF2 describes the training of RNN models, data being used and model selection; section SECREF3 describes the extraction of semantic features; section SECREF4 describes the task-specific workflows and scores. We review and summarize in section SECREF5 . Finally, section SECREF6 describes our future plans, mainly the development of an LSTM algorithm.

Question: Which Twitter sentiment treebank is used?
Answer: They built their own
Abstract: African languages are numerous, complex and low-resourced. The datasets required for machine translation are difficult to discover, and existing research is hard to reproduce. Minimal attention has been given to machine translation for African languages so there is scant research regarding the problems that arise when using machine translation techniques. To begin addressing these problems, we trained models to translate English to five of the official South African languages (Afrikaans, isiZulu, Northern Sotho, Setswana, Xitsonga), making use of modern neural machine translation techniques. The results obtained show the promise of using neural machine translation techniques for African languages. By providing reproducible publicly-available data, code and results, this research aims to provide a starting point for other researchers in African machine translation to compare to and build upon.

Introduction: Africa has over 2000 languages across the continent BIBREF0 . South Africa itself has 11 official languages. Unlike many major Western languages, the multitude of African languages are very low-resourced and the few resources that exist are often scattered and difficult to obtain. Machine translation of African languages would not only enable the preservation of such languages, but also empower African citizens to contribute to and learn from global scientific, social and educational conversations, which are currently predominantly English-based BIBREF1 . Tools, such as Google Translate BIBREF2 , support a subset of the official South African languages, namely English, Afrikaans, isiZulu, isiXhosa and Southern Sotho, but do not translate the remaining six official languages. Unfortunately, in addition to being low-resourced, progress in machine translation of African languages has suffered a number of problems. This paper discusses the problems and reviews existing machine translation research for African languages which demonstrate those problems. To try to solve the highlighted problems, we train models to perform machine translation of English to Afrikaans, isiZulu, Northern Sotho (N. Sotho), Setswana and Xitsonga, using state-of-the-art neural machine translation (NMT) architectures, namely, the Convolutional Sequence-to-Sequence (ConvS2S) and Transformer architectures. Section SECREF2 describes the problems facing machine translation for African languages, while the target languages are described in Section SECREF3 . Related work is presented in Section SECREF4 , and the methodology for training machine translation models is discussed in Section SECREF5 . Section SECREF6 presents quantitative and qualitative results.

Question: What datasets did they use?
Answer: English to Afrikaans, isiZulu, N. Sotho,
Setswana, and Xitsonga parallel corpora from the Autshumato project
Abstract: We show that an efficient and popular method for calculating bigram frequencies is unsuitable for bodies of short texts and offer a simple alternative. Our method has the same computational complexity as the old method and offers an exact count instead of an approximation.

Introduction: This short note is the result of a brief conversation between the authors and Joel Nothman. We came across a potential problem, he gave a sketch of a fix, and we worked out the details of a solution.

Question: What is the computational complexity of old method
Answer: O(2**N)
Abstract: We present here our approach to the GermEval 2019 Task 1 - Shared Task on hierarchical classification of German blurbs. We achieved first place in the hierarchical subtask B and second place on the root node, flat classification subtask A. In subtask A, we applied a simple multi-feature TF-IDF extraction method using different n-gram range and stopword removal, on each feature extraction module. The classifier on top was a standard linear SVM. For the hierarchical classification, we used a local approach, which was more light-weighted but was similar to the one used in subtask A. The key point of our approach was the application of a post-processing to cope with the multi-label aspect of the task, increasing the recall but not surpassing the precision measure score.

Introduction: Hierarchical Multi-label Classification (HMC) is an important task in Natural Language Processing (NLP). Several NLP problems can be formulated in this way, such as patent, news articles, books and movie genres classification (as well as many other classification tasks like diseases, gene function prediction). Also, many tasks can be formulated as hierarchical problem in order to cope with a large amount of labels to assign to the sample, in a divide and conquer manner (with pseudo meta-labels). A theoretical survey exists BIBREF0 discussing on how the task can be engaged, several approaches and the prediction quality measures. Basically, the task in HMC is to assign a sample to one or many nodes of a Directed Acyclic Graph (DAG) (in special cases a tree) based on features extracted from the sample. In the case of possible multiple parent, the evaluation of the prediction complicates heavily, for once since several paths can be taken, but only in a joining node must be considered. The GermEval 2019 Task 1 - Shared Task on hierarchical classification of German blurbs focus on the concrete challenge of classifying short descriptive texts of books into the root nodes (subtask A) or into the entire hierarchy (subtask B). The hierarchy can be described as a tree and consisted of 343 nodes, in which there are 8 root nodes. With about 21k samples it was not clear if deep learning methods or traditional NLP methods would perform better. Especially, in the subtask A, since for subtask B some classes had only a few examples. Although an ensemble of traditional and deep learning methods could profit in this area, it is difficult to design good heterogeneous ensembles. Our approach was a traditional NLP one, since we employed them successfully in several projects BIBREF1, BIBREF2, BIBREF3, with even more samples and larger hierarchies. We compared also new libraries and our own implementation, but focused on the post-processing of the multi-labels, since this aspect seemed to be the most promising improvement to our matured toolkit for this task. This means but also, to push recall up and hope to not overshot much over precision.

Question: Does the paper report F1-scores with and without post-processing for the second task?
Answer: With post-processing