UNIST-Eunchan/NLP-Paper-to-QA-Generation huggingface dataset:
Abstract: Due to the nature of human language, historical documents are hard to comprehend by contemporary people. This limits their accessibility to scholars specialized in the time period in which the documents were written. Modernization aims at breaking this language barrier by generating a new version of a historical document, written in the modern version of the document's original language. However, while it is able to increase the document's comprehension, modernization is still far from producing an error-free version. In this work, we propose a collaborative framework in which a scholar can work together with the machine to generate the new version. We tested our approach on a simulated environment, achieving significant reductions of the human effort needed to produce the modernized version of the document.

Introduction: In recent years, awareness of the importance of preserving our cultural heritage has increased. Historical documents are an important part of that heritage. In order to preserve them, there is an increased need in creating digital text versions which can be search and automatically processed BIBREF0. However, their linguistic properties create additional difficulties: due to the lack of a spelling convention, orthography changes depending on the time period and author. Furthermore, human language evolves with the passage of time, increasing the difficulty of the document's comprehension. Thus, historical documents are mostly accessible to scholars specialized in the time period in which each document was written. Modernization tackles the language barrier in order to increase the accessibility of historical documents. To achieve this, it generates a new version of a historical document in the modern version of the language in which the document was originally written (fi:Shakespeare shows an example of modernizing a document). However, while modernization has been successful in order to increase the comprehension of historical documents BIBREF1, BIBREF2, it is still far from creating error-free modern versions. Therefore, this task still needs to be carried out by scholars. Interactive machine translation (IMT) fosters human–computer collaborations to generate error-free translations in a productive way BIBREF4, BIBREF5. In this work, we proposed to apply one of these protocols to historical documents modernization. We strive for creating an error-free modern version of a historical document, decreasing the human effort needed to achieve this goal. The rest of this document is structured as follows: se:work introduces the related work. Then, in se:IMT we present our protocol. se:exp describes the experiments conducted in order to assess our proposal. The results of those experiments are presented and discussed in se:res. Finally, in se:conc, conclusions are drawn.

Question: What language(s) is the model tested on?
Answer: Dutch and Spanish
Abstract: We share a French-English parallel corpus of Foursquare restaurant reviews (this https URL), and define a new task to encourage research on Neural Machine Translation robustness and domain adaptation, in a real-world scenario where better-quality MT would be greatly beneficial. We discuss the challenges of such user-generated content, and train good baseline models that build upon the latest techniques for MT robustness. We also perform an extensive evaluation (automatic and human) that shows significant improvements over existing online systems. Finally, we propose task-specific metrics based on sentiment analysis or translation accuracy of domain-specific polysemous words.

Introduction: Very detailed information about social venues such as restaurants is available from user-generated reviews in applications like Google Maps, TripAdvisor or Foursquare. Most of these reviews are written in the local language and are not directly exploitable by foreign visitors: an analysis of the Foursquare database shows that, in Paris, only 49% of the restaurants have at least one review in English. It can be much worse for other cities and languages (e.g., only 1% of Seoul restaurants for a French-only speaker). Machine Translation of such user-generated content can improve the situation and make the data available for direct display or for downstream NLP tasks (e.g., cross-lingual information retrieval, sentiment analysis, spam or fake review detection), provided its quality is sufficient. We asked professionals to translate 11.5k French Foursquare reviews (18k sentences) to English. We believe that this resource will be valuable to the community for training and evaluating MT systems addressing challenges posed by user-generated content, which we discuss in detail in this paper. We conduct extensive experiments and combine techniques that seek to solve these challenges (e.g., factored case, noise generation, domain adaptation with tags) on top of a strong Transformer baseline. In addition to BLEU evaluation and human evaluation, we use targeted metrics that measure how well polysemous words are translated, or how well sentiments expressed in the original review can still be recovered from its translation.

Question: what are the human evaluation metrics?
Answer: The outputs are ranked by human evaluators, the wins, ties and losses are counted, then the Wilcoxon signed-rank test is applied.
Abstract: The versified play Henry VIII is nowadays widely recognized to be a collaborative work not written solely by William Shakespeare. We employ combined analysis of vocabulary and versification together with machine learning techniques to determine which authors also took part in the writing of the play and what were their relative contributions. Unlike most previous studies, we go beyond the attribution of particular scenes and use the rolling attribution approach to determine the probabilities of authorship of pieces of texts, without respecting the scene boundaries. Our results highly support the canonical division of the play between William Shakespeare and John Fletcher proposed by James Spedding, but also bring new evidence supporting the modifications proposed later by Thomas Merriam.

Introduction: In the first collection of William Shakespeare’s works published in 1623 (the so-called First Folio) a play appears entitled The Famous History of the Life of King Henry the Eight for the very first time. Nowadays it is widely recognized that along with Shakespeare, other authors were involved in the writing of this play, yet there are different opinions as to who these authors were and what the precise shares were of their authorial contributions. This article aims to contribute to the question of the play’s authorship using combined analysis of vocabulary and versification and modern machine learning techniques (as proposed in BIBREF0, BIBREF1).

Question: How confident is the conclusion about Shakespeare vs Flectcher?
Answer: very
Abstract: Clinical NLP has an immense potential in contributing to how clinical practice will be revolutionized by the advent of large scale processing of clinical records. However, this potential has remained largely untapped due to slow progress primarily caused by strict data access policies for researchers. In this paper, we discuss the concern for privacy and the measures it entails. We also suggest sources of less sensitive data. Finally, we draw attention to biases that can compromise the validity of empirical research and lead to socially harmful applications.

Introduction: The use of notes written by healthcare providers in the clinical settings has long been recognized to be a source of valuable information for clinical practice and medical research. Access to large quantities of clinical reports may help in identifying causes of diseases, establishing diagnoses, detecting side effects of beneficial treatments, and monitoring clinical outcomes BIBREF0 , BIBREF1 , BIBREF2 . The goal of clinical natural language processing (NLP) is to develop and apply computational methods for linguistic analysis and extraction of knowledge from free text reports BIBREF3 , BIBREF4 , BIBREF5 . But while the benefits of clinical NLP and data mining have been universally acknowledged, progress in the development of clinical NLP techniques has been slow. Several contributing factors have been identified, most notably difficult access to data, limited collaboration between researchers from different groups, and little sharing of implementations and trained models BIBREF6 . For comparison, in biomedical NLP, where the working data consist of biomedical research literature, these conditions have been present to a much lesser degree, and the progress has been more rapid BIBREF7 . The main contributing factor to this situation has been the sensitive nature of data, whose processing may in certain situations put patient's privacy at risk. The ethics discussion is gaining momentum in general NLP BIBREF8 . We aim in this paper to gather the ethical challenges that are especially relevant for clinical NLP, and to stimulate discussion about those in the broader NLP community. Although enhancing privacy through restricted data access has been the norm, we do not only discuss the right to privacy, but also draw attention to the social impact and biases emanating from clinical notes and their processing. The challenges we describe here are in large part not unique to clinical NLP, and are applicable to general data science as well.

Question: Other than privacy, what are the other major ethical challenges in clinical data?
Answer: sampling bias, unfair treatment due to biased data,  incomplete clinical stories, and reflection of health disparities.
Abstract: Many recent political events, like the 2016 US Presidential elections or the 2018 Brazilian elections have raised the attention of institutions and of the general public on the role of Internet and social media in influencing the outcome of these events. We argue that a safe democracy is one in which citizens have tools to make them aware of propaganda campaigns. We propose a novel task: performing fine-grained analysis of texts by detecting all fragments that contain propaganda techniques as well as their type. We further design a novel multi-granularity neural network, and we show that it outperforms several strong BERT-based baselines.

Introduction: Journalistic organisations, such as Media Bias/Fact Check, provide reports on news sources highlighting the ones that are propagandistic. Obviously, such analysis is time-consuming and possibly biased and it cannot be applied to the enormous amount of news that flood social media and the Internet. Research on detecting propaganda has focused primarily on classifying entire articles as propagandistic/non-propagandistic BIBREF0, BIBREF1, BIBREF2. Such learning systems are trained using gold labels obtained by transferring the label of the media source, as per Media Bias/Fact Check judgment, to each of its articles. Such distant supervision setting inevitably introduces noise in the learning process BIBREF3 and the resulting systems tend to lack explainability. We argue that in order to study propaganda in a sound and reliable way, we need to rely on high-quality trusted professional annotations and it is best to do so at the fragment level, targeting specific techniques rather than using a label for an entire document or an entire news outlet. Therefore, we propose a novel task: identifying specific instances of propaganda techniques used within an article. In particular, we design a novel multi-granularity neural network, and we show that it outperforms several strong BERT-based baselines. Our corpus could enable research in propagandistic and non-objective news, including the development of explainable AI systems. A system that can detect instances of use of specific propagandistic techniques would be able to make it explicit to the users why a given article was predicted to be propagandistic. It could also help train the users to spot the use of such techniques in the news.

Question: What are the propaganda types?
Answer: Although not all of the 18 types are listed, they include using loaded language or appeal to authority and slogans, using logical fallacies such as strawmen, hidden ad-hominen fallacies ad red herrings.
Abstract: This paper describes the Duluth system that participated in SemEval-2017 Task 6 #HashtagWars: Learning a Sense of Humor. The system participated in Subtasks A and B using N-gram language models, ranking highly in the task evaluation. This paper discusses the results of our system in the development and evaluation stages and from two post-evaluation runs.

Introduction: Humor is an expression of human uniqueness and intelligence and has drawn attention in diverse areas such as linguistics, psychology, philosophy and computer science. Computational humor draws from all of these fields and is a relatively new area of study. There is some history of systems that are able to generate humor (e.g., BIBREF0 , BIBREF1 ). However, humor detection remains a less explored and challenging problem (e.g., BIBREF2 , BIBREF3 , BIBREF4 , BIBREF5 ). SemEval-2017 Task 6 BIBREF6 also focuses on humor detection by asking participants to develop systems that learn a sense of humor from the Comedy Central TV show, @midnight with Chris Hardwick. Our system ranks tweets according to how funny they are by training N-gram language models on two different corpora. One consisting of funny tweets provided by the task organizers, and the other on a freely available research corpus of news data. The funny tweet data is made up of tweets that are intended to be humorous responses to a hashtag given by host Chris Hardwick during the program.

Question: How were the ngram models used to generate predictions on the data?
Answer: The n-gram models were used to calculate the logarithm of the probability for each tweet
Abstract: This paper describes our system that has been used in Task1 Affect in Tweets. We combine two different approaches. The first one called N-Stream ConvNets, which is a deep learning approach where the second one is XGboost regresseor based on a set of embedding and lexicons based features. Our system was evaluated on the testing sets of the tasks outperforming all other approaches for the Arabic version of valence intensity regression task and valence ordinal classification task.

Introduction: Sentiment analysis in Twitter is the problem of identifying people’s opinions expressed in tweets. It normally involves the classification of tweets into categories such as “positive”, “negative” and in some cases, “neutral”. The main challenges in designing a sentiment analysis system for Twitter are the following: Most of the existing systems are inspired in the work presented in BIBREF0 . Machine Learning techniques have been used to build a classifier from a set of tweets with a manually annotated sentiment polarity. The success of the Machine Learning models is based on two main facts: a large amount of labeled data and the intelligent design of a set of features that can distinguish between the samples. With this approach, most studies have focused on designing a set of efficient features to obtain a good classification performance BIBREF1 , BIBREF2 , BIBREF3 . For instance, the authors in BIBREF4 used diverse sentiment lexicons and a variety of hand-crafted features. This paper proposes the representation of tweets using a novel set of features, which include the information provided by seven lexicons and a bag of negated words (BonW). The concatenation of these features with a set of basic features improves the classification performance. The polarity of tweets is determined by a classifier based on a Support Vector Machine. The system has been evaluated on the Arabic and English language test sets of the Twitter Sentiment Analysis Track in SemEval 2017, subtask A (Message Polarity Classification). Our system (SiTAKA) has been ranked 8th over 36 teams in the English language test set and 2nd out of 8 teams in the Arabic language test set. The rest of the paper is structured as follows. Section SECREF2 presents the tools and the resources that have been used. In Section SECREF3 we describe the system. The experiments and results are presented and discussed in Section SECREF4 . Finally, in the last section the conclusions as well as further work are presented.

Question: What is the best performing model?
Answer: An ensemble of N-Channels ConvNet and XGboost regressor model
Abstract: Existing conversational datasets consist either of written proxies for dialog or small-scale transcriptions of natural speech. We introduce 'Interview': a large-scale (105K conversations) media dialog dataset collected from news interview transcripts. Compared to existing large-scale proxies for conversational data, language models trained on our dataset exhibit better zero-shot out-of-domain performance on existing spoken dialog datasets, demonstrating its usefulness in modeling real-world conversations. 'Interview' contains speaker role annotations for each turn, facilitating the development of engaging, responsive dialog systems. In fact, experiments on two dialog tasks show that leveraging such labels improves performance over strong speaker-agnostic baselines, and enabling models to generate more specific and inquisitive responses in interview-style conversations.

Introduction: Large repositories of textual communications (e.g. forum and microblog posts) have gained recent popularity as proxies for dialog BIBREF0, BIBREF1, BIBREF2. However, conversations in these settings differ from natural dialog: turns may be sparsely scattered over a large temporal span, contain distinct syntax and vocabulary BIBREF3, and differ greatly in formality and focus BIBREF4. In this paper, we investigate how appropriate such data is for modeling natural dialog, and introduce Interview, a new high-quality large-scale open-domain conversational dataset grounded in interview settings with annotations for specific speaker roles. We compare the performance of state-of-the-art language models fine-tuned on Interview and other popular conversational datasets, demonstrating that Interview contains more complex dialog and better models the characteristics of natural spoken conversations. Our dataset is an order of magnitude larger than existing high-quality natural dialog datasets and contains speaker role annotations for each turn, facilitating the development of conversational agents and assistive systems for settings involving specific speaker roles, such as doctor-patient interviews or hosted talk shows. In particular, we explore the tasks of role modeling in media dialog and role change detection on Interview and find that leveraging role information can enable more nuanced, on-topic and natural dialog generation, as well as improve role change classification performance. In summary, we present Interview, the first large-scale open-domain media dialog dataset. We explore two tasks for which it serves as a promising benchmark dataset: speaker role modeling and speaker change detection. We build simple yet strong models to show quantitatively that role labels from Interview improve performance on such tasks. Interview's scale, spoken origins, role diversity, and complex utterances make it a better source for grounded open-domain conversations.

Question: Which baselines did they compare to?
Answer: two models (GPT2 and DialoGPT) on two datasets (DailyDialog and CALLHOME)
Abstract: BERT, a pre-trained Transformer model, has achieved ground-breaking performance on multiple NLP tasks. In this paper, we describe BERTSUM, a simple variant of BERT, for extractive summarization. Our system is the state of the art on the CNN/Dailymail dataset, outperforming the previous best-performed system by 1.65 on ROUGE-L. The codes to reproduce our results are available at https://github.com/nlpyang/BertSum

Introduction: Single-document summarization is the task of automatically generating a shorter version of a document while retaining its most important information. The task has received much attention in the natural language processing community due to its potential for various information access applications. Examples include tools which digest textual content (e.g., news, social media, reviews), answer questions, or provide recommendations. The task is often divided into two paradigms, abstractive summarization and extractive summarization. In abstractive summarization, target summaries contains words or phrases that were not in the original text and usually require various text rewriting operations to generate, while extractive approaches form summaries by copying and concatenating the most important spans (usually sentences) in a document. In this paper, we focus on extractive summarization. Although many neural models have been proposed for extractive summarization recently BIBREF2 , BIBREF3 , BIBREF4 , BIBREF5 , BIBREF6 , BIBREF7 , the improvement on automatic metrics like ROUGE has reached a bottleneck due to the complexity of the task. In this paper, we argue that, BERT BIBREF0 , with its pre-training on a huge dataset and the powerful architecture for learning complex features, can further boost the performance of extractive summarization . In this paper, we focus on designing different variants of using BERT on the extractive summarization task and showing their results on CNN/Dailymail and NYT datasets. We found that a flat architecture with inter-sentence Transformer layers performs the best, achieving the state-of-the-art results on this task.

Question: What other evaluation metrics did they use other than ROUGE-L??
Answer: ROUGE-1 and ROUGE-2
Abstract: Skip-gram (word2vec) is a recent method for creating vector representations of words ("distributed word representations") using a neural network. The representation gained popularity in various areas of natural language processing, because it seems to capture syntactic and semantic information about words without any explicit supervision in this respect. We propose SubGram, a refinement of the Skip-gram model to consider also the word structure during the training process, achieving large gains on the Skip-gram original test set.

Introduction: Vector representations of words learned using neural networks (NN) have proven helpful in many algorithms for image annotation BIBREF0 or BIBREF1 , language modeling BIBREF2 , BIBREF3 and BIBREF4 or other natural language processing (NLP) tasks BIBREF5 or BIBREF6 . Traditionally, every input word of an NN is stored in the “one-hot” representation, where the vector has only one element set to one and the rest of the vector are zeros. The size of the vector equals to the size of the vocabulary. The NN is trained to perform some prediction, e.g. to predict surrounding words given a word of interest. Instead of using this prediction capacity in some task, the practice is to extract the output of NN's hidden layer of each word (called distributed representation) and directly use this deterministic mapping INLINEFORM0 of word forms to the vectors of real numbers as the word representation. The input one-hot representation of words has two weaknesses: the bloat of the size of the vector with more words in vocabulary and the inability to provide any explicit semantic or syntactic information to the NN. The learned distributed representation of words relies on much shorter vectors (e.g. vocabularies containing millions words are represented in vectors of a few hundred elements) and semantic or syntactic information is often found to be implicitly present (“embedded”) in the vector space. For example, the Euclidean distance between two words in the vector space may be related to semantic or syntactic similarity between them.

Question: How much were the gains they obtained?
Answer: Only 0.2% accuracy gain in morpho-sintactic questions in original test set, and 12.7% accuracy gain on their test set