UNIST-Eunchan/NLP-Paper-to-QA-Generation huggingface dataset:
Abstract: Unsupervised learning of cross-lingual word embedding offers elegant matching of words across languages, but has fundamental limitations in translating sentences. In this paper, we propose simple yet effective methods to improve word-by-word translation of cross-lingual embeddings, using only monolingual corpora but without any back-translation. We integrate a language model for context-aware search, and use a novel denoising autoencoder to handle reordering. Our system surpasses state-of-the-art unsupervised neural translation systems without costly iterative training. We also analyze the effect of vocabulary size and denoising type on the translation performance, which provides better understanding of learning the cross-lingual word embedding and its usage in translation.

Introduction: Building a machine translation (MT) system requires lots of bilingual data. Neural MT models BIBREF0 , which become the current standard, are even more difficult to train without huge bilingual supervision BIBREF1 . However, bilingual resources are still limited to some of the selected language pairs—mostly from or to English. A workaround for zero-resource language pairs is translating via an intermediate (pivot) language. To do so, we need to collect parallel data and train MT models for source-to-pivot and pivot-to-target individually; it takes a double effort and the decoding is twice as slow. Unsupervised learning is another alternative, where we can train an MT system with only monolingual corpora. Decipherment methods BIBREF2 , BIBREF3 are the first work in this direction, but they often suffer from a huge latent hypothesis space BIBREF4 . Recent work by unmt-artetxe and unmt-facebook train sequence-to-sequence MT models of both translation directions together in an unsupervised way. They do back-translation BIBREF5 back and forth for every iteration or batch, which needs an immensely long time and careful tuning of hyperparameters for massive monolingual data. Here we suggest rather simple methods to build an unsupervised MT system quickly, based on word translation using cross-lingual word embeddings. The contributions of this paper are: The proposed models can be efficiently trained with off-the-shelf softwares with little or no changes in the implementation, using only monolingual data. The provided analyses help for better learning of cross-lingual word embeddings for translation purpose. Altogether, our unsupervised MT system outperforms the sequence-to-sequence neural models even without training signals from the opposite translation direction, i.e. via back-translation.

Question: What is reordering in the context of the paper?
Answer: Re-arranging translated words so that they are in the correct order in the target language
Abstract: Previous studies have demonstrated the empirical success of word embeddings in various applications. In this paper, we investigate the problem of learning distributed representations for text documents which many machine learning algorithms take as input for a number of NLP tasks. We propose a neural network model, KeyVec, which learns document representations with the goal of preserving key semantics of the input text. It enables the learned low-dimensional vectors to retain the topics and important information from the documents that will flow to downstream tasks. Our empirical evaluations show the superior quality of KeyVec representations in two different document understanding tasks.

Introduction: In recent years, the use of word representations, such as word2vec BIBREF0 , BIBREF1 and GloVe BIBREF2 , has become a key “secret sauce” for the success of many natural language processing (NLP), information retrieval (IR) and machine learning (ML) tasks. The empirical success of word embeddings raises an interesting research question: Beyond words, can we learn fixed-length distributed representations for pieces of texts? The texts can be of variable-length, ranging from paragraphs to documents. Such document representations play a vital role in a large number of downstream NLP/IR/ML applications, such as text clustering, sentiment analysis, and document retrieval, which treat each piece of text as an instance. Learning a good representation that captures the semantics of each document is thus essential for the success of such applications. In this paper, we introduce KeyVec, a neural network model that learns densely distributed representations for documents of variable-length. In order to capture semantics, the document representations are trained and optimized in a way to recover key information of the documents. In particular, given a document, the KeyVec model constructs a fixed-length vector to be able to predict both salient sentences and key words in the document. In this way, KeyVec conquers the problem of prior embedding models which treat every word and every sentence equally, failing to identify the key information that a document conveys. As a result, the vectorial representations generated by KeyVec can naturally capture the topics of the documents, and thus should yield good performance in downstream tasks. We evaluate our KeyVec on two text understanding tasks: document retrieval and document clustering. As shown in the experimental section SECREF5 , KeyVec yields generic document representations that perform better than state-of-the-art embedding models.

Question: what dataset was used?
Answer: For the document retrieval task - the dataset of the document pool contained 669 academic papers published by IEEE. Fro the document clustering task -  the dataset of 850 academic papers, and 186 associated venues which are used as ground-truth for evaluation.
Abstract: What is an effective expression that draws laughter from human beings? In the present paper, in order to consider this question from an academic standpoint, we generate an image caption that draws a"laugh"by a computer. A system that outputs funny captions based on the image caption proposed in the computer vision field is constructed. Moreover, we also propose the Funny Score, which flexibly gives weights according to an evaluation database. The Funny Score more effectively brings out"laughter"to optimize a model. In addition, we build a self-collected BoketeDB, which contains a theme (image) and funny caption (text) posted on"Bokete", which is an image Ogiri website. In an experiment, we use BoketeDB to verify the effectiveness of the proposed method by comparing the results obtained using the proposed method and those obtained using MS COCO Pre-trained CNN+LSTM, which is the baseline and idiot created by humans. We refer to the proposed method, which uses the BoketeDB pre-trained model, as the Neural Joking Machine (NJM).

Introduction: Laughter is a special, higher-order function that only humans possess. In the analysis of laughter, as Wikipedia says, “Laughter is thought to be a shift of composition (schema)", and laughter frequently occurs when there is a change from a composition of receiver. However, the viewpoint of laughter differs greatly depending on the position of the receiver. Therefore, the quantitative measurement of laughter is very difficult. Image Ogiri on web services such as "Bokete" BIBREF0 have recently appeared, where users post funny captions for thematic images and the captions are evaluated in an SNS-like environment. Users compete to obtain the greatest number of “stars”. Although quantification of laughter is considered to be a very difficult task, the correspondence between evaluations and images on Bokete allows us to treat laughter quantitatively. Image captioning is an active topic in computer vision, and we believe that humorous image captioning can be realized. The main contributions of the present paper are as follows: BoketeDB In the experimental section, we compare the proposed method based on Funny Score and BoketeDB pre-trained parameters with a baseline provided by MS COCO Pre-trained CNN+LSTM. We also compare the results of the NJM with funny captions provided by humans. In an evaluation by humans, the results provided by the proposed method were ranked lower than those provided by humans (22.59% vs. 67.99%) but were ranked higher than the baseline (9.41%). Finally, we show the generated funny captions for several images.

Question: How are the results evaluated?
Answer: With a questionnaire asking subjects to rank methods according to its "funniness". Also, by posting the captions to Bokete to evaluate them by received stars
Abstract: We study two problems in neural machine translation (NMT). First, in beam search, whereas a wider beam should in principle help translation, it often hurts NMT. Second, NMT has a tendency to produce translations that are too short. Here, we argue that these problems are closely related and both rooted in label bias. We show that correcting the brevity problem almost eliminates the beam problem; we compare some commonly-used methods for doing this, finding that a simple per-word reward works well; and we introduce a simple and quick way to tune this reward using the perceptron algorithm.

Introduction: Although highly successful, neural machine translation (NMT) systems continue to be plagued by a number of problems. We focus on two here: the beam problem and the brevity problem. First, machine translation systems rely on heuristics to search through the intractably large space of possible translations. Most commonly, beam search is used during the decoding process. Traditional statistical machine translation systems often rely on large beams to find good translations. However, in neural machine translation, increasing the beam size has been shown to degrade performance. This is the last of the six challenges identified by BIBREF0 . The second problem, noted by several authors, is that NMT tends to generate translations that are too short. BIBREF1 and BIBREF0 address this by dividing translation scores by their length, inspired by work on audio chords BIBREF2 . A similar method is also used by Google's production system BIBREF3 . A third simple method used by various authors BIBREF4 , BIBREF5 , BIBREF6 is a tunable reward added for each output word. BIBREF7 and BIBREF8 propose variations of this reward that enable better guarantees during search. In this paper, we argue that these two problems are related (as hinted at by BIBREF0 ) and that both stem from label bias, an undesirable property of models that generate sentences word by word instead of all at once. The typical solution is to introduce a sentence-level correction to the model. We show that making such a correction almost completely eliminates the beam problem. We compare two commonly-used corrections, length normalization and a word reward, and show that the word reward is slightly better. Finally, instead of tuning the word reward using grid search, we introduce a way to learn it using a perceptron-like tuning method. We show that the optimal value is sensitive both to task and beam size, implying that it is important to tune for every model trained. Fortunately, tuning is a quick post-training step.

Question: Why does wider beam search hurt NMT?
Answer: Using a wider beam increases the probability of a shorter translation to remain in the top k variants and eventually score higher than any longer and more accurate translation variant
Abstract: For the purpose of automatically evaluating speakers' humor usage, we build a presentation corpus containing humorous utterances based on TED talks. Compared to previous data resources supporting humor recognition research, ours has several advantages, including (a) both positive and negative instances coming from a homogeneous data set, (b) containing a large number of speakers, and (c) being open. Focusing on using lexical cues for humor recognition, we systematically compare a newly emerging text classification method based on Convolutional Neural Networks (CNNs) with a well-established conventional method using linguistic knowledge. The advantages of the CNN method are both getting higher detection accuracies and being able to learn essential features automatically.

Introduction: The ability to make effective presentations has been found to be linked with success at school and in the workplace. Humor plays an important role in successful public speaking, e.g., helping to reduce public speaking anxiety often regarded as the most prevalent type of social phobia, generating shared amusement to boost persuasive power, and serving as a means to attract attention and reduce tension BIBREF0 . Automatically simulating an audience's reactions to humor will not only be useful for presentation training, but also improve conversational systems by giving machines more empathetic power. The present study reports our efforts in recognizing utterances that cause laughter in presentations. These include building a corpus from TED talks and using Convolutional Neural Networks (CNNs) in the recognition. The remainder of the paper is organized as follows: Section SECREF2 briefly reviews the previous related research; Section SECREF3 describes the corpus we collected from TED talks; Section SECREF4 describes the text classification methods; Section SECREF5 reports on our experiments; finally, Section SECREF6 discusses the findings of our study and plans for future work.

Question: How are the positive instances annotated? e.g. by annotators, or by laughter from the audience?
Answer: By laughter from the audience
Abstract: Building dialog agents that can converse naturally with humans is a challenging yet intriguing problem of artificial intelligence. In open-domain human-computer conversation, where the conversational agent is expected to respond to human responses in an interesting and engaging way, commonsense knowledge has to be integrated into the model effectively. In this paper, we investigate the impact of providing commonsense knowledge about the concepts covered in the dialog. Our model represents the first attempt to integrating a large commonsense knowledge base into end-to-end conversational models. In the retrieval-based scenario, we propose the Tri-LSTM model to jointly take into account message and commonsense for selecting an appropriate response. Our experiments suggest that the knowledge-augmented models are superior to their knowledge-free counterparts in automatic evaluation.

Introduction: In recent years, data-driven approaches to building conversation models have been made possible by the proliferation of social media conversation data and the increase of computing power. By relying on a large number of message-response pairs, the Seq2Seq framework BIBREF0 attempts to produce an appropriate response based solely on the message itself, without any memory module. In human-to-human conversations, however, people respond to each other's utterances in a meaningful way not only by paying attention to the latest utterance of the conversational partner itself, but also by recalling relevant information about the concepts covered in the dialogue and integrating it into their responses. Such information may contain personal experience, recent events, commonsense knowledge and more (Figure 1 ). As a result, it is speculated that a conversational model with a “memory look-up” module can mimic human conversations more closely BIBREF1 , BIBREF2 . In open-domain human-computer conversation, where the model is expected to respond to human utterances in an interesting and engaging way, commonsense knowledge has to be integrated into the model effectively. In the context of artificial intelligence (AI), commonsense knowledge is the set of background information that an individual is intended to know or assume and the ability to use it when appropriate BIBREF3 , BIBREF4 , BIBREF5 . Due to the vastness of such kind of knowledge, we speculate that this goal is better suited by employing an external memory module containing commonsense knowledge rather than forcing the system to encode it in model parameters as in traditional methods. In this paper, we investigate how to improve end-to-end dialogue systems by augmenting them with commonsense knowledge, integrated in the form of external memory. The remainder of this paper is as follows: next section proposes related work in the context of conversational models and commonsense knowledge; following, a section describes the proposed model in detail; later, a section illustrates experimental results; finally, the last section proposes concluding remarks and future work.

Question: Which domain are the conversations in?
Answer: open-domain Twitter dialogues
Abstract: We report a series of experiments with different semantic models on top of various statistical models for extractive text summarization. Though statistical models may better capture word co-occurrences and distribution around the text, they fail to detect the context and the sense of sentences /words as a whole. Semantic models help us gain better insight into the context of sentences. We show that how tuning weights between different models can help us achieve significant results on various benchmarks. Learning pre-trained vectors used in semantic models further, on given corpus, can give addition spike in performance. Using weighing techniques in between different statistical models too further refines our result. For Statistical models, we have used TF/IDF, TextRAnk, Jaccard/Cosine Similarities. For Semantic Models, we have used WordNet-based Model and proposed two models based on Glove Vectors and Facebook's InferSent. We tested our approach on DUC 2004 dataset, generating 100-word summaries. We have discussed the system, algorithms, analysis and also proposed and tested possible improvements. ROUGE scores were used to compare to other summarizers.

Introduction: Automatic Text Summarization deals with the task of condensing documents into a summary, whose level is similar to a human-generated summary. It is mostly distributed into two distinct domains, i.e., Abstractive Summarization and Extractive Summarization. Abstractive summarization( Dejong et al. ,1978) involves models to deduce the crux of the document. It then presents a summary consisting of words and phrases that were not there in the actual document, sometimes even paraphrasing BIBREF1 . A state of art method proposed by Wenyuan Zeng BIBREF2 produces such summaries with length restricted to 75. There have been many recent developments that produce optimal results, but it is still in a developing phase. It highly relies on natural language processing techniques, which is still evolving to match human standards. These shortcomings make abstractive summarization highly domain selective. As a result, their application is skewed to the areas where NLP techniques have been superlative. Extractive Summarization, on the other hand, uses different methods to identify the most informative/dominant sentences through the text, and then present the results, ranking them accordingly. In this paper, we have proposed two novel stand-alone summarization methods.The first method is based on Glove Model BIBREF3 ,and other is based on Facebook's InferSent BIBREF4 . We have also discussed how we can effectively subdue shortcomings of one model by using it in coalition with models which capture the view that other faintly held.

Question: What showed to be the best performing combination of semantic and statistical model on the summarization task in terms of ROUGE score?
Answer: Best result was obtained by using combination of: Jaccard/Cosine Similarity Matrix, TextRank and InferSent Based Model
Abstract: Knowledge Graph Embedding methods aim at representing entities and relations in a knowledge base as points or vectors in a continuous vector space. Several approaches using embeddings have shown promising results on tasks such as link prediction, entity recommendation, question answering, and triplet classification. However, only a few methods can compute low-dimensional embeddings of very large knowledge bases without needing state-of-the-art computational resources. In this paper, we propose KG2Vec, a simple and fast approach to Knowledge Graph Embedding based on the skip-gram model. Instead of using a predefined scoring function, we learn it relying on Long Short-Term Memories. We show that our embeddings achieve results comparable with the most scalable approaches on knowledge graph completion as well as on a new metric. Yet, KG2Vec can embed large graphs in lesser time by processing more than 250 million triples in less than 7 hours on common hardware.

Introduction: Recently, the number of public datasets in the Linked Data cloud has significantly grown to almost 10 thousands. At the time of writing, at least four of these datasets contain more than one billion triples each. This huge amount of available data has become a fertile ground for Machine Learning and Data Mining algorithms. Today, applications of machine-learning techniques comprise a broad variety of research areas related to Linked Data, such as Link Discovery, Named Entity Recognition, and Structured Question Answering. The field of Knowledge Graph Embedding (KGE) has emerged in the Machine Learning community during the last five years. The underlying concept of KGE is that in a knowledge base, each entity and relation can be regarded as a vector in a continuous space. The generated vector representations can be used by algorithms employing machine learning, deep learning, or statistical relational learning to accomplish a given task. Several KGE approaches have already shown promising results on tasks such as link prediction, entity recommendation, question answering, and triplet classification BIBREF0 , BIBREF1 , BIBREF2 , BIBREF3 . Moreover, Distributional Semantics techniques (e.g., Word2Vec or Doc2Vec) are relatively new in the Semantic Web community. The RDF2Vec approaches BIBREF4 , BIBREF5 are examples of pioneering research and to date, they represent the only option for learning embeddings on a large knowledge graph without the need for state-of-the-art hardware. To this end, we devise the KG2Vec approach, which comprises skip-gram techniques for creating embeddings on large knowledge graphs in a feasible time but still maintaining the quality of state-of-the-art embeddings. Our evaluation shows that KG2Vec achieves a vector quality comparable to the most scalable approaches and can process more than 250 million triples in less than 7 hours on a machine with suboptimal performances.

Question: What is the new metric?
Answer: Neighbour Similarity Test (NST) and Type and Category Test (TCT)