In this task, you are given a part of an article. Your task is to generate headline (title) for this text. Preferred headlines are under fifteen words.

Ex Input:
Fixed-vocabulary language models fail to account for one of the most characteristic statistical facts of natural language: the frequent creation and reuse of new word types. Although character-level language models offer a partial solution in that they can create word types not attested in the training corpus, they do not capture the “bursty” distribution of such words. In this paper, we augment a hierarchical LSTM language model that generates sequences of word tokens character by character with a caching mechanism that learns to reuse previously generated words. To validate our model we construct a new open-vocabulary language modeling corpus (the Multilingual Wikipedia Corpus; MWC) from comparable Wikipedia articles in 7 typologically diverse languages and demonstrate the effectiveness of our model across this range of languages.

Ex Output:
Learning to Create and Reuse Words in Open-Vocabulary Neural Language Modeling


Ex Input:
Recurrent neural network (RNN) based character-level language models (CLMs) are extremely useful for modeling unseen words by nature. However, their performance is generally much worse than the word-level language models (WLMs), since CLMs need to consider longer history of tokens to properly predict the next one. We address this problem by proposing hierarchical RNN architectures, which consist of multiple modules with different clock rates. Despite the multiclock structures, the input and output layers operate with the character-level clock, which allows the existing RNN CLM training approaches to be directly applicable without any modifications. Our CLM models show better perplexity than KneserNey (KN) 5-gram WLMs on the One Billion Word Benchmark with only 2% of parameters. Also, we present real-time character-level end-to-end speech recognition examples on the Wall Street Journal (WSJ) corpus, where replacing traditional mono-clock RNN CLMs with the proposed models results in better recognition accuracies even though the number of parameters are reduced to 30%.

Ex Output:
Character-Level Language Modeling with Hierarchical Recurrent Neural Networks


Ex Input:
This paper presents a system which creates and visualizes probabilistic semantic links between concepts in a thesaurus and classes in a classification system. For creating the links, we build on the Polylingual Labeled Topic Model (PLL-TM) [6]. PLL-TM identifies probable thesaurus descriptors for each class in the classification system by using information from the natural language text of documents, their assigned thesaurus descriptors and their designated classes. The links are then presented to users of the system in an interactive visualization, providing them with an automatically generated overview of the relations between the thesaurus and the classification system. Lisa Posch GESIS – Leibniz Institute for the Social Sciences Cologne, Germany Institute for Web Science and Technologies University of Koblenz-Landau, Germany E-mail: lisa.posch@gesis.org Philipp Schaer GESIS – Leibniz Institute for the Social Sciences Cologne, Germany E-mail: philipp.schaer@gesis.org Arnim Bleier GESIS – Leibniz Institute for the Social Sciences Cologne, Germany E-mail: arnim.bleier@gesis.org Markus Strohmaier GESIS – Leibniz Institute for the Social Sciences Cologne, Germany Institute for Web Science and Technologies University of Koblenz-Landau, Germany E-mail: markus.strohmaier@gesis.org

Ex Output:

A System for Probabilistic Linking of Thesauri and Classification Systems