In this task, you are given a part of an article. Your task is to generate headline (title) for this text. Preferred headlines are under fifteen words.
--------
Question: We investigate evaluation metrics for endto-end dialogue systems where supervised labels, such as task completion, are not available. Recent works in end-to-end dialogue systems have adopted metrics from machine translation and text summarization to compare a model’s generated response to a single target response. We show that these metrics correlate very weakly or not at all with human judgements of the response quality in both technical and non-technical domains. We provide quantitative and qualitative results highlighting specific weaknesses in existing metrics, and provide recommendations for future development of better automatic evaluation metrics for dialogue systems.

Answer: How NOT To Evaluate Your Dialogue System: An Empirical Study of Unsupervised Evaluation Metrics for Dialogue Response Generation


Question: Topic models provide a useful method for dimensionality reduction and exploratory data<lb>analysis in large text corpora. Most approaches to topic model inference have been based on<lb>a maximum likelihood objective. Efficient algorithms exist that approximate this objective,<lb>but they have no provable guarantees. Recently, algorithms have been introduced that provide<lb>provable bounds, but these algorithms are not practical because they are inefficient and not ro-<lb>bust to violations of model assumptions. In this paper we present an algorithm for topic model<lb>inference that is both provable and practical. The algorithm produces results comparable to the<lb>best MCMC implementations while running orders of magnitude faster.

Answer: A Practical Algorithm for Topic Modeling with Provable Guarantees


Question: Documents exhibit sequential structure at multiple levels of abstraction (e.g., sentences, paragraphs, sections). These abstractions constitute a natural hierarchy for representing the context in which to infer the meaning of words and larger fragments of text. In this paper, we present CLSTM (Contextual LSTM), an extension of the recurrent neural network LSTM (Long-Short Term Memory) model, where we incorporate contextual features (e.g., topics) into the model. We evaluate CLSTM on three specific NLP tasks: word prediction, next sentence selection, and sentence topic prediction. Results from experiments run on two corpora, English documents in Wikipedia and a subset of articles from a recent snapshot of English Google News, indicate that using both words and topics as features improves performance of the CLSTM models over baseline LSTM models for these tasks. For example on the next sentence selection task, we get relative accuracy improvements of 21% for the Wikipedia dataset and 18% for the Google News dataset. This clearly demonstrates the significant benefit of using context appropriately in natural language (NL) tasks. This has implications for a wide variety of NL applications like question answering, sentence completion, paraphrase generation, and next utterance prediction in dialog systems.

Answer:
Contextual LSTM (CLSTM) models for Large scale NLP tasks