In this task, you are given a part of an article. Your task is to generate headline (title) for this text. Preferred headlines are under fifteen words.

Q: Most existing Neural Machine Translation models use groups of characters or whole words as their unit of input and output. We propose a model with a hierarchical char2word encoder, that takes individual characters both as input and output. We first argue that this hierarchical representation of the character encoder reduces computational complexity, and show that it improves translation performance. Secondly, by qualitatively studying attention plots from the decoder we find that the model learns to compress common words into a single embedding whereas rare words, such as names and places, are represented character by character.

A: NEURAL MACHINE TRANSLATION WITH CHARACTERS AND HIERARCHICAL ENCODING
****
Q: Common activation functions used in neural networks can yield to training difficulties due to the saturation behavior of the activation function, which may hide dependencies which are not visible to first order (using only gradients). Gating mechanisms that use softly saturating activation functions to emulate the discrete switching of digital logic circuits are good examples of this. We propose to exploit the injection of appropriate noise so that some gradients may sometimes flow, even if the noiseless application of the activation function would yield zero gradient. Large noise will dominate the noise-free gradient and allow stochastic gradient descent to be more exploratory. By adding noise only to the problematic parts of the activation function we allow the optimization procedure to explore the boundary between the degenerate (saturating) and the well-behaved parts of the activation function. We also establish connections to simulated annealing, when the amount of noise is annealed down, making it easier to optimize hard objective functions. We find experimentally that replacing such saturating activation functions by by noisy variants helps training in many contexts, yielding state-of-the-art results on several datasets, especially when training seems to be the most difficult, e.g., when curriculum learning is necessary to obtain good results.

A: Noisy Activation Functions
****
Q: Until now, error type performance for Grammatical Error Correction (GEC) systems could only be measured in terms of recall because system output is not annotated. In this paper, we overcome this problem by using a linguisticallyenhanced alignment to automatically extract the edits between parallel original and corrected sentences and then classify them using a new dataset-independent rule-based classifier. As human experts rated the predicted error types as “Good” or “Acceptable” in at least 95% of cases, we applied our approach to the system output produced in the CoNLL-2014 shared task to carry out a detailed analysis of system error type performance for the first time.

A:
Automatic Annotation and Evaluation of Error Types for Grammatical Error Correction
****