Evaluating SequencetoSequence Models for Handwritten Text Recognition Encoderdecoder models have become an effective approach for sequence learning tasks like machine translation, image captioning and speech recognition, but have yet to show competitive results for handwritten text recognition. To this end, we propose an attentionbased sequencetosequence model. It combines a convolutional neural network as a generic feature extractor with a recurrent neural network to encode both the visual information, as well as the temporal context between characters in the input image, and uses a separate recurrent neural network to decode the actual character sequence. We make experimental comparisons between various attention mechanisms and positional encodings, in order to find an appropriate alignment between the input and output sequence. The model can be trained endtoend and the optional integration of a hybrid loss allows the encoder to retain an interpretable and usable output, if desired. We achieve competitive results on the IAM and ICFHR2016 READ data sets compared to the stateoftheart without the use of a language model, and we significantly improve over any recent sequencetosequence approaches.

A Comparative Study of AttentionBased EncoderDecoder Approaches to Natural Scene Text Recognition Attentionbased encoderdecoder approaches have shown promising results in scene text recognition. In the literature, models with different encoders, decoders and attention mechanisms have been proposed and compared on isolated word recognition tasks, where the models are trained on either synthetic word images or a small set of realworld images. In this paper, we investigate different components of the attention based framework and compare its performance with a CNNDBLSTMCTC based approach on largescale realworld scene text sentence recognition tasks. We train character models by using more than 1.6M realworld text lines and compare their performance on test sets collected from a variety of realworld scenarios. Our results show that (1) attention on a twodimensional feature map can yield better performance than onedimensional one and an RNN based decoder performs better than CNN based one; (2) attentionbased approaches can achieve higher recognition accuracy than CNNDBLSTMCTC based approaches on isolated word recognition tasks, but perform worse on sentence recognition tasks; (3) it is more effective and efficient for CNNDBLSTMCTC based approaches to leverage an explicit language model to boost recognition accuracy.