Analysis of Unsupervised Training Approaches for LSTMBased OCR In the context of historical documents, where labeled training data is especially expensive to acquire, the prospect of using unlabeled training data to improve and speed up the overall training is desirable. The most common way to use unlabeled data is unsupervised pretraining, which has been successfully applied to various CNN and RNN architectures in different domains. There is however not sufficient work for its application in the field of OCR. In this paper we investigate multiple architectures and how unlabeled data could be applied to them. We show that in combination with Connectionist Temporal Classification (CTC), a reconstruction objective has no apparent synergistic effect, with both objectives learning different representations. We therefore investigate the use of an LSTMbased Seq2Seq OCR architecture which shows promise regarding unsupervised pretraining.

Hybrid Training Data for Historical Text OCR Current optical character recognition (OCR) systems commonly make use of recurrent neural networks (RNN) that process whole text lines. Such systems avoid the task of character segmentation necessary for characterbased approaches. A disadvantage of this approach is a need of a large amount of annotated data. This can be solved by sing generated synthetic data instead of costly manually annotated ones. Unfortunately, such data is often not suitable for historical documents particularly for quality reasons. This work presents a hybrid approach for generating annotated data for OCR at a low cost. We first collect a small dataset of isolated characters from historical document images. Then, we generate historical looking text lines from the generated characters. Another contribution lies in the design and implementation of an OCR system based on a convolutionalLSTM network. We first pretrain this system on hybrid data. Afterwards, the network is finetuned with real printed text lines. We demonstrate that this training strategy is efficient for obtaining stateoftheart results. We also show that the score of the proposed system is comparable or even better in comparison to several stateoftheart systems.