# Understanding Unsupervised Deep Learning for Text Line Segmentation
###### Abstract  
We propose an unsupervised feature learning approach for segmenting text lines of handwritten document images with no labelling effort. Humans can easily group local text line features to global coarse patterns. We leverage this coherent visual perception of text lines as a supervising signal by formulating the feature learning as a global pattern differentiation task. The machine is trained to detect whether a document patch contains a similar global text line pattern with its identity or neighbours, and a different global text line pattern with its 90-degree-rotated identity or neighbours. Clustering the central windows of document image patches using their extracted features, forms blob lines which strike through the text lines. The blob lines guide an energy minimization function for extracting text lines in a binary image and guide a seam carving function for detecting baselines in a colour image. In identifying the aspect of the input patch that supports the actual prediction and clustering, we contribute toward the understanding of input patch functionality. We evaluate the method on several variants of text line segmentation datasets to demonstrate its effectiveness, visualize what it has learned, and enable it to comprehend its clustering strategy from a human perspective.  
text line segmentation; text line extraction; text line detection; unsupervised deep learning +
Footnote †: journal: Medical  
interrupted as pseudo-RGB colours, which mark the text line texture, the interline space texture, and the background texture. The pseudo-RGB image is thresholded to yield binary blob lines which strike through the text lines for binary input images. Meanwhile, for colour images, the pseudo-RGB image is transformed into an energy map that assists a seam carving function in extracting the baselines.  
The proposed method trains the model to discriminate global text line patterns, but leverages the patches' features for clustering their central windows. The question is: Which features guide the clustering central windows of the patches with similar patterns together? To provide a better understanding of the learned features, we visualize that the prevalent salient local parts contribute the most to the similarity between a pair. Hence, each document image patch contains a specific sequence of these features according to the global text line pattern they contain. This means the features from the penultimate fully connected layer embeds the document image patches to a compact space where distances correspond to a global text line pattern similarity.  
Part of the presented work in this paper has been published [11] for increased visibility. This paper is extended by the inclusion of a specific method for colour document images, addition of understanding regarding the method's behaviour, extensive experimental evaluation, and comparison with state-of-the-art methods.  
The paper is organized as follows. Section 2 provides a summary of the related work. In Section 3, the datasets used in our experiments are shown. The method is proposed in Section 4. In Section 5, a baseline experiment, ablation studies, and results are presented. Experiments to produce an understanding of the method's functionality are reported in Section 6. Finally, Section 7 provides concluding remarks and describes plans for future work.