# Understanding Unsupervised Deep Learning for Text Line Segmentation
## 2 Related Work  
Deep learning of handwritten text line segmentation has a recent history, starting from the Recurrent Neural Network (RNN) work of [PERSON] et al. [12]. The RNN is trained on document image columns as a vertical sequence of two labels, text line and interline. The sequential characteristic of RNN limits the input to horizontal text lines. Therefore, researchers have started focusing on the layout agnostic formulation of text line segmentation.  
Essentially, two main solutions from computer vision were considered, semantic segmentation using a Fully Convolutional Network (FCN) [13] and instance segmentation using a Mask-RCNN [14]. Instance segmentation labels each foreground pixel with an  
Figure 1: In the proposed method, a machine learning model is trained to determine whether two patches are identical or are the same when one patch is rotated 90 degrees. The network analyses the most frequent and salient features to reason the text line pattern similarity.  
object and instance. This can easily be adapted for the text line segmentation problem by labelling each foreground pixel with an instance of text line class [15]. However, this is not the case for semantic segmentation, which labels each pixel with an object and all its instances proceed with the same label. Therefore, the potential formulation has to define which pixels belong to a text line and predict whether or not a pixel is a text line pixel. Defining a text line with its raw foreground pixels would yield semantic segmentation to be paradoxically ineffective because the output will not discriminate a text line from the others. This inherent limitation has forced semantic segmentation methods to define a text line as a single component. There are two variants of this component: (1) Blob lines [15; 16; 17; 18; 19], a connected component that passes through the middle of the text line's characters. (2) A baseline [2], a line that passes through the bottom of the text lines' characters. However, both baselines and the blob lines extract the spatial location of the text lines, and do not label the text lines' pixels fully. Both variants can be evaluated by text line detection metrics for baselines [17; 19; 20] or blob lines [18]. The extraction methods of baselines and blob lines can be extended to allow segmentation of the text lines [15; 16]. Reference [16] is limited to the extraction of horizontal text lines only, whereas [15] can manage text lines with any orientation, script type, and font size. Such methods are evaluated using classical image segmentation metrics [20].  
Deep learning methods are very successful at segmenting handwritten text lines with different script types, sizes, and orientations, but require a large dataset of labelled text lines in order to learn comprehensive features directly from the pixels of the image. For example, rarely occurring curved text lines are poorly detected, as they are not well represented in the training data. This problem was addressed via augmentation [2], learning-free detection [21] or hybrid methods [3]. Alternatively, [PERSON] et al. [22] propose an unsupervised deep learning algorithm trained to predict whether a document patch contains a text line or interline. The label is based on an adjustable score and tuned by a human. Our method eliminates this score and assumes that a document image patch contains a similar global text line pattern with its identity or neighbours and a different global text line pattern with its 90-degree-rotated identity or neighbours.