# Understanding Unsupervised Deep Learning for Text Line Segmentation
## 4 Method
### Training  
The training phase produces a feature vector fc2 (Figure 3) of size 512 from the raw pixels of input images using a deep convolutional network. Input images are patches of size \(p\times p\) cropped from the document images. We train the network to reason about the similarity of the coarse trend of text lines between a pair of patches. The supervising signal is obtained from the assumption that identical or neighbouring patches (adjacent patches on the top, bottom, left, and right) contain similar coarse text line patterns, and contain different coarse text line patterns when one of them is rotated 90 degrees. The effective execution of this task requires the network to recognize the salient parts of text lines. During blob detection, the feature vectors of document patches are used to discriminate between differences in text pattern contained within the input patches.  
For the training phase, a pair of convolutional neural network (CNN) branches with identical weights are used, such that the feature vector is computed for both patches using the same function (Figure 3). However, the reasoning for each patch is performed separately through the CNN branch. Consequently, the feature vectors are concatenated and fed to the decision branch for predicting whether two image patches are similar or different in terms of the coarse text line patterns. All the fully connected layers are followed by a ReLU activation function, apart from fc5, which feeds into a sigmoid binary classifier.  
Figure 2: An overview of the method for binary and colour document images. Given a handwritten document image, the first phase, a feature vector for each image patch is formed such that, patches with similar global textual patterns are close together in the feature space. The second phase extracts the first three principal components from the feature vectors of image patches using PCA. The three principal components are interpreted as pseudo-RGB colours, producing a pseudo-RGB image. For binary document images, the third phase thresholds the pseudo-RGB image into binary blob lines that strike through the text lines. The binary blob lines assist an energy minimization function for extracting the pixel labels. Meanwhile, the third phase for colour document images transforms the pseudo-RGB image into an energy map based on grey level. The energy map assists a seam carving function for detecting the baselines.  
The model is trained on each dataset from scratch using \(N_{p}\) pairs:  
\[N_{p}=\frac{H_{a}\times W_{a}}{p\times p}\times N_{d} \tag{1}\]  
where \(w_{a}\) and \(H_{a}\) are the average document width and height. \(p\) is the size of the patch, and the number of documents in the dataset is denoted as \(N_{d}\). We set the training learning rate equal to \(1\times 10^{-5}\), the batch size equal to eight, and employed Adam as the optimization algorithm. The model was trained until the validation accuracy ceased improving. The best performing model on the validation subset is saved for the next phase.  
Pair Generation  
The proposed method basically assumes that a document image patch is similar to its identity or neighbours in terms of the text line pattern they contain. On the other hand, a patch is different from its 90-degree-rotated identity or neighbours in terms of the text line pattern they contain (Figure 1). In cases where the input document images contain almost horizontal text lines, the similarity and difference assumptions can be augmented by 180-degree rotation of one of the pairs or by flipping it horizontally. The similarity does not correlate with the proximity when there are fluctuating or skewed text lines; however, this represents a rare occurrence in document images composed almost entirely of horizontal text lines.