# Understanding Unsupervised Deep Learning for Text Line Segmentation
## 6 Understanding the Unsupervised Segmentation
### What Is Being Clustered? We have discussed the model details for treating the training phase as a black box. However, the text line segmentation in fact occurs during the pseudo-RGB image generation phase. This phase clusters the central windows according to the features of their patches. These features are the frequently occurring salient parts of the text lines in that patch. They form a signal specific to the global text line pattern of the patch. The frequency and the amplitude of this signal should be the same for all the patches, as they are cropped from similar documents. However, the phase of the signal differs according to the global text line pattern in the patch. For example, the patches that contain a text line region at its top frame form a signal at a different phase than the patches that contain an interline region at its top frame (Figure 17). Hence, the central windows are clustered into two clusters, one corresponds to the text line regions and the other corresponds to the interline regions (Figure 17). We have constructed an adversarial dataset to reveal how the pseudo-RGB image generation phase functions. This dataset contains digitally printed text lines with varying font sizes and interline spaces. Let this dataset be called the _heterogeneous printed_ dataset. We also constructed a homogeneous printed dataset that contains digitally printed text lines with homogeneous interline spaces. The network trained on the heterogeneous printed dataset and tested on both, the heterogeneous and the homogeneous. We observe that the network learns the frequently occurring salient features, therefore it can cluster the central windows of a homogeneous document into text line regions and interline regions. The network can also cluster the central windows of a heterogeneous document, however, into an indeterministic number of clusters (Figure 18). Therefore, we cannot threshold the blob lines conveniently. Figure 15: Visualization of the impact of network depth on the receptive field and the results. Each column shows the pseudo-RGB image (**top**) of the same document and visualizes the salient features of a patch from the document (**bottom**). The number of layers indicates the number of convolutional layers. The five-layer network is the AlexNet and the 13-layer network is the VGG16 network. Deeper networks have wider receptive fields and shallower networks have narrower receptive fields. All networks from the three layers up to 13 layers can be used to segment text lines. We can see that in all the pseudo-RGB images, the text lines are clearly marked. However, the quality of the text line markings declines as the number of layers increases. Furthermore, in salient feature visualization, we can see that the networks encode more complex features as the number of layers increases. We propose the use of component tree binarization that searches all possible thresholds and stops the search at the pixels once a blob line appears on them. The algorithm decides whether a set of pixels are a blob line or not according to the ligature score defined in Section 4.5.1. The resultant blob lines do not always strike through the text lines, as can be seen in Figure 10, where the blob line image contains more lines than the document image. We eliminate the spurious blob lines by adding the label cost to the energy function given in Equation (2), producing:  
\[E(f)=\sum_{c\in\mathcal{C}}D(c,\ell_{c})+\sum_{\{c,c^{\prime}\}\in\mathcal{N}} d(c,c^{\prime})\cdot\delta(\ell_{c}\
eq\ell_{c^{\prime}})+\sum_{\ell\in \mathcal{L}}h_{\ell} \tag{5}\]  
For every blob line \(\ell\in\mathcal{L}\), \(h_{\ell}\) is defined as \(\exp(-\alpha\cdot r_{\ell})\) where \(r_{\ell}\) is the normalized number of foreground pixels overlapping with blob line \(\ell\). Figure 16: Visualization of the resulting salient features using ResNet50. Patch saliency shows that the features from text lines and interline regions overlap, thus indicating that the network’s perception field is wide.