Segmentation Using Relative Location Features

In this paper we present a generic document segmentation algorithm that uses Relative Location Features to encode the spatial relationships between the different image entities. We prove that using this features improve the final segmentation result on documents with a Manhattan structure, in addition to obtain results up to the state of the art in other reference tasks. Problem definition Document segmentation is a well-known task in the Document Analysis field which consists in the identification of the different entities or regions that form the document image. In terms of computer vision, we can define it as a labeling problem where we want to assign to each image pixel one label from a finite set of k possible labels L = {c1, . , ck}. Some common issues from this task come from the great variability in types of documents to process, besides than other features that make this problem even more challenging, as the possible degradation on ancient documents or complex layouts to segment. Given this situation, it is common to find segmentation works devised to a particular type of documents. Among the usual techniques, projection profiles have proved to obtain good results on documents with a manhattan layout, while methods based in texture analysis or connected components are more used in documents with a non-manhattan layout. In addition to these aproaches, it has been proved that the inclusion of contextual and spatial information into the models improves the quality of the results [1]. In this scope, graphical models as Conditional Random Fields (CRF) offer a powerful framework to represent these relationships. Proposal In this paper we propose a CRF framework in order to find the optimal labelling configuration for an input image I that maximizes the a posteriori probability P (C|I), being C the set composed of the label values corresponding to each pixel p in the image given the set of class labels L. Here, the use of a CRF allow us to encode pairwise dependences between neighbouring pixels and compute the previous probability distribution in terms of energy minimization as follows: P (C|I) = 1 Z exp −∑ i Di(ci)− ∑ {i,j}∈N Vi,j(ci, cj)  (1) where the term Di(ci) represents how well the label ci fits in the i-th pixel of the image I . The term Vi,j will represent the pairwise potential between the set N of interacting neighbor pixels, in other words, the penalty of assigning pixel labels ci and cj to pixels pi and pj , respectively. Our proposal is based in compute the term Di(ci) as a combination of both texture features and Relative Location Features (RLF) [1]. In the first place, we extract texture information by computing Gabor filter responses for several frequencies and orientations. With these features we train a Gaussian Mixture Model (GMM) which will provide us with an initial class prediction for each image pixel. In second place, in order to compute the RLF we learn a set of probability maps for each pair of classes, which encode the probability of finding items of one class in a particular location according to the relative position of elements of another class. Then, combining the initial class prediction with the information provided by the probability maps we are able to encode the RLF, which we will call vother and vself . Finally, we are able to compute the term Di(ci) from (1) as: Di(ci) = w app logP (ci|pi)+ +w ci log v other ci (pi)+w self ci log v self ci (pi), (2) where P (ci|Si) is the probability returned by the GMM and the weights wapp, wother ci and w self ci are learned using a logistic regression model from he training dataset. Once we have built the proposed CRF, we perform energy minimization using the Graph Cut algorithm [2]. The segmentation process and the combination of these features can be seen in Figure 1. Discussion We have performed experiments devised to check whether the inclusion of RLF helps to improve the segmentation results either in documents with a Manhattan and non-Manhattan structure, at the same time that evaluate the effectiveness of our method on both types of documents. We have use two datasets that fit to these requirements, the 5CofM dataset as a set with Manhattan layout, and the PRImA dataset as a benchmark set with a non-Manhattan layout. The results on 5CofM confirm that the use of the RLF significantly improves the segmentation results for all the considered class entities. However, in the case of the PRImA dataset the inclusion of the RLF does not produce significant improvements, but our method still achieves results up to the state of the art. As an ongoing work we are working in several aspects to improve the reported results. Our current research lines are focused in the exploration of other inference algorithms as Loopy Belief Propagation and the use of 2D-SCFG (Stochastic context-free grammar) to obtain the final result, as well as other ways to include more spatial information between the entities. Gabor features + GMM Relative Location Features Probability Maps Training Set CRF-based final segmentation Figure 1: Segmentation process using Relative Location Features.