# Image Segmentation by Edge Partitioning over a Nonsubmodular Markov Random Field

Authors: Ho Yub Jung; Kyoung Mu Lee
Journal: Mathematical Problems in Engineering
 (2015)

---

## Abstract

Edge weight-based segmentation methods, such as normalized cut or minimum cut, require a partition number specification for their energy formulation. The number of partitions plays an important role in the segmentation overall quality. However, finding a suitable partition number is a nontrivial problem, and the numbers are ordinarily manually assigned. This is an aspect of the general partition problem, where finding the partition number is an important and difficult issue. In this paper, the edge weights instead of the pixels are partitioned to segment the images. By partitioning the edge weights into two disjoints sets, that is, cut and connect, an image can be partitioned into all possible disjointed segments. The proposed energy function is independent of the number of segments. The energy is minimized by iterating the QPBO-α-expansion algorithm over the pairwise Markov random field and the mean estimation of the cut and connected edges. Experiments using the Berkeley database show that the proposed segmentation method can obtain equivalently accurate segmentation results without designating the segmentation numbers.


---

## Body

## 1. Introduction

There are numerous approaches and applications for unsupervised image segmentation in computer vision. Many different theories are proposed for varying the roles of the unsupervised segmentation. As a low level vision problem, an image can be simplified by oversegmentation using a number of different approaches, such as mode-seeking mean shift, multilevel thresholding, histogram-based neural networks, superpixel algorithms, and various graph-based methods [1–4]. Conversely, semantic segmentation is attempted for simultaneous detection, recognition, and segmentation [5].Generally, the role of unsupervised segmentation falls between image simplification and full semantic segmentation, where semantically meaningful segments are expected to be found but not necessarily recognized. Segmentation is posed as an image-coloring problem that minimizes specific energy functions. Energy functions can be optimized using stochastic methods such as deterministic annealing and stochastic clustering [6–10]. For graph theoretic segmentation approaches, the spectral method and graph cut are efficient deterministic optimization methods [11–13]. Another traditional segmentation method is the variational method, which evolves boundary contours in a level set framework [14, 15].The edge weight-based segmentation methods have evolved together with graph partition problems. When edge weights are all positive, the minimum cut can be found; however, the minimum cut has bias toward smaller cuts. Adding negative edge weights can prevent the problem so the graph becomes nonsubmodular; however, the problem becomes NP-hard [16]. Different algorithms have been introduced to estimate the correlation in clustering problem [17, 18]. In contrast, Shi and Malik normalized nonnegative edge weights so the bias toward smaller cuts was eliminated [11].For the graph theoretic segmentation and level set methods, the number of segments must be predefined. The segment number choice greatly influences the quality of segmentation, especially for a normalized cut. Nonetheless, there have been attempts to solve this problem. The number of segments can be controlled by setting the threshold value to the recursive normalized cut [11]. For level set approaches, a four-color theorem was used to segment images with an arbitrary number of phases with one or two level set functions [19]. However, these methods are still functions of K, the number of segments.In this paper, transforming the pixel clustering problem into an edge partition problem circumvents the segment number selection problem. Edges among adjacent pixels can represent dissimilarity or similarity weights. Two edge partitions are always sufficient for pixel-partitioning problems. An edge can be in a cut set or connected set, which can then be translated into a unique segmentation, as in Figure1(c). The cut edges indicate that the two node labels are different, whereas the connected edges indicate that two nodes have the same labels. In most cases, however, the cut or connect assignments on the edges are not enough to define a specific segmentation configuration, as in Figure 1(d). Random cut and connect assignments on the edges may result in contradiction of the node labels. However, under the pixel coloring framework, cut and connect assignments on the edges are defined concurrently with pixel labels, and inconsistencies, such as those in Figure 1(d), are prevented.Figure 1
An image can be segmented by partitioning edges into two sets. Cut (dotted red) and connected (solid black) edge sets can be translated into a unique segmentation as in (c). However, it is also possible to have edge partitions that contradict the label assignments as in (d). By finding the image labeling that minimizes the edge partition energy, edge partitions like (d) are prevented, and a consistent image segmentation becomes possible as shown in (a) and (b).
(a)
(b)
(c)
(d)Under the pixel-labeling framework, a label number selection problem arises. Although the label number selection might seem similar to the segment number selection problem, there are subtle differences. First, pixels do not need to use all label assignments; thus, low numbers of segments are possible with large numbers of labels. Second, under the four-color map theorem, the maximum number of labels for two-dimensional (2D) segmentation can be as low as four. The four-color map theorem states that any 2D map can be colored with intact borders using a maximum of four colors [26]. This theorem can be translated directly to the segmentation problem; any 2D image segmentation can be represented using four labels [19].In the following sections, a new energy function is introduced for image segmentation through the edge partition. The edge partitions can uniquely define the image segmentation with the hard constraints enforced by the image-labeling framework. Next, an energy minimization algorithm is proposed for the edge partitioning. The experimental section discusses tests of the proposed algorithm using the Berkeley image segmentation database.


## 2. Pixel Clustering

Image segmentation can be viewed as a pixel-partitioning problem. Many image segmentation methods borrow their ideas from the general partitioning techniques. TheK-means algorithm minimizes the following function and segments the image into K regions. Consider(1)Ek=∑i=1k∑wj∈Siwj-μi2,where wj is the pixel feature value and μi is the mean values of the respective partition Si. The K-means algorithm minimizes the sum of the squared distance from the mean of each partition. The energy function must have a fixed segmentation number. However, estimating the number of segments is a difficult task, and the number of partitions is often designated by human discretion.


## 3. Edge Partitions

An image can be represented as a set of nodes and edges by a graphG=(V,E). An edge (u,v)∈E is assigned with weight w between nodes u,v∈V. For each node v∈V, a label from xv∈{1,2⋯} is assigned to define a segmentation.


### 3.1. Energy Function

The segmentation problem is formulated in terms of edge partitions. The edges can be partitioned into two setsCoff (cut) and Con (connect), such that Coff∪Con=E and Coff∩Con=∅. If an edge is in Con, the pixel nodes connected by the edge have the same label. Otherwise, if an edge is in Coff, the pixel nodes connected by the edge have a different label(2)Coff=wu,v∈Es.txu≠xv,Con=wu,v∈Es.txu=xv.In (2), w(u,v) is an edge between pixel nodes u and v. The pixel labels for pixels u and v are denoted by xu and xv, respectively. w(u,v) is a positive edge weight between pixel nodes u and v. w(u,v) can be a similar or dissimilar measure between the two pixel nodes. A simple example of w(u,v) is the absolute difference between two pixel colors. Thus, if the colors between the two pixels have a large difference, the edge will likely be in Coff. If the two pixel colors have a small difference, the edge should be in Con. The mean edge weight values of the Con and Coff edge sets are found in the following equation:(3)μCoff=∑u,v∈Coffwu,vCoff,μCon=∑u,v∈Conwu,vCon,and then the energy function associated with the edge partitions can be defined by following equation:(4)E=∑u,v∈Coffwu,v-μCoff2+∑u,v∈Conwu,v-μCon2.· is the cardinality of the set. The energy function (4) is the same as the K-means algorithm in (1) except that the number of partitions is set to K=2. The proposed energy function has two mean centers, but it also has hard constraints in (2). Regardless of the segmentation number, there can only be two partitions for the edges, cut Coff and connected edges Con.The proposed energy function breaks down into an image-labeling problem in order to maintain the label consistency conditions of (2). The image label state x=(x1,x2,…,xV) that minimizes (4) under (3) and (2) constraints is the proposed segmentation state. The number of labels must be at least two to avoid division by zero in (3). Under the well-known four-color map theorem, four labels {1,2,3,4} are sufficient to define all possible segment configurations for 2D images [19].


### 3.2. Optimization

Given the image label statex, the mean values μCoff, μCon can be estimated as in (3). Otherwise, if μCoff and μCon are kept constant, the image label state x can be found by optimizing the following pairwise energy function:(5)EL=∑u,v∈Eθuvxu,xv,(6)θuvxu,xv=wu,v-μCoff2ifxu≠xv,θuvxu,xv=wu,v-μCon2ifxu=xv.If the labels between edges are not the same, the edge is considered to be in the Coff cut set; otherwise, it belongs to the Con connected set. With μCoff and μCon constants, minimizing (5) is equivalent to minimizing the edge partition function (4).The multilabel pairwise energy function (5) can be solved by QPBO-α-expansion. QPBO-α-expansion optimizes the multilabel MRFs by iteratively expanding a single label using graph cut [27]. Graph cut can find the optimal expansion if the expansion is submodular. In this problem, the expansions are nonsubmodular. The pairwise potentials for QPBO-α-expansion, where xu,xv∈x is the current label state, can be defined as follows:(7)θuvxu,xv=wu,v-μCoff2ifxu≠xv,θuvxu,xv=wu,v-μCon2ifxu=xv,θuvxu,αv=wu,v-μCoff2ifxu≠αv,θuvxu,αv=wu,v-μCon2ifxu=αv,θuvαu,xv=wu,v-μCoff2ifαu≠xv,θuvαu,xv=wu,v-μCon2ifαu=xv,θuvαu,αv=wu,v-μCon2.This nonsubmodular binary labeling problem can be approached using the QPBO algorithm [28] with the possibility of a large number of unlabeled nodes. Recently introduced, QPBO improve (QPBOI) algorithm can cope with unlabeled regions [28]; however, this algorithm is not as efficient as the graph cut which minimizes the submodular potentials. The QPBOI algorithm can randomly improve the solution, but iterations of the improved steps can be time-consuming for large numbers of nodes.Similar to the originalK-means algorithm, good initialization is helpful to the optimization. The initial estimation of the means, μCoff and μCon, can be found by a K-means algorithm minimization of edge partitions (4) without the labeling constraint of (2). To estimate the initial state x, the pixel clustering K-means algorithm (1) can be used. The general framework is illustrated in Algorithm 1.Algorithm 1:Minimizing edge partition.
(1)   Estimate image label state x with the K-means algorithm on pixels, Kp=4.
(2)   Estimate μCoff and μCon by the K-means algorithm minimization of (4)
without the labeling constraint of (2).
(3)    Estimate the image label state x using the QPBOI-α-expansion.
(keep μCoff and μCon constant)
(4)    Estimate μCoff and μCon from the image label state x.
(5)   If μCoff and μCon are unchanged, terminate.
Else, repeat steps 3 and 4.


### 3.3. Edge Weights

Various examples of the edge partition segmentation results using the color distance edge weights are shown in Figure2 for the MSRC image database [29]. The color distance from the neighboring pixels is sufficient for some image segmentation problems, but more rigorous weight calculations are often suited for semantic segmentation. Instead of proposing new edge weight calculations, an existing state-of-the-art contour detection algorithm is incorporated.Figure 2
Even with the simple color distance weight, the edge partitions can produce adequate segmentation results. Some of the segmentation results from the MSRC database are shown.The global probability of the boundary (GPB) edge detection method [25, 30], which scored best for the Berkeley database (http://www.cs.berkeley.edu/projects/vision/bsds), is employed as the edge weights. The edge weights can be connected between the pixel nodes, and the proposed edge partitioning algorithm can be implemented. Figure 3 shows the other segmentation results under the pixel-to-pixel edge connections. Although Figures 3(a) and 3(b) show a good segmentation result, the QPBOI algorithm cannot obtain a good segmentation in Figures 3(c) and 3(d). The QPBOI algorithm often fails in the presence of a large number of nodes. Thus, to reduce both the computational time and the chance of failure in the QPBOI algorithm, the oversegmentation process is adopted from [25] in this segmentation. The edges are connected between the superpixels instead of the pixels. The number of oversegments is between 400 and 1000. The edge partitioning algorithm segments a BSDS image average in under 5 seconds.Figure 3
The QPBOI optimization scheme can be efficient for (a) and (b). However, for many cases, the nonsubmodular potentials are too strong, and the QPBOI optimization fails in (c) and (d). More iterations of QPBOI can improve the result; however, iterations are time-consuming without improvement guarantees. Superpixel images are used in this study to reduce the computation time and increase the QPBOI iterations.
(a)
(b)
(c)
(d)


## 3.1. Energy Function

The segmentation problem is formulated in terms of edge partitions. The edges can be partitioned into two setsCoff (cut) and Con (connect), such that Coff∪Con=E and Coff∩Con=∅. If an edge is in Con, the pixel nodes connected by the edge have the same label. Otherwise, if an edge is in Coff, the pixel nodes connected by the edge have a different label(2)Coff=wu,v∈Es.txu≠xv,Con=wu,v∈Es.txu=xv.In (2), w(u,v) is an edge between pixel nodes u and v. The pixel labels for pixels u and v are denoted by xu and xv, respectively. w(u,v) is a positive edge weight between pixel nodes u and v. w(u,v) can be a similar or dissimilar measure between the two pixel nodes. A simple example of w(u,v) is the absolute difference between two pixel colors. Thus, if the colors between the two pixels have a large difference, the edge will likely be in Coff. If the two pixel colors have a small difference, the edge should be in Con. The mean edge weight values of the Con and Coff edge sets are found in the following equation:(3)μCoff=∑u,v∈Coffwu,vCoff,μCon=∑u,v∈Conwu,vCon,and then the energy function associated with the edge partitions can be defined by following equation:(4)E=∑u,v∈Coffwu,v-μCoff2+∑u,v∈Conwu,v-μCon2.· is the cardinality of the set. The energy function (4) is the same as the K-means algorithm in (1) except that the number of partitions is set to K=2. The proposed energy function has two mean centers, but it also has hard constraints in (2). Regardless of the segmentation number, there can only be two partitions for the edges, cut Coff and connected edges Con.The proposed energy function breaks down into an image-labeling problem in order to maintain the label consistency conditions of (2). The image label state x=(x1,x2,…,xV) that minimizes (4) under (3) and (2) constraints is the proposed segmentation state. The number of labels must be at least two to avoid division by zero in (3). Under the well-known four-color map theorem, four labels {1,2,3,4} are sufficient to define all possible segment configurations for 2D images [19].


## 3.2. Optimization

Given the image label statex, the mean values μCoff, μCon can be estimated as in (3). Otherwise, if μCoff and μCon are kept constant, the image label state x can be found by optimizing the following pairwise energy function:(5)EL=∑u,v∈Eθuvxu,xv,(6)θuvxu,xv=wu,v-μCoff2ifxu≠xv,θuvxu,xv=wu,v-μCon2ifxu=xv.If the labels between edges are not the same, the edge is considered to be in the Coff cut set; otherwise, it belongs to the Con connected set. With μCoff and μCon constants, minimizing (5) is equivalent to minimizing the edge partition function (4).The multilabel pairwise energy function (5) can be solved by QPBO-α-expansion. QPBO-α-expansion optimizes the multilabel MRFs by iteratively expanding a single label using graph cut [27]. Graph cut can find the optimal expansion if the expansion is submodular. In this problem, the expansions are nonsubmodular. The pairwise potentials for QPBO-α-expansion, where xu,xv∈x is the current label state, can be defined as follows:(7)θuvxu,xv=wu,v-μCoff2ifxu≠xv,θuvxu,xv=wu,v-μCon2ifxu=xv,θuvxu,αv=wu,v-μCoff2ifxu≠αv,θuvxu,αv=wu,v-μCon2ifxu=αv,θuvαu,xv=wu,v-μCoff2ifαu≠xv,θuvαu,xv=wu,v-μCon2ifαu=xv,θuvαu,αv=wu,v-μCon2.This nonsubmodular binary labeling problem can be approached using the QPBO algorithm [28] with the possibility of a large number of unlabeled nodes. Recently introduced, QPBO improve (QPBOI) algorithm can cope with unlabeled regions [28]; however, this algorithm is not as efficient as the graph cut which minimizes the submodular potentials. The QPBOI algorithm can randomly improve the solution, but iterations of the improved steps can be time-consuming for large numbers of nodes.Similar to the originalK-means algorithm, good initialization is helpful to the optimization. The initial estimation of the means, μCoff and μCon, can be found by a K-means algorithm minimization of edge partitions (4) without the labeling constraint of (2). To estimate the initial state x, the pixel clustering K-means algorithm (1) can be used. The general framework is illustrated in Algorithm 1.Algorithm 1:Minimizing edge partition.
(1)   Estimate image label state x with the K-means algorithm on pixels, Kp=4.
(2)   Estimate μCoff and μCon by the K-means algorithm minimization of (4)
without the labeling constraint of (2).
(3)    Estimate the image label state x using the QPBOI-α-expansion.
(keep μCoff and μCon constant)
(4)    Estimate μCoff and μCon from the image label state x.
(5)   If μCoff and μCon are unchanged, terminate.
Else, repeat steps 3 and 4.


## 3.3. Edge Weights

Various examples of the edge partition segmentation results using the color distance edge weights are shown in Figure2 for the MSRC image database [29]. The color distance from the neighboring pixels is sufficient for some image segmentation problems, but more rigorous weight calculations are often suited for semantic segmentation. Instead of proposing new edge weight calculations, an existing state-of-the-art contour detection algorithm is incorporated.Figure 2
Even with the simple color distance weight, the edge partitions can produce adequate segmentation results. Some of the segmentation results from the MSRC database are shown.The global probability of the boundary (GPB) edge detection method [25, 30], which scored best for the Berkeley database (http://www.cs.berkeley.edu/projects/vision/bsds), is employed as the edge weights. The edge weights can be connected between the pixel nodes, and the proposed edge partitioning algorithm can be implemented. Figure 3 shows the other segmentation results under the pixel-to-pixel edge connections. Although Figures 3(a) and 3(b) show a good segmentation result, the QPBOI algorithm cannot obtain a good segmentation in Figures 3(c) and 3(d). The QPBOI algorithm often fails in the presence of a large number of nodes. Thus, to reduce both the computational time and the chance of failure in the QPBOI algorithm, the oversegmentation process is adopted from [25] in this segmentation. The edges are connected between the superpixels instead of the pixels. The number of oversegments is between 400 and 1000. The edge partitioning algorithm segments a BSDS image average in under 5 seconds.Figure 3
The QPBOI optimization scheme can be efficient for (a) and (b). However, for many cases, the nonsubmodular potentials are too strong, and the QPBOI optimization fails in (c) and (d). More iterations of QPBOI can improve the result; however, iterations are time-consuming without improvement guarantees. Superpixel images are used in this study to reduce the computation time and increase the QPBOI iterations.
(a)
(b)
(c)
(d)


## 4. Evaluation

The proposed edge partition approach is evaluated using the popular Berkeley image database. The set contains 300 images with at least four human segment annotations per image. The three quantitative evaluation methods used are as follows: Probabilistic Rand Index (PRI) [31], Variation of Information (VoI) [32], and Boundary Displacement Error (BDE) [33]. Global Consistency Error (GCE) [34] is not included in this evaluation. GCE measures the extent to which one segmentation can be viewed as a refinement of another. However, one pixel per segment and one segment for an entire image can give zero error for GCE [31]. GCE favors extremely oversegmented or undersegmented results, and both cases are unwanted for a semantic segmentation. GCE is deemed to be an inconsistent evaluation method.The evaluation methods used in this study are PRI, VoI, and BDE. PRI counts the number of consistent labels between the segmentation and the ground truth. VoI measures the segmentation randomness that cannot be explained by the ground truth. BDE is the average displacement error or the boundary pixels between two segmentation results. PRI counts the correctness in segmentation, while VoI and BDE measure the errors between the segmentation and ground truth. In the first subsection, the proposed method is evaluated against various segmentation methods. In the second subsection, the comparison between the proposed and the merge-threshold methods is demonstrated using the same edge weights.


### 4.1. Comparison to the Previous Segmentation Methods

Generally, the parameters are constant for the entire database and test methods. This evaluation includes mean shift (MShift) [1], graph-based segmentation (GBIS) [21], JSEG [20], Normalized Tree Partitioning (NTP) [22], saliency-based segmentation (Saliency) [23], Boundary Encoding Based Segmentation (TBES) [24], normalized cut (Ncut) [11], and fully connected spectral segmentation (SpecSeg) [13]. Additionally, contour to region (CtoR) [25] uses the same edge weights. Table 1 summarizes the performance of these methods. Many of the evaluation results are obtained from [13].Table 1
Constant parameters are maintained for the Berkeley image set and the test methods. The top ranking results are written in bold, and the rankings are in parenthesis. For segmentation methods with a∗, a different segmentation number is assigned optimally for each image.
Method
PRI
VoI
BDE
MShift [1]
0.7958
1.9725
14.41
JSEG [20]
0.7756
2.3217
14.40 (3)
GBIS [21]
0.7139
3.3949
16.67
NTP [22]
0.7521
2.4954
16.30
Saliency [23]
0.7758
1.8165
16.24
TBES [24]
0.80 (3)
1.76 (3)
—
CtoR [25]
0.806 (1)
1.717 (2)
11.57 (2)
EPartition
0.804 (2)
1.700 (1)
11.16 (1)
NCut∗ [11]
0.772
2.259
12.94
SpecSeg∗ [13]
0.8146
1.8545
12.21For PRI measurements, the merge-threshold method of CtoR ranks first. The proposed segmentation ranks first for VoI and BDE. The CtoR method is available to the public by the authors. The threshold value for the CtoR method was chosen to be 80 for its highest average ranking. A number of segmentation results of CtoR and of the proposed EPartition are shown in Figure4. For the normalized cut and fully connected spectral segmentation, the segmentation number is chosen for each image and is excluded from the rankings.Figure 4
The segmentation results from Table1 are shown for CtoR and the proposed EPartition.
(a)
Image
(b)
CtoR
(c)
EPartitionCtoR and EPartition use the same edge weights; thus, their performances are similar. However, in CtoR, a merge-threshold algorithm is used for segmentation. Different thresholds among integer intervals{1,…,255} are shown for the PRI, VoI, and BDE evaluation methods in Figure 5. Generally, PRI and BDE favor oversegmentation and VoI favors undersegmentation. The optimal threshold value is generally smaller for PRI and BDE than VoI.Figure 5
The segmentation evaluation (PRI, VoI, and BDE) versus threshold values is plotted for CtoR. Different threshold values give an optimal score for each segmentation evaluation approach. In contrast, the proposed EPartition is independent of the threshold values and finds the approximate optimal segmentation for all evaluation approaches.
(a)
PRI
(b)
VoI
(c)
BDEIn contrast, the edge partitioning segmentation is independent of a threshold value. Figure5 shows the performance of the CtoR merge-threshold method in terms of threshold values. The proposed EPartition segmentation evaluation scores for PRI, BDE, and VoI are very close to the highest evaluation score of CtoR. However, the merge-threshold method in CtoR requires a specific threshold value for each segmentation evaluation method. The advantage of EPartition is that correct segmentation is possible without the designation of segmentation number or a threshold value.


### 4.2. Comparison to Trained Threshold

In previous experiments, EPartition was shown to have competitive performance with CtoR when the optimal threshold value is hand-picked for CtoR. In this section, the threshold value is trained from the Berkeley 300 set and the segmentation performances are compared to the Weizmann segmentation set [35]. The Weizmann set contains 100 images with three human segmentation annotations.In Table2, the segmentation evaluations of the CtoR and EPartition methods are compared. There is a minuscule difference for PRI and small differences in the VoI evaluation methods. For BDE evaluation, EPartition clearly outperforms CtoR method. The trained threshold value was not robust for different segmentation evaluation approaches. By partitioning the edges through minimizing the mean squared distance, the proposed EPartition shows adaptive performance among the three evaluation methods. Various comparative segmentation results are shown in Figure 6.Table 2
The segmentation results for the Weizmann image set are summarized. The threshold value of CtoR is trained from the Berkeley set. Superior results are written in bold.
Method
PRI
VoI
BDE
CtoR [25]
0.7170
1.017
22.01
EPartition
0.7169
1.225
13.67Figure 6
The segmentation results from Table2 are shown for CtoR and the proposed EPartition.
(a)
CtoR
(b)
EPartition
(c)
CtoR
(d)
EPartition


## 4.1. Comparison to the Previous Segmentation Methods

Generally, the parameters are constant for the entire database and test methods. This evaluation includes mean shift (MShift) [1], graph-based segmentation (GBIS) [21], JSEG [20], Normalized Tree Partitioning (NTP) [22], saliency-based segmentation (Saliency) [23], Boundary Encoding Based Segmentation (TBES) [24], normalized cut (Ncut) [11], and fully connected spectral segmentation (SpecSeg) [13]. Additionally, contour to region (CtoR) [25] uses the same edge weights. Table 1 summarizes the performance of these methods. Many of the evaluation results are obtained from [13].Table 1
Constant parameters are maintained for the Berkeley image set and the test methods. The top ranking results are written in bold, and the rankings are in parenthesis. For segmentation methods with a∗, a different segmentation number is assigned optimally for each image.
Method
PRI
VoI
BDE
MShift [1]
0.7958
1.9725
14.41
JSEG [20]
0.7756
2.3217
14.40 (3)
GBIS [21]
0.7139
3.3949
16.67
NTP [22]
0.7521
2.4954
16.30
Saliency [23]
0.7758
1.8165
16.24
TBES [24]
0.80 (3)
1.76 (3)
—
CtoR [25]
0.806 (1)
1.717 (2)
11.57 (2)
EPartition
0.804 (2)
1.700 (1)
11.16 (1)
NCut∗ [11]
0.772
2.259
12.94
SpecSeg∗ [13]
0.8146
1.8545
12.21For PRI measurements, the merge-threshold method of CtoR ranks first. The proposed segmentation ranks first for VoI and BDE. The CtoR method is available to the public by the authors. The threshold value for the CtoR method was chosen to be 80 for its highest average ranking. A number of segmentation results of CtoR and of the proposed EPartition are shown in Figure4. For the normalized cut and fully connected spectral segmentation, the segmentation number is chosen for each image and is excluded from the rankings.Figure 4
The segmentation results from Table1 are shown for CtoR and the proposed EPartition.
(a)
Image
(b)
CtoR
(c)
EPartitionCtoR and EPartition use the same edge weights; thus, their performances are similar. However, in CtoR, a merge-threshold algorithm is used for segmentation. Different thresholds among integer intervals{1,…,255} are shown for the PRI, VoI, and BDE evaluation methods in Figure 5. Generally, PRI and BDE favor oversegmentation and VoI favors undersegmentation. The optimal threshold value is generally smaller for PRI and BDE than VoI.Figure 5
The segmentation evaluation (PRI, VoI, and BDE) versus threshold values is plotted for CtoR. Different threshold values give an optimal score for each segmentation evaluation approach. In contrast, the proposed EPartition is independent of the threshold values and finds the approximate optimal segmentation for all evaluation approaches.
(a)
PRI
(b)
VoI
(c)
BDEIn contrast, the edge partitioning segmentation is independent of a threshold value. Figure5 shows the performance of the CtoR merge-threshold method in terms of threshold values. The proposed EPartition segmentation evaluation scores for PRI, BDE, and VoI are very close to the highest evaluation score of CtoR. However, the merge-threshold method in CtoR requires a specific threshold value for each segmentation evaluation method. The advantage of EPartition is that correct segmentation is possible without the designation of segmentation number or a threshold value.


## 4.2. Comparison to Trained Threshold

In previous experiments, EPartition was shown to have competitive performance with CtoR when the optimal threshold value is hand-picked for CtoR. In this section, the threshold value is trained from the Berkeley 300 set and the segmentation performances are compared to the Weizmann segmentation set [35]. The Weizmann set contains 100 images with three human segmentation annotations.In Table2, the segmentation evaluations of the CtoR and EPartition methods are compared. There is a minuscule difference for PRI and small differences in the VoI evaluation methods. For BDE evaluation, EPartition clearly outperforms CtoR method. The trained threshold value was not robust for different segmentation evaluation approaches. By partitioning the edges through minimizing the mean squared distance, the proposed EPartition shows adaptive performance among the three evaluation methods. Various comparative segmentation results are shown in Figure 6.Table 2
The segmentation results for the Weizmann image set are summarized. The threshold value of CtoR is trained from the Berkeley set. Superior results are written in bold.
Method
PRI
VoI
BDE
CtoR [25]
0.7170
1.017
22.01
EPartition
0.7169
1.225
13.67Figure 6
The segmentation results from Table2 are shown for CtoR and the proposed EPartition.
(a)
CtoR
(b)
EPartition
(c)
CtoR
(d)
EPartition


## 5. Conclusion and Future Works

In this paper, image segmentation by edge partitioning is proposed. In contrast with previous edge weight-based segmentation methods, such as normalized cut, the proposed method is independent of the number of segments. Furthermore, compared with the previous segmentation techniques, edge partitioning remains competitive without the need for the segmentation number selection. Segmentation by edge partitioning has shown to be competitive with previous segmentation techniques in the Berkeley database. The advantage of the proposed method lies in its adaptive nature for handling edge weights without threshold values or segment number assignments.The proposed algorithm can be extended to general partitioning problems. Four labels are sufficient when segmenting 2D images. However, for fully connected graphs, the number of labels can be arbitrarily large. If a maximum number of labels are chosen, the edge partitioning method can be incorporated into a general partition problem without designating the specific number of partitions among nodes.



---

*Source: 683176-2015-12-22.xml*