IEEE TRANSACTIONS ON CYBERNETICS, VOL. 43, NO. 2, APRIL 2013

725

Image Cosegmentation by Incorporating Color
Reward Strategy and Active Contour Model
Fanman Meng, Hongliang Li, Senior Member, IEEE, Guanghui Liu, and King Ngi Ngan, Fellow, IEEE

Abstract—The design of robust and efficient cosegmentation
algorithms is challenging because of the variety and complexityof the objects and images. In this paper, we propose a new cosegmentation model by incorporating a color reward strategy andan active contour model. A new energy function corresponding
to the curve is first generated with two considerations: the foreground similarity between the image pairs and the backgroundconsistency in each of the image pair. Furthermore, a new foreground similarity measurement based on the rewarding strategy
is proposed. Then, we minimize the energy function value via amutual procedure which uses dynamic priors to mutually evolvethe curves. The proposed method is evaluated on many images
from commonly used databases. The experimental results demonstrate that the proposed model can efficiently segment the common
objects from the image pairs with generally lower error rate thanmany existing and conventional cosegmentation methods.
Index Terms—Active contour model, cosegmentation, reward
strategy.

I. I NTRODUCTION

D

URING the past decades, the method of single-imagebased segmentation has been developed in a variety of
directions. According to the necessity of the object prior, we
can classify the directions into two classes, i.e., unsupervised
segmentation method and supervised segmentation method.
The unsupervised segmentation method aims to segment theimage into many uniform and homogeneous regions with respect to texture or color properties [1]–[7]. The supervised
segmentation method segments the object using object prior.
Compared with the unsupervised segmentation method, thesupervised segmentation method can achieve semantic segmentation depending on whether the object prior is precisely
modeled or not. To generate precise object prior, several object
prior generation methods were proposed in the previous work,
such as saliency-detection-based methods [8]–[12], learningManuscript received February 2, 2012; revised August 2, 2012; accepted
August 11, 2012. Date of publication September 14, 2012; date of current
version April 16, 2013. This work was supported in part by the National Natural
Science Foundation of China under Grants 60972109, 61101091 and 61271289,
by the Ph.D. Programs Foundation of the Ministry of Education of China
under Grant 20110185110002, and by the Fundamental Research Funds for theCentral Universities under Grant E022050205. This paper was recommended
by Associate Editor I. Bloch.
F. Meng, H. Li, and G. Liu are with the School of Electronic Engineering,
University of Electronic Science and Technology of China, Chengdu 610073,
China (e-mail: hlli@uestc.edu.cn).
K. N. Ngan is with the Department of Electronic Engineering, The
Chinese University of Hong Kong, Shatin, Hong Kong (e-mail: knngan@
ee.cuhk.edu.hk).
Color versions of one or more of the figures in this paper are available online
at http://ieeexplore.ieee.org.
Digital Object Identifier 10.1109/TSMCB.2012.2215316

based methods [13]–[15], and human-interaction-based methods [16]–[18]. Although accurate segmentation is achievedby the supervised segmentation method, it may be hard togenerate the corresponding object prior for complex cases.
Furthermore, the huge workload will be produced for the userwhen the number of the target images is high.
Recently, segmenting objects from multiple images has attracted researchers’ attention. Generally, the multiple-imagebased segmentation method first provides several other images
containing the object and then segments the objects by extracting common objects among the images. Compared with thesingle-image-based segmentation method, the common-object
segmentation method is convenient in terms of user interaction
and can achieve accurate segmentation results. In this paper,
we restrict the focus to segment common objects from a pair ofimages.
In recent years, extracting common objects from a groupof images has become an active research topic. A number of solutions were proposed to solve common-object segmentation problem, such as matching-based method, cosegmentation method, and mutual segmentation method. The
matching-based method considers the common-object segmentation as a matching task which generally consists of two steps:
1) locate the common objects among original images and2) segment the common objects based on the matching results.
Since the successful segmentation for the matching-based
method depends on the precise location of the common object,
the well-known local region descriptors were employed forthe locating, such as scale-invariant feature transform (SIFT)
descriptor [19], saliency descriptor [20] and Harris-affine interest operator [21]. Other methods, such as conditional random
field method and latent Dirichlet allocation (LDA)-based topic
discovery method, were also introduced for common-object
locating. In [22], SIFT feature was used for the commonobject locating. Then, the common objects were segmented
by the corresponding growing method. In [23], conditionalrandom field method was used to adapt generic knowledge tothe new class. The obtained new knowledge was used to locateand segment the common objects. LDA was used to extractcommon objects in [24], which performed topic discovery oneach image and segmented the common objects based on how
well they were explained by the topic. In [25], Harris-affineinterest operator and SIFT descriptor were extracted to locate
the common objects. Then, the common objects were extracted
by the spectral clustering method.
Unlike the aforementioned methods where common objects
are segmented based on two steps, cosegmentation [26] simultaneously segments the common objects from a pair of images.

2168-2267/$31.00 © 2012 IEEE

726

The idea of the cosegmentation is to consider an additional
foreground similarity constraint on single-image-based segmentation method to result in the segmentation of the common
objects. Several cosegmentation methods were proposed, such
as Markov random field (MRF)-based cosegmentation method[26]–[30] and discriminative-clustering-based cosegmentation
method [31].
The MRF-based cosegmentation method adds the constraintof the foreground similarity into traditional MRF-based segmentation model to simultaneously consider common-object
segmentation and single image segmentation. The key ofthe MRF-based cosegmentation method is how to measure
foreground similarity. Furthermore, since the foreground similarity measurement affects the optimization of the energyfunction, designing efficient optimization method is another key
problem. In the existing MRF-based cosegmentation method,
similarity measurements such as L1-norm [26], L2-norm [27],reward model [28], and Boykov–Jolly model [29], [32] were
proposed. The corresponding optimization methods, namely,
trust-region graph-cut method, quadratic pseudo-Boolean optimization method, maximum flow procedure of graph, anddual decomposition, were also proposed for optimization of theenergy function.
Apart from MRF-based cosegmentation method,
discriminative clustering and spectral clustering methods
were combined to segment common objects in [31], whichused spectral clustering technique and positive definite kernels
to train a classifier for the common-object classification. In[32], an interactive cosegmentation method was proposed.
The authors used human interaction guided by an automatic
recommendation system to achieve more accurate commonobject segmentation. An object cosegmentation method was
proposed in [33]. The authors first used the unsupervised
segmentation method to segment the images into a series of
local regions. Then, the common objects were selected fromthe local regions by A∗ -search algorithm. Since the common
objects can be similar to each other on many features, random
forest regressor method was used to select the useful featuresfrom a total of 33 features (such as color histogram, textons,
SIFT descriptors, boundary curvature, and area). The authorsin [34] presented a scale-invariant cosegmentation method byconsidering the scale variation among the common objects. The
fact that the matrix composed of the common objects shouldhave a rank of one was used for modeling the energy function,and the common objects were finally segmented throughsearching image pixels that made the rank of corresponding
matrix be equal to one. The algorithm in [35] solved
cosegmentation by generating a novel global energy term
which considered the foreground similarity and the backgrounddissimilarity. Meanwhile, a cosaliency map was employed in[35] to construct the unary term of the energy function.
Similar to cosegmentation, the mutual segmentation algorithm given by Riklin-Raviv et al. in [36] and [37] simultaneously locates and segments common objects from a pair ofimages. In mutual segmentation, common objects are extracted
by a mutually supporting evolution process, in which the information gained in the evolving segmentation of one image
is a dynamic prior for the other. Thus, the segmentation andIEEE TRANSACTIONS ON CYBERNETICS, VOL. 43, NO. 2, APRIL 2013

the prior information are refined along the process [36]. In[36], the shape feature similarity between the foregrounds was
represented by a homograph matrix transform which is robust
to the shape variations such as occlusion, noising, and rotation.
The significant improvement of the segmentation results was
gained.
In this paper, we propose a new cosegmentation model which
integrates active contours method and rewarding strategy. We
first generate a new energy function with two conflicting goals.
One is foreground similarity among the images, which is related to common-object segmentation. The other is backgroundconsistency in each image, which corresponds to single image
segmentation. We use a color histogram for region representation and employ a rewarding strategy to measure the foregroundsimilarity and the background consistency. Then, we formulate
the energy function by level set and use a mutual evolution
approach to minimize the energy function value. We evaluate our approach on many images. The experimental results
demonstrate the effectiveness of the proposed model.
The contributions of this paper are listed as follows.
1) An active-contour-based model is proposed for the cosegmentation problem. The proposed model can segmentcommon objects from a pair of images.
2) A new reward-strategy-based foreground similarity measurement is proposed to evaluate foreground similarity,
which makes the proposed energy function easy for optimization (be optimized by the method of calculus ofvariations). In addition, a rewarding strategy consideringboth the foreground reward and the background reward is
proposed.
The rest of this paper is organized as follows. The background including the rewarding strategy and the active contour
model is introduced in Section II. In Section III, we present our
proposed cosegmentation algorithm. Experimental results areprovided in Section IV to validate the efficiency of the proposedmodel. Finally, Section V gives the conclusion of this paper.
II. BACKGROUND
Our method is related to two previous methods, i.e., therewarding strategy and the active contour model. In this section,
we introduce the background, advantages, and challenges of thetwo types of methods.
A. Rewarding Strategy
Unlike traditional single image segmentation, cosegmentation [26] aims at simultaneously segmenting common objects
from a group of images. This method can segment multiple
images jointly instead of segmenting each image independently
based on the co-occurrence of objects in the images [38]. It
has many potential applications in computer vision, such asimage classification [39], object recognition [40], and image
retrieval [33].
To achieve common-object segmentation, the cosegmentation is usually formulated as an optimization problem which
consists of two terms. One is the single image segmentation
term which considers the smoothness of the segmentation inMENG et al.: IMAGE COSEGMENTATION

Fig. 1.

Example to illustrate the rewarding strategy.

each image. The other is the common-object segmentation term
which focuses on extracting similar regions as foregrounds. The
key of the cosegmentation model is how to design efficient
foreground similarity measurement to form common-object
segmentation term.
In the existing cosegmentation approaches, a penalizing
strategy is usually used to measure foreground similarity. The
penalizing strategy measures foreground similarity by penalizing the differences between the foregrounds, which leads
to a small difference between the features of the segmented
objects. For example, in Rother’s method [26], the similarity
between two regions is evaluated by L1-norml |h1 (b) −h2 (b)|, where h1 and h2 are the features of the two regionswith the length l. A small L1-norm value corresponds to large
similarity. It is seen from the measurement that the similarity
measurement is constructed by the differences between thefeatures. A large difference is penalized. Penalizing-strategybased methods can achieve accurate common-object segmentation results. However, the penalizing-strategy-based foregroundsimilarity measurement leads to nondeterministic-polynomialtime-hard optimization problem.
Instead of penalizing the foreground difference, the rewarding strategy in [28] rewards the similarities between the foregrounds. For example,
the similarity between the foregroundsis measured by l |h1 (b) · h2 (b)|, which rewards the similarity instead of penalizing the differences. Compared with thepenalizing strategy, the rewarding strategy is advantageous inits effective solution for the energy optimization, since therewarding-strategy-based foreground similarity measurement
leads to simpler optimization problem, such as submodular
problem that can be efficiently optimized with graph cuts.
However, the rewarding strategy can lead to unexpected
results in the case where the background with uniform color
occupies a large area of one image while there are a few regionsthat are similar to the background in the other image [29]. In
this case, the rewarding strategy will classify the background as
the common objects, because the reward of background regionis larger than the reward of foreground region. As an example,
Fig. 1 shows the original images, ground truth masks, and theresults of the reward strategy. It can be seen in Fig. 1 that thecommon objects are the black regions and the blue regions areshared by the two images. Since the blue region takes a big partof the first image, the reward of the blue region will be larger
than the reward of the black region. Hence, the final results by
the rewarding strategy are the blue regions instead of the black
regions.

727

imum value of the energy function, and the curve serves as
the boundary of the object to be segmented. Then, the corresponding curve decreases the energy criterion and converges
toward a (local) minimum of the criterion by solving thepartial differential equation (PDE) constructed from the energy function. The existing active-contour-based segmentation
methods can be classified into two categories, i.e., geometricbased active contour method (geometric-based method)[41]–[45] and region-based active contour method (regionbased method) [46]–[51]. The geometric-based methods are
based on an energy defined as a curve integral, while region
methods have region terms in the energy which are integrals
on the regions inside the curves. Compared with the geometricbased method, the region-based method is more robust to theinitial curve placement.
One of the well-known region-based methods is the C–Vmodel [47]. The major components of the energy function in
the C–V model are shown as
E(C) = (I(x, y)−vi )2 dxdy+ (I(x, y)−ve )2 dxdy (1)ωoωiwhere I is an image, I(x, y) denotes the color value in location(x, y), E(C) is the energy function that corresponds to thecurve C, ω i and ω o denote the regions inside and outside thecurve C, respectively, and vi and ve are the mean color valuesof regions ω i and ω o , respectively.
For the right-hand side of (1), the first term represents theforeground consistency, which can be called interior term. The
second term, namely, exterior term, measures the backgroundconsistency. Based on the assumption that the textures of foreground and background are uniform, such as medical images,
the integrals in interior term and exterior term are equal tozero only if the foreground and background are accurately segmented. Therefore, the object is segmented through searching
the curve corresponding to the smallest energy value.
Note that the assumption of the C–V model that the texturesof foreground and background are uniform may not be used fornatural images, where the foreground and background may becomposed of multiple colors. This means that the C–V model
may not be suitable to segment objects from natural images.
The reason is that the mean value is no longer suitable for representing a region composed of multiple colors. Furthermore, theobject information provided by a single image is not sufficient
to distinguish foreground and background from an image inthe case of complex foreground and background. Thus, segmenting object from the image with complex foreground andbackground is a challenge for the C–V model.
III. P ROPOSED M ETHOD
In this section, we introduce our proposed model. The energyfunction is first introduced. Then, the level set formulation andthe optimization are illustrated.

B. Region-Based Active Contour Model

A. Proposed ModelIn active-contour-based segmentation method, the energyfunction is related to a curve which corresponds to the min-Different from the traditional region-based method, two
images are considered in the cosegmentation problem. Thus,

728

IEEE TRANSACTIONS ON CYBERNETICS, VOL. 43, NO. 2, APRIL 2013

the constraint of foregrounds similarity is considered in the
proposed model, which is performed at the region level.
1) Energy Function: We form the energy function for curve
Ck as
 
Ek (Ck ) = μ · Length(Ck ) + ν · Area ωki i 
− λik f Ik (x, y), g ω1−k
dxdy
i
ωk



− λok

f [Ik (x, y), g (ωko )] dxdy

(2)4) Analysis of Energy Function: To analyze the proposedenergy function, we assume that only the common objects
in two images have the same colors. In addition, most of
the areas of the common objects locate within the curves.
For a pixel located in the common object in image Ik , wei)] > f [Ik (i, j), g(ωko )], where
can see that f [Ik (i, j), g(ω1−k(i, j) denotes the pixel location. On the contrary, we obi)] < f [Ik (i, j), g(ωko )] for a backgroundtain f [Ik (i, j), g(ω1−kpixel. Hence, the curve energy is minimum when the curve
accurately separates common objects from images. Otherwise,
the curve energy increases. For Ik , the common-object segmentation thus can be depicted asoωkwhere Ik , k ∈ {0, 1}, is the kth image of the image pair,
Ck denotes the curve in the image Ik , Ek (Ck ) is the energyfunction corresponding to Ck , ωki and ωko depict the regions
inside and outside the curve Ck , g(ω) is the representationof the region ω, f (p, g(ω)) is used to measure the similarity
between pixel p and region ω represented by g(ω), Length(Ck )denotes the length of the curve Ck , Area(·) denotes the area ofiis the region inside the curve in the othera region, and ω1−kimage I1−k . μ > 0, ν > 0, λik > 0, and λ0k > 0 are parameters.
At the right-hand side of (2), the first term and the second
term describe the length and the area of the region inside thecurve, respectively. The two terms are related to curve intrinsic
properties. The third term, namely, interior term, is composedof the integral of similarities between the foreground pixels ofiin I1−k . The interior term describes the
Ik and the region ω1−kforeground similarity. The last term is the exterior term whichis the integral of similarities between the background pixels
in Ik and the exterior region ωko . The exterior term represents
background consistency. It is seen from (2) that the major
components of the energy function are composed of an interior
term and an exterior term.
2) Region Representation: In the proposed model, we use acolor histogram h to represent a region ω, i.e., g(ω) = h. For
a red (R)–green (G)-blue (B) image, color value is represented
by a 3-D vector. A histogram is generated by considering theprobability of each color’s occurrence in a region. Since a
region feature is used for region representation, our method is aregion-based method.
3) Foreground Similarity Measurement: We set foregroundsimilarity measurement f as f (p, g(ω)) = h(p) for a pixel
value p. A large foreground similarity value will be given if apixel value is with a large histogram value. Otherwise, a small
foreground similarity value is given. In practice, the pixel values
are commonly slightly different from each other for a color.
Thus, two pixels will have the same color if their color distanceis less than β. For a pixel value p, f (p, g(ω)) is then given byf (p, g(ω)) =h(p )(3)|p −p|≤βwhere |p − p| denotes the distance between two pixel color
values p and p. Furthermore, |p − p| defines the distance as thelargest difference among the differences in R, G, and B color
channels.

Ck∗ = arg min Ek (Ck ).
Ck(4)

It is seen from (2) that foreground similarity is designed byithe reward strategy, i.e., ωi f [Ik (x, y), g(ω1−k)]dxdy. Furtherk
more, the background is also designed by the reward strategy,i.e., ωo f [Ik (x, y), g(ωko )]dxdy. Hence, rewarding strategy iskemployed for region similarity measurement. Compared withthe reward strategy used in [28], we can see that both foreground similarity and background consistency are rewarded,
which improves the traditional rewarding strategy. The improvement is resulted by the fact that, for an image pair where
the background with uniform color occupies a large area of
one image while there are a few regions that are similar tothe background in the other image, a pixel of backgroundi)] also haswith large foreground reward f [Ik (x, y), g(ω1−koa large background reward f [Ik (x, y), g(ωk )], which is incontrast with foreground reward. When background reward
is larger than foreground reward, i.e., f [Ik (x, y), g(ωko )] >i)], the pixel will be considered as backf [Ik (x, y), g(ω1−kground instead of foreground. Thus, the results of traditional
rewarding strategy are improved.
B. Level Set Formulation
By manipulating (2), the level-set-based energy function of
Ik is expressed asEk (φk ) = μ δ (φk (x, y)) |∇φk (x, y)| dxdy
Ωk



+ν

H (φk (x, y)) dxdy
Ωk



− λik


 i 
H (φk (x, y)) dxdy
f Ik (x, y), g ω1−k

Ωk



− λok

f [Ik (x, y), g (ωko )] (1−H (φk (x, y))) dxdy
Ωk(5)
where φk denotes the zero level set function, δ(x) is the
1-D Dirac function, H(x) is the Heaviside function,
δ(φk (x, y)) represents Ck , H(φk (x, y)) denotes the interior
region ωki , and 1 − H(φk (x, y)) means exterior region ωko .

MENG et al.: IMAGE COSEGMENTATION

729

C. Energy Function Minimization

φnk , n ∈ Z, which can provide more accurate common-object
information in the minimization process. Hence, the dynamic
method usually achieves good performance for the image
cosegmentation compared with the fixed method. In this paper,
we use the dynamic method for common-object segmentation.
The steps of the proposed algorithm is described in
Algorithm (1).

To minimize the proposed energy function, we derive thecorresponding Euler–Lagrange equation. For the last termin (5), when f [Ik (x, y), g(ωko )] is dependent on φk , theEuler–Lagrange equation is represented as−λok ·∂f(1 − H(φk )) + λok · δ(φk )f [Ik , g (ωko )] = 0 (6)∂φkwhere the term −λok · (∂f /∂φk )(1 − H(φk )) is related to theshape derivative as shown in [44]. In this paper, we use thei) and g(ωko ) fixed, which means
strategy in [47] to keep g(ω1−kothat f [Ik (x, y), g(ωk )] is independent of φk . Hence, ∂f /∂φk =0, and we can rewrite (6) as
λok · δ(φk )f [Ik , g (ωko )] = 0.

(7)

Based on (7), the Euler–Lagrange equation of (5) for φk isshown as∇φk
∂φk
= δ(φk )· μ · div− ν + λik∂t|∇φk | i  o−λk ·f [Ik , g(ωko )] = 0·f Ik , g ω1−k

(9)

For (9), the corresponding discrete form by explicit scheme canbe expressed by
(i, j) − φnk (i, j)
φn+1k= δ(φnk (i, j)) · −μ · κ(i, j) − ν + λikt i 
− λok · f [Ik (i, j), g (ωko )]·f Ik (i, j), g ω1−k(10)where φ0k is the initial curve. Setting t = 1, the final evolving
form of our model is shown as
φn+1(i, j) = φnk (i, j)+δ (φnk (i, j))k i 
· −μ · κ(i, j) − +λik · f Ik (i, j), g ω1−k−λok · f [Ik (i, j), g (ωko )]] .

• Set initial curves φ0k , k = 0, 1, n = 0, and initial parameters.
• Repeati) and1) For each image Ik , k = 0, 1, calculate g(ω1−kog(ωk ).
i) and g(ωko ), solve PDE in2) Based on the obtained g(ω1−kn+1
φ through (11) to obtain φk , k = 0, 1.
Until stop condition is satisfied.
> 0 are• Extract common objects: The locations with φn+1kforeground pixels of Ik .

(8)

where t ≥ 0 is an artificial time in φk (t, x, y). Since curvatureκ can be calculated by κ = −div(∇φ/|∇φ|) in the differential
geometry theory, we may rewrite (8) as i ∂φk= δ(φk ) · −μ · κ − ν + λik · f Ik , g ω1−k∂t−λok · f [Ik , g (ωko )]} = 0.

Algorithm 1 The proposed algorithm(11)

Equation (11) is used to minimize the energy function value.
i) can be fixed
In the minimization process, the value g(ω1−k(fixed method) or updated (dynamic method) along the iteri) is calculated using theation. For the fixed method, g(ω1−kiwhole image of I1−k . Thus, g(ω1−k ) is a constant value. For thei) is dynamically updated along
dynamic method, g(ω1−kthe evolution of C1−k . Compared with the dynamic method,
the fixed method has lower computational cost since there arei) in each iteration.
no additional operators to calculate g(ω1−kMeanwhile, for the dynamic method, the curves tend to evolveto the boundary of common objects along the iterations of

IV. E XPERIMENTS
Based on the subjective and objective assessments of the segmentation results, we verify that the proposed cosegmentation
algorithm can achieve more accurate segmentation comparedwith traditional cosegmentation methods.
A. Parameter Setting
In the experiment, we set the initial curve as a rectangle,
where each edge has small distance (γ) to the corresponding
image edge. We set γ = 5 for all test image pairs to cover themost area of the common objects. We normalize κ and f in(11) to [0, 1]. Since the sum of histogram g (corresponding
to β = 15) is used for measuring foreground similarity f ,we employ the normalized histogram for g to guarantee thati)] is in the range of [0, 1]. Furthermore, wef [Ik (i, j), g(ω1−k
set μ = 0.01, λik = λok = 1, β = 15, and ν = 0.001 for all
image pairs. The number of the iterations is set to Ni = 1500,
since there are complex foregrounds and backgrounds in some
images.
B. Results of the Proposed Method
We collect 30 image pairs from the data sets commonly usedin cosegmentation research and other databases such as Microsoft Research Cambridge image database and the Caltech256 Object Categories database. The image pairs with complex
foregrounds and backgrounds and the image pairs with multiple
common objects are selected for the experiment. In addition, theimage pairs with similar backgrounds are selected to verify ourrewarding strategy. Fig. 2 shows the test image pairs and theground truth masks (binary masks). It can be seen that some
images have complex foreground and background, such as
person (the fourth image pair in the first row), some image pairs

730

IEEE TRANSACTIONS ON CYBERNETICS, VOL. 43, NO. 2, APRIL 2013

Fig. 2. Original image pairs and the ground truth masks used in this paper.

Fig. 3. Segmentation results. (a) and (b) Original image pairs. (c) and(d) Segmentation results by the fixed method. (e) and (f) Segmentation results by the
proposed method. (g) and (h) Final segmentation results after postprocessing.

have multiple common objects, such as duck (the fourth pair in
the third row), and some image pairs have similar backgrounds,
such as mcow (the third image pair in the fourth row).

The segmentation results for nine image pairs are shown in
Fig. 3. The original images are shown in Fig. 3(a) and (b), andthe results of our dynamic method are shown in Fig. 3(e) and (f).

MENG et al.: IMAGE COSEGMENTATION

731

TABLE IR ESULTS OF THE F IXED M ETHOD AND DYNAMIC M ETHOD AND THE F INAL R ESULTS IN T ERMS OF E RROR R ATE

It can be seen in Fig. 3(e) and (f) that the proposed dynamic
method segments common objects from these image pairs. For
example, the common objects “dog” in the image pair dog withcomplex texture are segmented from the original image pair(the third image pair in Fig. 3).
To compare with the segmentation results of the proposed
dynamic method, Fig. 3(c) and (d) shows the results of the
fixed segmentation method with the same parameter setting as
in the proposed dynamic method. We can see that the fixed
method leads to the loss of some regions of the foreground. For
example, the segmentation results of the image pair banana (thefirst image pair) are composed of a number of region fragments.
The reason is that the interior term of the fixed model is weak.
A weak interior term is resulted by the fact that the informationof the common objects is interfered by the backgrounds. The
proposed dynamic method provides more accurate commonobject information than the fixed method, because the curvesof the two images evolve to the objects in the energy function
minimization process.
From Fig. 3, we can see that the advantages of the proposed method are as follows: 1) A fixed initial curve cansegment the common objects from the original image pairs, as
shown in Fig. 3, 2) the proposed model can segment multiple
common objects from these original images because all theseobjects have large foreground rewards based on our reward
measurement in (2). For example, the two “cows” with different
colors are segmented as common objects by using the proposed
dynamic method in the seventh image pair.
Note that the image pairs mcow (the seventh image pair)and duck (the eighth image pair) have similar backgrounds“meadow” in Fig. 3. It is seen that the backgrounds “meadow”
have uniform color and occupy most of the area of the two
images, which result in a large foreground similarity reward.
We can see from the results in Fig. 3(e) and (f) that the proposed
method accurately segments the “meadow” as the backgrounddue to the reward of the background similarity in our model.

Furthermore, Fig. 3(e) and (f) shows that there are redundantregions or isolated pixels in the segmentation results, suchas image pair banana (the first image pair). There are two
reasons. One is that the isolated pixels have similar colors to
the common objects, which results in a large foreground rewardof the isolated pixels. For example, the redundant regions inimage pair banana have colors of black and yellow, which
are similar to the colors of the “banana.” The other reason
is that the weak smoothness effort [the first term and thesecond term in (5)] cannot generate enough shrink force toeliminate the redundant regions. The weak smoothness effort
is resulted by the small μ (μ = 0.01) and ν (ν = 0.001) thatare used in the experiments. Large μ (i.e., μ = 0.1) and ν (i.e.,ν = 0.1) can eliminate the redundant regions. However, large
μ and ν will also eliminate parts of the common objects. In
the proposed method, we use postprocessing to eliminate theredundant regions which are usually small while guaranteeing
the common objects to be unaffected, as shown in Fig. 3(g)and (h). In the postprocessing method, all disconnected regions
are first identified. Then, the areas of the disconnected regions
are calculated D = {D0 , D1 , . . . , Da }, a ∈ Z, where a is thenumber of the disconnected regions. To eliminate the redundantregions, we consider the disconnected regions with Di > T as
the regions of the common objects, where T is a threshold.
In the experiment, we set T = 0.2 · max(D) for all the image
pairs.
The result of the segmentation is represented by a binarymask, where 0 means background and 1 means foreground. We
use the mean error rate for objective evaluation of an image
pair, since there are two images in each image pair. The error
rate of an image is the ratio of the number of incorrect pixels
to the total number of the pixels of the original image. A lowerror rate indicates an accurate segmentation. Table I shows theresulted error rates of the fixed method, the proposed dynamic
method, and the final results (by the postprocessing method).
We can see in Table I that the error rates of the dynamic method

732IEEE TRANSACTIONS ON CYBERNETICS, VOL. 43, NO. 2, APRIL 2013

Fig. 4. Results of [28], [31], [36], [52], and the proposed method. The first row displays the original images. The results of [28], [31], [36], [52], and the proposed
method are shown from the second row to the last row, respectively.

are lower than those of the fixed method for most of the test
image pairs (25 of the 30 image pairs). Furthermore, the final
results in Table I show that the error rates are further decreasedby the postprocessing method for most of the image pairs (26of the 30 image pairs). In addition, in some cases, the fixed
method gets the lowest error rates, such as cbleopardb andperson. As described in Section III-C, the fixed method uses theother whole image to generate the object prior. When it provides
good reference for the common-object prior, the fixed method
can also achieve better segmentations.
C. Comparison With Other Methods
We compare our model with the existing work for cosegmentation in [28], [31], and [36]. The comparison is shown in
Fig. 4 and Table II. The authors in [28] present a semisupervised
MRF-based cosegmentation method which uses a rewarding
strategy for cosegmentation. We implement this method using
the source code provided by the authors.1 As the performanceof the method in [28] is determined by the initial pixels, weselect the most suitable seed pixels for accurate segmentation.
In addition, we change parameter λ for more accurate segmentation. The method in [31] aims to classify the common-object
regions from all the local regions. Discriminative clustering is
employed in [31] for the classification. We use the source codeprovided by the authors.2 Parameter μ is varied to achieve moreaccurate segmentation. In the implementation of the method in[31], we consider the color descriptor and obtain superpixel
using the Ncuts method [1]. The authors in [36] proposeda mutual segmentation approach using a shape feature. We
implement the approach with Matlab codes. The parameters forthe experiment are set according to [36]. Furthermore, we set
the initial curve to be inside the common objects.
In our experiments, the proposed postprocessing method isimplemented in [28], [31], and [36] to improve the corresponding segmentation results and compare with the proposed
1 http://www.biostat.wisc.edu/~vsingh/restricted/myfiles/
coseg_vikas_maxflow_distrib.tar.gz
2 http://www.di.ens.fr/~joulin/code/coseg.zip

dynamic method. The related segmentation results are shown in
Fig. 4. The original images are shown in the first row of Fig. 4,
containing amira, duck, pvochorseb, llama, cdboara, cddoga,kim, and pvoctraind. By using the postprocessing method, thesegmentation results of [28] are shown in the second row ofFig. 4. We can see that the common objects are accurately
segmented from the image pairs such as duck and cddoga. Furthermore, the common objects are not successfully segmented
from other image pairs such as llama and kim, because it isdifficult to set precise initial color values of the common objects
and backgrounds when the foreground and background have
similar colors. The third row of Fig. 4 shows the segmentation
results of [31] using the postprocessing method. It can be seenthat the common objects are accurately segmented for the image
pairs, such as duck and amira. As the classifier training canbe affected by the similarity of the backgrounds, unsuccessful
segmentations are obtained for image pairs such as pvochorseb
and llama, as shown in the third row of Fig. 4. The fourth row
of Fig. 4 shows the results of [36] applying the postprocessing.
We can see that the method in [36] with the postprocessing
can achieve successful segmentations for image pairs suchas cdboara and kim. Meanwhile, unsuccessful segmentation
is observed for image pairs such as duck and cddoga, since
the precise homograph matrix representing shape similarity isaffected by shape variations, such as the variations of pose,scale, and rotation that exist in the data set. For the proposed
dynamic method, the segmentation results are shown in thesixth row of Fig. 4. It is illustrated that the common objects
are successfully segmented by the proposed cosegmentation
method.
Furthermore, we compare our method with the single-imagebased segmentation method Grabcut [52] which segments objects through iteratively using graph-cut-based segmentation
method. For segmentation result comparison in the experiment,
the same initial curve is used in our method and the Grabcut
method in [52] with postprocessing. The results of Grabcut are
shown in the fifth row of Fig. 4. We can observe that the accurate segmentations are achieved for image pairs such as duckand cdboara. Furthermore, unsuccessful segmentations are obtained such as amira and kim due to the initialization sensitivity

MENG et al.: IMAGE COSEGMENTATION

733

TABLE IIR ESULTS OF [28], [31], [36], [52], AND THE P ROPOSED M ETHOD IN T ERMS OF E RROR R ATE

TABLE IIIC OMPUTATIONAL C OMPLEXITIES FOR THE C OMPARISON M ETHODS AND O UR M ETHOD

of the Grabcut method to the initial curve placement. The initial
curve placement determines the background which defines thecorresponding Gaussian mixture model and affects the segmentation results. Note that the enhanced segmentation results canbe obtained when the initial curve is set manually in [52].
For all the images, the error rates of the methods analyzed
in Section IV-C are summarized in Table II. It shows thatthe proposed method segments the common objects with theminimum error rates for most of the image pairs. Furthermore,
we calculate the mean error rate by taking the average ofover 30 image pairs for comparison. The mean error rate ofour method (5.80% error rate) is about 2.88% less than themethod in [52] (8.68% error rate) which obtains the minimum
mean error rate among the other methods. In some cases, theexisting method achieves the lowest error rates compared with
our method, such as cdcora and knut. The reason is that our
method uses fixed parameters for all image pairs in order toperform fair comparison. More accurate segmentation can beobtained by adjusting these parameters for each image pair.

We compare the time complexity of the proposed methodwith those of the methods in [28], [31], [36], and [52]. In
our method, the number of iterations is related to the image
size n. Also, the pixel number and the calculation of thehistogram for the inner region at each iteration correspond ton. Thus, the time complexity of our method is proportional ton × (n + n) = 2n2 , which means that the time complexity is
proportional to O(n2 ). Table III shows the time complexitiesof the proposed method and the other methods. In Table III, it
can be seen that the methods in [28] and [52] have lower timecomplexities, i.e., O(n log n), since the graph-cut algorithm isused for the optimization. The proposed method has the sametime complexity with [31] and [36].
D. Discussion
The proposed cosegmentation model considers the initializations of two curves, since (11) is related to two curves Ck andC1−k for k ∈ {0, 1}. For image Ik , we assume that P1 and P2

734

IEEE TRANSACTIONS ON CYBERNETICS, VOL. 43, NO. 2, APRIL 2013

Fig. 5. Segmentation results under various initial curves.

Fig. 6. Unsuccessful segmentation results under specific initializations.

denote the sets of the common-object pixels and the backgroundpixels located on the curve Ck . To facilitate the selection of theinitial curve in the proposed method, the requirements of theinitial curves are described as follows.
1) For a pixel Ik (i, j) ∈ P1 with location (i, j),
the initial curves should make the value ofi)] greater than f [Ik (i, j), g(ωko )],f [Ik (i, j), g(ω1−ki)] > f [Ik (i, j), g(ωko )].
i.e., f [Ik (i, j), g(ω1−k2) On the contrary, for a pixel Ik (i, j) ∈ P2 with location (i, j), the initial curves should guarantee thati)] ≤ f [Ik (i, j), g(ωko )].
f [Ik (i, j), g(ω1−kBased on the assumption that the original images have difi)] = 0 for
ferent backgrounds, we can obtain f [Ik (i, j), g(ω1−kIk (i, j) ∈ P2 , which means that requirement 2) can always besatisfied.
To fulfill requirement 1), we assume that the initial curves
cover most of the regions of the objects and set the initial
curve as a rectangle in our model. Each edge of the rectangle has small distance to the corresponding image edge. It
is seen that our curve initialization satisfies requirement 1).
The reason is that f [Ik (i, j), g(ωko )] ≈ 0 for Ik (i, j) ∈ P1 andi)] > 0.
f [Ik (i, j), g(ω1−kFig. 5 shows the results of our method with various initial curves. Both artificial image pairs (the first image pair)and natural image pairs (the remaining three image pairs) areconsidered. For each image pair, initial curves partially or
fully covering the common objects are considered. They are
shown in the first and second columns for each image pair
block in Fig. 5(a)–(d). We can see that some initial curves
cover the whole regions of the common objects, and the others
partially cover the common objects. For example, the firstinitial curve for stone covers all the region of the common

object. For the third initial curve in the butterfly, only a small
region of the common object locates inside. We can see fromthe results that the curves (except the third initial curve inmotor) evolve to the boundary of the common objects. For
example, the fourth initial curves in Fig. 5(a) successfully
segment the common objects from the original images, although the initial curves both partially cover the regions of thecommon objects. The successful segmentations are contributedby the fulfillment of requirement 1) by the initial curves. For
the incorrect segmentation in motor, only the red regions aresegmented as common objects, because the initial curves cover
only the red regions of the common objects, which violates
requirement 1).
Furthermore, we show several unsuccessful segmentations
by the specific initializations. These results are shown in Fig. 6,
where the initial curves are shown in Fig. 6(a) and (b). It
can be seen that the initial curve does not cover the common
objects in one of the images. The initial segmentation results(without postprocessing) are shown in Fig. 6(c) and (d). The
final segmentation results are shown in Fig. 6(e) and (f). We
can see from Fig. 6(c)–(f) that incorrect initialization could
lead to wrong segmentation results, since the curve dose notinclude the common objects. Compared with the results in
Fig. 5, we can see that good performance can be achieved if
the initial curves contain enough foreground information. Thiscondition can be easily satisfied when the initial curve is set as
the rectangle with small distance to the image edge.
We discuss the curve evolution process along the iterations.
Fig. 7 shows the results containing four image pairs. For each
image pair, five curve evolution states corresponding to iteration
numbers 1, 101, 201, 301, and 401 are displayed. Fig. 7 demonstrates that the curves for the images evolve to the boundaryof the common objects. For example, the curves evolve to theMENG et al.: IMAGE COSEGMENTATION

735

Fig. 7. Curve evolution processes under different iterations. There are five curve states for each original image corresponding to iteration numbers 1, 101, 201,
301, and 401.

boundary of the objects at the 300th iteration in the first image
pair.
Compared with the well-known C–V model, the proposedmodel has two advantages. First, we use color histogram insteadof mean value for region representation, which is more suitableto represent a region with multiple colors. Second, for an image
pair, we introduce the color information of one image into
the energy function of the other image, which increases therobustness to the initial curve replacement. The reason is that,in the C–V model, the foreground information is modeled by
the interior region. Thus, the foreground information is affectedby the initial curve placement. A careful initial curve setting
is necessary for obtaining the precise foreground information.
In our model, the foreground information of an image Ik isprovided by the other image I1−k in the image pair. Based on
the assumption that most areas of the common objects locate
within the curve in I1−k , precise foreground information can beprovided to Ik . Note that it is simple to set the initial curve as arectangle to cover most areas of I1−k .
There are small regions in the initial segmentation. These
small regions are caused by the topology changes of the levelsets. Limiting the topology changes can remove these small
regions and avoid the need of the postprocessing. In our model,
we do not limit the topology changes of the level set because
we use the topology changes to cope with the variations among
the common objects, such as the variations of the shape andthe pose. Limiting the topology changes may also limit the
proposed method to handle these variations. Meanwhile, thesesmall regions can be removed by a simple postprocessing.
Note that limiting the topology change is suitable to solve
the cosegmentation with similar shapes. In the future, we will
extend the model to segment common objects with similar
shapes by introducing the limitation of the topology changes.
In this paper, we use level sets to minimize the energyfunction, which searches the minimum in a local manner. Note
that, in previous works, some authors use graph techniques tosearch for a global optimization, such as [28] and [35]. In thesemethods, the successful segmentation depends on the accurate
object prior to build the unary term of the energy function,

which usually needs user’s interaction to initialize the object
location or appearance [28]. Our method is statical-regionbased method where the contour is not evolved by fitting tolocal gradient information (as in the Snakes) but rather by fitting
statistical models to intensity, color, texture, or motion withineach of the separated regions [53]. It has been studied thatthe cost functionals of the statical-region-based method tend tohave less local minima for most realistic images [53]. Hence,
the level sets are suitable to solve the proposed energy function
in realistic images. The experimental results also demonstrate
the effectiveness of the level-set-based minimization.
V. C ONCLUSION
In this paper, we have proposed a novel cosegmentation
method to segment the common objects from image pairs. In the
proposed model, the energy function is modeled by considering
the foreground similarity and the background consistency. A
color histogram is used to represent a region, and the rewarding
strategy is used to evaluate the foreground similarity. The
background reward is used to improve the classical reward
strategy. Furthermore, the level set is used to represent the
curve. Then, a mutual optimization procedure is used for theminimization of the energy function value. The experimental
results demonstrate that the proposed model can efficiently
segment the common objects from the image pairs with generally lower error rates than many existing and conventionalcosegmentation methods. In the future, we will select otherregion features to obtain more accurate segmentation, and wewill modify the proposed method to further reduce the timecomplexity.
ACKNOWLEDGMENT
The authors would like to thank all the anonymous reviewers
for the constructive comments and useful suggestions that led to
improvements in the quality, presentation, and organization ofthis paper. The authors would also like to thank T. Riklin-Raviv
for her kindness, advice, generosity, and great help.

736IEEE TRANSACTIONS ON CYBERNETICS, VOL. 43, NO. 2, APRIL 2013

R EFERENCES[1] J. Shi and J. Malik, “Normalized cuts and image segmentation,” IEEE
Transactions on Pattern Analysis and Machine Intelligence, vol. 22, no. 8,
pp. 888–905, 2000.
[2] G. Mori, X. Ren, A. Efros, and J. Malik, “Recovering human body configurations: Combining segmentation and recognition,” in IEEE Conference
on Computer Vision and Pattern Recognition, 2004, vol. 2, pp. 326–333.
[3] P. F. Felzenszwalb and D. P. Huttenlocher, “Efficient graph-based image
segmentation,” International Journal of Computer Vision, vol. 59, no. 2,
pp. 167–181, 2004.
[4] P. Arbelaez, M. Maire, C. Fowlkes, and J. Malik, “From contours to regions: An empirical evaluation,” in IEEE Conference on Computer Vision
and Pattern Recognition, 2009, pp. 2294–2301.
[5] L. Chen, C. Chen, and M. Lu, “A multiple-kernel fuzzy c-means algorithm
for image segmentation,” IEEE Transactions on Systems, Man, and Cybernetics, Part B: Cybernetics, vol. 41, no. 5, pp. 1263–1274, Oct. 2011.
[6] S.-Y. Chien, Y.-W. Huang, and L.-G. Chen, “Predictive watershed: A
fast watershed algorithm for video segmentation,” IEEE Transactions on
Circuits and Systems for Video Technology, vol. 13, no. 5, pp. 453–461,
May 2003.
[7] M. Segundo, L. Silva, O. Bellon, and C. Queirolo, “Automatic face segmentation and facial landmark detection in range images,” IEEE Transactions on Systems, Man, and Cybernetics, Part B: Cybernetics, vol. 40,
no. 5, pp. 1319–1330, Oct. 2010.
[8] H. Li and K. N. Ngan, “Unsupervised video segmentation with low depthof field,” IEEE Transactions on Circuits and Systems for Video Technology, vol. 17, no. 12, pp. 1742–1751, Dec. 2007.
[9] N. Jacobson, Y. L. Lee, V. Mahadevan, N. Vasconcelos, and T. Q. Nguyen,“A novel approach to FRUC using discriminant saliency and frame segmentation,” IEEE Transactions on Image Processing, vol. 19, no. 11,
pp. 2924–2934, Nov. 2010.
[10] J. Han, K. N. Ngan, M. Li, and H. J. Zhang, “Unsupervised extraction ofvisual attention objects in color images,” IEEE Transactions on Circuits
and Systems for Video Technology, vol. 16, no. 1, pp. 141–145, Jan. 2006.
[11] S. Goferman, L. Z. Manor, and A. Tal, “Context-aware saliency detection,” in IEEE Conference on Computer Vision and Pattern Recognition,
2010, pp. 2376–2383.
[12] C. Guo and L. Zhang, “A novel multiresolution spatiotemporal saliency
detection model and its applications in image and video compression,”
IEEE Transactions on Image Processing, vol. 19, no. 1, pp. 185–198,
Jan. 2010.
[13] S. Paisitkriangkrai, C. Shen, and J. Zhang, “Fast pedestrian detection
using a cascade of boosted covariance features,” IEEE Transactions on
Circuits and Systems for Video Technology, vol. 18, no. 8, pp. 1140–1151,
Aug. 2008.
[14] F. Jing, M. Li, H. J. Zhang, and B. Zhang, “Relevance feedback in regionbased image retrieval,” IEEE Transactions on Circuits and Systems forVideo Technology, vol. 14, no. 5, pp. 672–681, May 2004.
[15] H. Li, K. N. Ngan, and Q. Liu, “Faceseg: Automatic face segmentation
for real-time video,” IEEE Transactions on Multimedia, vol. 11, no. 1,
pp. 77–88, Jan. 2009.
[16] Y. Y. Boykov and M. P. Jolly, “Interactive graph cuts for optimal boundary and region segmentation of objects in n-d images,” in International
Conference on Computer Vision, 2001, pp. 105–112.
[17] L. Grady, “Random walks for image segmentation,” IEEE Transactions
on Pattern Analysis and Machine Intelligence, vol. 28, no. 11, pp. 1768–
1783, Nov. 2006.
[18] J. Zhang, J. Zheng, and J. Cai, “A diffusion approach to seeded image
segmentation,” in IEEE Conference on Computer Vision and Pattern
Recognition, 2010, pp. 2125–2132.
[19] D. G. Lowe, “Distinctive image features from scale-invariant keypoints,”
International Journal of Computer Vision, vol. 60, no. 2, pp. 91–110,
2004.
[20] T. Kadir and M. Brady, “Saliency, scale and image description,” International Journal of Computer Vision, vol. 45, no. 2, pp. 83–105, 2001.
[21] K. Mikolajczyk and C. Schmid, “Scale and affine invariant interest point
detectors,” International Journal of Computer Vision, vol. 60, no. 1,
pp. 63–86, 2004.
[22] M. Cho, Y. M. Shin, and K. M. Lee, “Unsupervised detection and segmentation of identical objects,” in IEEE Conference on Computer Vision
and Pattern Recognition, 2010, pp. 1617–1624.
[23] T. Deselaers, B. Alexe, and V. Ferrari, “Localizing objects while learning
their appearance,” in European Conference on Computer Vision, 2010,
pp. 452–466.
[24] B. C. Russell, W. T. Freeman, A. A. Efros, J. Sivic, and A. Zisserman,“Using multiple segmentations to discover objects and their extent inimage collections,” in IEEE Conference on Computer Vision and Pattern
Recognition, 2006, pp. 1605–1614.
[25] K. Grauman and T. Darrell, “Unsupervised learning of categories fromsets of partially matching image features,” in IEEE Conference on Computer Vision and Pattern Recognition, 2006, pp. 19–25.
[26] C. Rother, V. Kolmogorov, T. Minka, and A. Blake, “Cosegmentation ofimage pairs by histogram matching—Incorporating a global constraint
into MRFs,” in IEEE Conference on Computer Vision and Pattern Recognition, 2006, pp. 993–1000.
[27] L. Mukherjee, V. Singh, and C. R. Dyer, “Half-integrality based algorithms for cosegmentation of images,” in IEEE Conference on Computer
Vision and Pattern Recognition, 2009, pp. 2028–2035.
[28] D. S. Hochbaum and V. Singh, “An efficient algorithm for cosegmentation,” in International Conference on Computer Vision, 2009,
pp. 269–276.
[29] S. Vicente, V. Kolmogorov, and C. Rother, “Cosegmentation revisited:
Models and optimization,” in European Conference on Computer Vision,
2010, pp. 465–479.
[30] D. Batra, D. Parikh, A. Kowdle, T. Chen, and J. Luo, “Seed image selection in interactive cosegmentation,” in IEEE International Conference on
Image Processing, 2009, pp. 2393–2396.
[31] A. Joulin, F. Bach, and J. Ponce, “Discriminative clustering for image
co-segmentation,” in IEEE Conference on Computer Vision and Pattern
Recognition, 2010, pp. 1943–1950.
[32] D. Batra, A. Kowdle, D. Parikh, J. Luo, and T. Chen, “Icoseg: Interactive
co-segmentation with intelligent scribble guidance,” in IEEE Conference
on Computer Vision and Pattern Recognition, pp. 3169–3176.
[33] S. Vicente, C. Rother, and V. Kolmogorov, “Object cosegmentation,” inIEEE Conference on Computer Vision and Pattern Recognition, 2011,
pp. 2217–2224.
[34] L. Mukherjee, V. Singh, and J. Peng, “Scale invariant cosegmentation
for image groups,” in IEEE Conference on Computer Vision and Pattern
Recognition, 2011, pp. 1881–1888.
[35] K. Chang, T. Liu, and S. Lai, “From co-saliency to co-segmentation:
An efficient and fully unsupervised energy minimization model,” in IEEE
Conference on Computer Vision and Pattern Recognition, 2011, pp. 2129–
2136.
[36] T. Riklin-Raviv, N. Sochen, and N. Kiryati, “Shape-based mutual
segmentation,” International Journal of Computer Vision, vol. 79, no. 3,
pp. 231–245, 2008.
[37] T. Riklin-Raviv, N. Sochen, and N. Kiryati, “Mutual segmentation withlevel-sets,” in IEEE CVPR Workshop on Perceptual Organization in
Computer Vision, 2006.
[38] G. Kim and E. P. Xing, “On multiple foreground cosegmentation,” inIEEE Conference on Computer Vision and Pattern Recognition, 2012.
[39] Y. Chai, V. Lempitsky, and A. Zisserman, “Bicos: A bi-level cosegmentation method for image classification,” in IEEE International
Conference on Computer Vision, 2011, pp. 2579–2586.
[40] A. C. Gallagher and T. Chen, “Clothing cosegmentation for recognizing people,” in IEEE Conference on Computer Vision and Pattern
Recognition, 2008, pp. 1–8.
[41] M. Kass, A. Witkin, and D. Terzopoulos, “Snakes: Active contour
models,” International Journal of Computer Vision, vol. 1, no. 4, pp. 321–
331, 1987.
[42] L. D. Cohen, “On active contour models and balloons,” CVGIP: Image
Understanding, vol. 53, no. 2, pp. 211–218, 1991.
[43] V. Caselles, R. Kimmel, and G. Sapiro, “Geodesic active contours,” International Journal of Computer Vision, vol. 22, no. 1, pp. 61–79, 1997.
[44] S. J. Besson, M. Barlaud, and G. Aubert, “Shape gradients for histogram segmentation using active contours,” in International Conference
on Computer Vision, 2003, pp. 408–415.
[45] X. Gao, B. Wang, D. Tao, and X. Li, “A relay level set method forautomatic image segmentation,” IEEE Transactions on Systems, Man,
and Cybernetics, Part B: Cybernetics, vol. 41, no. 2, pp. 518–525,
Apr. 2011.
[46] L. D. Cohen, “Avoiding local minima for deformable curves in image analysis,” in Curves and Surfaces With Applications in CAGD,
A. Le Méhauté, C. Rabut, and L. L. Schumaker, Eds, 1997, pp. 77–84.
[47] T. F. Chan and L. A. Vese, “Active contours without edges,” IEEE Transaction on Image Processing, vol. 10, no. 2, pp. 266–277, Feb. 2001.
[48] T. F. Chan, B. Y. Sandberg, and L. A. Vese, “Active contours without edge
for vector-valued images,” Journal of Visual Communication and Image
Representation, vol. 11, no. 2, pp. 130–141, Jun. 2000.
[49] N. Paragios and R. Deriche, “Geodesic active regions: A new framework
to deal with frame partition problems in computer vision,” Journal of
Visual Communication and Image Representation, vol. 13, no. 1/2,
pp. 249–268, Mar. 2002.

MENG et al.: IMAGE COSEGMENTATION[50] Q. Chen, Q. Sun, P. A. Heng, and D. Xia, “Two-stage object tracking
method based on kernel and active contour,” IEEE Transactions on Circuits and Systems for Video Technology, vol. 20, no. 4, pp. 605–609,
Apr. 2010.
[51] B. Wang, X. Gao, D. Tao, and X. Li, “A unified tensor level set for image
segmentation,” IEEE Transactions on Systems, Man, and Cybernetics,
Part B: Cybernetics, vol. 40, no. 3, pp. 857–867, Jun. 2010.
[52] C. Rother, V. Kolmogorov, and A. Blake, “Grabcut: Interactive foregroundextraction using iterated graph cuts,” ACM Transactions on Graphics,
vol. 23, no. 3, pp. 309–314, Aug. 2004.
[53] D. Cremers, M. Rousson, and R. Deriche, “A review of statistical approaches to level set segmentation: Integrating color, texture, motion
and shape,” International Journal of Computer Vision, vol. 72, no. 2,
pp. 195–215, 2007.

737

Guanghui Liu received the M.Sc. and Ph.D. degrees in electronic engineering from the University of Electronic Science and Technology of China(UESTC), Chengdu, China, in 2002 and 2005 respectively.
In 2005, he joined Samsung Electronics, Suwon,
South Korea, as a Senior Engineer. Since 2009, hehas been an Associate Professor with the Schoolof Electronic Engineering, UESTC. His general research interests include digital signal processing andtelecommunications, with emphasis on digital video
transmission, and orthogonal frequency-division multiplexing techniques. In
these areas, he has published tens of papers in refereed journals or conferences
and received more than ten patents (four U.S. granted patents).
Dr. Liu served as the Publication Chair of the 2010 International Symposiumon Intelligent Signal Processing and Communication Systems.

Fanman Meng received the B.Sc. degree in computer science and technology in Shandong Agricultural University, Taian, China, 2006, and the
M.Sc. degree in computer software and theory in
Xihua University, Chengdu, China, 2009, respectively. Since 2009, he has been working toward the
Ph.D. degree in the Intelligent Visual Information
Processing and Communication Laboratory, University of Electronic Science and Technology of China,
Chengdu, China.
His research interests include image segmentation,
object detection, and visual attention.

Hongliang Li (SM’12) received the Ph.D. degree inelectronics and information engineering from Xi’an
Jiaotong University, Xi’an, China, in 2005.
From 2005 to 2006, he was a Research Associatewith the Visual Signal Processing and Communication Laboratory, The Chinese University of Hong
Kong, Shatin, Hong Kong, where he was a Postdoctoral Fellow from 2006 to 2008. He is currently aProfessor with the School of Electronic Engineering,
University of Electronic Science and Technology
of China, Chengdu, China. He is a Coeditor of a
Springer book titled “Video segmentation and its applications.” He is a Memberof the Editorial Board of the Journal on Visual Communications and Image
Representation. He has authored or coauthored numerous technical articles
in well-known international journals and conferences. His research interests
include image segmentation, object detection, image and video coding, visual
attention, and multimedia communication system.
Dr. Li was involved in many professional activities. He served as a Technical Program Committee member in a number of international conferences,e.g., the 2005 International Symposium on Intelligent Signal Processing andCommunication Systems (ISPACS), PCM 2007, Pacific Rim Conference on
Multimedia 2009, and Visual Communications and Image Processing 2010;as a Technical Program Cochair in ISPACS2009; and as a General Cochairof the 2010 International Symposium on Intelligent Signal Processing andCommunication Systems. He will serve as a Local Chair of the 2014 IEEE
International Conference on Multimedia and Expo. He was selected as the New
Century Excellent Talents in University, Chinese Ministry of Education, China,
in 2008.

King Ngi Ngan (F’00) received the Ph.D. degree inelectrical engineering from Loughborough University, Loughborough, U.K.
He was a Full Professor with Nanyang Technological University, Singapore, and The University ofWestern Australia, Perth, Australia. He is currently
a Chair Professor with the Department of Electronic
Engineering, The Chinese University of Hong Kong,
Shatin, Hong Kong. He holds honorary and visiting
professorships in numerous universities in China,
Australia, and southeast Asia. He served as an Associate Editor of the Journal on Visual Communications and Image Representation, European Association for Signal Processing Journal of Signal Processing:Image Communication, and Journal of Applied Signal Processing. He has
published extensively, including three authored books, six edited volumes, over
300 refereed technical papers, and edited nine special issues in journals. In
addition, he is the holder of ten patents in the areas of image/video coding andcommunications.
Prof. Ngan is a fellow of the Institution of Engineering and Technology(U.K.) and The Institution of Engineers Australia (Australia) and an IEEE
Distinguished Lecturer in 2006–2007. He served as an Associate Editor ofIEEE T RANSACTIONS ON C IRCUITS AND S YSTEMS FOR V IDEO T ECHNOL OGY . He chaired a number of prestigious international conferences on video
signal processing and communications and served on the advisory and technicalcommittees of numerous professional organizations. He cochaired the IEEE
International Conference on Image Processing held in Hong Kong in September
2010.