# Multivariate Self-Dual Morphological Operators Based on Extremum Constraint

Authors: Tao Lei; Yi Wang; Weiwei Luo
Journal: Mathematical Problems in Engineering
 (2015)

---

## Abstract

Self-dual morphological operators (SDMO) do not rely on whether one starts the sequence with erosion or dilation; they treat the image foreground and background identically. However, it is difficult to extend SDMO to multichannel images. Based on the self-duality property of traditional morphological operators and the theory of extremum constraint, this paper gives a complete characterization for the construction of multivariate SDMO. We introduce a pair of symmetric vector orderings (SVO) to construct multivariate dual morphological operators. Furthermore, utilizing extremum constraint to optimize multivariate morphological operators, we construct multivariate SDMO. Finally, we illustrate the importance and effectiveness of the multivariate SDMO by applications of noise removal and segmentation performance. The experimental results show that the proposed multivariate SDMO achieves better results, and they suppress noises more efficiently without losing image details compared with other filtering methods. Moreover, the proposed multivariate SDMO is also shown to have the best segmentation performance after the filtered images via watershed transformation.


---

## Body

## 1. Introduction

One of the main issues for mathematical morphology is its extension to the multivariable case [1]. So far, a lot of effort has been made to design multivariate morphological operators which can be applied in multichannel images [2–5]. The key point of constructing morphological operators for multichannel images consists in the definition of the ordering scheme for multivariate data. Thus, researchers have proposed many vector orderings, which can be divided into four categories: marginal ordering, lexicographical ordering, partial ordering, and reduced ordering in which the lexicographical ordering is one of the most popular algorithms [6]. Louverdis presented multivariate morphological operators based on lexicographical ordering in HSV color space [7]. As lexicographical ordering is most suitable for the situations in which an order of “importance” exists on the available channels, the vector ordering based on the combinations of the different vector distances (e.g. Mahalanobis distance, Euclidean distance) and lexicographical ordering was proposed by Angulo [8]. Recently, Aptoula et al. proposed α-trimmed lexicographical extremum estimation algorithms, based on the new lexicographical ordering, and constructed the new multivariate morphological operators [9, 10]. In order to make further improvement on the performance of multivariate morphological operators for color image filtering and segmentation, Angulo studied quaternion properties and applied quaternion decomposition to color image representation and then proposed a new lexicographical ordering based on quaternion decomposition, which can provide better results in image processing [11]. According to [10, 11], Lei et al. proposed vector ordering based on fuzzy extremum estimation algorithm, and then constructed multivariate morphological operators. The proposed operators have a high performance on color image filtering [12].In recent years, various complex mathematical tools, such as machine learning [13], rand projection depth function [14], principal component analysis [15], hypercomplex [16], probabilistic extrema estimation [17], and group-invariant frames [18], are employed to develop multivariate morphological theory and improve the performance on color image filtering and segmentation. Lezoray proposed nonlocal vector mathematical morphology. First, unsupervised vector ordering was proposed according to dictionary learning and machine learning. And then the improved multivariate morphological operators were proposed [19]. Velasco-Forero studied multivariate mathematical morphology based on random projection depth function and proposed vector ordering algorithm based on P-ordering. The proposed algorithm solved the problem of extending morphological theory to multichannel images [14].A lot of multivariate morphological operators have been proposed, but few researches on the properties have been reported. Most of the morphological operators occur in pairs of dual operators, such as erosion/dilation and opening/closing. In the binary case, erosion on the background of an image is equivalent to the dilation on its foreground [20]. Also in the grayscale case, the complement of erosion on the original image is equivalent to dilation on its complement. The traditional morphological operators will lead to a heavy deviation of the gray values of filtered images [21]. Therefore, the final results depend on whether erosion or the dual dilation is used as the first operator in the sequence. For example, images operated by those morphological operators of which the first operator is erosion will tend to be darker; in contrast, images operated by operators of which the first operator is the dual dilation will have the opposite results. In order to treat the image foreground and background identically, self-dual morphological operators can be applied in image processing [22]. Bouaynaya et al. studied SDMO in the case of the general theory of lattice morphology and spatially variant morphology and illustrated the importance of the self-duality property by an application to speckle noise removal in radar images [23]. Self-dual morphological operators have been widely used in binary and grayscale images. However, it is difficult to extend the applications of SDMO to multichannel images.To address this issue, we study the existing multivariate morphological operators and find that vector ordering is the key factor for the duality. By defining a pair of SVO in RGB color space, we construct multivariate dual morphological operators and multivariate SDMO. However, it is well known that VMF (vector median filters) is able to provide better results than multivariate morphological operators for color image filtering. Therefore, multivariate SDMO based on extremum constraint are finally presented. The experimental results demonstrate that the proposed multivariate SDMO performs better than the popular VMFs. As image filtering is the foundation of image segmentation and understanding, we applied watershed transformation to the images filtered by various filters. The segmentation results show that the proposed multivariate SDMO also has the best segmentation performance.The structure of this paper is organized as follows. Section2 introduces the traditional self-dual morphological operators. Section 3 analyzes the reasons why the existing multivariate morphological operators are not dual and then gives the solution of the duality of multivariate morphological operators. Multivariate self-dual morphological operators based on extremum constrain are presented in Section 4. Section 5 presents the experimental results and analysis, and Section 6 gives conclusions of this paper.


## 2. Self-Dual Morphological Operators

Before entering a general discussion on the construction of self-dual operators, the related concepts of self-dual morphological transformation are proposed.Definition 1.
The complement of an imagef, denoted by fc, is defined for each pixel h as the maximum value of the data type used for storing the image minus the value of the image f at position h: (1)fch=hmax⁡-fh,where h=h1,h2,…,hn-1,hn, hmax=maxh1,h2,…,hn-1,hn.Definition 2.
Two transformationψ and Φ are dual with respect to complementation if applying ψ to an image is equivalent to applying Φ to the complement of the image and taking the complement of the result:(2)ψ,Φare  dual  with  respect  to  complementation⟺ψ=CΦC.Definition 3.
A transformationψ is self-dual with respect to complementation C, if its dual transformation with respect to the complementation is ψ itself: (3)ψis  self-dual  with  respect  to  complementationC⟺ψ=CψC.The median operation is a self-dual neighborhood image transformation since it replaces each pixel value by the median value of the original image pixels located in a window centered at the considered pixel. The detailed proof is proposed as below. For color images, Zanoguera and Meyer studied vector leveling which are self-dual vector morphological filers and proposed a convenient algorithm for calculating a nonseparable rotation-invariant vector leveling because the algorithm which derives from the definition is rather costly. The proposed algorithm is applied to integer images without compromising convergence issues and provides better results than those obtained when independently applying a scalar leveling to the three color components [24].The traditional morphological filtering operators always have the problem that the filtered image is brighter or darker than the original image. Fortunately, self-dual morphological transformation can solve the problem; it can maintain the gray value of the filtered image. Based on dual and self-dual transformation, Heijmans proposed morphological median operators, which are also called self-dual morphological operators [22], to address the problem that the classical morphological filters are unfair for the foreground and background of an image.Proposition 4.
Letψε,ψ∗δ be a pair of dual transformation, and let ε and δ denote morphological erosion and dilation, respectively. Let ψ be an increasing transformation and let ψ∗ be the dual transformation of ψ. Thus, morphological median denoted by ω can be represented as follows:(4)ω=id∨ψε∧ψ∗δ.
Or(5)ω=id∧ψ∗δ∨ψε,where id denotes identity operator. We can demonstrate that ω is a self-dual transformation.Self-dual morphological filtering operators are applied in image filtering, and the results are shown in Figure1, where ω is the self-dual filtering operators based on opening-closing and closing-opening and the structuring element B5×5d is a disk of size 5 × 5, B5×5d=B3×3d⊕B3×3d, B3×3d=[010;111;010].Figure 1
Filtering results using morphological dual filtering operators and self-dual filtering operators. (a) Original image. (b) Image filtered by opening-closing. (c) Image filtered by closing-opening. (d) Image filtered by self-dual filtering operatorω.
(a)
(b)
(c)
(d)Figures1(b)-1(c) show that the image filtered by opening-closing turns darker, while the image filtered by closing-opening turns brighter than before. Obviously, the conventional morphological filters lead to a drift of the gray value of filtered image. Self-dual morphological filters overcome the problem; it can keep the gray value of an image since self-dual morphological filters equally treat the foreground and background of the image as shown in Figure 1(d).


## 3. Multivariate Self-Dual Morphological Operators

We firstly exhibit a group of experimental results from the existing multivariate morphological filters, as shown in Figure2, before researching multivariate dual morphological operators (γ→, ϕ→ denote multivariate opening and closing, respectively. The structuring element is a disk of size 3 × 3. Table 1 shows the popular vector orderings corresponding to filters).Table 1
The popular vector orderings corresponding to filters.
Vector orderings
Filters
Reference
VSH (value-saturation-hue)
γ
→
VSH, ϕ→VSH
[7]
DLSH (distance-luminance-saturation-hue)
γ
→
DLSH, ϕ→DLSH
[8]
a-TLO (lexicographical ordering based onα-trimmed, luminance-saturation-hue) α=0.25
γ
→
a-TLO, ϕ→a-TLO
[9]
LOQD (lexicographical ordering based on quaternion decomposition)
γ
→
LOQD, ϕ→LOQD
[11]Figure 2
Images filtered by the existing excellent multivariate morphological filters (opening and closing). (a) Original image. (b) Original image contained 20% salt and pepper noise. (c) Image filtered byγ→VSH [7]. (d) Image filtered by ϕ→VSH [7]. (e) Image filtered by γ→DLSH [8]. (f) Image filtered by ϕ→DLSH [8]. (g) Image filtered by γ→a-TLO (a=0.25) [9]. (h) Image filtered by ϕ→a-TLO (a=0.25) [9]. (i) Image filtered by γ→LOQD [11]. (j) Image filtered by ϕ→LOQD [11].
(a)
(b)
(c)
(d)
(e)
(f)
(g)
(h)
(i)
(j)As can be seen from Figure2, multivariate opening and closing have different filtering performances. γ→ that started with erosion is able to effectively suppress noise, but ϕ→ that started with dilation does not have the similar effect. ϕ→ is unable to effectively suppress noise and even enhance noise. Accordingly, we conclude that the existing multivariate morphological operators are not dual. To overcome the problem, multivariate dual morphological operators based on symmetric vector ordering (SVO) are proposed as below.Letv be a vector set, v=v1,v2,…vn, 1≤i,j≤n, vi=vir,vig,vib, and then we can define SVO (≤BED,≤WED) as (6)vi≤BEDvj⟺dBEDvi,b<dBEDvj,bordBEDvi,b=dBEDvj,b,dWEDvi,w>dWEDvj,wordBEDvi,b=dBEDvj,b,dWEDvi,w=dWEDvj,w,gi<gjorgi=gj,ri<rjorgi=gj,ri=rj,bi≤bj,vi≤WEDvj⟺dWEDvi,w>dWEDvj,wordWEDvi,w=dWEDvj,w,dBEDvi,b<dBEDvj,bordWEDvi,w=dWEDvj,w,dBEDvi,b=dBEDvj,b,gi<gjorgi=gj,ri<rjorgi=gj,ri=rj,bi≤bj,where(7)dBEDvi,b=vir-br2+vig-bg2+vib-bb2,dWEDvi,w=wr-vir2+wg-vig2+wb-vib2and b=br,bg,bb=0,0,0, w=wr,wg,wb=255,255,255. The multivariate extremum can be defined according to (6)-(7)(8)∧SVOv=vi∈v∣vi≤BEDvj,∀vj∈v,∨SVOv=vi∈v∣vi≤WEDvj,∀vj∈v.Furthermore, multivariate erosion and dilation based on SVO can be defined as(9)ε→SVOfx=fy:fy=∧SVOft,t∈Bx,δ→SVOfx=fy:fy=∨SVOft,t∈Bx,where f represents a multichannel image and B denotes a structuring element. Depending on the duality of morphological operators, δ→SVOfcc=ε→SVOf and ε→SVO,δ→SVO is therefore a pair of dual morphological operators.Based on the classical morphological combination operators and the definition ofε→SVO,δ→SVO, multivariate opening and closing are denoted by γ→SVO and ϕ→SVO, respectively,(10)γ→SVOf=δ→SVOε→SVOf,(11)ϕ→SVOf=ε→SVOδ→SVOf.We proposed dual multivariate morphological operatorsε→SVO,δ→SVO. According to Definition 3, we can propose the definition of multivariate self-dual morphological operators.Proposition 5.
Assuming thatψ→SVOε→SVO,ψ→SVO∗δ→SVO is a pair of dual transformation, ε→SVO and δ→SVO denote multivariate morphological erosion and dilation, respectively. ψ→SVO is an increasing transformation, ψ→SVO∗ is the dual transformation of ψ→SVO, the multivariate self-dual morphological transformation ω→ can be defined as follows:(12)ω→SVO=id∨SVOψ→SVOε→SVO∧WEDψ→SVO∗δ→SVO.Or(13)ω→SVO=id∧WEDψ→SVO∗δ→SVO∨SVOψ→SVOε→SVO.
To demonstrate thatω→ is able to maintain the brightness, saturation, and hue of color image, Figure 3 shows images processed by multivariate dual morphological operators and multivariate self-dual morphological operators (the structuring element is a square of size 3×3, B3×3s=[111;111;111]).Figure 3
Images filtered by multivariate dual and self-dual morphological filters. (a) Original image. (b1) Image filtered byε→SVO. (c1) Image filtered by δ→SVO. (d1) Image filtered by ω→SVOε→SVO,δ→SVO. (b2) Image filtered by γ→SVO. (c2) Image filtered by ϕ→SVO. (d2) Image filtered by ω→SVOγ→SVO,ϕ→SVO. (b3) Image filtered by ϕ→SVOγ→SVO. (c3) Image filtered by γ→SVOϕ→SVO. (d3) Image filtered by ω→SVOϕ→SVOγ→SVO,γ→SVOϕ→SVO.For multivariate self-dual morphological operators,ω→SVOε→SVO,δ→SVO=id∨BEDε→SVO∧WEDδ→SVO, where ψ→SVO=ψSVO∗=id; ω→SVOγ→SVO,ϕ→SVO=id∨BEDγ→SVO∧WEDϕ→SVO, where ψ→SVO=δ→SVO, ψ→SVO∗=ε→SVO; ω→SVOϕ→SVOγ→SVO,γ→SVOϕ→SVO=id∨BEDϕ→SVOγ→SVO∧WEDγ→SVOϕ→SVO, where ψ→SVO=ϕ→SVOδ→SVO, ψ→SVO∗=γ→SVOε→SVO. As can be seen from Figure 3, images, shown in Figures 3(b1), 3(b2), and 3(b3), filtered by filtering operators that started with erosion will be darker than before. Similarly, images, seen in Figures 3(c1), 3(c2), and 3(c3), filtered by filtering operators that started with dilation will be brighter than before. However, Figures 3(d1), 3(d2), and 3(d3) show that images filtered by ω→ maintain the gray value. In order to display them clearer, the histograms of Figure 3 in HSV color space are shown in Figure 4.Figure 4
Histograms of Figure3 in HSV color space. (a) H component. (b) S component. (c) V component.
(a)
(b)
(c)


## 4. Multivariate Self-Dual Morphological Operators

As SDMO treats both bright and dark structures of an image in an identical way, it can suppress noise effectively and preserve the edges. However, for multichannel image filtering, VMF is more popular than SDMO since the former can achieve better effect than the latter. For different VMF, the popular method detects noise first and uses a switch to control filters. Consequently, the intelligent algorithm must be considered as a key step in multivariate SDMO in order to improve their performances.We analyze the basic theory of traditional morphological operators and soft morphological operators. Multivariate soft morphological erosion and dilation based on SVO are defined as [25, 26](14)ε→SVOsoftf,B,A,rx=r⋄fa:a∈Ax∪fb:b∈B∖Ax︸Therth  smallest  value  of  the  vector  set,δ→SVOsoftf,B,A,rx=r⋄fa:a∈Ax∪fb:b∈B∖Ax︸Therth  largest  value  of  the  vector  set,where r⋄x=x,…,x︸rtimes, ⋄ represents repetitive operation of any object. B is a structuring element. A is called hard center; B∖A is called soft boundary and r is the repetitive parameter.Figures5(a)-5(b) explain the basic principle of ε→SVO and ε→SVOsoft. We can see that the basic morphological operator ε→SVO outputs the first vector v1′ and soft morphological operator ε→SVOsoft outputs the rth vector vr′ (when r=1, ε→SVO=ε→SVOsoft). As an example, Figure 6 illustrates the results of ε→SVO and δ→SVO for a color block, where ε→SVO=131,0,109, δ→SVO=142,131,255. ε→SVO and δ→SVO are all noise pixels and ε→SVOsoft is noise pixel in the case of r=1,2,3,4; δ→SVOsoft is noise pixel in the case of r=1,2,3; ε→SVOsoft and δ→SVOsoft are all nonnoise pixels when r=5. However, how to decide r is a problem in soft morphological operators.Figure 5
The principle of different erosion operation. (a) Erosion. (b) Soft erosion. (c) Erosion based on extremum constraint.
(a)
(b)
(c)Figure 6
Results from multivariate erosion and dilation, multivariate soft-erosion, and dilation based on≤BED and ≤WED (the structuring element is a square of size 7 × 7, B7×7d=B5×5d⊕B3×3d). (a) A color block of size 7 × 7. (b) Figure 3(a) contained 5% salt and pepper noise. (c) Sorting result of color pixels using ε→SVO. (d) Sorting result of color pixels using δ→SVO. (e) Sorting result of color pixels using ε→SVOsoft (in the case of different r). (f) Sorting result of color pixels using δ→SVOsoft (in the case of different r).
(a)
(b)
(c)
(d)
(e)
(f)For the problem mentioned above, Evans and Liu proposed a novel pairwise pixel rejection scheme which works by removing the two pixels that are furthest apart and then finding the color morphological gradient of the remaining pixels. Based on the scheme, a morphological gradient approach to color edge detection is proposed to improve the performance in the presence of noise [27]. In fact, this method employs extremum removal to reduce noise. In addition, Gimenez and Evans believed that the extrema definition used by color sieve is of critical importance for its subsequent performance [28]. Inspired by these ideas, we proposed a novel algorithm of extremum computation based on extremum constraint. In the new algorithm, a constraint condition is used in extremum computation according to the principal of switched vector median filter. Figure 5(c) illustrates the principal of ε→SVOEC. And a more formal description for computing the extremum based on this procedure is given in Algorithm 1.Algorithm 1:MEEEC (multivariate extremum estimation based on extremum constraint) aims to improve multivariate morphological filters (as an example, the infimum based on extremum constraint ∧SVOEC is proposed).
Input:  Letv={vj}1≤j≤n be a p-dimension vector set.  vj=(vj1,vj2,…,vjp) denotes
ap-dimension vector, i=1, vmax⁡=max⁡{vjh}1≤j≤n,1≤h≤p;
Output:vi′ is the infimum of v;
(1) Sort in ascending order the vectors ofv with respect to the ≤BED;
v
′
←
v
v
j
∣
≤
SVO;
(2) Extremum constraint
Ifvmax⁡∈vi′
If i≥n-1/2
i=n-1/2; go to step (3);
else
i=i+1, back to the step (2);
end
Else
Go to step (3);
end
(3)∧SVOEC=vi′As to the supremum, it can be obtained in a likewise mode by simply sorting in an ascending order≤WED. Consequently, the resulting extrema can be used in order to define the operators in (9) for multivariate pixels. An illustration of this approach is given in Figure 5(c). And depending on Algorithm 1, ∧SVOEC and ∨SVOEC are defined as(15)∧SVOECv=vi∈v∣vi≤SVOvj,vi∉vEC,vj∉vEC,∀vj∈vvEC≠ϕvi∈v∣vi≤SVOvj,∀vj∈vvEC=ϕ∨SVOECv=vi∈v∣vj≤SVOvi,vi∉vEC,vj∉vEC,∀vj∈vvEC≠ϕvi∈v∣vj≤SVOvi,∀vj∈vvEC=ϕ,where vEC=ECv=vi∣maxvjh1≤j≤n,1≤h≤p∉vi,∀vi∈v.According to (9) and (15), the basic multivariate morphological operators based on extremum constrain are defined as follows:(16)ε→SVOECfx=fy:fy=∧SVOECft,t∈Bx,δ→SVOECfx=fy:fy=∨SVOECft,t∈Bx,whereε→BEDEC and δ→WEDEC are performed on Figure 6(b) and the results are shown in Figure 7.Figure 7
Sorting the results usingε→SVOEC and δ→SVOEC on Figure 6(b). (a) Sorting the result using ε→SVOEC. (b) Sorting the result using δ→SVOEC.
(a)
(b)Figure7 shows that the optimized ε→SVOEC and δ→SVOEC output antinoise pixels. Therefore, we can conclude that MMO-EC can provide better antinoise filtering results than the existing multivariate morphological operators. According to the definitions of SDMO and ε→SVOEC,δ→SVOEC, we can define multivariate self-dual morphological operators based on extremum constraint (MSDMO-EC) as follows:(17)ω→SVOEC=id∨SVOψSVOε→SVOEC∧SVOψSVO∗δ→SVOEC,where ψSVO is a multivariate morphological operators composed by ε→SVOsoft and δ→SVOsoft and ψSVO∗ is the dual operation of ψSVO. According to Definition 3, ω→SVOEC is also a multivariate self-dual morphological operator because ω→SVOECf=ω→SVOECfcc is obtained according to ω→SVOECfc=ω→SVOECfc.


## 5. Experimental Results and Analysis

### 5.1. Noise Reduction

In this section, the proposed MSDMO-EC has been tested on more than 50 color images taken from public image databases. The image “Butterfly” is selected to show the capability of the proposed MSDMO-EC and is evaluated in detail to highlight the advantages of the algorithm. As mentioned above, VMF is one of the most popular approaches for impulsive noise removing, originally introduced by Astola et al. [29]. The process consists of replacing the pixel value by the median of the pixels contained in a window around it. The existing VMF can provide better filtering results than various multivariate morphological filters. Consequently, for the evaluation of the efficiency of the proposed MSDMF-EC, the comparison with the following filtering operators was performed:(1)
VMF: vector median filter (1990) [29](2)
AVMF: adaptive vector median filter (2003) [30](3)
FPGF: fast peer group filter (2005) [31](4)
LABSVMF: switching vector median filter in CIELAB color space (2007) [32](5)
RSVMF: robust switching vector median filter (2008) [33](6)
PSGF: peer group switching filter (2010) [34](7)
FRF: fuzzy rank-ordered differences filter (2010) [35](8)
QSF: quaternion switching filter (2012) [36].Figures8(a) and 8(a1) show the original image “Butterfly,” and Figures 8(b) and 8(b1) show the original image contaminated by 25% salt and pepper noise. Figures 8(c)–8(j) and 8(c1)–8(j1) show images filtered by different VMF and the proposed MSDMO-EC (for comparing the performance of various filters fairly, the window of VMF is a square of size 3 × 3 and the structuring element of MSDMO-EC is also a square of size 3 × 3. ψ=δ→WEDsoft, ψ∗=ε→WEDsoft, r=2).Figure 8
Filtering results for (b, b1) using different methods. ((a), (a1)) Original image. ((b), (b1)) The original image contained 25% salt and pepper noise. ((c), (c1)) Image filtered by VMF. ((d), (d1)) Image filtered by AVMF. ((e), (e1)) Image filtered by FPGF. ((f), (f1)) Image filtered by LABSVMF. ((g), (g1)) Image filtered by RSVMF. ((h), (h1)) Image filtered by FRF. ((i), (i1)) Image filtered by QSF. ((j), (j1)) Image filtered by MSDMO-EC.Since VMF and the proposed MSDMO-EC are all self-dual operation, they can preserve the brightness, saturation, and hue of original image shown in Figures8(c)–8(j) and 8(c1)–8(j1). Thus, the proposed MSDMO-EC overcomes the problem that the traditional vector morphological operators will lead to a heavily migration of the brightness, saturation, and hue of original image.We can see that MSDMO-EC has the best performance in suppressing noise and preserving edges and structural features. In Figures8(c)–8(i) and 8(c1)–8(i1), a lot of noise pixels are preserved or amplified while few noise pixels are preserved in Figures 8(j) and 8(j1). Obviously, the proposed MSDMO-EC is suitable and excellent for impulsive noise removing in multichannel image processing.Figure9 shows the dependence of the average SSIM, MSE, and NCD for 50 test images in the noise intensity ranging from 0.05% to 40%. The comparison with different filtering operators for noise reduction shows that the proposed MSDMF-EC can provide better results in the case of noise level over 10%.Figure 9
Comparison of various morphological filters in terms of the average SSIM, MSE, and NCD for 50 test images in different levels of salt and pepper noise. (a) SSIM. (b) MSE. (c) NCD.
(a)
(b)
(c)Three quantitative measures: peak signal-to-noise ratio (PSNR), mean square error (MSE), and normalized color difference (NCD) are commonly used for the measurement of the restoration quality. Because structural similarity index measurement system (SSIM) provides a better measurement to image quality assessment, we use SSIM to replace PSNR [37, 38](18)MSEf,g=13M×N∑i=1M∑j=1Nfi,j-gi,j2,NCDf,g=∑i=1M∑j=1NfL∗a∗b∗i,j-gL∗a∗b∗i,j2∑i=1M∑j=1NfL∗a∗b∗i,j2,SSIMf,g=2μfμg+C12μfg+C2μf2+μg2+C1σf2+σg2+C2,where μf and μg denote the average value (average brightness) of f and g, respectively. σf and σg represent the variance of f and g. σfg is the covariance between f and g. C1 and C2 are the extremely minimum constant to avoid the case in which denominator is zero.As can be seen from Figure9, the various VMFs and MSDMF-EC are employed to restore image contaminated by different noise levels. Among these methods, the proposed MSDMF-EC provides higher SSIM and lower MSE and NCD. In addition, the experimental data show that the performance of different filtering operators is relatively close in the case of low noise levels. But with the increase of noise levels, the performance of various improved VMF declines quickly, even lower than the classical VMF. Obviously, the proposed MSDMF-EC is better than the traditional multivariate morphological filtering operators for maintaining brightness, hue, and saturation of images. Moreover, they have a significant advantage in handling salt and pepper noise pollution.


### 5.2. Color Image Segmentation

Watershed transformation is an efficient tool for image segmentation. However, a direct computation of the watersheds of the segmentation function produces an oversegmentation which is due to the presence of spurious minima. Consequently, the image must be filtered before computing its watersheds so as to remove all irrelevant minima. It is obvious that the result of watershed transformation relies on the filtering technique. In the experiments, we applied watershed transformation to filtered images to compare the performance of different filters. According to the popular watershed approaches reported in [39–41], we propose the general steps of color image segmentation based on watershed transformation:(1)
image filtering using the filter;(2)
calculating gradient using vector gradient approach proposed by di Zenzo [42];(3)
gradient reconstruction using morphological closing-opening reconstruction operators [40];(4)
modifying gradient image using minimal imposition [43];(5)
watershed transformation.According to the proposed segmentation approach and filtered images in Figure8, we can compare the performance of various filters for color image segmentation. Figure 10 illustrates the segmentation results in terms of different approaches. It is obvious that the proposed MSDMF-EC is able to provide better preprocessing result for subsequent image segmentation (in step (3), the structuring element is a disk of size 4 × 4; in step (4), the gradient image is normalized and h=0.03).Figure 10
Comparison of various morphological filters in terms of the segmentation results for Figures8(c1)–8(j1). (a) Segmentation results for Figure 8(c1). (b) Segmentation results for Figure 8(d1). (c) Segmentation results for Figure 8(e1). (d) Segmentation results for Figure 8(f1). (e) Segmentation results for Figure 8(g1). (f) Segmentation results for Figure 8(h1). (g) Segmentation results for Figure 8(i1). (h) Segmentation results for Figure 8(j1).
(a)
(b)
(c)
(d)
(e)
(f)
(g)
(h)The proposed MSDMF-EC has a good performance for noise reduction and subsequent image segmentation. In order to further prove the performance of the proposed MSDMF-EC, we test the methods on natural image. Since natural images include various kinds of noise, it is necessary to apply the proposed morphological operators to natural images such as the standard Berkeley segmentation dataset. Figure11 shows the segmentation results of natural images from BSD500. Comparing the performance of various filters fairly, the window of VMF is a square of size 3 × 3; the structuring element of opening-closing filter and MSDMO-EC is also a square of size 3 × 3 (for image segmentation, in step (3), the structuring element is a disk of size 4 × 4; in step (4), the gradient image is normalized and h=0.03).Figure 11
Comparison of various morphological filters in terms of the segmentation results for natural image. (a) Ground-truth boundaries. (b) Segmentation results for images filtered by morphological opening-closing filter. (c) Segmentation results for images filtered by the VMF. (d) Segmentation results for images filtered by the MSDMO-EC.
(a)
(b)
(c)
(d)As can be seen from Figure11, Figure 11(d) is the closest to Figure 11(a). In other words, the proposed MSDMO-EC can provide the best filtered images for watershed transformation.


## 5.1. Noise Reduction

In this section, the proposed MSDMO-EC has been tested on more than 50 color images taken from public image databases. The image “Butterfly” is selected to show the capability of the proposed MSDMO-EC and is evaluated in detail to highlight the advantages of the algorithm. As mentioned above, VMF is one of the most popular approaches for impulsive noise removing, originally introduced by Astola et al. [29]. The process consists of replacing the pixel value by the median of the pixels contained in a window around it. The existing VMF can provide better filtering results than various multivariate morphological filters. Consequently, for the evaluation of the efficiency of the proposed MSDMF-EC, the comparison with the following filtering operators was performed:(1)
VMF: vector median filter (1990) [29](2)
AVMF: adaptive vector median filter (2003) [30](3)
FPGF: fast peer group filter (2005) [31](4)
LABSVMF: switching vector median filter in CIELAB color space (2007) [32](5)
RSVMF: robust switching vector median filter (2008) [33](6)
PSGF: peer group switching filter (2010) [34](7)
FRF: fuzzy rank-ordered differences filter (2010) [35](8)
QSF: quaternion switching filter (2012) [36].Figures8(a) and 8(a1) show the original image “Butterfly,” and Figures 8(b) and 8(b1) show the original image contaminated by 25% salt and pepper noise. Figures 8(c)–8(j) and 8(c1)–8(j1) show images filtered by different VMF and the proposed MSDMO-EC (for comparing the performance of various filters fairly, the window of VMF is a square of size 3 × 3 and the structuring element of MSDMO-EC is also a square of size 3 × 3. ψ=δ→WEDsoft, ψ∗=ε→WEDsoft, r=2).Figure 8
Filtering results for (b, b1) using different methods. ((a), (a1)) Original image. ((b), (b1)) The original image contained 25% salt and pepper noise. ((c), (c1)) Image filtered by VMF. ((d), (d1)) Image filtered by AVMF. ((e), (e1)) Image filtered by FPGF. ((f), (f1)) Image filtered by LABSVMF. ((g), (g1)) Image filtered by RSVMF. ((h), (h1)) Image filtered by FRF. ((i), (i1)) Image filtered by QSF. ((j), (j1)) Image filtered by MSDMO-EC.Since VMF and the proposed MSDMO-EC are all self-dual operation, they can preserve the brightness, saturation, and hue of original image shown in Figures8(c)–8(j) and 8(c1)–8(j1). Thus, the proposed MSDMO-EC overcomes the problem that the traditional vector morphological operators will lead to a heavily migration of the brightness, saturation, and hue of original image.We can see that MSDMO-EC has the best performance in suppressing noise and preserving edges and structural features. In Figures8(c)–8(i) and 8(c1)–8(i1), a lot of noise pixels are preserved or amplified while few noise pixels are preserved in Figures 8(j) and 8(j1). Obviously, the proposed MSDMO-EC is suitable and excellent for impulsive noise removing in multichannel image processing.Figure9 shows the dependence of the average SSIM, MSE, and NCD for 50 test images in the noise intensity ranging from 0.05% to 40%. The comparison with different filtering operators for noise reduction shows that the proposed MSDMF-EC can provide better results in the case of noise level over 10%.Figure 9
Comparison of various morphological filters in terms of the average SSIM, MSE, and NCD for 50 test images in different levels of salt and pepper noise. (a) SSIM. (b) MSE. (c) NCD.
(a)
(b)
(c)Three quantitative measures: peak signal-to-noise ratio (PSNR), mean square error (MSE), and normalized color difference (NCD) are commonly used for the measurement of the restoration quality. Because structural similarity index measurement system (SSIM) provides a better measurement to image quality assessment, we use SSIM to replace PSNR [37, 38](18)MSEf,g=13M×N∑i=1M∑j=1Nfi,j-gi,j2,NCDf,g=∑i=1M∑j=1NfL∗a∗b∗i,j-gL∗a∗b∗i,j2∑i=1M∑j=1NfL∗a∗b∗i,j2,SSIMf,g=2μfμg+C12μfg+C2μf2+μg2+C1σf2+σg2+C2,where μf and μg denote the average value (average brightness) of f and g, respectively. σf and σg represent the variance of f and g. σfg is the covariance between f and g. C1 and C2 are the extremely minimum constant to avoid the case in which denominator is zero.As can be seen from Figure9, the various VMFs and MSDMF-EC are employed to restore image contaminated by different noise levels. Among these methods, the proposed MSDMF-EC provides higher SSIM and lower MSE and NCD. In addition, the experimental data show that the performance of different filtering operators is relatively close in the case of low noise levels. But with the increase of noise levels, the performance of various improved VMF declines quickly, even lower than the classical VMF. Obviously, the proposed MSDMF-EC is better than the traditional multivariate morphological filtering operators for maintaining brightness, hue, and saturation of images. Moreover, they have a significant advantage in handling salt and pepper noise pollution.


## 5.2. Color Image Segmentation

Watershed transformation is an efficient tool for image segmentation. However, a direct computation of the watersheds of the segmentation function produces an oversegmentation which is due to the presence of spurious minima. Consequently, the image must be filtered before computing its watersheds so as to remove all irrelevant minima. It is obvious that the result of watershed transformation relies on the filtering technique. In the experiments, we applied watershed transformation to filtered images to compare the performance of different filters. According to the popular watershed approaches reported in [39–41], we propose the general steps of color image segmentation based on watershed transformation:(1)
image filtering using the filter;(2)
calculating gradient using vector gradient approach proposed by di Zenzo [42];(3)
gradient reconstruction using morphological closing-opening reconstruction operators [40];(4)
modifying gradient image using minimal imposition [43];(5)
watershed transformation.According to the proposed segmentation approach and filtered images in Figure8, we can compare the performance of various filters for color image segmentation. Figure 10 illustrates the segmentation results in terms of different approaches. It is obvious that the proposed MSDMF-EC is able to provide better preprocessing result for subsequent image segmentation (in step (3), the structuring element is a disk of size 4 × 4; in step (4), the gradient image is normalized and h=0.03).Figure 10
Comparison of various morphological filters in terms of the segmentation results for Figures8(c1)–8(j1). (a) Segmentation results for Figure 8(c1). (b) Segmentation results for Figure 8(d1). (c) Segmentation results for Figure 8(e1). (d) Segmentation results for Figure 8(f1). (e) Segmentation results for Figure 8(g1). (f) Segmentation results for Figure 8(h1). (g) Segmentation results for Figure 8(i1). (h) Segmentation results for Figure 8(j1).
(a)
(b)
(c)
(d)
(e)
(f)
(g)
(h)The proposed MSDMF-EC has a good performance for noise reduction and subsequent image segmentation. In order to further prove the performance of the proposed MSDMF-EC, we test the methods on natural image. Since natural images include various kinds of noise, it is necessary to apply the proposed morphological operators to natural images such as the standard Berkeley segmentation dataset. Figure11 shows the segmentation results of natural images from BSD500. Comparing the performance of various filters fairly, the window of VMF is a square of size 3 × 3; the structuring element of opening-closing filter and MSDMO-EC is also a square of size 3 × 3 (for image segmentation, in step (3), the structuring element is a disk of size 4 × 4; in step (4), the gradient image is normalized and h=0.03).Figure 11
Comparison of various morphological filters in terms of the segmentation results for natural image. (a) Ground-truth boundaries. (b) Segmentation results for images filtered by morphological opening-closing filter. (c) Segmentation results for images filtered by the VMF. (d) Segmentation results for images filtered by the MSDMO-EC.
(a)
(b)
(c)
(d)As can be seen from Figure11, Figure 11(d) is the closest to Figure 11(a). In other words, the proposed MSDMO-EC can provide the best filtered images for watershed transformation.


## 6. Conclusions

In this study, we aim to find out why the existing multivariate morphological operators do not satisfy the duality. Motivated from this, we introduced a pair of symmetric vector orderings and applied it to construct multivariate dual morphological operators which can be used for multichannel image processing. Besides, we proposed MSDMO-EC based on the principle of extremum constraint to improve the performance of multivariate morphological operators for noise removal in multichannel images. Moreover, the advantages of the proposed operators are illustrated by two examples of color image filtering and segmentation. The experimental results demonstrate that the proposed MSDMO-EC is able to not only suppress noise efficiently but also provide better segmentation results.



---

*Source: 596348-2015-07-13.xml*