Improved framework of many‐objective evolutionary algorithm to handle cloud detection problem in satellite imagery

IET Image ProcessingVolume 14, Issue 17 p. 4795-4807 Research Article Free Access Improved framework of many-objective evolutionary algorithm to handle cloud detection problem in satellite imagery Rachana Gupta, Corresponding Author 2015rec9517@mnit.ac.in Department of Electronics and Communication Engineering, Malaviya National Institute of Technology, Jaipur, 302017 Rajasthan, IndiaSearch for more papers by this authorSatyasai Jagannath Nanda, orcid.org/0000-0002-4005-5589 Department of Electronics and Communication Engineering, Malaviya National Institute of Technology, Jaipur, 302017 Rajasthan, IndiaSearch for more papers by this author Rachana Gupta, Corresponding Author 2015rec9517@mnit.ac.in Department of Electronics and Communication Engineering, Malaviya National Institute of Technology, Jaipur, 302017 Rajasthan, IndiaSearch for more papers by this authorSatyasai Jagannath Nanda, orcid.org/0000-0002-4005-5589 Department of Electronics and Communication Engineering, Malaviya National Institute of Technology, Jaipur, 302017 Rajasthan, IndiaSearch for more papers by this author First published: 24 March 2021 https://doi.org/10.1049/iet-ipr.2020.0535 AboutSectionsPDF ToolsRequest permissionExport citationAdd to favoritesTrack citation ShareShare Give accessShare full text accessShare full-text accessPlease review our Terms and Conditions of Use and check box below to share full-text version of article.I have read and accept the Wiley Online Library Terms and Conditions of UseShareable LinkUse the link below to share a full-text version of this article with your friends and colleagues. Learn more.Copy URL Share a linkShare onEmailFacebookTwitterLinked InRedditWechat Abstract Automatic cloud detection algorithm based on supervised learning approach has emerged due to its effectiveness in extracting weather information in satellite images. However, algorithm requires field-expert intervention with huge database of training samples to evaluate its clustering performance. Moreover, lacking in availability of labelled data makes difficult to train the input samples. Therefore, this article puts forward unsupervised many-objective evolutionary clustering technique to discriminate cloudy regions on varying characteristic of underlying surfaces. The study begins with the modification in search capability of θ -NSGA-III optimisation algorithm by incorporating penalised vector angle concept in associate operator. The analysis of proposed approach has been carried out on benchmark many-objective DTLZ test problems, compared against original θ -NSGA-III and NSGA-III algorithms. The proposed modified θ -NSGA-III has been further utilised as clustering technique to solve unsupervised cloud detection problem. Optimal centroid vector for clustering using proposed approach is obtained through modified crossover operator, mutation operator and environmental selection method. Experimental results reveal that proposed approach outperforms comparative many-objective algorithms, MOEA/D and NSGA-III for Landsat 8, MODIS and NOAA satellite images with lower classification average error of 2.44 % in cloud detection for most of the evaluated test cases. 1 Introduction Remote sensing images provide significant information on global change monitoring [1], land-use [2], agriculture science [3] and examining marine environment and climatic studies [4]. However, the images with cloud contaminated regions do not give efficient surface and atmospheric information. This affects various post-processing of satellite images such as segmentation [5], generation of three-dimensional surface [6] and image matching [7]. Moreover, cloudless satellite images is important for temporal data to obtain large observation range. Therefore, cloud detection is an important initial approach to assess information from satellite images. Initially, few studies of cloud detection algorithm have been observed by using spectral parameters such as irradiance and reflectance to obtain thickness, density and height of clouds [8, 9]. Further, textural parameters were used to obtain features of clouds within a region like rough, striped, ruffle and so on [10, 11]. Some studies were observed based on albedo parameter which has been evaluated by measuring the difference in the heating effect between land and ocean territories and to get the climatic information [12, 13]. Thereafter, the 1992 kick started an era of supervised neural network learning technique where spectral, textural and albedo parameters were used as input attributes in order to train the artificial network system and detect the cloud contaminated regions from satellite images [14-16]. As a confinement of research on cloud detection, this paper poses the problem of cloud detection as clustering problem in the raw intensity space without computing attributes which reduces complexity of the algorithm. Moreover, cloud detection problem is handled by using clustering technique because of lacking in availability of cloudy labelled data to train the supervised system. Clustering is an unsupervised classification technique where the input space is distinguished into K different clusters based on similarity metric. The main aim of any clustering technique is to generate a centroid vector, C K λ of input data X ∈ x 1 λ , x 2 λ , … , x n λ , with n samples and λ decision variables. Many clustering algorithms have been developed by computing the dissimilarity between the points of a database using standard Euclidean distance such as K-means, K-medoids and Hierarchical algorithms (e.g. average linkage) [17]. It has been reported that conventional clustering techniques get stuck at local optimal solution while solving multi-modal problems. Therefore, neural network supervised learning approach has attracted many researchers to handle cloud detection problem in satellite images [15] by using spectral and textural information of clouds. However, these information are difficult to obtain due to high complexity of underlying surfaces. This leads to increase the number of neurons and layers in the neural network which increases computational complexity of the algorithm. Moreover, unavailability of labelled data in satellite images makes difficult to train the cloud detection system. The mitigation of this problem is to develop optimisation technique as clustering algorithm for effective performance. This inspired us to focus around meta-heuristics optimisation method to obtain optimal solution in order to differentiate clouds with least computational complexity and high performance accuracy. One of the recently accessible meta-heuristics technique is parallel social spider clustering algorithm for multi-spectral satellite images [18]. The algorithm uses intra-cluster distance which measures the distance among each element of a cluster with their cluster head. However, optimisation of single cluster validity parameter is not efficient for categorical data. Hence, it is important to optimise several performance measures simultaneously in order to capture different characteristics of earth's surfaces (e.g. change detection application [19]). Therefore, the extended version of social spider algorithm has been proposed, namely multi-objective social spider clustering algorithm (MOSSO) [20], to distinguish cloud in multi-spectral remote sensing images by optimising centroids using similarity (SM) and variance (VAR) measures as objective functions. However, the cloud detection using MOSSO approach found difficult over soil region due to their similar spectral features in the visible range. Therefore, it is desirable to optimise increased number of objectives simultaneously [21] which is difficult to be solved by multi-objective techniques due to two major problems. The first problem is a convergence pressure where most of the solutions become non-dominated which causes failure in dominance-based selection strategy. The second problem is the diversity among solutions in high-dimensional objective space. Thus, the idea is to enhance convergence pressure by modifying conventional Pareto dominance method which is observed in studies like ϵ -dominance [22], θ -dominance [23], fuzzy dominance, niching indicator, preference order ranking [24], L-optimality [25], sequence-based deterministic initialisation using differential evolution, NSGA-II, multi-objective immune algorithms [26], knee point driven evolutionary algorithm [27], archive-guided velocity update method in MOPSO [28], grid-based evolutionary algorithm [24], Karush–Kuhn–Tucker proximity measure [29], generalised VEGA [30], dynamic multi-objectives optimisation [31] and integration of preferences based technique to steer the search procedure in the region of interest inspite of considering entire Pareto front [32]. The second strategy to solve diversity problem is observed in algorithms like diversity ranking method [33], adaptive weight vector adjustment based MOEA/D (MOEA/D-AWA), MOEA/D with stable matching (MOEA/DSTM), archive based MOEA/D (EAG-MOEA/D), MOEA/D with online diversity metric and distance based updating MOEA/D (MOEA/D-DU) [34], NSGA-III algorithm (reference point strategy is introduced in conventional NSGA-II) [35, 36], dominance and decomposition based MOEA (MOEA/DD) [37], resource allocation strategy in decomposition based MOEA [38], multi-swarm multi-objective intelligence-based algorithm (MSMO/2D) [39], reference vector guided evolutionary algorithm [40] and recently introduced θ -NSGA-III by Gupta and Nanda [41]. Literature reveals that use of centroid vectors in decision space and associate operator of θ -NSGA-III [41] for convergence and diversity properties provide a straight forward idea to handle clustering problem for multi-spectral remote sensing images in high-dimensional objective space. The major deliverable of the present work are discussed below: A modified θ -NSGA-III method is proposed by removing perpendicular distance and using penalised vector angle concept in associate operator. θ -NSGA-III clustering technique is proposed as an efficient many-objective cloud clustering algorithm by introducing alternative crossover operator, mutation operator and a new environmental selection method to get optimal centroid vector from non-dominated solution set in many-objective space. Five distinguish objective functions are proposed as fitness function in order to increase the search capability of θ -NSGA-III clustering method to handle unsupervised cloud detection problem. Performance evaluation of proposed approach is realised on satellite images, compared with two most widely utilised many-objective methods (i.e. NSGA-III and MOEA/D). The paper is organised as follows. Section 2 narrates the idea of the proposed modified θ -NSGA-III approach. Section 3 outlines the introduction of modified θ -NSGA-III as clustering algorithm. Experimental results of the proposed θ -NSGA-III clustering technique is discussed in Section 4 to evaluate the clustering quality on multi-spectral test images. Section 5 concludes and highlights the important findings of this paper. 2 Proposed modified θ -NSGA-III algorithm Before proposed modification in θ -NSGA-III procedure is described, a brief idea of recently published many-objective θ -NSGA-III is discussed from original paper [41]. θ -NSGA-III begins with description of assuming the size of parent population ( P t ) and their offspring ( Q t ) in t th generation as N. The initial step is to sort R t ( P t ∪ Q t ) of 2 N popsize into different non-dominated fronts ( F 1 , F 2 , F 3 , …, F l ) to select optimal N members. Each non-dominated front is chosen from F 1 until last front, F l where P t + 1 first time surpasses with N members. Therefore, partial and diversified solutions are required from F l . This is accomplished by introducing a concept of associate operator in NSGA-III algorithm [35]. Associate operator in (1) incorporates perpendicular distance of each search agent ( s x ) from reference line (w) (divides objective space uniformly) to uniformly diversify the solutions in objective space d ⊥ ( s x , w ) = s x − w T s x w w 2 (1) where s x is a normalised fitness vector of obtained individual. In a different way, θ -NSGA-III has introduced a new concept of minimum vector-angle penalised distance scheme in (2) in which perpendicular distance and angle of each solution of F l is computed from each reference line [41] d θ ⊥ ( s x , w ) = min d ⊥ ( s x , w ) . 1 + t − 1 t m a x − 1 α cos ⁡ θ s x , w θ s x , w = arccos s x ∙ w ∑ i = 1 M s x 2 ∑ i = 1 M w 2 ; θ s x , w ∈ 0 , π 2 (2) where t and t max are the current and maximum iteration of the algorithm, respectively. 2.1 Proposed modification in associate operator As already discussed, minimum vector-angle penalised distance scheme has been utilised in associate operator to maintain diversity among solutions of F l in original θ -NSGA-III method. In this work, vector angle d θ ( s x , w ) concept is introduced to promote diversity among the solutions by using angular parameter and removing perpendicular distance parameter from associate operator. Here, associate operator is utilised for the solutions corresponding to F 1 to F l unlike existing algorithms where associate operator is utilised only to maintain diversity for the solutions of F l . This is mathematically formulated in (3) which demonstrates the vector product on two linearly independent vectors, s x and w d θ ( s x , w ) = s x w sin Π θ s x , w 180 ; θ s x , w ∈ 0 , π 2 (3) Fig. 1 illustrates the associate operator in three-dimensional normalised objective space. As stated by vector angle concept, obtained solutions are associated with each reference line. If these vectors have same direction or either of these vectors has zero length, then d θ ( s x , w ) becomes zero. Hence, smaller d θ ( s x , w ) indicates more similar direction of s x and w. This shows that minimum value of d θ ( s x , w ) is used to associate a reference line with the closest population member in the normalised objective space. If a reference point is associated with multiple population members then random population member is selected for the next generation. Fig. 1Open in figure viewerPowerPoint Illustration of modified associate operation of obtained solutions (note: perpendicular distance is removed) The overall procedure of the proposed modified θ -NSGA-III is represented in Algorithm 1 (see Fig. 2) as a pseudo-code. It is important to note that entire procedure is equivalent to unconstrained θ -NSGA-III except associate operator in the formulation of optimisation algorithm to solve many-objective problems. Fig. 2Open in figure viewerPowerPoint Algorithm 1: procedure of the proposed modified θ -NSGA-III 2.2 Validation based on many-objective performance metric A quantitative assessment is carried out by considering two performance metrics to minimise the distance among the solutions of true Pareto front and approximated Pareto front (to maintain convergence) and maximise the equally distribution of solutions (to maintain diversity). 2.2.1 Inverted generational distance (IGD) The IGD metric (refer [42]) computes convergence and diversity of obtained solution set from the solutions of true Pareto front which is mathematical representation as IGD = ∑ i = 1 N d i N (4) where d i represents Euclidean distance among solutions of true Pareto and obtained fronts and N is a population size. 2.2.2 Delta ( Δ ) Delta ( Δ ) measures the solution spread of approximated Pareto front (AP) that covers the true Pareto front (TP) [43] which is mathematically formulated as Δ = 1 M ∑ i = 1 M min ( max ( T P , A P ) ) − max ( min ( T P , A P ) ) max ( T P ) − min ( T P ) 2 (5) where M shows the total number of objective functions. It is important to mention that small value of IGD provides a combine measure of converge and diverse solution set, whereas large Δ value represents diversity property. 2.3 Experimental parameter settings The effect of population size and number of generations are explored on three-objective DTLZ test problems in terms of IGD performance metric. The logarithmic plot of its IGD value versus population size is illustrated in Fig. 3. It is observed that population with 50 individuals give lower IGD value for most of the test problems. Moreover, population with more than 50 members give almost equal efficiency for IGD value. However, it is obvious that performance of modified θ -NSGA-III depends on rise in number of generations (t). Therefore, Fig. 4 reflects graphical representation of number of generations versus IGD performance on DTLZ test problems. Small IGD value for most of the test cases is observed with more than 250 number of generations. The evidence of Figs. 3 and 4 has encouraged to set population size with 50 individuals ( N = 50 ) and 250 number of generations ( t = 250 ) for the proposed modified θ -NSGA-III algorithm. Fig. 3Open in figure viewerPowerPoint Effect of population size on IGD performance metric Fig. 4Open in figure viewerPowerPoint Effect on IGD performance validity parameter over DTLZ test suite with N = 50 before the generation update 2.4 Results This section addresses the experimental investigation and verification of the proposed modified θ -NSGA-III algorithm by comparing with original θ -NSGA-III and NSGA-III algorithms due to their similar framework. The performance is evaluated on DTLZ test suite in terms of IGD and Δ metrics to verify convergence towards true Pareto solutions and to maintain diversity among the obtained solutions. The reference points with respect to number of objective functions and other considered parameters are tabulated in Table 1 [35]. Table 1. Parameters used in modified θ -NSGA-III method Parameters θ -NSGA-III reference points for N = 50 M = 3 55 M = 5 70 M = 8 36 crossover probability ( μ c ) 0.6 mutation probability ( μ m ) 0.6 no. of offspring ( p c ) 30 no. of mutants ( p m ) 30 All compared algorithms (modified θ -NSGA-III, θ -NSGA-III and NSGA-III) are tested on three to eight objective versions of DTLZ test problems. Fig. 5 shows that modified θ -NSGA-III consists of convergence property and finds a well-distributed solution set on entire optimal Pareto front for three-objective D T L Z 2 problem. However, it is evident from Figs. 5b–c that θ -NSGA-III and NSGA-III are also able to find better distribution of solutions but unable to converge towards true Pareto front for D T L Z 2 problem. Fig. 5Open in figure viewerPowerPoint Obtained solutions on 3-objective D T L Z 2 problem by using (a) Modified θ -NSGA-III, (b) θ -NSGA-III and, (c) NSGA-III Figs. 6–8-6–8 show the value path plot of obtained solution set for three to eight versions of D T L Z 2 problem by all the compared algorithms. The converge property corresponds to f i ∈ [0,1] and spread of solutions by proposed approach is clear from plot shown in Figs. 6–8b-6–8b. In contrast, Figs. 6–8b-6–8b show that NSGA-III is unable to obtain the converge solution set for five and eight objective problems. Moreover, Fig. 8c represents that θ -NSGA-III finds difficult to maintain diversity among the solutions for eight objective problem. Fig. 6Open in figure viewerPowerPoint Obtained solutions in value path format for three-objective DTLZ2 problem (a) Modified θ -NSGA-III, (b) θ -NSGA-III, (c) NSGA-III Fig. 7Open in figure viewerPowerPoint Obtained solutions in value path format for five-objective DTLZ2 problem (a) Modified θ -NSGA-III, (b) θ -NSGA-III, (c) NSGA-III Fig. 8Open in figure viewerPowerPoint Obtained solutions in value path format for eight-objective DTLZ2 problem (a) Modified θ -NSGA-III, (b) θ -NSGA-III, (c) NSGA-III Table 2 shows that the proposed approach is better than compared algorithms in terms of IGD value for most of the cases of DTLZ test problems with increased number of objectives, except for D T L Z 6 problem. Additionally, we have computed Δ metric by the proposed modified θ -NSGA-III and tabulated the mean and standard deviation for 20 independent runs. Large Δ value indicates well-distribution of solutions on entire optimal Pareto front using modified θ -NSGA-III. Corresponding Δ values of θ -NSGA-III and NSGA-III have also been presented. It is observed that the proposed approach produces almost similar Δ values for all the test problems. Therefore, the performance of original θ -NSGA-III is consistently observed better as compared to NSGA-III for DTLZ test problems, however proposed modified θ -NSGA-III gives more converge and diverse solution set for large number objectives over both the compared algorithms. Table 2. Mean ( ± standard deviation) of IGD and Δ by modified θ -NSGA-III, θ -NSGA-IIIand NSGA-III on M-objective D T L Z 1 , D T L Z 2 , D T L Z 3 , D T L Z 4 , D T L Z 6 and D T L Z 7 test problems Problem M Modified θ -NSGA-III θ -NSGA-III NSGA-III IGD DTLZ1 3 3.09 × 10 − 3 ( ± 7.02 × 10 − 4 ) 4.03 × 10 − 3 ( ± 5.06 × 10 − 4 ) 2.06 × 10 − 2 ( ± 8.6 × 10 − 4 ) 5 4.67 × 10 − 3 ( ± 4.16 × 10 − 4 ) 3.92 × 10 − 3 ( ± 7.09 × 10 − 4 ) 5.29 × 10 − 2 ( ± 1.7 × 10 − 4 ) 8 3.39 × 10 − 3 ( ± 8.07 × 10 − 4 ) 4.66 × 10 − 3 ( ± 4.36 × 10 − 4 ) 9.42 × 10 − 2 ( ± 6.1 × 10 − 4 ) DTLZ2 3 9.75 × 10 − 3 ( ± 8.44 × 10 − 4 ) 1.11 × 10 − 2 ( ± 9.43 × 10 − 4 ) 5.24 × 10 − 2 ( ± 1.0 × 10 − 4 ) 5 1.01 × 10 − 2 ( ± 8.98 × 10 − 4 ) 1.13 × 10 − 2 ( ± 1.22 × 10 − 3 ) 1.66 × 10 − 1 ( ± 2.3 × 10 − 4 ) 8 8.49 × 10 − 3 ( ± 1.38 × 10 − 3 ) 8.78 × 10 − 3 ( ± 6.50 × 10 − 4 ) 3.44 × 10 − 1 ( ± 9.6 × 10 − 4 ) DTLZ3 3 1.27 × 10 − 2 ( ± 1.08 × 10 − 2 ) 1.74 × 10 − 2 ( ± 1.75 × 10 − 3 ) 5.47 × 10 − 2 ( ± 7.7 × 10 − 3 ) 5 2.30 × 10 − 3 ( ± 7.1 × 10 − 3 ) 1.52 × 10 − 2 ( ± 1.91 × 10 − 3 ) 1.66 × 10 − 1 ( ± 3.5 × 10 − 4 ) 8 1.15 × 10 − 2 ( ± 1.90 × 10 − 3 ) 1.44 × 10 − 2 ( ± 2.45 × 10 − 3 ) 3.46 × 10 − 1 ( ± 5.7 × 10 − 3 ) DTLZ4 3 7.69 × 10 − 3 ( ± 4.10 × 10 − 3 ) 1.55 × 10 − 2 ( ± 3.21 × 10 − 3 ) 5.24 × 10 − 2 ( ± 2.7 × 10 − 4 ) 5 7.78 × 10 − 3 ( ± 3.62 × 10 − 3 ) 1.62 × 10 − 2 ( ± 2.59 × 10 − 3 ) 1.66 × 10 − 1 ( ± 3.5 × 10 − 4 ) 8 1.00 × 10 − 2 ( ± 2.98 × 10 − 3 ) 1.52 × 10 − 2 ( ± 2.28 × 10 − 3 ) 3.46 × 10 − 1 ( ± 5.7 × 10 − 3 ) DTLZ6 3 4.24 × 10 − 3 ( ± 8.43 × 10 − 4 ) 5.81 × 10 − 3 ( ± 5.84 × 10 − 4 ) 3.57 × 10 − 3 ( ± 2.86 × 10 − 4 ) 5 4.18 × 10 − 3 ( ± 6.78 × 10 − 4 ) 5.85 × 10 − 3 ( ± 4.16 × 10 − 4 ) 4.12 × 10 − 3 ( ± 4.95 × 10 − 4 ) 8 4.59 × 10 − 3 ( ± 2.44 × 10 − 4 ) 4.86 × 10 − 3 ( ± 2.33 × 10 − 4 ) 4.23 × 10 − 3 ( ± 2.61 × 10 − 4 ) DTLZ7 3 2.03 × 10 − 1 ( ± 1.06 × 10 − 2 ) 2.11 × 10 − 1 ( ± 1.06 × 10 − 2 ) 2.08 × 10 − 1 ( ± 1.07 × 10 − 2 ) 5 2.04 × 10 − 1 ( ± 2.27 × 10 − 3 ) 2.24 × 10 − 1 ( ± 4.16 × 10 − 4 ) 2.28 × 10 − 1 ( ± 1.70 × 10 − 3 ) 8 2.00 × 10 − 1 ( ± 3.16 × 10 − 3 ) 2.07 × 10 − 1 ( ± 2.19 × 10 − 3 ) 2.25 × 10 − 1 ( ± 4.08 × 10 − 3 ) Delta DTLZ1 3 9.78 × 10 − 1 ( ± 1.08 × 10 − 1 ) 8.61 × 10 − 1 ( ± 1.45 × 10 − 1 ) 8.38 × 10 − 1 ( ± 1.67 × 10 − 1 ) 5 1.22 × 10 ( ± 7.15 × 10 − 2 ) 1.18 × 10 ( ± 2.07 × 10 − 2 ) 8.92 × 10 − 1 ( ± 1.98 × 10 − 1 ) 8 1.22 × 10 ( ± 8.04 × 10 − 2 ) 1.17 × 10 ( ± 6.22 × 10 − 2 ) 6.05 × 10 − 1 ( ± 2.23 × 10 − 1 ) DTLZ2 3 1.08 × 10 ( ± 1.53 × 10 − 1 ) 1.04 × 10 ( ± 9.38 × 10 − 2 ) 1.03 × 10 ( ± 1.24 × 10 − 1 ) 5 1.22 × 10 ( ± 1.67 × 10 − 1 ) 1.05 × 10 ( ± 7.44 × 10 − 2 ) 1.07 × 10 ( ± 7.92 × 10 − 2 ) 8 1.01 × 10 ( ± 1.64 × 10 − 1 ) 9.24 × 10 − 1 ( ± 9.32 × 10 − 2 ) 7.98 × 10 − 1 ( ± 9.45 × 10 − 2 ) DTLZ3 3 1.22 × 10 ( ± 1.27 × 10 − 1 ) 1.06 × 10 ( ± 8.29 × 10 − 2 ) 1.02 × 10 ( ± 1.06 × 10 − 1 ) 5 1.22 × 10 ( ± 1.42 × 10 − 1 ) 1.20 × 10 ( ± 3.21 × 10 − 2 ) 1.12 × 10 ( ± 8.92 × 10 − 2 ) 8 1.22 × 10 ( ± 3.34 × 10 − 2 ) 1.22 × 10 ( ± 2.28 × 10 − 2 ) 1.05 × 10 ( ± 3.72 × 10 − 1 ) DTLZ4 3 1.22 × 10 ( ± 4.14 × 10 − 2 ) 1.05 × 10 ( ± 9.93 × 10 − 2 ) 1.21 × 10 ( ± 2.52 × 10 − 2 ) 5 1.24 × 10 ( ± 6.10 × 10 − 2 ) 1.13 × 10 ( ± 5.45 × 10 − 2 ) 1.08 × 10 ( ± 9.66 × 10 − 2 ) 8 1.22 × 10 ( ± 1.67 × 10 − 2 ) 1.11 × 10 ( ± 5.44 × 10 − 2 ) 1.22 × 10 ( ± 2.34 E − 16 ) DTLZ6 3 1.00 × 10 ( ± 1.21 × 10 − 1 ) 9.46 × 10 − 1 ( ± 9.69 × 10 − 2 ) 2.53 × 10 − 1 ( ± 1.41 × 10 − 1 ) 5 1.00