# Personal Communication Technologies for Smart Spaces Density-Based Clustering for Content and Color Adaptive Tone Mapping

Authors: Maleeha Javed; Hassan Dawood; Muhammad Murtaza Khan; Ameen Banjar; Riad Alharbey; Hussain Dawood
Journal: Mobile Information Systems
 (2020)

---

## Abstract

Tone mapping operators are designed to display high dynamic range (HDR) images on low dynamic range devices. Clustering-based content and color adaptive tone mapping algorithm aims to maintain the color information and local texture. However, fine details can still be lost in low dynamic range images. This paper presents an effective way of clustering-based content and color adaptive tone mapping algorithm by using fast search and find of density peak clustering. The suggested clustering method reduces the loss of local structure and allows better adaption of color in images. The experiments are carried out to evaluate the effectiveness and performance of proposed technique with state-of-the-art clustering techniques. The objective and subjective evaluation results reveal that fast search and find of density peak preserves more textural information. Therefore, it is most suitable to be used for clustering-based content and color adaptive tone mapping algorithm.


---

## Body

## 1. Introduction

The demand for high dynamic range (HDR) images is rapidly increasing with the advent of sensors, display technology, and availability of multiple exposure photography. HDR images provide better visual quality as compared to their counterpart low dynamic range (LDR) images on account of their large ranged brightness levels and better preservation of color variations. This enables to store and visualize bright and dark objects in the same image which would normally require multiple images at different exposures to be in low dynamic range. Visualization of HDR images on standard display devices (made for displaying LDR images) is challenging as they are incapable of displaying the high dynamic range of the image. To address this issue, tone mapping operators are used to convert HDR images to LDR images. However, the technology that directly displays HDR images is becoming more cost-effective. Recently, most of the available display devices have LDR, which is why HDR to LDR conversion still has practical applications.For visualization of HDR images on LDR devices, Retinex theory has been widely used, which slices the image into its components: base layer and detail layer. Edge preservation filtering techniques are used for the approximation of base layer in Retinex image decomposition. However, these methods have resulted in false coloring and halo artifacts because filtering is unable to capture complex local structure of the image. To overcome this problem, we need a method that can better adopt the local structure and color variation of HDR images in LDR images.Furthermore, tone mapping operators are generally divided into two main categories: global tone mapping operators and local tone mapping operators. Global tone mapping operators use the same monotonic curve for dynamic range compression of the entire image [1, 2]. Initially, the focus of the research was on the design of global tone mapping operators. Drago et al. [3] proposed a tone mapping algorithm that used an adaptive logarithmic base for luminance compression while maintaining the image’s details and contrast. Kim and Kautz [4] introduced a tone mapping operator based on the hypothesis that the human visual system adopts Gaussian distribution. Reinhard et al. [5] proposed the use of spontaneous dodging and burning for dynamic range compression. The terms “dodging” and “burning” originate from printing, where holding backlight from a region of print is called dodging and the addition of light to image is referred to as burning. This can be used to increase or decrease the brightness of a captured image effectively. Reinhard and Devlin [6] introduced the light adaptive tone mapping which provides satisfying visual effects. Khan et al. [7] introduced a lookup table-based approach, where they have utilized the histogram of luminance for tone mapping. To produce the low dynamic range image appropriate for various viewing conditions, Han et al. [8] modify Khan et al.’s [7] method by considering the impact of ambient light on HVS. Generally, global tone mapping operators are computationally efficient [9]. However, they are unable to preserve the local characteristics and dynamic range variations as they ignore local pixel intensity variations in an image [9, 10].Local tone mapping operators solve the issue of global tone mapping operators by incorporating the ratio between neighboring pixels and compressing each pixel. Gu et al. [11] and Min et al. [12] proposed local tone mapping operators based on layer decomposition, where the input image is divided into base layer and detail layer. The layer-decomposition-based approach has also been utilized by authors in [13–16]. Durand and Dorsey [17] proposed a method that preserves the detail layer by encoding large-scale variations of the base layer. Debevec and Gibson [18] proposed a method that helped to preserve brightness and details of image by applying a local luminance adaption function for compression of dynamic range, followed by reinjecting details in the low dynamic range image. Image color appearance model (iCAM06) [19] employed tone mapping by using iCAM06 color appearance model by considering the viewing conditions to generate optimal results. However, iCAM06 [19] led to the poor visual quality due to introduction of halo artifact, color saturation, and gradient reversal at the edges. To eliminate these problems, the authors in [20] proposed an iCAM06 based model. In their proposed algorithm guided filter [20], color adaptive transformation matrix and HPE primitives were altered to enhance the effectiveness of model.Krawczyk et al. [21] introduced an algorithm based on anchoring theory that decomposed image luminance into patches and then calculated the lightness for each patch. Li et al. [22] proposed a symmetrical analysis-synthesis filter for reducing the intensity range. Jia and Zhang [23] used the guided image filter for tone mapping. Meylan et al. [24] suggested a method based on the characteristic of human retina to reduce dynamic range while increasing the local contrast of the image. Another retina inspired range compression algorithm is used [25] to overcome the halo artifacts, loss of details, different visualization across different displays, and color saturation.Parraga and Otazu [26] developed a tone mapping method based on human color perception by dividing the image intensity into multiresolution contrast. Also, they had used a nonlinear saturation model for dynamic range reduction based on visual cortex behavior. Chen et al. [27] introduced Earth mover’s distance to segment HDR image and then applied local tone mapping on each component of the image. This maintained local texture and balanced perceptional impression. To remove artifacts and contrast enhancement, Liang et al. [28] used nonlinear diffusion. l2-based retime model [29] was used for contrast enhancement. Ahn et al. [30] proposed a Retinex-based adaptive local tone mapping algorithm according to which the use of guided filter reduced halo artifacts. Recently, Liang et al. [31] proposed a hybrid l1 − l0 norm-based layer decomposition model. l1 sparsity term was imposed on the base layer and l0 on the detail layer. In [32], the authors proposed the use of decomposed multiscale Retinex for information preservation in tone-mapped image. Shu and Wu [33] employed an optimal local tone mapping operator to avoid halo artifacts and double edges. Rana et al. [34] proposed a pixel-wise adaptive tone mapping operator based on support vector regression to overcome the problem of drastic illumination variation.El Mezeni and Saranovac [35] introduced an enhanced local tone mapping operator (ELTM). ELTM decomposed an image into detail and base layers, where the base layer was compressed into both logarithmic and linear domain. Local tone mapping operator presented in [36] was operationally similar to HVS. Li and Zheng [37] presented a saliency-aware local tone mapping operator that preserved edges using guided filters. Ferradans et al. [38] proposed a two-stage algorithm incorporating both global tone mapping and local tone mapping. In the first step, they applied a global tone mapping operator based on the human visual system and followed by local method for contrast enhancement in the second stage. To make the local and global tone mapping algorithm more effective, Ambalathankandy et al. [39] used histogram equalization method implemented on FPGA with efficient resource usage. To resolve the overenhancement, a nonuniform quantization technique is proposed for CT image enhancement [40]. Recently, deep convolution neural network is also used for range compression [41]. The authors in [41] used the output of existing tone mapping operators as training set and therefore inherited the best properties of all tone mapping operators. Hence, they performed robustly well for all testing images compared to current methods of tone mapping.Li et al. [42] introduced a clustering-based content and color adaptive tone mapping algorithm that preserved the local structure and naturalness of image. They used K-means algorithm for the clustering of color structure. However, K-means is difficult to adopt for a specific problem due to prespecified number of clusters. In addition, the utilization of K-means for tone mapping has led to the loss of complex local detail in LDR images. Furthermore, the number of available clustering algorithms encourages the use of the most effective algorithm for clustering-based tone mapping algorithm. Therefore, to preserve the complex local details, we have adopted a better clustering technique.In this paper, we have proposed a clustering-based tone mapping algorithm to overcome the problems of false coloring, halo artifacts, and loss of complex local structure. For that purpose, we have utilized the fast search and find of density peak for content and color adaptive tone mapping. This clustering technique automatically recognizes clusters regardless of dimensionality of data. The following study compares various available state-of-the-art clustering algorithms for tone mapping operation. The effectiveness of proposed technique is compared with different clustering algorithms through subjective and objective evaluation techniques. The experimental analysis suggests that the fast search and find of density peak provides visually appealing result without compromising the local structure of an image.The rest of the paper is composed of the following sections. The second section provides a brief introduction to the clustering-based content and color adaptive tone mapping algorithm. The third section describes different applied clustering techniques. Furthermore, the fourth section discusses the experimentation conducted to evaluate the performance of different clustering techniques. Our overall work is concluded in the fifth section.


## 2. Clustering-Based Content and Color Adaptive Tone Mapping Algorithm

Li et al. [42] proposed clustering-based content and color adaptive tone mapping algorithm. Most of conventional tone mapping operators split an image into chrominance and luminance channels. Instead, Li et al. divided an image into overlapped color patches. The overall algorithm consists of training and testing phases. The algorithm [42] utilizes training phase to learn PCA transform matrix for each color structure cluster from HDR training images, while the testing phase is used to project the HDR test images and to find the closest match in the training set. In the first step, logarithmic transform is applied on each HDR image to enhance the contrast and brightness of low luminance values in a channel while compressing the higher ones. Afterward, each image is divided into overlapped color patches to avoid artifacts such as local graying out, hue shift, or color fringes [43]. Each patch is further divided into approximately uncorrelated components: color structure, color variation, and patch mean. Patches with varied intensity level may have similar structures [42]. Therefore, patches are grouped together into different clusters based on color structure and then PCA transform matrix is obtained for each cluster. In the testing phase, for each patch color structure, similarity measure is calculated with cluster centers extracted from the training data and then relevant PCA matrix is retrieved. For tone mapping, S curve arctan function is applied on PCA projection matrix. The patch mean is compressed by a linear function and the color variation is controlled by an arctan function. The image is reconstructed by processed patches. At the end, postprocessing step is performed for contrast enhancement by clamping image at its maximum and minimum intensity value.


## 3. Compared Clustering Techniques

To explore the influence of different clustering algorithms,K-means is a reference clustering algorithm and the results are compared with Gaussian Mixture Model (GMM) [44], DBSCAN [45], and fast search and find of density peak (FSFDP) [46] clustering algorithms. Before presenting the results obtained by using each of these clustering schemes, a brief description of each clustering technique is presented with its strengths and shortcomings.


### 3.1.K-Means

K-means is an extensively used partition-based clustering technique proposed by MacQueen. Partitioning can be done based on minimizing the objective function known as square error [47] which can be calculated as follows:(1)JV=∑i=1c∑j=1cixi−vi2,where xi−vi, “c,” and “ci” represented Euclidean distance, number of cluster centers, and the number of points in ith cluster, respectively. K-means was considered by Li et al. [42] for the clustering of the color structure. K-means is a relatively simple and computationally efficient algorithm. However, it suffers from the restriction of requiring human interaction for selecting the number of clusters, which provides easy implementation and effective results. Results are influenced by the initialization and may affect the performance of the tone mapping algorithm. Furthermore, K-means is sensitive to the cluster with a single data point due to its square distance [48].


### 3.2. Fast Search and Find of Density Peak

Fast search and find of density peak (FSFDP) [46] assumes that the density of data at the center of a cluster is relatively high and the center of one cluster is far away from the center of another cluster. Based on this assumption, FSFDP calculates the number of clusters and their centers automatically. Moreover, it detects and removes outliers intuitively. Clusters of nonspherical shape are easy to be recognized using FSFDP. In FSFDP, the cutoff distance used for calculating the density of each data point has a great influence on the effectiveness of FSFDP. For a data point i, FSFDP calculates local density, ρi, and distance, δi, from its nearest center as follows:(2)ρi=∑jXdij−dc,where dij, denotes the distance from data point i to j and dc is cutoff distance used to calculate the density of each data point. For assigning a data point i to the nearest cluster center, distance is calculated as(3)δi=minj:ρj>ρidij,if∃j,s,tρj>ρi,maxj:ρj>ρidij,otherwise.Equations (2) and (3) represent that the cluster centers are points with greater distance “δ” from other cluster centers and high data density “ρ.” The fast search and find of density peak is further experimented on two scale implementations [42] and represented as two-scale fast search and find of density peak (TFSFDP).


### 3.3. Gaussian Mixture Model

UnlikeK-means clustering, Gaussian Mixture Model (GMM) [44] allows clusters of different shapes. The cluster orientation is dependent on the Gaussian distribution. GMM sets the k number of Gaussians according to the data distribution. GMM learns the cluster belonging through the probability of each data point using its parameters such as variance Σk, mean μk, and weight of the cluster πk. Mathematically, it is computed by using (4)px=∑k=1KπkXxμk,Σk.GMM allows cluster overlapping; however, it is computationally extensive. Therefore, GMM is not applicable for high-dimensional data. Moreover, GMM is dependent on Gaussian distribution (clusters).


### 3.4. Density-Based Spatial Clustering of Applications with Noise

Similar to FSFDP, Density-Based Spatial Clustering of Application with Noise (DBSCAN) [45] is a density-based clustering algorithm that does not require the number of clusters as an initial parameter. DBSCAN identifies the clusters of various shapes and sizes based on data connectivity. Furthermore, DBSCAN is widely adopted because of its efficient computation. However, the algorithm of DBSCAN depends on the initial radius “r” of the cluster as follows:(5)Nr=i∈D|disti,j≤r.If the value of the radius is set to be small, all points that belong to the sparse cluster are considered as noise. On the other hand, if the radius is set to be large, all points lie within a single cluster. Therefore, to obtain appropriate results, the algorithm is executed for a number of times with different values of “r.”


## 3.1.K-Means

K-means is an extensively used partition-based clustering technique proposed by MacQueen. Partitioning can be done based on minimizing the objective function known as square error [47] which can be calculated as follows:(1)JV=∑i=1c∑j=1cixi−vi2,where xi−vi, “c,” and “ci” represented Euclidean distance, number of cluster centers, and the number of points in ith cluster, respectively. K-means was considered by Li et al. [42] for the clustering of the color structure. K-means is a relatively simple and computationally efficient algorithm. However, it suffers from the restriction of requiring human interaction for selecting the number of clusters, which provides easy implementation and effective results. Results are influenced by the initialization and may affect the performance of the tone mapping algorithm. Furthermore, K-means is sensitive to the cluster with a single data point due to its square distance [48].


## 3.2. Fast Search and Find of Density Peak

Fast search and find of density peak (FSFDP) [46] assumes that the density of data at the center of a cluster is relatively high and the center of one cluster is far away from the center of another cluster. Based on this assumption, FSFDP calculates the number of clusters and their centers automatically. Moreover, it detects and removes outliers intuitively. Clusters of nonspherical shape are easy to be recognized using FSFDP. In FSFDP, the cutoff distance used for calculating the density of each data point has a great influence on the effectiveness of FSFDP. For a data point i, FSFDP calculates local density, ρi, and distance, δi, from its nearest center as follows:(2)ρi=∑jXdij−dc,where dij, denotes the distance from data point i to j and dc is cutoff distance used to calculate the density of each data point. For assigning a data point i to the nearest cluster center, distance is calculated as(3)δi=minj:ρj>ρidij,if∃j,s,tρj>ρi,maxj:ρj>ρidij,otherwise.Equations (2) and (3) represent that the cluster centers are points with greater distance “δ” from other cluster centers and high data density “ρ.” The fast search and find of density peak is further experimented on two scale implementations [42] and represented as two-scale fast search and find of density peak (TFSFDP).


## 3.3. Gaussian Mixture Model

UnlikeK-means clustering, Gaussian Mixture Model (GMM) [44] allows clusters of different shapes. The cluster orientation is dependent on the Gaussian distribution. GMM sets the k number of Gaussians according to the data distribution. GMM learns the cluster belonging through the probability of each data point using its parameters such as variance Σk, mean μk, and weight of the cluster πk. Mathematically, it is computed by using (4)px=∑k=1KπkXxμk,Σk.GMM allows cluster overlapping; however, it is computationally extensive. Therefore, GMM is not applicable for high-dimensional data. Moreover, GMM is dependent on Gaussian distribution (clusters).


## 3.4. Density-Based Spatial Clustering of Applications with Noise

Similar to FSFDP, Density-Based Spatial Clustering of Application with Noise (DBSCAN) [45] is a density-based clustering algorithm that does not require the number of clusters as an initial parameter. DBSCAN identifies the clusters of various shapes and sizes based on data connectivity. Furthermore, DBSCAN is widely adopted because of its efficient computation. However, the algorithm of DBSCAN depends on the initial radius “r” of the cluster as follows:(5)Nr=i∈D|disti,j≤r.If the value of the radius is set to be small, all points that belong to the sparse cluster are considered as noise. On the other hand, if the radius is set to be large, all points lie within a single cluster. Therefore, to obtain appropriate results, the algorithm is executed for a number of times with different values of “r.”


## 4. Experiments and Results

This section describes the details of the dataset and the evaluation criteria used in our experiments. As different clustering methods have different parameters, the details of these are discussed in parameter setting section. The focus is to elaborate the influence of different state-of-the-art clustering algorithms on clustering-based content and color adaptive tone mapping algorithm, which is examined in performance and effectiveness section.


### 4.1. Dataset and Evaluation Criteria

For the training phase, Kodak’s database [49] is used, which consists of a total of 24 images of size of 512 × 768 × 3. All images are of true color (24 bits per pixel) and, for tone mapping, these images are used as standard set suit [49]. In the testing phase, we performed a series of experiments on Fun and Shi [50] HDR dataset to evaluate the influence of different clustering algorithms on clustering-based content and color adaptive tone mapping [42]. This dataset contains both indoor and outdoor HDR images of 105 scenes captured through Nikon D700 digital camera. Sample images from the dataset are shown in Figure 1.Figure 1
HDR tone-mapped source images [50].We compared the results of different clustering algorithms both qualitatively and quantitatively. Tone mapping operators attempt to preserve desirable characteristics, including local structure and naturalness, and to avoid the halo artifacts while converting an HDR image into LDR images. The classical evaluation parameters for objective measurements like PSNR cannot be used for the evaluation of tone mapping due to the unavailability of reference LDR images. So, we used two metrics for quantitative evaluation of dynamic range reduction with different clustering techniques: tone-mapped image quality index (TMQI) [51] and feature similarity index for tone-mapped images (FSITM) [52]. TMQI [51] is used to compute the structural resemblance and indexes of naturalness, and FSITM [52] is to measure the local phase similarity between HDR and LDR images.


### 4.2. Parameters Setting

Appearance regeneration of an image is dependent not only on the intensity relationship but also on many local weighted attributes including brightness, contrast, gray level, and color relationship. Clustering-based content and color adaptive tone mapping [42] is a patch base model in which the size and shape of the patch depend upon the window used for the calculation of mean and color structure. In this study, we have used the default window with size of 7 × 7 as defined by Li et al. [42]. We have used default values of the control parameter for color appearance, local structure, and luminance within the tone mapping process as used by [42].For clustering algorithms, the control parameters are set with the most optimal values. The details of these parameters for each clustering method are as follows.


#### 4.2.1. ForK-means

ForK-means clustering, the number of clusters is initially set to be 100.


#### 4.2.2. For Fast Search and Find of Density Peak

For fast search and find of density peak, cut-off distance “dc” given in equation (2) is significant to be set. However, varying its default value that is “1.4141” has no difference in objective evaluation as shown in Table 1; therefore we set it as 1.4141.Table 1
Effect of cutoff distance on FSITM and TMQI parameters for an image of size 1419 × 2130 × 3.
Objective evaluationdc = 1.3838dc = 1.4141dc = 1.9192dc = 2.3465dc = 2.8979dc = 3.2348dc = 4FSITM0.8710.8710.8710.8710.8710.8710.871TMQI0.8810.8810.8810.8810.8810.8810.881


#### 4.2.3. For Density-Based Spatial Clustering of Applications with Noise

Furthermore, for DBSCAN, we execute the algorithm several times to get to the appropriate value of “r” as mentioned in equation (5). On the default value of “r,” which is 10 for DBSCAN, the algorithm considers sparse datapoints as noise and therefore selected value of “r” to be 0.2.


#### 4.2.4. For Gaussian Mixture Model

For GMM, we used the default value of Gaussian distribution “k” in equation (4) as (2). The experimentation shows that the different values of Gaussian distribution have none or little effect on objective evaluation. However, Gaussian distribution’s larger values cause high computational cost. This is illustrated in Figure 2.Figure 2
Effect of Gaussian distribution “k” on time and FSITM and TMQI parameters.


### 4.3. Performance and Effectiveness

#### 4.3.1. Objective Evaluation

Tables2 and 3 present the results of different clustering techniques applied with the method of [42] in terms of TMQI and FSITM, respectively. The best results are highlighted in both tables. Green, blue, and yellow numbers represent the first, second, and third best scores, respectively; likewise red number represents the worst score. The effectiveness of tone mapping operators depends upon the contrast, brightness, and local structure of HDR image scenes. So, a tone mapping operator cannot be equally effective for all HDR images [53]. Table 2 depicts that TFSFDP performs best for 9 images; however, considering the average of each method, K-means and FSFDP can be used alternatively. Table 3 has strengthened the results further, as FSITM shows that TFSFDP performs best, while K-means and FSFDP stood in the second and third positions, respectively. Furthermore, DBSCAN performs worst in terms of both FSITM and TMQI for every image. Based on the obtained results, DBSCAN is not recommended for tone mapping.Table 2
Tone-mapped image quality index score.
S. no.With FSFDPWith TFSFDPWithK-meansWith GMMWith DBSCAN10.7580.7610.7660.7150.37320.7720.7800.7740.7230.22930.8180.8350.8210.6460.49040.8240.8200.8290.7810.49050.8230.8300.8260.7900.36560.8810.8930.8780.8200.46070.7620.7810.7540.7780.25880.7270.7300.7280.6250.40190.8800.8890.8780.8000.486100.7890.8040.7830.7440.426110.9170.9160.9070.7690.520120.8740.8730.8800.8180.358Avg.0.8190.8260.8190.6830.405Table 3
Feature similarity index for tone-mapped image score.
S. no.With FSFDPWith TFSFDPWithK-meansWith GMMWith DBSCAN10.8770.8910.8820.7750.67120.8700.8860.8730.7500.67030.8620.8710.8640.7020.62440.8590.8800.8640.8260.57450.8600.8740.8610.8300.57260.8710.8850.8750.7840.61870.8550.8610.8560.7070.71480.8860.8920.8880.7550.61890.8490.8650.8550.7540.643100.8420.8610.8450.7850.589110.8430.8600.8450.7920.667120.8270.8500.8340.7820.624Avg.0.8580.8730.8620.7700.632Performance of clustering-based tone mapping techniques is also compared in terms of their execution time. The execution times of different clustering-based tone mapping techniques are presented in Figure3, which highlights that GMM has high computation cost irrespective of the provided TMQI and FSITM results. Therefore, it is not a preferred clustering algorithm to be used for tone mapping operation. In terms of fastest execution time, K-means outperforms the other methods with least execution time of about “6.598 seconds,” whereas DBSCAN and FSFDP can also be considered for tone mapping operation as their execution times are about “11.31 seconds” and “12.218 seconds,” respectively.Figure 3
Average execution time with different clustering algorithm applied on five images with size of 535 × 401 × 3.


#### 4.3.2. Subjective Evaluation Assessment

Subjective evaluation is performed through visual assessment and scoring. Figures4 and 5 exhibit a visual comparison of clustering-based techniques using tone-mapped indoor and outdoor scene images. FSFDP, TFSFDP, and K-means produced the visually appealing results as these clustering methods better preserve the local structure and color saturation as shown in Figures 4(a)–4(c) and Figures 5(a)–5(c). In case of FSFDP, minor details like clouds near the shaded corner shown in Figures 4(a) and 4(b) and wood pattern of cutting board as in Figures 5(a) and 5(b) remain better preserved than K-means that can be clearly illustrated from Figures 4(c) and 5(c). GMM suffers from information loss and unpleasant effect. In Figures 4(d) and 5(d), details on a glass of the window and shelve are not visible. In case of DBSCAN, colors are oversaturated and result in unnatural visual appearances of image as depicted in Figures 4(e) and 5(e).Figure 4
Visual comparison of outdoor tone-mapped images with (a) FSFDP, (b) TFSFDP, (c)K-means, (d) GMM, and (e) DBSCAN.
(a)(b)(c)(d)(e)Figure 5
Visual comparison of indoor tone-mapped images with (a) FSFDP, (b) TFSFDP, (c)K-means, (d) GMM, and (e) DBSCAN.
(a)(b)(c)(d)(e)Subjective evaluation of 12 images is performed by 8 volunteers currently working on image processing and machine learning. We used a monitor with a spatial resolution of 2560 × 1600 to display images, side by side using the image viewer customizable window. We showed the resultant images of all methods to volunteers and asked them to score image from 1 to 10. “1” represents the worst score, while “10” is the highest score. Ratings are recorded by using score sheets provided to each volunteer during the subjective assessment. Figure6 shows that, for most of the images, FSFDP has the highest averaged score.Figure 6
Average subjective evaluation score.


## 4.1. Dataset and Evaluation Criteria

For the training phase, Kodak’s database [49] is used, which consists of a total of 24 images of size of 512 × 768 × 3. All images are of true color (24 bits per pixel) and, for tone mapping, these images are used as standard set suit [49]. In the testing phase, we performed a series of experiments on Fun and Shi [50] HDR dataset to evaluate the influence of different clustering algorithms on clustering-based content and color adaptive tone mapping [42]. This dataset contains both indoor and outdoor HDR images of 105 scenes captured through Nikon D700 digital camera. Sample images from the dataset are shown in Figure 1.Figure 1
HDR tone-mapped source images [50].We compared the results of different clustering algorithms both qualitatively and quantitatively. Tone mapping operators attempt to preserve desirable characteristics, including local structure and naturalness, and to avoid the halo artifacts while converting an HDR image into LDR images. The classical evaluation parameters for objective measurements like PSNR cannot be used for the evaluation of tone mapping due to the unavailability of reference LDR images. So, we used two metrics for quantitative evaluation of dynamic range reduction with different clustering techniques: tone-mapped image quality index (TMQI) [51] and feature similarity index for tone-mapped images (FSITM) [52]. TMQI [51] is used to compute the structural resemblance and indexes of naturalness, and FSITM [52] is to measure the local phase similarity between HDR and LDR images.


## 4.2. Parameters Setting

Appearance regeneration of an image is dependent not only on the intensity relationship but also on many local weighted attributes including brightness, contrast, gray level, and color relationship. Clustering-based content and color adaptive tone mapping [42] is a patch base model in which the size and shape of the patch depend upon the window used for the calculation of mean and color structure. In this study, we have used the default window with size of 7 × 7 as defined by Li et al. [42]. We have used default values of the control parameter for color appearance, local structure, and luminance within the tone mapping process as used by [42].For clustering algorithms, the control parameters are set with the most optimal values. The details of these parameters for each clustering method are as follows.


### 4.2.1. ForK-means

ForK-means clustering, the number of clusters is initially set to be 100.


### 4.2.2. For Fast Search and Find of Density Peak

For fast search and find of density peak, cut-off distance “dc” given in equation (2) is significant to be set. However, varying its default value that is “1.4141” has no difference in objective evaluation as shown in Table 1; therefore we set it as 1.4141.Table 1
Effect of cutoff distance on FSITM and TMQI parameters for an image of size 1419 × 2130 × 3.
Objective evaluationdc = 1.3838dc = 1.4141dc = 1.9192dc = 2.3465dc = 2.8979dc = 3.2348dc = 4FSITM0.8710.8710.8710.8710.8710.8710.871TMQI0.8810.8810.8810.8810.8810.8810.881


### 4.2.3. For Density-Based Spatial Clustering of Applications with Noise

Furthermore, for DBSCAN, we execute the algorithm several times to get to the appropriate value of “r” as mentioned in equation (5). On the default value of “r,” which is 10 for DBSCAN, the algorithm considers sparse datapoints as noise and therefore selected value of “r” to be 0.2.


### 4.2.4. For Gaussian Mixture Model

For GMM, we used the default value of Gaussian distribution “k” in equation (4) as (2). The experimentation shows that the different values of Gaussian distribution have none or little effect on objective evaluation. However, Gaussian distribution’s larger values cause high computational cost. This is illustrated in Figure 2.Figure 2
Effect of Gaussian distribution “k” on time and FSITM and TMQI parameters.


## 4.2.1. ForK-means

ForK-means clustering, the number of clusters is initially set to be 100.


## 4.2.2. For Fast Search and Find of Density Peak

For fast search and find of density peak, cut-off distance “dc” given in equation (2) is significant to be set. However, varying its default value that is “1.4141” has no difference in objective evaluation as shown in Table 1; therefore we set it as 1.4141.Table 1
Effect of cutoff distance on FSITM and TMQI parameters for an image of size 1419 × 2130 × 3.
Objective evaluationdc = 1.3838dc = 1.4141dc = 1.9192dc = 2.3465dc = 2.8979dc = 3.2348dc = 4FSITM0.8710.8710.8710.8710.8710.8710.871TMQI0.8810.8810.8810.8810.8810.8810.881


## 4.2.3. For Density-Based Spatial Clustering of Applications with Noise

Furthermore, for DBSCAN, we execute the algorithm several times to get to the appropriate value of “r” as mentioned in equation (5). On the default value of “r,” which is 10 for DBSCAN, the algorithm considers sparse datapoints as noise and therefore selected value of “r” to be 0.2.


## 4.2.4. For Gaussian Mixture Model

For GMM, we used the default value of Gaussian distribution “k” in equation (4) as (2). The experimentation shows that the different values of Gaussian distribution have none or little effect on objective evaluation. However, Gaussian distribution’s larger values cause high computational cost. This is illustrated in Figure 2.Figure 2
Effect of Gaussian distribution “k” on time and FSITM and TMQI parameters.


## 4.3. Performance and Effectiveness

### 4.3.1. Objective Evaluation

Tables2 and 3 present the results of different clustering techniques applied with the method of [42] in terms of TMQI and FSITM, respectively. The best results are highlighted in both tables. Green, blue, and yellow numbers represent the first, second, and third best scores, respectively; likewise red number represents the worst score. The effectiveness of tone mapping operators depends upon the contrast, brightness, and local structure of HDR image scenes. So, a tone mapping operator cannot be equally effective for all HDR images [53]. Table 2 depicts that TFSFDP performs best for 9 images; however, considering the average of each method, K-means and FSFDP can be used alternatively. Table 3 has strengthened the results further, as FSITM shows that TFSFDP performs best, while K-means and FSFDP stood in the second and third positions, respectively. Furthermore, DBSCAN performs worst in terms of both FSITM and TMQI for every image. Based on the obtained results, DBSCAN is not recommended for tone mapping.Table 2
Tone-mapped image quality index score.
S. no.With FSFDPWith TFSFDPWithK-meansWith GMMWith DBSCAN10.7580.7610.7660.7150.37320.7720.7800.7740.7230.22930.8180.8350.8210.6460.49040.8240.8200.8290.7810.49050.8230.8300.8260.7900.36560.8810.8930.8780.8200.46070.7620.7810.7540.7780.25880.7270.7300.7280.6250.40190.8800.8890.8780.8000.486100.7890.8040.7830.7440.426110.9170.9160.9070.7690.520120.8740.8730.8800.8180.358Avg.0.8190.8260.8190.6830.405Table 3
Feature similarity index for tone-mapped image score.
S. no.With FSFDPWith TFSFDPWithK-meansWith GMMWith DBSCAN10.8770.8910.8820.7750.67120.8700.8860.8730.7500.67030.8620.8710.8640.7020.62440.8590.8800.8640.8260.57450.8600.8740.8610.8300.57260.8710.8850.8750.7840.61870.8550.8610.8560.7070.71480.8860.8920.8880.7550.61890.8490.8650.8550.7540.643100.8420.8610.8450.7850.589110.8430.8600.8450.7920.667120.8270.8500.8340.7820.624Avg.0.8580.8730.8620.7700.632Performance of clustering-based tone mapping techniques is also compared in terms of their execution time. The execution times of different clustering-based tone mapping techniques are presented in Figure3, which highlights that GMM has high computation cost irrespective of the provided TMQI and FSITM results. Therefore, it is not a preferred clustering algorithm to be used for tone mapping operation. In terms of fastest execution time, K-means outperforms the other methods with least execution time of about “6.598 seconds,” whereas DBSCAN and FSFDP can also be considered for tone mapping operation as their execution times are about “11.31 seconds” and “12.218 seconds,” respectively.Figure 3
Average execution time with different clustering algorithm applied on five images with size of 535 × 401 × 3.


### 4.3.2. Subjective Evaluation Assessment

Subjective evaluation is performed through visual assessment and scoring. Figures4 and 5 exhibit a visual comparison of clustering-based techniques using tone-mapped indoor and outdoor scene images. FSFDP, TFSFDP, and K-means produced the visually appealing results as these clustering methods better preserve the local structure and color saturation as shown in Figures 4(a)–4(c) and Figures 5(a)–5(c). In case of FSFDP, minor details like clouds near the shaded corner shown in Figures 4(a) and 4(b) and wood pattern of cutting board as in Figures 5(a) and 5(b) remain better preserved than K-means that can be clearly illustrated from Figures 4(c) and 5(c). GMM suffers from information loss and unpleasant effect. In Figures 4(d) and 5(d), details on a glass of the window and shelve are not visible. In case of DBSCAN, colors are oversaturated and result in unnatural visual appearances of image as depicted in Figures 4(e) and 5(e).Figure 4
Visual comparison of outdoor tone-mapped images with (a) FSFDP, (b) TFSFDP, (c)K-means, (d) GMM, and (e) DBSCAN.
(a)(b)(c)(d)(e)Figure 5
Visual comparison of indoor tone-mapped images with (a) FSFDP, (b) TFSFDP, (c)K-means, (d) GMM, and (e) DBSCAN.
(a)(b)(c)(d)(e)Subjective evaluation of 12 images is performed by 8 volunteers currently working on image processing and machine learning. We used a monitor with a spatial resolution of 2560 × 1600 to display images, side by side using the image viewer customizable window. We showed the resultant images of all methods to volunteers and asked them to score image from 1 to 10. “1” represents the worst score, while “10” is the highest score. Ratings are recorded by using score sheets provided to each volunteer during the subjective assessment. Figure6 shows that, for most of the images, FSFDP has the highest averaged score.Figure 6
Average subjective evaluation score.


## 4.3.1. Objective Evaluation

Tables2 and 3 present the results of different clustering techniques applied with the method of [42] in terms of TMQI and FSITM, respectively. The best results are highlighted in both tables. Green, blue, and yellow numbers represent the first, second, and third best scores, respectively; likewise red number represents the worst score. The effectiveness of tone mapping operators depends upon the contrast, brightness, and local structure of HDR image scenes. So, a tone mapping operator cannot be equally effective for all HDR images [53]. Table 2 depicts that TFSFDP performs best for 9 images; however, considering the average of each method, K-means and FSFDP can be used alternatively. Table 3 has strengthened the results further, as FSITM shows that TFSFDP performs best, while K-means and FSFDP stood in the second and third positions, respectively. Furthermore, DBSCAN performs worst in terms of both FSITM and TMQI for every image. Based on the obtained results, DBSCAN is not recommended for tone mapping.Table 2
Tone-mapped image quality index score.
S. no.With FSFDPWith TFSFDPWithK-meansWith GMMWith DBSCAN10.7580.7610.7660.7150.37320.7720.7800.7740.7230.22930.8180.8350.8210.6460.49040.8240.8200.8290.7810.49050.8230.8300.8260.7900.36560.8810.8930.8780.8200.46070.7620.7810.7540.7780.25880.7270.7300.7280.6250.40190.8800.8890.8780.8000.486100.7890.8040.7830.7440.426110.9170.9160.9070.7690.520120.8740.8730.8800.8180.358Avg.0.8190.8260.8190.6830.405Table 3
Feature similarity index for tone-mapped image score.
S. no.With FSFDPWith TFSFDPWithK-meansWith GMMWith DBSCAN10.8770.8910.8820.7750.67120.8700.8860.8730.7500.67030.8620.8710.8640.7020.62440.8590.8800.8640.8260.57450.8600.8740.8610.8300.57260.8710.8850.8750.7840.61870.8550.8610.8560.7070.71480.8860.8920.8880.7550.61890.8490.8650.8550.7540.643100.8420.8610.8450.7850.589110.8430.8600.8450.7920.667120.8270.8500.8340.7820.624Avg.0.8580.8730.8620.7700.632Performance of clustering-based tone mapping techniques is also compared in terms of their execution time. The execution times of different clustering-based tone mapping techniques are presented in Figure3, which highlights that GMM has high computation cost irrespective of the provided TMQI and FSITM results. Therefore, it is not a preferred clustering algorithm to be used for tone mapping operation. In terms of fastest execution time, K-means outperforms the other methods with least execution time of about “6.598 seconds,” whereas DBSCAN and FSFDP can also be considered for tone mapping operation as their execution times are about “11.31 seconds” and “12.218 seconds,” respectively.Figure 3
Average execution time with different clustering algorithm applied on five images with size of 535 × 401 × 3.


## 4.3.2. Subjective Evaluation Assessment

Subjective evaluation is performed through visual assessment and scoring. Figures4 and 5 exhibit a visual comparison of clustering-based techniques using tone-mapped indoor and outdoor scene images. FSFDP, TFSFDP, and K-means produced the visually appealing results as these clustering methods better preserve the local structure and color saturation as shown in Figures 4(a)–4(c) and Figures 5(a)–5(c). In case of FSFDP, minor details like clouds near the shaded corner shown in Figures 4(a) and 4(b) and wood pattern of cutting board as in Figures 5(a) and 5(b) remain better preserved than K-means that can be clearly illustrated from Figures 4(c) and 5(c). GMM suffers from information loss and unpleasant effect. In Figures 4(d) and 5(d), details on a glass of the window and shelve are not visible. In case of DBSCAN, colors are oversaturated and result in unnatural visual appearances of image as depicted in Figures 4(e) and 5(e).Figure 4
Visual comparison of outdoor tone-mapped images with (a) FSFDP, (b) TFSFDP, (c)K-means, (d) GMM, and (e) DBSCAN.
(a)(b)(c)(d)(e)Figure 5
Visual comparison of indoor tone-mapped images with (a) FSFDP, (b) TFSFDP, (c)K-means, (d) GMM, and (e) DBSCAN.
(a)(b)(c)(d)(e)Subjective evaluation of 12 images is performed by 8 volunteers currently working on image processing and machine learning. We used a monitor with a spatial resolution of 2560 × 1600 to display images, side by side using the image viewer customizable window. We showed the resultant images of all methods to volunteers and asked them to score image from 1 to 10. “1” represents the worst score, while “10” is the highest score. Ratings are recorded by using score sheets provided to each volunteer during the subjective assessment. Figure6 shows that, for most of the images, FSFDP has the highest averaged score.Figure 6
Average subjective evaluation score.


## 5. Conclusion

Tone mapping is a complex field in which the conversion of HDR image to LDR image without information loss is needed. The intention of this article is to present the best clustering technique among existing techniques for preserving complex local details lost in case of clustering-based content and color adaptive tone mapping method. For this purpose, the influence of different clustering techniques is examined. The effectiveness is measured in terms of subjective evaluation and FSITM and TMQI values. Experiments show that fast search and find of density peak results in more appealing results for tone mapping with acceptable computational cost. In the future, this model can be extended by using a feature other than color structure for clustering of patches. Besides, any other function in place ofS shape arctan curve can be used for range compression.



---

*Source: 8846033-2020-08-17.xml*