Image contrast enhancement using triple clipped dynamic histogram equalisation based on standard deviation

IET Image ProcessingVolume 13, Issue 7 p. 1081-1089 Research ArticleFree Access Image contrast enhancement using triple clipped dynamic histogram equalisation based on standard deviation Majid Zarie, Majid Zarie orcid.org/0000-0002-1786-4521 Electrical Engineering Department, Khatam Al-anbia (pbuh) University, Tehran, IranSearch for more papers by this authorAli Parsayan, Ali Parsayan orcid.org/0000-0002-2573-4655 Electrical Engineering Department, Amirkabir University of Technology, Tehran, IranSearch for more papers by this authorHassan Hajghassem, Corresponding Author Hassan Hajghassem hajghassem@ut.ac.ir orcid.org/0000-0001-8950-8481 Faculty of New Sciences & Technology, University of Tehran, Tehran, IranSearch for more papers by this author Majid Zarie, Majid Zarie orcid.org/0000-0002-1786-4521 Electrical Engineering Department, Khatam Al-anbia (pbuh) University, Tehran, IranSearch for more papers by this authorAli Parsayan, Ali Parsayan orcid.org/0000-0002-2573-4655 Electrical Engineering Department, Amirkabir University of Technology, Tehran, IranSearch for more papers by this authorHassan Hajghassem, Corresponding Author Hassan Hajghassem hajghassem@ut.ac.ir orcid.org/0000-0001-8950-8481 Faculty of New Sciences & Technology, University of Tehran, Tehran, IranSearch for more papers by this author First published: 29 April 2019 https://doi.org/10.1049/iet-ipr.2018.5395Citations: 15AboutSectionsPDF ToolsRequest permissionExport citationAdd to favoritesTrack citation ShareShare Give accessShare full text accessShare full-text accessPlease review our Terms and Conditions of Use and check box below to share full-text version of article.I have read and accept the Wiley Online Library Terms and Conditions of UseShareable LinkUse the link below to share a full-text version of this article with your friends and colleagues. Learn more.Copy URL Share a linkShare onFacebookTwitterLinkedInRedditWechat Abstract Many histogram equalisations-based methods have been proposed to be applied in contrast enhancement. However, a few methods that can simultaneously create a natural enhancement in images with low, median, and high brightness ranges are suggested. Here, a robust contrast enhancement algorithm, which is called triple clipped dynamic histogram equalisation based on standard deviation (TCDHE-SD), is proposed. In the proposed method, the histogram of the input image is partitioned into three parts with approximately the same number of pixels based on the standard deviation. Then, the process of histogram clipping is performed on each sub-histogram. After that, all histograms will be mapped to a new dynamic range by means of applying some simple calculations, and finally, the sub-histogram equalisation process will be performed independently. This method is proposed to achieve the multiple purposes of the maximum average information content (entropy), controlling the enhancement ratio and preserving reasonable brightness. It also provides a natural enhancement by generating clear images with maximum details. The performance assessment of the proposed method in terms of entropy as well as visual quality based on the mean opinion score (MOS) indicates a significant advantage of it over the previous ones. 1 Introduction One of the important qualitative factors in the processing of different images is the contrast. If the contrast of an image is heavily concentrated in the dark, very bright, or in a particular range, image information is lost in regions concentrated excessively and uniformly. Therefore, the image contrast should be enhanced to show all the input image information [1]. Clearly, generally the main dynamic range of a scene is limited in the smaller dynamic range of the acquisition system [2]. Therefore, these imaging devices require using some processing algorithms known as dynamic range compression (DRC) to achieve a proper display range of intensity. Despite compressing the dynamic range, they introduce some unwanted effects such as saturated colours, halo, and so on. Some techniques are applicable as post-processing steps for images with exposure problems and important details hidden in the noticeable dark area. Most of these techniques use brightness adjustment and global contrast. Examples of the include point conversion, such as logarithmic transformation, non-linear gamma correction, automatic gain/offset, and global transforms such as histogram equalisation [3]. The gain/offset correction is a linear method that could generate artefacts with respect to saturation and clipping. Some details of darker regions are revealed by non-linear gamma correction or logarithmic correction, but the information stored in brighter areas is destroyed [2]. The most common method for enhancing contrast is the global histogram equalisation, which is used for almost all types of images due to its simple and appropriate performance. However, it is inefficient on images that contain too dark and bright regions [3]. A combination of these techniques is presented to compress the effective dynamic range and compensate for shadows in images with still colour [3]. This method is based on contrast stretching and recursive-filtering approaches conducted with respect to statistical measurements of the image and implemented under a logarithmic image processing. They result in enhancing low light areas while retaining colour and detail without creating visual artefacts. General histogram equalisation (GHE) is the most common method for enhancing contrast, which is used for almost all types of images due to its simple and fairly good performance. The original idea of ​​the GHE is to remap the grey levels of the input image based on the uniform expansion of the probability density function. GHE flattens and loosens the dynamic range of the histogram and increases the input image contrast as much as possible [4, 5]. The GHE method is not suitable for implementation in electronic devices, including TVs because this method leads to the production of annoying artefacts and noise amplification in the output image. One of the reasons for this problem is that it generally changes the brightness of the input image significantly. Moreover, in some uniform regions, it creates severely bright or dark saturation phenomenon [6-8]. Various methods have been proposed to overcome the limitations of the GHE method. One of the first suggestions was a method called brightness preserving bi-histogram equalisation (BBHE) [7]. In BBHE, the main histogram is partitioned into two sub-histograms based on the mean values of intensity. Then, the equalisation process is performed in both sub-histograms separately. Compared with the GHE method, the average brightness of the input image is preserved, but in some cases, it has an intensity saturation problem. After that, the dualistic sub-image histogram equalisation (DSIHE) is presented [9]. In this method, the main histogram is partitioned into two sub-histograms with equal number of pixels in which the median of the intensity values is used as the separator point. The DSIHE method claims to be superior to BBHE in terms of preserving the mean brightness and information (entropy). In some cases, BBHE and DSIHE methods do not enhance contrast or require a higher degree of mean brightness preservation. In this way, the recursive mean-separate histogram equalisation (RMSHE) was presented [10]. In RMSHE method similar to BBHE, the main histogram is partitioned into two parts. Subsequently, these sub-histograms are again partitioned into two parts based on their mean values. This process is repeated recursively to reach the optimal number of sub-histograms. Then, the equalisation process is performed separately in each sub-histogram. Similar to RMSHE another method called recursive sub-image histogram equalisation (RSIHE) is proposed [11]. In the RSIHE method, the histogram is recursively partitioned based on the median intensity values instead of the mean. In multiple-histogram partitioning methods, there is no significant enhancement in the presence of a large number of sub-histograms. It is also a great challenge to find the optimal value of the repeat factor in these methods. In order to overcome these issues, it is possible to perform partitioning based on histogram form such as multi-peak histogram equalisation with brightness preserving (MPHEBP) [12]. In the MPHEBP method, the original histogram is initially smoothed with a one-dimensional smoothing filter and then it is partitioned into several partitions based on the local maximum and each sub- histogram is equalised independently. Although the histogram-based methods preserve brightness well, but in sub-histograms with a narrow range, an insignificant enhancement results in the loss of image details, and in the sub-histogram with a wide range excessive enhancement creates annoying artefacts in the image because the sub-histogram does not extend beyond the specified dynamic range. To overcome this problem, the dynamic histogram equalisation (DHE) was presented [13]. In the DHE method, first the input histogram is smoothed by a smoothing filter and then it is partitioned based on the local minimum points. Then, before each equalisation process, each sub-histogram is mapped to a new dynamic range. The mapping process is a function of the dynamic range of the sub-histogram and the number of pixels forming each partition. An extension of the two lateral methods was presented as brightness preserving DHE (BPDHE) [8, 14]. In the BPDHE method after using the Gaussian smoothing filter, the local maxima points are used as separation points and claims that the maximum points preserve the mean brightness better than the minimum points. Before the equalisation process, each sub-histogram is mapped to a new dynamic range. To maintain the average image brightness, brightness normalisation is done in the last step. In this method, brightness normalisation ratio plays an important role in the output image because the small ratio causes an insignificant contrast enhancement and a larger ratio than one result exceeding the pixels from the output maximum dynamic range and causes saturation severity problem. Then, the singular value decomposition (SVD) is presented for enhancing the contrast of the images [15]. The singular value matrix represents the intensity information of the given image and any change on the singular values change the intensity of the input image. The proposed technique converts the image into the SVD domain and after normalising the singular value matrix, it reconstructs the image in the spatial domain by using the updated singular value matrix. The technique is called the singular value equalisation (SVE). All of the above methods have no control over the enhancement. Therefore, the bi-histogram equalisation with a plateau limit (BHEPL) method was proposed based on histogram clipping [16]. Similar to the BBHE in BHEPL, the main histogram is partitioned into two sub-histograms, then the control of enhancement ratio is done by setting the plateau limit as the mean of the occurrence of intensity in each sub-histogram. Bars in the histogram at each intensity level higher than the plateau limit are replaced by the plateau limit; otherwise the same histogram bars remain. Finally, the equalisation process is performed independently on both clipped sub-histograms. Then, the adaptive contrast enhancement method with brightness preserving including two enhanced methods of BHEPL called dynamic quadrants histogram equalisation plateau limit (DQHEPL) and bi-histogram equalisation median plateau limit (BHEPL-D) are proposed [17]. In the DQHEPL method, the main histogram is partitioned into four sub-histograms based on the median values of intensity. Then, the clipping process is performed in each sub-histogram according to the mean number of intensities. Then, before the equalisation process, each sub-histogram is mapped into a new dynamic range, while the second partitioning point is preserved as the mean brightness preserving. The limitation for brightness preserving prevents proper enhancement in images with up or down concentrated histogram bars. The steps of the BHEPL-D method are similar to those of the BHEPL method with the difference that the median number of intensity is used for histogram clipping. The median-mean-based sub-image-clipped histogram equalisation (MMSICHE) is presented [18]. In MMSICHE, the main histogram is initially divided into two partitions based on the median of the intensity values. Then, based on the mean intensity values in each sub-histogram, it is divided into four partitions. Then, the clipping process of the main histogram is carried out based on the median of the number of the occurrences of intensity and finally each partition is equalised separately. After that, the exposure-based sub-image histogram equalisation (ESIHE) was presented to enhance the images exposed to low bright [19]. In the ESIHE method, the exposure threshold is used to partition the original histogram into two parts [20]. Then, before performing the equalisation process, the process of the main histogram clipping is performed according to the mean number of intensities. Following that, the expanded ESIHE called recursively separated exposure-based sub-image histogram equalisation (RS-ESIHE) was proposed [21]. In the RS-ESIHE method, the original histogram is reciprocally partitioned into several sub-histograms based on the exposure threshold. Before performing the equalisation process, the process of main histogram clipping is done generally based on the mean number of intensities. Recently the entropy-based dynamic sub-histogram equalisations (EDSHE) method is proposed [22]. In the EDSHE method, the original histogram is first partitioned into two sub-histograms with equal entropy. Then, these sub-histograms are partitioned into two parts again according to entropy. This process is recursively repeated to achieve the optimal number of sub-histograms until meeting the stop condition. Then, before each equalisation process, each sub-histogram is mapped to a new dynamic range. There are two major disadvantages in this method. First, not performing the clipping process leads to the situation where no control in present on the enhancement ratio. Second, due to recursive and high number of partitioning, significant enhancements are not observed in some parts. Each of the above methods has been proposed to solve a special problem in the field of contrast enhancement. The most popular goals in most methods are to achieve a clear image in the output with maximum detail, away from annoying artefacts such as intensity saturation and natural enhancement. Nevertheless, to what extent they have reached these goals should be sought in evaluations and output images. The proposed TCDHE-SD method creates a reasonable and balanced relationship between enhancement ratio control, preserving the mean brightness and preserving average information content. One of the strengths of this article, which distinguishes it from similar articles, is the use of images with varied lighting changes. In other words, it can simultaneously create an image with maximum details and natural enhancement in images with low, median, and high brightness ranges, which is away from annoying artefacts and saturation severity problem. In this method, first the histogram of the input image is partitioned into three partitions with almost equal number of pixels based on standard deviation. Then, in order to control the enhancement ratio, the process of clipping the histogram is then applied to the same threshold level as the mean of occurring intensities in each sub-histogram. Subsequently, each sub-histogram is mapped to a new dynamic range and finally the equalisation process is performed independently in each sub-histogram. Other parts of the article are organised as follows. In the second section, the proposed TCDHE-SD method is addressed in detail. In the third section, the results of the tests are discussed based on the visual quality and the content of the average information. Conclusion is presented in the fourth section. 2 Triple-clipped dynamic histogram equalisation based on standard deviation The proposed TCDHE-SD method consists of the following four sections, which are discussed in full detail. Triple partitioning of the histogram Histogram clipping Mapping each partition to a new dynamic range Equalising each partition independently 2.1 Triple partitioning of the histogram As mentioned in the introduction section, the histogram of the input image is partitioned into several sub-histograms using the mean, median, the maxima points, the minima points, the exposure level, and entropy. In images with noise, even the use of a smoother filter in the input histogram prior to the partitioning will not help to select the most suitable maxima and minima points and these points will be detected incorrectly. Also, there should be significant valleys or peaks in the histogram of the input image in these methods. On the other hand, in case of using the mean, median, and exposure level once for partitioning the histogram into two partitions, abnormal enhancement may be created in some areas such as the general histogram equalisation (GHE). Meanwhile, the use of the mean, median, entropy, and exposure levels, recursively, that partition the histogram into four or more partitions may not result in significant enhancement in some partitions. Also, finding the optimal reciprocal level is a major challenge in these methods. So to overcome the above cases, the proposed TCDHE-SD method partitions the histogram of the input image into three partitions so that the number of pixels in the three sub-histograms is almost the same. The triple histogram partitioning process consists of the following two steps. In the first step, first use the estimated Gaussian distribution of each type of input image, which is modelled using its mean and standard deviation, each pixel of the image can be considered as a random variable with a distribution function. According to the central limit theorem, the sum of a sequence of random variables tends to the Gaussian distribution [1]. Therefore, the estimated Gaussian distribution of the input image can be used and partition the histogram into three equal partitions with a probability of 0.33. Accordingly, the mean value and standard deviation of the given input image X is obtained using (1) and (2) [4]. (1) (2)In the above equations X (i, j) represents the intensity of the image, μ is the mean, σ is standard deviation and M and N are the input image dimensions. In order that, the estimated Gaussian distribution histogram in the input image is partitioned into three equal partitions with a probability of about 0.33: (3)In the above equation, L–1 is the highest pixel intensity in the n-bit image and it is equal to 2n and is a positive integer. In order to obtain its value as an example, the middle interval is considered and a new variable Z is created by standardisation. (4)Given the equations between distributions, . is a standard normal random variable Z∼N (0, 1). Thus, (5)In the above equation is the standard normal cumulative distribution. Using table of standard normal cumulative probabilities the value is obtained. It should be noted that the value of has the same value for the other two intervals. In the second step, the m1 and m2 partitioning points are placed equal to the intensity levels, which are obtained according to (6) and (7). (6) (7)Thus, if the histogram of the input image before the partitioning process has the intensity range [m0: m3], m0 and m3 have the lowest and highest values in the input image. In the proposed TCDHE-SD method, the histogram of the input image W is partitioned into three sub-histograms with approximately equal number of pixels called WL, WM, and WU. The intensity range of each sub-histogram is equal to [m0: m1–1], [m1: m2–1] and [m2: m3], respectively. Fig. 1 shows an example of the triple partitioning. Fig. 1Open in figure viewerPowerPoint Example of a triple partitioning in the proposed TCDHE-SD method 2.2 Histogram clipping The main idea behind histogram clipping is that it controls the enhancement ratio in the histogram equalisation process so that it prevents creating intensity saturation and excessive enhancement in the enhanced image and leads ultimately to natural appearance in the enhanced image. To perform the histogram clipping process, it is necessary to limit the first derivative of the histogram or the histogram of the input image. Therefore, in the clipped histogram, the shape of the input histogram is changed based on a specified threshold level before performing the equalisation process. The bars of the histogram, which have a surface larger than the threshold, are limited by threshold levels. In some cases, controlling the enhancement ratio fails by using threshold level based on the median of the number of the occurrences of intensity. Also, in method with multiple sub-histograms, using a clipping process on the main histogram instead of clipping on each sub-histogram leads to lack of control of a proper enhancement ratio. Given the above cases histogram clipping in the TCDHE-SD method is such that the mean of the intensity at each sub-histogram is considered to be the histogram clipping threshold [16]. The clipping threshold levels are calculated using (8)–(10). (8) (9) (10)In the above equations, h (k) is the histogram of the input image and T1, T2, and T3 are the clipping threshold levels calculated for each of the three sub-histograms. In Fig. 2, the threshold levels calculated for each of the three sub-histograms are observed. Then, the bars of any sub-histogram with a greater level for that histogram are limited by the threshold level. The clipped histogram is obtained using (11)–(13). (11) (12) (13) Fig. 2Open in figure viewerPowerPoint Example of Calculation of clipping threshold in the TCDHE-SD method In the above equations, hc (k) indicates the clipped histogram. Therefore, the histogram clipping method used in the TCDHE-SD method enables to control the enhancement ratio well throughout the histogram. Fig. 3 shows a clipped histogram. Fig. 3Open in figure viewerPowerPoint Example of clipped histogram in the TCDHE-SD method 2.3 Mapping each partition to a new dynamic range As stated in the partitioning section, each sub-histogram has a specific intensity range. In the case of an equalisation process for each part independently, each of the sub-histograms cannot perform histogram equalisation and expansion beyond the specified range. In most cases, the sub-histogram with limited range are not significantly enhanced due to lack of space for enhancement. On the other hand, in a sub-histogram with a large range, the problem of over-expansion is created. These reasons cause the loss of image detail and saturation of the intensity in the output image. In order to balance the equalisation process and overcome the above problems, the TCDHE-SD method with the idea of dynamic histogram equalisation [13] carries out the mapping process based on the dynamic range of grey level used in each sub-histogram and assigns a new dynamic range for each sub-histogram. However, unlike the dynamic histogram equalisation method, which makes the calculations complex by applying the number of pixels in each sub-histogram, in the TCDHE-SD method, the number of pixels in the sub-histogram is roughly equal and does not have a significant effect on the calculation of the new dynamic range. To simplify the mapping steps, it is possible to use the number of pixels in the calculation. The mapping process is performed in the TCDHE-SD method using (14)–(17). (14) (15) (16) (17)In the above equations, L–1 is the highest pixel intensity in the n-bit image and it is equal to 2n. Therefore, after performing the mapping process, the new dynamic ranges of the first, second, and third sub-histograms are [n0: n1–1], [n1: n2–1], and [n2: n3], respectively. The mapping process for the new dynamic range is shown in Fig. 4. Fig. 4Open in figure viewerPowerPoint Example of a mapping process to a new dynamic range in the TCDHE-SD method 2.4 Equalising each partition independently After specifying a new dynamic range for each sub-histogram, each part must be independently equalised in the final stage of the TCDHE-SD method. The process of equalising each sub-histogram consists of four steps. In the first step, the PDF for each sub-histogram is determined according to (18)–(20) [4]. (18) (19) (20)In the above equations, NL, NM, and NM are the total number of pixels in the clipped sub-histogram. In the second step, the cumulative density function (CDF) of each sub-histogram is defined by (21)–(23) using the corresponding PDF. (21) (22) (23)In the third step, the transfer function of each sub-histogram is determined using the CDF for each sub-histogram according to (24)–(26). (24) (25) (26)Using the obtained transfer functions, each partition is equalised independently. The fourth step involves integrating all three sub-images to create a complete image. Therefore, the output image of the TCDHE-SD method is produced by combining all three transfer functions. 3 Results of experiments and discussions In this section, the simulation results of the proposed TCDHE-SD method are compared with the methods as GHE [4], BBHE [7], RSIHE [11], BPDHE [8], SVE [15], BHEPL [16], DQHEPL [17], and MMSICHE [18]. In order to evaluate and compare the existing methods, four images of the Tomb of Saadi, tank, aircraft, and road with histogram concentrated in very dark, median to dark, median to bright, and very bright ranges are selected as the test images. The simulation results of the proposed TCDHE-SD method and other methods are shown in Figs. 5-8. In order to visualise the capability of preserving the details and to demonstrate the obvious effect of different contrast enhancement methods on fading or expansion of different intensity levels, the histogram results for each of the four images are shown in Fig. 9. In the final part of this section, the resistance of the proposed algorithm to noise is investigated on the two sample images of aircraft and the road. The simulation results of the proposed TCDHE-SD and other methods are shown in Fig. 9 and 10. Fig. 5Open in figure viewerPowerPoint Image of Tomb of Saadi with histogram concentrated in the very dark range Fig. 6Open in figure viewerPowerPoint Image of a tank with histogram concentrated in the median to dark range Fig. 7Open in figure viewerPowerPoint Image of an aircraft with histogram concentrated in the median to bright range Fig. 8Open in figure viewerPowerPoint Image of a road with histogram concentrated in the very bright range Fig. 9Open in figure viewerPowerPoint Histogram of the images (axis X: intensity and axis: Y number of pixels) (a) Tomb of Saadi, (Fig. 5), (b) Tank (Fig. 6), (c) Aircraft (Fig. 7), (d) Road (Fig. 8) Fig. 10Open in figure viewerPowerPoint Image of an aircraft with histogram concentrated in the median to bright range (input image is contaminated by a Gaussian noise with mean 0 and variance 0.001) 3.1 Performance evaluation based on average information content If the contrast of an image is heavily concentrated in the dark, bright, or in a particular range, information may be lost in areas that are concentrated excessively and uniformly. Among the contrast enhancement methods, the most appropriate is the one that, while enhancing the natural contrast, can display all the input image information. Therefore, in order to demonstrate the capability of the proposed TCDHE-SD method to extract information from the image, it is possible to use the average information content (entropy) as a quantitative evaluation. Entropy measures the amount of richness of the image details and is usually measured in bits. The larger entropy value indicates that more information is available from the image. The entropy is defined by the following equation. [22-24]. (27)In the (27), PDF (k) is the probability density function (PDF) from an image at the intensity level k and L–1 is the highest pixel intensity in the image. Therefore, in an image with a high entropy value, more features can be extracted from the image. However, for a better understanding of the image addition to the actual information (entropy), the redundant information is also required. The proposed algorithm improves visual quality by changing the type and size of the redundant information. For optimal performance, the entropy should be close to the original image so that the actual image information is preserved. In Table 1, the entropy results are tabled using different methods for each of the four images. According to the above table, the proposed TCDHE-SD method produces the highest entropy for all images compared to other methods. Therefore, in all images, the produced entropy is very close to the original image. The average of the entropy generated by the proposed TCDHE-SD method is 5.250, which is close to 5.264 and is the average entropy of the original images. Therefore, given the high and close to the original image entropy, the proposed TCDHE-SD method is the best way to extract information content in the image. It also guarantees natural enhancement in the output image. Table 1. Entropy results in different methods Method INPUT GHE BBHE RSIHE BPDHE SVE BHEPL DQHEPL MMSICHE TCDHE-SD Fig. 5 5.220 5.054 5.095 5.012